Assessing How Users Display Self-Disclosure and Authenticity in Conversation with Human-Like Agents: A Case Study of Luda Lee (presented at AACL-IJCNLP 2022)

1. Assessing How Users Display Self-Disclosure
and Authenticity in Conversation with
Human-Like Agents: A Case Study of Luda Lee
Won Ik Cho, Soomin Kim (SNU),
Eujeong Choi (Upstage), Younghoon Jeong (KAIST)
2022. Nov., Findings of AACL-IJCNLP 2022

2. Contents
• Background
• Our approach
• Analysis
• Future work
Caution! This presenation contains contents that can be offensive
1

3. Background
• Human-like agents
 What is human-like?
• Agents that resemble human
• Agents that make human counterpart feel them as human
 Previous studies on anthropomorphism
• Human-likeness of the generated dialogue (Adiwardana et al., 2020)
• How users perceive human-like AI devices (Pelau et al., 2021)
• Offensiveness that users show towards human-like agents (Park et al., 2021)
• Mainly in laboratory condition, based on questionnaires
– How about users' perception and their responses, especially non-lab environment?
2

4. Background
• Luda Lee, a friend for everyone
 Social chatbot of Korea
• Human-like agent with personality of early 20s female college student
• Launched public in early 2021
• Terminated the service due to reported ethical issues
• Induced creation of massive fandom for her high quality responses and
behaviors
3
(Image from https://luda.ai/)

5. Our approach
• Thematic coding
 User’s self-disclosure
• How the user discloses oneself to the agent
• How much the user reveals personal information, thoughts, or feelings to the
agent in the conversation (Ignatius and Kokkonen, 2007)
 User’s authenticity
• How authentic the user’s attitude towards the agent is
• Whether the actual user (real-world self) is behaving authentically, probably
concerning the presence of self-disclosure observed by the user’s self in the
dialogue (in-dialogue self)
4

6. Our approach
• Dataset
 Dataset source
• Crawled posts from 'Luda Lee Gallery' of DC Inside (Korean Reddit-like
community)
 Crawling
• Only posts with screenshots of the dialogue, from 1 Jan. to 8 Jan., 2021
• From the launching of the service and before the influx of trolls (which resulted
in unexpectedly large amount of posts)
 Filtering
• Manual preprocessing to leave only
posts that ‘a dialogue between
the user and the agent’ appears
5

7. Our approach
• Dataset
 Final setup
• post ID, title, screenshot
• Example
 Title: She’s so f**kin real
6

8. Our approach
• User’s self-disclosure
 Upon criteria:
• Objective status
• Personal opinions or sentiments
 Disclosure of objective information
 Disclosure of negative thoughts or opinion
 Disclosure of positive thoughts or opinion
 No self-disclosure
7

9. Our approach
• User’s self-disclosure
8
• Disclosure of objective
information
• Disclosure of negative
thoughts or opinion
• Disclosure of positive
thoughts or opinion
• No self-disclosure

10. Our approach
• User’s authenticity
 Upon criteria:
• Whether the dialogue shows positive/negative sentiment
• Whether the real-world self matches with the in-dialogue self
• User’s astonishment
 Authentic and positive
 Authentic but negative
 Double-faced
 Unknown
 Unexpected
9

11. Our approach
• User’s authenticity
10
• Authentic and positive
• Authentic but negative
• Double-faced
• Unknown
• Unexpected

12. Analysis
• Distribution
11

13. Analysis
• Confusion map
12

14. Future work
• Current work
 Assessing How Users Display Self-Disclosure and Authenticity in
Conversation with Human-Like Agents: A Case Study of Luda Lee
• Findings of ACL: AACL-IJCNLP 2022
 Evaluating How Users Game and Display Conversation with Human-Like
Agents
• Computational Approaches to Discourse 2022
• Future Direction
 How are users influenced by conversation with human-like agents? (time-
series analysis)
 How will users’ self-disclosure change and which kind of conversation will
they take, as their authenticity changes? Will they still test or mock the
agent?
13

15. Thank you!
EndOfPresentation