Virtual Assistant Personality Preference Among Urban English Speakers - An investigation into how bot personality impacts customer experience ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Virtual Assistant Personality Preference
Among Urban English Speakers
An investigation into how bot personality impacts customer experience
during interactions with AI-powered assistants
By Brinda Mehra
Report Summary Virtual assistants, chatbots and other artificially intelligent conversational agents have been gaining an immense amount of traction in recent times. Many market research companies pre- dict that interactions with these AI-powered assistants will boom over the next few years. How- ever, the current satisfaction levels with virtual assistant interactions are not as high as they could be, as these assistants sometimes fail to deliver on the quality of the interaction. This paper looks at the possibility of integrating personality into virtual assistants in order to understand whether personality can have an actual positive impact on user perceptions of and experiences with these assistants. While the concept of ‘chatbot personality’ has been well-documented in the context of general conversational agents, there is a lack of exploration with regards to enterprise virtual assistants and personality –– a gap which this paper attempts to address.
Table of Contents SECTION I - OVERVIEW 1 Understanding the need to improve the quality of human-bot interactions through imbued personality. SECTION II - THEORETICAL BACKGROUND 2 Exploring the theoretical framework underlying the study, as well as successful implementations of bot personality in the industry. SECTION III - DEFINING THE BOT PERSONALITIES 4 The selection of three distinct bot personality types for the purpose of the study - Transactional, Prosocial and Friendly. SECTION IV - EXPERIMENTAL DESIGN 6 The methodology for the two experiments conducted as part of the study. SECTION V - RESULTS 8 A summary and analysis of the findings derived from the two experiments. SECTION VI - CONCLUSION 12 The implications of the study and the road ahead. REFERENCES 13
Section I - Overview
By 2020, the average person was predicted to have more daily interactions with an AI-powered
chatbot or virtual assistant than with their own spouse [1]. This prediction, made in 2016, has not
entirely borne out, but at the time it would have been great news for the AI industry. After all,
one of the key aims for artificial intelligence is to provide a conversational experience that is
indistinguishable from that of a human being [2]. One can argue then that people holding more
conversations with their virtual assistants than their spouses would certainly achieve that goal,
if not exceed it.
However, as we draw closer to 2020, it does not seem like the current level of satisfaction with
chatbot interactions heralds the beginning of a golden age for virtual assistants.
The general reception of virtual assistants is more positive than negative – 38% of people
globally rate their perception of virtual assistants as positive whereas only 11% rate it as neg-
ative - but the majority remain neutral [3]. This suggests that AI companies must understand
how to leverage the conversational experience in order to both gain and retain users.
Although there may be multiple ways in which companies are able to augment the conversa-
tional experience in order to achieve their goal of user engagement and retention, this paper
will attempt to make an argument for the development of personality within the virtual assistant
to achieve this goal. Research on the role of personality in humans suggest that it plays a key
role in maintaining social relationships as well as making social interactions productive and
enjoyable, which has major implications for user interactions with virtual assistants[4].
This paper explores how personality can influence customer experiences with AI assistants
across two different use-cases: food-ordering and education.
1Section II - Theoretical Background
Current research on the conversational abilities of chatbots and virtual assistants suggests that
there is a major gap between the level of the conversation that these assistants can hold and
user expectations of the interaction. Although such agents promise human-like dialogue, many
of these interactions often fall short of the mark [5].
Virtual assistants are able to embody the functional aspects of human conversation but
often fail to embody and encapsulate the major characteristics that define human conver-
sation, thus causing users to feel that interactions with such software agents are dry, stilt-
ed and uncomfortable [6].
In fact, many feel that the key attributes of a human conversation that make a social interaction
successful (such as the ability to understand intent, trust, active listening and humor) cannot be
organically developed in a conversational interaction with a virtual assistant and express their
doubts that a software agent could ever achieve the ability to hold conversations like a human
[7].
Naturally, this poses several problems for the widespread adoption of virtual assistants in daily
life, as users are unlikely to continue interacting with these assistants if all their interactions are
constrained, task-based and leave no scope for any further interaction. Moreover, when this
lack of conversational abilities is coupled with the fact that users tend to personify computer
agents, often treating them like real people and expecting empathy, humor and other human
characteristics from the interaction, there is an underlying need for these AI assistants to devel-
op more anthropomorphic qualities in order to fulfill user expectations [8].
Perhaps the easiest way to anthropomorphize chatbots and virtual assistants is to imbue
them with a personality.
Although recent reports suggest that personality is not a key requirement for a satisfactory
interaction, with an estimated 48% of users stating that they would ‘rather interact with a virtual
assistant that solves their problems’ than one with personality, one can make an argument for
the impact that personality can have [3]. Perhaps the most glowing recommendation for the
development of personality comes with the phenomenal success of Microsoft’s Xiaoice, a gen-
eral conversational chatbot deployed on the Chinese social media network WeChat. Developed
using an Empathic Computing Framework, NLG and context retention, Xiaoice mimics the per-
sonality of an eighteen year old girl who is witty, self-effacing and has a wonderful sense of
humor, often flirting with her users and engaging them in deep conversations [9]. The presence
of this personality plays a major role in Xiaoice’s success – it boasts of around 660 million users
worldwide and a Conversation Turns Per Unit metric (CPS) of 23, which is a significantly higher
number than that of many other chatbots [10].
2Naturally, this leads many to question whether personality is the main factor behind Xiaoice’s
success. Although multiple other factors, such as the presence of image recognition or senti-
ment analysis, play an equally important role in its success, the key role personality plays in
human social relationships suggests that it may also be the reason behind Xiaoice’s immense
popularity.
Research on the role of personality in human social interactions suggests that it can influence
how satisfying a social interaction can be, with an interaction between compatible personalities
being far more productive and enjoyable than any other interactions [11], [4]. Similarly, personality
is also responsible for helping classify other people’s behaviour into simplifying traits, which
then allows us to predict other people’s behaviour and guide our own responses to it, making it
a linchpin for the success of human relationships [12], [13].
Although research on the importance of personality has focused exclusively on human relation-
ships until now, it becomes imperative to understand how this information can be applied to
human-computer interaction (HCI) because of new developments in understanding how the
human brain reacts to technology. According to the Media Equation theory, humans tend to
treat technology and technologically mediated communication in the same way they would
According
treat to the Media Equation theory, humans tend to treat technology and technologi-
real people.
cally mediated communication in the same way they would treat real people.
This process happens subconsciously and automatically, with studies suggesting this reaction
may occur because human brains have not evolved to deal with twenty-first century technology
and therefore are unable to distinguish many facets of the real world from the virtual [14], [15].
Given the spate of research that supports Media Equation Theory and the importance of per-
sonality in human social interactions, there has naturally been a lot of interest garnered in how
AI companies and researchers can capitalize upon these psychological phenomena to amelio-
rate virtual assistant interactions, with chatbot personality slowly becoming a key area of focus.
Building upon this, Haptik carried out a small but in-depth research experiment on how person-
ality can influence user experiences in order to contribute to the existing literature on this topic,
as well as broach the discussion of deploying virtual assistant personality in an enterprise con-
text, which has historically been wary of bringing in any sort of personality into bot interactions
until fairly recently.
3Section III - Defining The Bot Personalities
On the basis of previous research into virtual assistant personality, Haptik’s study also utilized
the Five-Factor Model of personality (FFM) in order to develop the three key personalities it
would test. In psychological literature, FFM consists of five broad trait dimensions that are the
most common descriptors used to describe people. These five dimensions, namely Openness
to Experience, Conscientiousness, Extraversion, Agreeableness and Neuroticism represent a
range between the two extremes of the trait, where most people tend to fall (Goldberg, 1993).
The three key bot personality types were developed using IBM’s Primary and Secondary Char-
acteristics Table, which is also based on the FFM and used three different combinations of the
traits of Extraversion, Agreeableness and Conscientiousness to define the bot personalities, as
described below.
Transactional Personality
I'd like to book
Transactional Personality: A transactional personality would a flight.
be defined as one that is serious, highly principled and restrained.
According to the IBM chart, this chatbot would score high on Con-
scientiousness and low on Agreeableness, while the interaction
with this personality would be highly goal-oriented and have little-
Sure. Give me a
to no social niceties involved, with sharp, to-the-point text. minute, while I
search for flights.
Prosocial Personality
I'd like to book
a flight.
Prosocial Personality: A prosocial personality, by definition, is one
that is extremely helpful. It is also cooperative, polite and consider-
ate, scoring high on both Conscientiousness and Agreeableness in
the IBM Chart. Interactions with this personality would contain sev-
eral social niceties, repeated assurances of its desire to help and
Hi! I'd be glad to some markers of engagement like exclamation marks and emoti-
help you search
for a flight. cons.
4Friendly Personality
I'd like to book
Friendly Personality: Much like its name suggests, interactions a flight.
with this bot are meant to be reminiscent of an interaction with a
friend. Vibrant, enthusiastic and social, this personality will score
high on the dimensions of Extraversion and Agreeableness, utilis-
ing several markers of engagement like images, gifs, emoticons,
Hey there! Just
letter reduplication and slang in order to make the interaction sit back and relax
more casual in nature while I find you
a flight!
Since Haptik’s Bot Builder Tool operates on a rule-based system, these personalities were
developed manually rather than generativally via a corpus database. In order to test for accura-
cy, sections of the chatbot copy were processed via IBM’s emotion analyser to check whether
the correct sentiment was being expressed. The results suggested that the personalities were
fairly well defined, with the Transactional personality receiving a dominant tone of Analytical
(0.66); Prosocial personality, the dominant tones of Joy (0.61); and Friendly personality, the
dominant tones of Joy (0.63) and Confidence (0.66). In order to test for the applicability of per-
sonality across use-cases, two use cases were tested to demonstrate how personality impacts
user experience.
5Section IV - Experimental Design
EXPERIMENT I
The first study focused on a food-ordering use-case, with users interacting with three virtual
assistants for the fictional company of WcDonalds. These bots were developed on Haptik’s Bot
Builder Tool and were in accordance with the description of the three personality types men-
tioned in the previous section. Forty users aged 18-53 were asked to participate in this experi-
ment (M = 31.25) in a repeated-measures design, which entailed all users interacting with all
three versions of the WcDonalds chatbot, namely: WcBot (the transactional bot), Roland Wc-
Donald (the prosocial bot) and Ron (the friendly bot). Following their interactions with each of
the chatbots, they were asked to rate their experience for each of the chatbots across three
key dimensions on a Likert type item questionnaire. These dimensions were:
Likelihood of Re-interaction
Comfort Level
Productivity & Efficiency
WcBot is online Roland WcDonald is online Ron is online
A.I Powered WcDonalds Chatbot Wc Donald’s very own Chatbot! Wc Donald’s Chatbot at your service
IT’S TIME TO GET THIS PAR-TAY
STARTED
Hey there, I’m the WcDonald’s
Chatbot Roland WcDonald
Get Started YESS LET’S GO!
I would love to help you get the best
meal of your life today!
Hello, I am the WcDonald’s Chatbot So, what do you feel like starting
with?? Pick one even tho it’s hard
Please choose how you would like to
This is how I can assist you:
start:
Place Orders Directly Show Extra Value Deals HUNGRY! Let me order now!! Deal Me Up!
Let’s Order Directly! Check Extra Value Deals
Store Locator FMY Please!
Locate Stores
6EXPERIMENT II
The second study was a conceptual replication of the first and tested both a different use-case
as well as a different age group in order to understand whether personality could affect user
experiences across use-cases and demographics. Twelve participants aged 11-15 were asked to
watch three .mp4 files of an interaction with a chat-based dictionary on a Facebook Messenger
interface, which was developed using BotSociety. Much like the previous study, this study also
tested all three previously defined personalities. However in this case, the names of the bots
were WordHelper (the transactional bot), WordAid (the prosocial bot) and Word!Baby (the
friendly bot). Unlike the previous study, participants only watched the video of the interaction
and did not actually interact with it. In order to understand which chatbot interaction appealed
to the participants the most, they were asked to pick which chatbot they would have liked to
interact with and explain why. All responses were collected via WhatsApp.
7Section V - Results
There was an overwhelming preference for the friendly personality across both studies, as is
evident from the findings.
EXPERIMENT I
The friendly bot Ron scored higher than the transactional and prosocial personalities across all
three dimensions tested, suggesting that the introduction of a fun, exuberent personality does
positively impact user experience. Further evidence of the overwhelming preference for the
friendly personality across all three criteria is provided below:
Likelihood of Re-interaction:
The friendly bot personality had the highest mean rating across all three personalities,
receiving 4.05 out of a scale of 5, which was much higher than the mean ratings for the trans-
actional or the prosocial personalities.
42.5% of participants gave the friendly bot the highest possible ranking of ‘5’ for likelihood
of reinteraction, compared to 12.5% for the prosocial bot and 15% for the transactional bot.
Participants were almost three times as likely to state that they would definitely interact with
the friendly bot than with the transactional or prosocial bots.
Transactional Bot
Prosocial Bot
Friendly Bot
0 1 2 3 4 5
Fig.1: Average Ratings for Users’ Likelihood of Re-interaction Across Bot Personalities
8Comfort Level:
Participants also seemed to feel more comfortable with the friendly bot than with the transac-
tional or prosocial bots
40% of participants said that they felt extremely comfortable interacting with the friendly
bot, in comparison to 17.5% for the transactional tbot and 7.5% for the prosocial bot.
In terms of average ratings, the friendly bot also had the highest mean rating for users’ com-
fort levels, scoring 4.05 in comparison to the significantly lower scores for the transactional
bot (3.325) and prosocial bots (3.3).
Average ranking of the friendly chatbot was almost 22% higher than that of the transac-
tional chatbot.
5
4
3
2
1
0
Transactional Prosocial Friendly
Bot Bot Bot
Fig.2: Mean Ratings for Users’ Comfort Levels Across Bot Personalities
9Productivity and Efficiency:
In the context of productivity and efficiency, the friendly bot was almost four times as likely to
be considered ‘extremely productive and efficient’ than the transactional bot.
45% of participants gave the friendly bot personality the highest ranking, compared to only
12.5% for the transactional bot and 17.5% for the prosocial bot.
The friendly bot had the highest mean score in productivity and efficiency as well, scoring a
rating of 4.175 compared to the transactional bots rating of 3.525 or the prosocial bots rating
of 3.625.
Transactional
Bot
Prosocial
Bot
Friendly
Bot
Fig.3: Average Ratings for Productivity & Efficiency Across Bot Personalities
10EXPERIMENT II
There was a similar preference expressed by participants towards the friendly bot in this experi-
ment. 66.67% of the participants expressed a preference for interacting with the friendly bot
Word!Baby over the other two, citing that they enjoyed the interaction because it felt just
like talking to another human being. Participants also commented on the fact that the use of
gifs and emojis made it easier to understand the definition of the word, as did the fact that it was
disseminated in a manner that was far easier to understand than that of the transactional or pro-
social bots. Some of the participants reviews are mentioned below:
“I would have liked to talk to the first chatbot (Word!Baby) because I think that it
explains the definition of the word in a simpler manner with an explanation which
everyone can understand. Also I like the fact that it talks exactly like a human
because it seems as if the chatbot is your friend.”
–– Male, 13
“ [I like] this one [Word!Baby] because [I] felt like I was chatting with someone
real and who knew who [I am]”
–– Male, 12
“ I would like to chat with Word!Baby because it was like more like a casual
conversation and could deliver the meaning very easily and was to the point.
Also the added visual pictures/gif helps you remember the word better. “
–– Female, 15
WordHelper
25%
WordAid
8.3%
Word!Baby
66.7%
Fig. 4: User Preferences for Interactions Across Bot Personalities
However, unlike the previous experiment, the transactional bot personality scored higher
ratings than the prosocial personality in this study. 25% of the participants stated that they
would like to interact with the transactional bot WordHelper, stating that they found the
interaction objective, concise and good for people in a hurry. Only 8.33% of users stated that
they would like to interact with the prosocial bot WordAid.
11Section VI - Conclusion
This study strongly supports the idea that personality can positively affect customer experienc-
es, with the friendly bot personality being preferred by participants across ages, use-cases,
domains and chatbot types.
The results suggest that users may enjoy interacting with a virtual assistant that can converse
at a level that is evocative of an interaction between friends as opposed to an interaction that is
highly goal-oriented and leaves no scope for further developments.
Although a fairly small study that is focused on the Indian consumer market, the outcome of this
study does call for a renewed exploration into how AI companies can integrate personality into
their chatbots or other digital agents in an enterprise context –– especially since the incorpora-
tion of anthropomorphic qualities seems to be the ‘secret sauce’ in making human-AI interac-
tions successful.
12References
Beaver, L. (2017, May 11). Chatbots are gaining traction. Retrieved August 6, 2019, from
https://www.businessinsider.com/chatbots-are-gaining-traction-2017-5?IR=T
Churches, O., Nicholls, M., Thiessen, M., Kohler, M., & Keage, H. (2014). Emoticons in mind: An
event-related potential study. Social neuroscience, 9(2), 196-202.
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., ... & Wade, V. (2019, April).
What Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents. In
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (p. 475).
ACM.
Clark, L. (2018, July). Social boundaries of appropriate speech in HCI: A politeness perspective.
In The 32nd International BCS Human Computer Interaction COnference (HCI 2018), Belfast,
United Kingdom, 4-6 July 2018. BCS Learning and Development.
Dryer, D. C. (1999). Getting personal with computers: how to design personalities for agents.
Applied artificial intelligence, 13(3), 273-295.
Goldberg, L. R. (1993). The structure of phenotypic personality traits. American psychologist,
48(1), 26.
Hornigold, T. (2019, July 14). This Chatbot has Over 660 Million Users-and It Wants to Be Their
Best Friend. Retrieved August 6, 2019, from https://singularityhub.com/2019/07/14/this-chat-
bot-has-over-660-million-users-and-it-wants-to-be-their-best-friend/
Horowitz, L. M., Dryer, D. C., & Krasnoperova, E. N. (1997). The circumplex structure of inter-
personal problems. Circumplex models of personality and emotions, 347-384.
IBM. (2015, September 11). How conversation (with context) will usher in the AI future.
Retrieved August 6, 2019, from https://www.ibm.com/watson/advantage-reports/future-of-ar-
tificial-intelligence/ai-conversation.html
Levy, H. P. (2016, October 18). Gartner Predicts a Virtual World of Exponential Change.
Retrieved August 6, 2019, from https://www.gartner.com/smarterwithgartner/gartner-pre-
dicts-a-virtual-world-of-exponential-change/
Luxton, D., Anderson S. L., & Anderson, M. (2016). Chapter 11 - Ethical Issues and Artificial
Intelligence Technologies in Behavioral and Mental Health Care. In Artificial Intelligence in
Behavioural and Mental Health Care (pp 255-276). Cambridge, Massachusetts: Academic
Press.
13McTear, M., Callejas, Z., & Griol, D. (2016). The Conversational Interface: Talking to Smart Devic-
es: Springer International Publishing.
Press, G. (2017, May 17). AI By The Numbers: 33 Facts And Forecasts About Chatbots And Voice
Assistants. Retrieved August 6, 2019, from
https://www.forbes.com/sites/gilpress/2017/05/15/ai-by-the-numbers-33-facts-and-
forecasts-about-chatbots-and-voice-assistants/#5c028577731f
Reeves, B., & Nass, C. I. (1996). The media equation: How people treat computers, television, and
new media like real people and places. Cambridge university press.
Schultz, D. and Schultz, S. (1998). Theories of Personality. 6th ed. Pennsylvania: Brooks/Cole
Publishing, p.348.
Spencer, G. (2018, November 01). Much more than a chatbot: China’s Xiaoice mixes AI with ...
Retrieved July 10, 2019,from https://news.microsoft.com/apac/features/much-more-
than-a-chatbot-chinas-xiaoice-mixes-ai-with-emotions-and-wins-over-millions-of-fans/
Uleman, J. S., Hon, A., Roman, R. J., & Moskowitz, G. B. (1996). On-line evidence for spontaneous
trait inferences at encoding. Personality and Social Psychology Bulletin, 22(4), 377-394.
Zhou, L., Gao, J., Li, D., & Shum, H. Y. (2018). The design and implementation of XiaoIce, an em-
pathetic social chatbot. arXiv preprint arXiv:1812.08989.
14Thank You!
Interesting in developing a Conversational AI solution for your enterprise?
Reach out to our team at enterprise@haptik.aiYou can also read
