The Use and Promise of Conversational Agents in Digital Health
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Published online: 2021-09-03
191
© 2021 IMIA and Georg Thieme Verlag KG
The Use and Promise of Conversational
Agents in Digital Health
Tilman Dingler1, Dominika Kwasnicka2, Jing Wei1, Enying Gong2, Brian Oldenburg2
1
NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, School of Computing
and Information Systems, University of Melbourne, Parkville, Australia
2
NHMRC CRE in Digital Technology to Transform Chronic Disease Outcomes, Melbourne School of
Population and Global Health, University of Melbourne, Melbourne, Australia
tants, such as Google Home and Amazon’s
Summary Conclusion: In this article, we describe the state-of-the-art of
these and other enabling technologies for speech and conver- Alexa, have been widely adopted. In 2020,
Objectives: To describe the use and promise of conversational
sation and discuss ongoing research efforts to develop conver- more than 83 million people in the U.S. use
agents in digital health—including health promotion and-
sational agents that “live” with patients and customize their such ‘smart’ speakers, a 13.7% increased
prevention—and how they can be combined with other new
service offerings around their needs. These agents can function as uptake compared to 2019. Conversational
technologies to provide healthcare at home.
‘digital companions’ who will send reminders about medications agents interact with their users using a
Method: A narrative review of recent advances in technologies
and appointments, proactively check in to gather self-assess- text-based interface (i.e., chatbots) or a
underpinning conversational agents and their use and potential
ments, and follow up with patients on their treatment plans. speech-based one (i.e., voice assistants).
for healthcare and improving health outcomes.
Together with an unobtrusive and continuous collection of other Commanded by users’ language, they present
Results: By responding to written and spoken language,
health data, conversational agents can provide novel and deeply a natural user interface and thus, can make
conversational agents present a versatile, natural user interface
personalized access to digital health care, and they will continue their services and applications accessible to
and have the potential to make their services and applications
to become an increasingly important part of the ecosystem for a wide range of the population [10].
more widely accessible. Historically, conversational interfaces for
future healthcare delivery. The functionality of conversational
health applications have focused mainly on mental health, but
agents ranges from triggering notifications
with an increase in affordable devices and the modernization of
Keywords to responding to simple commands and
health services, conversational agents are becoming more widely
Conversational agents, digital healthcare, telehealth, healthcare questions [11]. More sophisticated versions
deployed across the health system. We present our work on con-
ecosystem retain the progression and context of a con-
text-aware voice assistants capable of proactively engaging users
versation across multiple sessions, which
and delivering health information and services. The proactive
Yearb Med Inform 2021:191-9 allows them to provide highly customized
voice agents we deploy, allow us to conduct experience sampling
http://dx.doi.org/10.1055/s-0041-1726510 interactions. As more and more sensors and
in people’s homes and to collect information about the contexts
smart devices are being used in consumers’
in which users are interacting with them.
homes as part of the context of the Internet
of Things, this is creating a new ecosystem
of technology-enabled devices and services
[12]. Conversational agents offer an intuitive
1 Introduction the COVID-19 pandemic accelerated the
introduction of virtual healthcare delivery
interface in this ecosystem between the user,
the user’s data, and their other home appli-
In 2020, telehealth experienced an unprec- in many countries, it also prompted the ances and devices.
edented uptake around the globe, with the rapid development of many other diverse In this article, we discuss the potential of
COVID-19 pandemic acting as a catalyst [1]. technology-enabled systems and processes these trends being enabled by conversational
Telehealth effectively reduces face-to-face for delivering virtual healthcare to patients agents to seamlessly integrate user needs,
consultations and, with it, the risk of viral [6]. For patients with mental health and/or data, and health services. Further, by tapping
exposure [1–3]. At the same time, Telehealth chronic conditions requiring ongoing care into personal health data that are available
can maintain the delivery of many important and follow-up, COVID-19 has presented from wearable devices, such as Fitbit and
health services and can increase the number many challenges [7, 8]. Apple Health, these smart agents can also
of interactions that patients have with their One new technology development is the trigger tailored conversations about people’s
health providers; thus, making the services widespread uptake of conversational agents health states and can also triage support to
more accessible and more cost-efficient in people’s lives, and these now also have be delivered by other health services. We
to the patient and provider [1–5]. While many health applications [9]. Voice assis- describe how such exchanges of information
IMIA Yearbook of Medical Informatics 2021192
Dingler et al.
about a person’s health and related issues that involved slightly rephrasing and repeat- desktop computers and web applications
can become a continuous, seamless “con- ing what patients had just said. And while and have been developed for a range of
versation” between patients, their data, and Weizenbaum ended up being surprised by different conditions [18]. Voice assistants
their health providers. The delivery of health users’ tendency to attribute human-like communicate specifically through spoken
services in this way can be personalized to feelings to the chatbot, today’s systems and language and can but do not have to come
a person’s specific needs and preferences. devices are often seen as companions and with a visual representation. Amazon
Furthermore, the semi-automated nature human-like entities with social elements Alexa, Google Home, Apple’s Siri, and
of the conversation can relieve some of the and personalities [14]. Personality traits are Microsoft’s Cortana are the most promi-
pressures from the healthcare system. This increasingly being added to conversational nent and widely used voice agents [19],
approach can be scalable, and people can interfaces to build trust with users [10] by and they do not have associated visual
access health services 24 hours/7 days a adapting to users’ personal preferences [15]. representations. In the case of Siri and Cor-
week and from any location. We also present Subsequently, in 1995, Richard Wallace tana, they are shipped out with the phone’s
some of our own work on context-aware created the Artificial Linguistic Internet operating system. Alexa and Google Home
voice assistants that are capable of proac- Computer Entity (ALICE), an award-win- can be controlled through the phone as well
tively engaging users and delivering health ning chatbot capable of processing natural but also reside in special hardware devices
information and services. language and engaging in conversations at users’ homes.
We begin with a brief history of the with humans using pattern-matching [16].
origins of conversational agents to explain Nowadays, conversational agents are widely
common shortcomings that lead us to en- deployed in the form of chatbots supporting
hance commercially available voice agents customers with bookings, helping with 2.1 Towards Proactive
with proactive features. We then present banking tasks, and answering frequently Conversational Agents
common usages of conversational agents in asked questions. Voice assistants constantly listen for user
healthcare, as evidenced by related work. Early chatbots followed static scripts commands and thus currently, play a rela-
From here, we draw a picture of the future and based on the user response, would tively passive role. To initiate a conversation,
applications of interconnected health ser- choose the most fitting continuation of the Alexa can light up an LED ring to signal a
vices to which conversational agents can conversation. This pattern reflects current notification, which then needs to be actively
provide the user interface in the form of online question-answer assistants where received and answered by the user. Howev-
ongoing, highly personalized conversations users ask questions, and the agent triggers a er, other more proactive features are still
about the user’s health and well-being. response based on simple pattern matching. missing. Activation at inconvenient times
Such chatbots are most widely deployed as can quickly be perceived as intrusive and
their implementation does not require spe- annoying [20]. Proactive agents are currently
cific programming knowledge and is rather being researched and developed in two ways:
based on tools, such as Google’s Dialogflow as 1) a context-aware proactive voice assis-
2 Conversational Agents: [17]. More sophisticated agents store and tant; and 2) as autonomous agents acting on
Evolution in Personalization retrieve user information in databases and
personalise the conversation around the us-
behalf of their user.
The idea to converse with computers us- er’s profile. A profile can consist of personal
ing natural language goes back to Alan data, such as name and age, of interaction
Turing’s seminal Imitation Game [13]. He history, i.e., information gathered in previous 2.2 Context-aware Voice Assistants
devised what is today known as the Turing conversations, and context, such as time of Google Home and Amazon Echo are
Test, which challenges people to deter- day, the medium through which the user currently limited to sending simple no-
mine whether they are talking to another converses, location, or device proximity. tifications or setting off alarms. Such
human or, in fact, a machine. A machine The more user data is available, the more the interactions merely transfer smartphone‘s
passes the test when the distinction is no agent can adjust the conversation around the notifications to speakers and usually, do not
longer clear. In 1966, Joseph Weizenbaum user’s goals and needs. employ a back-and-forth multi-turn-taking
developed ELIZA, a chatbot, which he orig- Types of conversational agents range interaction with the user. To enable more
inally created to demonstrate how limited from dialogue systems (e.g., a common fluent human-like conversations, however,
the communication between humans and system used by hospitals to triage incom- conversations agents need to be able to
machines were at the time. Using simple ing phone calls regarding severity, urgency, initiate conversations and engage users in
pattern matching and substitutions, the and pathology) to previously mentioned longer turn-taking conversations [21, 22].
chatbot conveyed the mere illusion of un- online chatbots and voice assistant. Compared with notifications on smart-
derstanding its users. The chatbot took the Sometimes, agents come with human-like phones, which can be silenced or limited
form of a Rogerian psychotherapist named avatars to signal trust and connectedness. to vibrations, voice notifications from con-
after Carl Rogers, using his famous method These embodied agents are often used on versational agents are more interruptive as
IMIA Yearbook of Medical Informatics 2021193
The Use and Promise of Conversational Agents in Digital Health
voice or audio are more difficult to ignore.
Users may only engage in conversations with
the conversational agent if the agent initiates
the conversation at an opportune moment.
Conversational agents, which can conduct
bi-directional conversations, need, therefore,
to become aware of the user’s current context
so that they can detect appropriate times for
initiating conversations.
In our research, we investigate under
which circumstances smart speakers can
trigger interactions with their users based
on contextual factors [23]. These factors
include device location, time of day, user
proximity, ambient light levels and current
noise levels. Figure 1 shows our current
prototype, which consists of a Google
Home assistant we modified. Two ear-
phones attached to the speaker can deliver
pre-recorded voice commands inaudible to
the user. Those pre-recorded voice com-
mands invoke our custom Google Action
(voice applications). This setup allows
us to run experience sampling surveys,
which provide subjective user assessments Fig. 1 A proactive Google Home assistant. We attached earphones to the speaker, which deliver inaudible sound bites for activating the device
throughout the day [24]. Our current cus- and invoking custom applications, such as triggering a health survey or invoking a medication reminder.
tom Google Action implements a 4-item
mini-survey. To gauge the users’ current
context, the speaker asks about people’s
availability, boredom level, mood, and
current activities. Invoking the survey is
done in regular intervals but, with the help
of sensor data, surveys can be triggered by 2.3 Autonomous Agents 3 Application and Evidence
certain events as well, such as the presence
of a person, when the user wakes up in the
The second type of active agents is a fully
autonomous system acting on behalf of its
of Conversational Agents in
morning or before leaving their home. For users. In 2018, Google CEO Sundar Pichai Healthcare
patients living with chronic health condi- introduced a new Google assistant, which
tions, specific types of mini-surveys and booked an appointment at a hair salon via Conversational agents are being increas-
reminders can be implemented in voice phone for its user [26]. Using speech syn- ingly applied in the healthcare field as a
applications and be deployed on our system thesis with natural language elements, the promising tool to improve delivery and
to collect data about patients’ medical or system was close to indistinguishable to a quality, stimulated in part by the advent of
mental conditions and support medication human voice over the phone. The system COVID-19. Many observations have been
adherence. As this system is deployed in called Google Duplex is the first example of made regarding the potential of conversa-
people’s home, we can, therefore, study a conversational agent that is, indeed, a per- tional agents in addressing the temporal,
how proactive conversational agents can sonal assistant to its user [27]. For people geographical and organizational barriers
adapt to the needs of patients with different living with chronic conditions, or indeed, [28,29]. Patients and healthcare profes-
chronic conditions in their daily life. The people experiencing a more acute illness, sionals have also demonstrated a general
system can learn when patients are more the ability of an agent to book a doctor’s acceptance of conversational agents and
available and receptive to voice-based appointment or call an ambulance could high perceived usefulness, convenience
interactions with conversational agents be lifesaving. We expect to see more of and engagement in overcoming logistical
under different contexts. Thus, just-in-time such autonomous systems being commonly and service barriers. However, studies
data collection or interventions [25] can be used in the future as more health services have also raised concerns regarding the
delivered through context-aware conversa- expand their online offerings and provide quality of information, responsiveness,
tional agents. online bookings. and potential ethical risks have also been
IMIA Yearbook of Medical Informatics 2021194
Dingler et al.
raised [28–30]. Errors may occur when the of building a relationship with the users in • Treatment assistant, disease manage-
conversational agents misrecognize users’ order to provide personalized education, ment and consultations: Conversational
input; the users express an intent that the coaching and monitoring. One recent review agents could provide consultations to
system cannot handle, or the system pro- developed a taxonomy for conversational empower patients to engage more fully
vides incomplete or inaccurate information agents in health. It classified the health before and during the clinical appoint-
to users. Many of these errors are quite purpose of conversational agents into six ment. The conversational agents were de-
common for non-health domains, but they groups: training, education, assistance, veloped by mimicking human interaction
may bring significant harm to patients if prevention, diagnosis, and elderly assistance between healthcare providers and patients
the conversations are about medicine use [34]. Car and colleagues also described a using natural language processing and
or emergencies. A multi-layered escalation shifting trend in health goals of conversa- machine learning. By exchanging infor-
from a conversational agent to connecting to tional agents from monitoring and tracking, mation and providing live responses, con-
a health professional is, therefore, required. to connecting patients to health care services versational agents may collect patients’
In addition to the program errors, conver- and supporting users in achieving long-term information before their appointment to
sational agents may also introduce biases behaviour change goals and chronic disease provide tailored counselling and improve
and ethical challenges if the sampling of management [32]. the quality of visits [42]. Conversational
the data sets used to train the algorithms • Screening, diagnosis and triage: agents are increasingly applied in the field
were biased to particular racial or ethnic Several conversational agents, mainly of mental health. Provoost and colleagues
groups or the prediction models are not health chatbots, have been developed for reviewed studies that used embodied con-
disproportionately accurate for certain screening patients with cancers, mental versational agents in clinical psychology
subgroups of consumers. health disorders and risk of chronic and found that conversational agents were
Several recent review articles have de- diseases [35–38]. Some chatbots were used for social skills training, cognitive
scribed the scope of conversational agents in developed following the finite-state behaviour therapy, and counselling for
healthcare. Laranjo and colleagues identified dialogue management system. The depression and anxiety, though more than
17 conversational agents with unconstrained agents controlled the conversation flow half of identified studies were focused on
natural language input capabilities in health- and contained structured questions autism treatment [43].
care. They found that conversational agents and scripts with symptom checklists • Assisting self-management and be-
in healthcare are still in the early stage com- or scales to assess the risk of diseases. haviour change: Pereira and colleagues
pared to other fields [31]. Carr et al. updated During the COVID-19 pandemic, an reviewed studies that used health chatbots
the review and yielded 47 studies covering all increasing number of health chatbots for behaviour change [44,45]. They em-
types of conversational agents in healthcare, were released based on mobile commu- phasized the role of chatbots in monitor-
indicating a rising trend of studies in this nication apps such as WhatsApp, LINE, ing bio-signals, coaching and advising
field since 2016 [32]. This recent scoping WeChat to assist self-triage and personal users by improving their self-efficacy and
review also found that about half of the iden- risk assessment [39]. Besides, empow- cognition, providing encouragement and
tified agents were delivered via smartphone ered by AI, the screening and triage reinforcement through mood analysis and
apps or smartphone-embedded software as system could fulfil more complex re- influencing patients’ behaviour through
the major delivery channels, followed by quirements to achieve automated triage. distraction and encouragement. These
web-based chatbots. Conversational agents For example, the Babylon AI-powered approaches and behaviour change tech-
are also increasingly designed to increase the Triage and Diagnostic system allows niques were applied to promote patients’
program’s engagement and personalization, people to be automatically triaged as self-management in chronic conditions,
either as an authority and knowledgeable their first contact point with the health- mental and neurological disorders and
identity as a virtual health professional or care system before linking with GPs or addictions
coach or as an informal human-like friend specialists. The Babylon chatbots offer • Training and education: The purpose of
and peer [32, 33]. AI-empowered triage based on users’ training and education is achieved mainly
Other reviews have undertaken a con- self-reported symptoms, personal med- by providing information to users via con-
tent analysis of conversational agents in ical history and a medical knowledge versations and coaching users to acquire
healthcare. Conversational agents have database reviewed by clinical experts. skills. An increasing number of studies
been developed to support patients and The chatbots triage users and provide have applied virtual coaches or virtual
healthcare professionals in specific tasks; them with an action plan to link fur- nurses via an Avatar, embodied conver-
thus, a large proportion of conversational ther appointment-making with GPs or sational agents to provide personalised
agents in healthcare have been considered as specialists [40]. Studies have shown its coaching for people with chronic diseases
goal-oriented agents [32]. The health goals general safety and accuracy compared and healthy elderly [46]. In addition,
of conversational agents can be as short as with human doctors, which indicated a virtual patients have also been developed
answering immediate health-related queries great potential in saving health work- to simulate real clinical scenarios to train
or they can be much more long-term as part force and resources [41]. medical students in performing physical
IMIA Yearbook of Medical Informatics 2021195
The Use and Promise of Conversational Agents in Digital Health
exams and clinical judgment to build In another project, we designed and As little real human intervention is
medical students’ capacity before they implemented agents that are part of a smart needed for triaging health services, booking
perform clinical practice [46]. home environment of people living with appointments, sending medication remind-
chronic heart failure [51]. Up to two-thirds of ers, or following up on the development of
Despite the increasing use of conversational heart failure hospitalizations are preventable symptoms, conversational agents are highly
agents in healthcare, the evidence on the as indicated by hospital data [52]. Supporting scalable as their ability to converse with an
efficacy and effectiveness of conversational self-management of this condition is the fo- entire patient population is virtually unlim-
agents in improving health and well-being is cus of the strand of conversational agents we ited. Communication can happen asynchro-
still mostly lacking and piecemeal. Milne- are currently developing. They help patients nously, meaning patients can request health
Ives and colleagues conducted a review on monitor their symptoms, adhere to medica- service information and support at any time
the general effectiveness of AI-empowered tion plans, stick to diet and exercise regimens with a conversational agent, without the
conversational agents in healthcare through and manage symptoms by recognizing need to wait for a dedicated connection or
synthesizing findings from experiments and symptom changes and initiating measures, appointment with a health provider unless
trials. They found that only three-quarters out such as behaviour change or reaching out to there is a need to do so. Agents’ support,
of 31 identified studies reported positive or attain appropriate assistance. therefore, is immediate as they issue prompt
mixed evidence of effectiveness in clinical or Over the last five years, chatbots have en- responses. Triaging services is objective,
behavioural outcomes, and only five studies tered mainstream messaging services, such as and assessments are generally unbiased
evaluated the cost-effectiveness of the pro- Facebook Messenger and WhatsApp. These during the interaction. However, biases can
gram. Studies with robust design and com- bots autonomously chat with their users inside be introduced in other ways, for example,
prehensive evaluation are desperately needed the messaging app itself without the need to during the training of the language model or
to provide evidence before scaling up the install another application. They “live” right of the recommendation algorithms. Special
innovations into the real-world setting (47). next to private conversations users have with care needs to be applied to make sure that
friends and family providing easy access and training data is as unbiased as possible, an
lowering the threshold to interact. As most active field for research around the ethics of
users interact with messaging apps several artificial intelligence.
3.1 Continuous Access to Health times a day [53], chatbot conversations are of While the goal of personalization is to
Services high visibility. For older phone versions that create a more personal and customized re-
In our current work at the NHMRC Centre do not support a modern app marketplace, lationship between patient and health care
for Research Excellence in Digital Technolo- conversational agents can communicate services, conversational agents can also
gy to Transform Chronic Disease Outcomes, through conventional SMS as well. Depend- offer the safety of anonymity [11]. Users will
we are researching and developing proactive ing on the available infrastructure, agents are, sometimes feel more comfortable interacting
conversational agents with the goal to make therefore, available in rural regions providing with an “anonymous” conversation agent
healthcare services more personalized and access to health service to people across the compared to a healthcare professional [54].
easier to navigate. For instance, our recent geographical and economic spectrum. Stigma and barriers to healthcare advice
trial tested and showed that conversational Easy access to and the ability to keep access can be significantly reduced, and
agent ‘Laura’ could support people liv- track of patients’ conversations and data that can then allow exploration of several
ing with type II diabetes [48]. The agent allows these agents to personalize the challenging healthcare topics [55]. This is
provides brief interactions to support the information and information delivery to particularly relevant for patients seeking
person with diabetes in goal setting, action an unprecedented degree. If the agent has mental health support and explains why the
planning and then, following up and asking access to the patient’s clinical and health uptake has been so impressive in this regard.
relevant questions about progress with the services history and, once authorized, the As evident by the examples given, con-
self-management of diabetes. Conversational system does not need to repeatedly request versational agents can also handle a range
agents, such as ‘Laura’ can have multiple patients’ credentials as is the case with of highly sensitive data. Hence, they need
uses in supporting people who are living current consultations over the phone. This to be designed in a way that supports and
with chronic conditions. They can even can save considerable time and conveys guarantees patients’ privacy and data securi-
provide emotional and social support, use the idea to the patient of having a personal ty. We recommend following tried and tested
effective behaviour change techniques to health coach literally “in their pocket”. protection strategies as they are employed in
improve current health behaviours, such as Often anthropomorphic elements, such as other web and mobile technologies, includ-
physical activity, healthy eating, medication a human-like avatar or natural language ing the use of login credentials, two-factor
adherence [49]. In addition to supporting use, make interactions more humane and authentication (i.e., patients need to verify
individuals, conversational agents may also personal. By incorporating personality into their identity through two separate channels),
save the healthcare system costs by reducing conversational agents and emotional aspects biometric authentication (e.g., fingerprint,
unnecessary hospital attendance and emer- into dialogues, the agents are also capable facial recognition), and data encryption
gency department visits [50]. of building trust and rapport with patients. techniques. While the exchange of some
IMIA Yearbook of Medical Informatics 2021196
Dingler et al.
patient data is indispensable to integrate interface to actively inform patients of frameworks, such as RASA [56], which
data sources and communicate patient states useful health data collected from wearable build a starting point to create custom agents.
to health care professionals, other data can and IoT devices such as steps, calories This variant provides a higher degree of
be collected, processed, and stored on the burned, and heart rate or provide them with customization and flexibility but requires a
patients’ local devices. necessary and evidence-based advice and certain expertise and development resources.
Such privacy-by-design architecture interventions based on in-situ or long-term And while the market and its technologies
can enable appropriate compromises being monitoring of patients’ health states. are still relatively young, there is much in-
made between data integration, privacy, and To create such an ecosystem to fully sup- novation, product development, and research
ownership. Another compromise is usually port and enable patients to manage complex around bringing smart conversational agents
struck at the interface level, where usability chronic conditions at home will benefit from to people with the goal to support health and
and security requirements collide. A secure an interconnected environment, a network of well-being.
system is safe from password guessing or smart devices. The Internet of Things (IoT) As speech recognition becomes more
impersonation attempts. Usable security describes the vision of such a network of sophisticated and human-like, commercial
research looks at how to make systems both connected devices that includes sensors, voice assistants increasingly enter people’s
safe and user-friendly. To make conversation- software, services, and output modalities. homes, and the ecosystem of tools to build
al agents accessible to a broad population, Access to sensor and device information conversational agents grows, we anticipate a
they need to be easy to set up yet safe to in patients’ homes will allow the connected steep proliferation of health applications and
use. Home assistants can employ voice agent to make inferences about the user state services in the next years, opening the way
recognition technology to authenticate users and context, including the person’s health for more digital innovations in healthcare.
based on their voice and speech patterns, for condition and circumstances. Third-party In the last Section, we will outline some
example, even in a household with several data streams from devices, such as Fitbit scenarios in which conversational agents
people. We, thereby, need to consider the or the Apple Watch, can also be accessed could become the primary interface between
delicate interplay between household resi- and integrated to provide a more holistic patients and health services.
dents, conversational agents, and the home picture. The agent provides the interface
and wearable devices. for continuous engagement: an ongoing
conversation between the user, user data,
and health services. 3.4 Triaging Health Issues
There is already a range of dedicated health
3.2 The Interconnected Interface apps, such as MyDiabetesCoach [57] or the
Effective patient monitoring includes Woebot [58] which is a digital mental health
continuous, unobtrusive data collection
3.3 Enabling Technologies for therapist that provides self-management
with explicit check-ins. Creating a technol- Healthcare in the Home support. Yet, we see the biggest potential of
ogy-enabled healthcare ecosystem can be Software disciplines, such as natural lan- conversational agents in their ability to help
understood as a multilateral conversation guage processing (NLP), have made signif- patients effectively navigate health services,
between users, their data, and healthcare icant leaps in recent decades. Throughout connect to health resources, and engage
services. To ensure continuous information most of the late 20th century, NLP software in ongoing conversations about health
exchange, conversational agents need to was mainly rule-based when in the 1990’s, and well-being. The aim of conversational
be able to initiate and actively undertake machine-learning techniques were used agents is, therefore, not to replace health
such conversations. Chatbots can trigger to statistically process natural language. professionals but rather, to help patients to
conversations using notifications and With advances in machine learning and the more effectively navigate the health system
sending reminders in chat based on time eventual switch to neural networks, NLP and to empower them to better manage
intervals. Event-triggered reminders, on started becoming on par with transcriptions their own health. This should save time and
the other hand, can be based on contextual by humans. These advances paved the way money on both sides: patients and health
information, such as the number of steps for commercial voice assistants, such as professionals. Additionally, conversational
taken. Current smart speakers, such as Amazon Alexa (introduced in 2012) and agents may support healthcare professionals
Google Home or Amazon’s Alexa, usually Google Nest (released in 2014). in healthcare provision, e.g., by providing
behave in a passive way; however, they Conversational agents with their natural patients with relevant reminders, booking
respond to activation commands (e.g., user interface have the potential to become their clinical appointments, allowing them to
“Hey Google”) explicitly initiated by the the primary user interface for text- and voice- contact healthcare professional via telehealth
user. Users, therefore, need to actively based interactions with apps and services. consultations, supporting efficient informa-
invoke current smart speakers to request New tools and development frameworks tion exchange with hospital information
data (e.g., steps) or to report events (e.g., make it possible to create agents without system (such as HIS, PACS and CIS), and
have taken medications). Proactive smart much domain expertise in machine learn- providing regular checking and monitoring
speakers, in contrast, can provide a voice ing. Further, there is a range of open-source at large scale.
IMIA Yearbook of Medical Informatics 2021197
The Use and Promise of Conversational Agents in Digital Health
For example, let us consider the example ‘live’ in people’s messaging services, such the collection of frequent data points and
of symptom tracking of active COVID-19 as WhatsApp, Facebook Messenger, or Ap- retain the connectedness between health
cases, which requires monitoring by having ple’s iMessage. They are easily subscribed institutions and patients. Figure 2 shows how
medical professionals regularly checking to by sending a simple welcome message our conversational agent ‘lives’ on patients’
in for symptoms development. The sheer and remind their users to frequently answer devices and acts as starting point for checking
number of active cases may already be over- questions about their possible symptoms and their health state or requesting health services.
whelming for a regional hospital but moni- report if any new symptoms develop over The agent provides highly customized recom-
toring active cases only may not be sufficient. time. Figure 2 depicts the information flow mendations, stores and integrates the patient’s
For effective COVID tracing, the broader between patients and health care services various data points from other sources, such
circle of people who have been in contact with the conversational agents triaging most as wearable technologies, and proactively
with active cases need to be monitored as of the traffic. In case symptoms continue checks on patients’ symptoms and health. It
well. Therefore, the number of people who to deteriorate over time, the chatbot either shares data where necessary with the patient’s
require regular check-ins increases expo- recommends contacting local healthcare GP and autonomously books appointments
nentially as the circle of contacts increases services or proactively forwards the user’s when needed. Once the patient explicitly
and this makes manual tracking by medical contact to be reached out to. Dialogue sys- grants the GP data access, the patient’s state
professionals (or other service providers) tems have long been in use, triaging patients’ can be inspected at any point in time using a
almost impossible. calling hospitals, but the sheer number of data dashboard. In the case of deteriorating
Automated systems can send out surveys triage services makes the system slow and or ambiguous health conditions, the GP can
and reminders to ask people to report symp- often causes patient frustration [59]. take over the conversation and initiate a
toms, but other than email, solid technology In our work, we design and develop personal chat or phone call. Health will be a
infrastructure is missing to coordinate these, conversational agents that support health continuous conversation between patients,
store the data, and follow-up with missed care services, scale their accessibility, and caretakers, and their data, facilitated by
reports. Chatbots can help here as they can install a rigorous check-in regime to ensure conversational agents.
general information
Conversational
Agent
active check-ins
report symptoms
General Public
user data
manual check-ins &
consultations
register condition
Case Dashboard
Fig. 2 Conversational agents effectively triage patient requests while customizing their support around the patient’s personal information and interaction history.
IMIA Yearbook of Medical Informatics 2021198
Dingler et al.
4 Conclusion 2. Monaghesh E, Hajizadeh A. The role of telehealth
during COVID-19 outbreak: a systematic review
18. Gong E, Baptista S, Russell A, Scuffham P, Riddell
M, Speight J, et al. My Diabetes Coach, a Mobile
based on current evidence. BMC Public Health App–Based Interactive Conversational Agent
The use of conversational agents is slowly to Support Type 2 Diabetes Self-Management:
2020;20(1):1–9.
becoming more common in healthcare 3. Rush KL, Hatt L, Janke R, Burton L, Ferrier M, Randomized Effectiveness-Implementation Trial.
delivery. These agents have a variety of Tetrault M. The efficacy of telehealth delivered J Med Internet Res 2020 Nov 5;22(11):e20322.
essential characteristics, which make them educational approaches for patients with chronic 19. Kepuska V, Bohouta G. Next-generation of virtual
diseases: A systematic review. Patient Educ Couns personal assistants (microsoft cortana, apple siri,
an increasing focus of study and commercial amazon alexa and google home). In IEEE; 2018.
interest in digital health and healthcare de- 2018;101(8):1310–21.
p. 99–103.
4. Jin K, Khonsari S, Gallagher R, Gallagher P, Clark
livery. Conversational agents can be widely AM, Freedman B, et al. Telehealth interventions
20. Park C, Lim J, Kim J, Lee S-J, Lee D. Don’t
accessible and highly personalised health Bother Me. I’m Socializing! A Breakpoint-Based
for the secondary prevention of coronary heart Smartphone Notification System. In: CSCW 2017.
“companions”. They can be accessed on disease: a systematic review and meta-analysis. Proceedings of the 2017 ACM Conference on
a person’s phone, through a website visit Eur J Cardiovasc Nurs 2019;18(4):260–71. Computer Supported Cooperative Work and Social
on a desktop computer, or via a dedicated 5. Orlando JF, Beard M, Kumar S. Systematic re- Computing; 2017. p. 541–54.
view of patient and caregivers’ satisfaction with 21. Sacks H, Schegloff EA, Jefferson G. A simplest
device, such as Amazon Alexa. The now telehealth videoconferencing as a mode of service systematics for the organization of turn taking for
almost ubiquitous availability of mobile delivery in managing patients’ health. PloS One conversation. In: Studies in the organization of
devices and recent commercialization of 2019;14(8):e0221848. conversational interaction. Elsevier; 1978. p. 7–55.
smart speakers make these agents more 6. Chudasama YV, Gillies CL, Zaccardi F, Coles B, 22. Liao QV, Davis M, Geyer W, Muller M, Shami NS.
Davies MJ, Seidu S, et al. Impact of COVID-19 on What can you do? Studying social-agent orienta-
widely accessible. With language and speech tion and agent proactive interactions with an agent
routine care for chronic diseases: a global survey
being natural interaction modalities, this of views from healthcare professionals. Diabetes for employees. In: DIS ‘16. Proceedings of the
enhances the intuitiveness of their use and Metab Syndr Clin Res Rev 2020;14(5):965–7. 2016 ACM Conference on Designing Interactive
makes them easy to learn. While chatbots 7. Wright A, Salazar A, Mirica M, Volk LA, Schiff Systems; 2016. p. 264–75.
GD. The invisible epidemic: neglected chronic 23. Wei, J., Dingler, T. and Kostakos, V. Developing
are traditionally text-based, voice assistants the Proactive Speaker Prototype Based on Google
use speech to interact with their users. The disease management during COVID-19. J Gen
Intern Med 2020;35(9):2816–7. Home. In: Extended Abstracts of the 2021 CHI
use and promise of conversational agents in 8. Druss BG. Addressing the COVID-19 pandemic Conference on Human Factors in Computing
the healthcare ecosystem is rapidly increas- Systems; 2021. p. 1-6.
in populations with serious mental illness. JAMA 24. Consolvo S, Walker M. Using the experience sam-
ing. All the advantages notwithstanding, Psychiatry 2020;77(9):891–2. pling method to evaluate ubicomp applications.
researchers, users and technology developers 9. Sezgin E, Huang Y, Ramtekkar U, Lin S. Readiness IEEE Pervasive Comput 2003;2(2):24–31.
also do need to be aware of the extra layers for voice assistants to support healthcare delivery 25. Nahum-Shani I, Hekler EB, Spruijt-Metz D.
during a health crisis and pandemic. Npj Digit Med Building health behavior models to guide the
of complexity in terms of privacy, security 2020;3(1):1–4. development of just-in-time adaptive interven-
and data access when using conversational 10. Braun M, Mainz A, Chadowitz R, Pfleging B, Alt tions: A pragmatic framework. Health Psychol
agents in healthcare settings. F. At Your Service: Designing Voice Assistant Per- 2015;34(S):1209.
sonalities to Improve Automotive User Interfaces. 26. Google Duplex: A.I. Assistant Calls Local
In: Proceedings of the 2019 CHI Conference on Businesses To Make Appointments - YouTube
Acknowledgements Human Factors in Computing Systems [Internet]. [Internet]. [cited 2021 Feb 2]. Available from:
The NHMRC Centre of Research Excellence Glasgow Scotland Uk: ACM; 2019 [cited 2021 https://www.youtube.com/atch?v=D5VN56jQM-
(CRE) in Digital Technology to Transform Feb 2]. p. 1–11. Available from: https://dl.acm. WM&t=3s&ab_channel=JeffreyGrubb
Chronic Disease Outcomes aims to trans- org/doi/10.1145/3290605.3300270 27. Hancock JT, Naaman M, Levy K. AI-mediated
11. Laranjo L, Dunn AG, Tong HL, Kocaballi AB, communication: definition, research agenda, and
form Australia’s digital health ecosystem. It Chen J, Bashir R, et al. Conversational agents in ethical considerations. J Comput-Mediat Commun
is funded by the NHMRC (2020-2024: ID healthcare: a systematic review. J Am Med Inform 2020;25(1):89–100.
1170937) and involves researchers from the Assoc 2018;25(9):1248–58. 28. Palanica A, Flaschner P, Thommandram A, Li M,
12. Singh RP, Javaid M, Haleem A, Suman R. Inter- Fossat Y. Physicians’ perceptions of chatbots in
University of Melbourne, Deakin University, net of things (IoT) applications to fight against health care: Cross-sectional web-based survey. J
Griffith University and Macquarie Univer- COVID-19 pandemic. Diabetes Metab Syndr Clin Med Internet Res 2019;21(4):e12887.
sity in Australia and Palo Alto and Stanford Res Rev 2020;14(4):521–4. 29. Househ M, Doha R. Perceptions and Opinions of
Universities in USA. The centre is leveraging 13. Machinery C. Computing machinery and intelli- Patients About Mental Health Chatbots: Scoping
gence-AM Turing. Mind 1950;59(236):433. Review. J Med Internet Res 2021;23(1):e17828.
the ubiquitous availability of smart devices 14. Reeves B, Nass C. The media equation: How 30. Luxton DD. Ethical implications of conversational
and digital technology to improve the health people treat computers, television, and new me- agents in global public health. Bull World Health
and wellbeing of the population, particularly dia like real people. Cambridge university press Organ 2020;98(4):285.
those with chronic conditions. Cambridge, UK; 1996. 31. Laranjo L, Dunn AG, Tong HL, Kocaballi AB,
15. Mairesse F, Walker MA. Towards personality-based Chen J, Bashir R, et al. Conversational agents in
user adaptation: psychologically informed stylistic healthcare: a systematic review. J Am Med Inform
language generation. User Model User-Adapt Assoc 2018;25(9):1248–58.
References Interact 2010;20(3):227–78.
16. Wallace RS. The anatomy of ALICE. In: Parsing
32. Car LT, Dhinagaran DA, Kyaw BM, Kowatsch T,
Rayhan JS, Theng YL, et al. Conversational agents
1. Wosik J, Fudim M, Cameron B, Gellad ZF, Cho the Turing Test. Springer; 2009. p. 181–210. in health care: Scoping review and conceptual
A, Phinney D, et al. Telehealth Transformation: 17. Dialogflow Google Cloud [Internet]. [cited 2021 analysis. J Med Internet Res 2020;22(8):e17158.
COVID-19 and the rise of Virtual Care. J Am Med Feb 2]. Available from: https://cloud.google.com/ 33. Kocaballi AB, Berkovsky S, Quiroz JC, Laranjo
Inform Assoc 2020;27(6):957–62. dialogflow/ L, Tong HL, Rezazadegan D, et al. The per-
IMIA Yearbook of Medical Informatics 2021199
The Use and Promise of Conversational Agents in Digital Health
sonalization of conversational agents in health 2017;19(5):e151. 2020;10(8).
care: systematic review. J Med Internet Res 44. Pereira J, Díaz Ó. Using health chatbots for 52. The Heart Foundation. Australian heart disease
2019;21(11):e15360. behavior change: A mapping Study. J Med Syst statistics [Internet]. The Heart Foundation. 2020
34. Montenegro JLZ, da Costa CA, da Rosa Righi R. 2019;43(5):135. [cited 2020 Apr 22]. Available from: https://www.
Survey of conversational agents in health. Expert 45. Williams ED, Bird D, Forbes AW, Russell A, Ash heartfoundation.org.au/about-us/what-we-do/
Syst Appl 2019;129:56–67. S, Friedman R, et al. Randomised controlled trial heart-disease-in-australia/australian-heart-dis-
35. Chang P, Sheng Y-H, Sang Y-Y, Wang D-W. of an automated, interactive telephone interven- ease-statistics
Developing a wireless speech-and touch-based tion (TLC Diabetes) to improve type 2 diabetes 53. Böhmer M, Hecht B, Schöning J, Krüger A, Bauer
intelligent comprehensive triage support system. management: baseline findings and six-month G. Falling asleep with angry birds, facebook and
CIN Comput Inform Nurs 2008;26(1):31–8. outcomes. BMC Public Health 2012;12(1):1–11. kindle: a large scale study on mobile application
36. Xu R, Mei G, Zhang G, Gao P, Judkins T, Can- 46. Tropea P, Schlieter H, Sterpi I, Judica E, Gand K, usage. In: Proceedings of the 13th Conference on
nizzaro M, et al. A voice-based automated system Caprino M, et al. Rehabilitation, the great absentee Human-Computer Interaction with Mobile Devices
for PTSD screening and monitoring. Stud Health of virtual coaching in medical care: scoping review. and Services, Mobile HCI; 2011. p. 47–56.
Technol Inform 2012;173:552-8. J Med Internet Res 2019;21(10):e12805. 54. Meadows R, Hine C, Suddaby E. Conversational
37. Spänig S, Emberger-Klein A, Sowa J-P, Canbay 47. Milne-Ives M, de Cock C, Lim E, Shehadeh MH, agents and the making of mental health recovery.
A, Menrad K, Heider D. The virtual doctor: An de Pennington N, Mole G, et al. The Effectiveness Digit Health 2020;6:2055207620966170.
interactive clinical-decision-support system based of Artificial Intelligence Conversational Agents in 55. Sebastian J, Richards D. Changing stigmatizing
on deep learning for non-invasive prediction of Health Care: Systematic Review. J Med Internet attitudes to mental health via education and contact
diabetes. Artif Intell Med 2019;100:101706. Res 2020;22(10):e20346. with embodied conversational agents. Comput
38. Ghosh S, Bhatia S, Bhatia A. Quro: facilitating user 48. Gong E, Baptista S, Russell A, Scuffham P, Riddell Hum Behav 2017;73:479–88.
symptom check using a personalised chatbot-ori- M, Speight J, et al. My Diabetes Coach, a mobile 56. Open source conversational AI [Internet]. Rasa.
ented dialogue system. Stud Health Technol Inf application-based interactive conversational agent 2020 [cited 2021 Feb 2]. Available from: https://
2018;252:51–6. to support type 2 diabetes self-management: A rasa.com/
39. Almalki M, Azeez F. Health Chatbots for Fighting randomized effectiveness-implementation trial. 57. Gong E, Baptista S, Russell A, Scuffham P, Riddell
COVID-19: a Scoping Review. Acta Inform Med- Forthcoming 2021. M, Speight J, et al. My Diabetes Coach, a Mobile
ica 2020;28(4):241. 49. Kramer LL, Ter Stal S, Mulder BC, de Vet E, App–Based Interactive Conversational Agent
40. Middleton K, Butt M, Hammerla N, Hamblin S, van Velsen L. Developing Embodied Conversa- to Support Type 2 Diabetes Self-Management:
Mehta K, Parsa A. Sorting out symptoms: design tional Agents for Coaching People in a Healthy Randomized Effectiveness-Implementation Trial.
and evaluation of the’babylon check’automated Lifestyle: Scoping Review. J Med Internet Res J Med Internet Res 2020 Nov 5;22(11):e20322.
triage system. ArXiv Prepr ArXiv160602041. 2016. 2020;22(2):e14058. 58. Mental Health Chatbot [Internet]. Woebot Health.
41. Razzaki S, Baker A, Perov Y, Middleton K, Bax- 50. Baptista S, Wadley G, Bird D, Harris M, Riddell [cited 2021 Feb 2]. Available from: https://woe-
ter J, Mullarkey D, et al. A comparative study of M, Russell A, et al. Acceptability of an embodied bothealth.com/
artificial intelligence and human doctors for the conversational agent for type 2 diabetes self-man- 59. Liao K, Liu Q, Wei Z, Peng B, Chen Q, Sun W, et
purpose of triage and diagnosis. ArXiv Prepr agement education and support via a smartphone al. Task-oriented Dialogue System for Automatic
ArXiv180610698. 2018. app: a mixed-methods study. Diabetes Technol Disease Diagnosis via Hierarchical Reinforcement
42. Adams WG, Phillips BD, Bacic JD, Walsh Ther 2020;22:A221–2. Learning. ArXiv Prepr ArXiv200414254. 2020.
KE, Shanahan CW, Paasche-Orlow MK. Au- 51. Gao L, Maddison R, Rawstorn J, Ball K, Oldenburg
tomated conversation system before pediatric B, Chow C, et al. Protocol: Economic evaluation
primary care visits: a randomized trial. Pediatrics protocol for a multicentre randomised controlled
2014;134(3):e691–9. trial to compare Smartphone Cardiac Rehabil- Correspondence to:
43. Provoost S, Lau HM, Ruwaard J, Riper H. itation, Assisted self-Management (SCRAM) Tilman Dingler
Embodied conversational agents in clinical psy- versus usual care cardiac rehabilitation among Tel.: +61 413 873 517
chology: a scoping review. J Med Internet Res people with coronary heart disease. BMJ Open E-mail: tilman.dingler@unimelb.edu.au
IMIA Yearbook of Medical Informatics 2021You can also read
