"Alexa, Can I Trust You?" - CYBERTRUST - Cyber Intrusion
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
CYBERTRUST
“Alexa, Can I
Trust You?”
Hyunji Chung, Michaela Iorga, and Jeffrey Voas, NIST
Sangjin Lee, Korea University
Several recent incidents highlight significant For example, in January 2017, a
6-year-old Dallas girl sharing her
security and privacy risks associated with love of dollhouses and cookies with
intelligent virtual assistants (IVAs). Better the family’s new Amazon Echo Dot
prompted Alexa to order—much to
diagnostic testing of IVA ecosystems can her parents’ surprise—a $160 Kid-
Kraft Sparkle Mansion and four
reveal such vulnerabilities and lead to more pounds of sugar cookies. After re-
porting the story, the anchor of a San
trustworthy systems. Diego TV morning show remarked,
“I love the little girl saying ‘Alexa or-
I
dered me a dollhouse.’” Several Echo
ntelligent virtual assistants (IVAs) have opened up a owners watching the broadcast reported that, after hear-
new world where you can ask a machine questions as ing the anchor’s comment, their own devices also tried to
if it’s a human and request it to perform certain tasks. order pricey dollhouses.2
For example, upon waking up: “Hey, what’s on my The following month, during the Super Bowl, a Goo-
schedule for today?” Before you leave the house for work: gle Home ad using the system’s voice-search-activation
“What’s my commute time?” At dinner: “Have one large phrase “OK, Google” reportedly set off many viewers’ own
pepperoni pizza delivered from Luigi’s.” When you go to devices.3 Capitalizing on the incident, in April, Burger
sleep: “Turn off the bedroom lights.” Ideally, such interac- King ran an ad for the Whopper in which an actor playing
tions should be solely between you and the device assist- an employee at one of its restaurants says that 15 seconds
ing you. But are they? How do you know for sure? isn’t enough time to describe the sandwich and instead
IVAs are becoming increasingly popular: accord- asks Google, which cites the defi nition from Wikipedia—
ing to Gartner, the IVA market will reach $2.1 billion by prompting viewers’ devices to repeat the question and
2020.1 However, recent news reports have revealed that thus essentially extend the ad.4 Ironically, after publi-
popular voice-activated assistants such as Google Home, cally exploiting the system’s vulnerability, the marketing
Apple’s Siri, and Amazon Alexa aren’t always reliable or stunt backfi red—someone altered the Wikipedia entry for
trustworthy. the product to say that it contained cyanide and caused
40 COM PUTE R PUBLISHED BY THE IEEE COMPUTER SOCIET Y 0 0 1 8 - 9 1 6 2 / 1 7/ $ 3 3 .0 0 © 2 0 1 7 I E E E
EDITOR JEFFREY VOAS
NIST; j.voas@ieee.org
cancer5—and became a sobering les- music and videos, purchase items, On the cloud side is the IVA—the soft-
son that a hijacked IVA could cause make recommendations, provide di- ware that processes text and voice
real harm. rections, turn on lights, open garage commands and carries out requested
Here we explore the nature of IVAs doors, and so on (en.wikipedia.org actions. There are two user-side compo-
and some of the security and privacy /wiki/ Virtual_assistant_(artificial nents: IVA-enabled devices—for exam-
concerns associated with this emerg- _intelligence)). We use the term intelli- ple, an Echo Dot (Alexa) or a PC running
ing technology. Are IVAs secure? Are gent virtual assistant, but other names Windows 10 (Cortana)—and compan-
they recording our conversations? If are also commonly used such as smart ion applications installed on the device
so, where is this voice data stored? The assistant, intelligent personal assis- that communicate with the IVA.
presence of IVAs in homes makes this tant, digital assistant, and personal Requests sent to an IVA, whether in
a public-facing challenge, and one that virtual assistant. Regardless of the ter- text format (for example, through on-
attracts instant—and unwelcome— minology, the system’s “brain”—the line chat) or voice format, along with
media attention when problems arise. intelligence that converts human voice the system’s responses are stored in
to text, performs linguistic analysis, the cloud. These user–IVA “conversa-
INTELLIGENT VIRTUAL and carries out the requested action— tions” are usually accessible through
ASSISTANTS is a cloud-hosted service; the devices a companion app. Obviously, the con-
IVAs evolved from chatbots, software themselves run agent programs and, tent of such conversations could con-
agents programmed to converse with whether communicating with the ser- tain revealing details—for example,
humans through either text or voice vice by default or configured to do so, questions about health symptoms.
(en.wikipedia.org/wiki/Chatbot). The have no embedded intelligence. However, user voice recordings them-
fi rst chatbot, ELIZA, was developed by IVAs can communicate with mul- selves also pose a privacy risk because
Joseph Weizenbaum at MIT 16 years af- tiple compatible IoT devices running they constitute personally identifi-
ter Alan Turing fi rst proposed his test a supported OS. Siri works exclusively able information—unauthorized en-
of artificial intelligence in 1950. ELIZA with Apple products—iPhone, iPad, tities could use such data to identify
used natural-language processing to iPod Touch, HomePod, Mac, Apple the user, maliciously obtain access
recognize key words in typed input Watch, and Apple TV devices. Micro- to systems that implement voice rec-
and generate pre-scripted responses soft Cortana works with Windows ognition, or simply process data and
that to some users resembled human 10, Android, Xbox One, Skype, iOS, construct voice artifacts that could be
understanding. PARRY, introduced in Cyanogen, and Windows Mixed Re- used to impersonate the user.6
1972 by psychiatrist Kenneth Colby, ality devices. Alexa works with Am- IVA software can be integrated
convinced a number of trained experts azon’s Echo, Fire, and Dash product into IoT device operating systems—
that it was a real person with paranoid families and various smart devices for example, the latest versions of
schizophrenia. running Android and iOS including iOS and OS X have the Siri agent in-
Over time, chatbots such as Alice smartphones, smart speakers and stalled by default, and Windows 10
(the inspiration for the fi lm Her), Jab- headphones, smartwatches, and smart- has the Cortana agent as one of its
berwacky, and Cleverbot incorporated home devices including TVs, inter- default processes—or downloaded
increasingly sophisticated algorithms coms, lights, thermostats, and refrig- and installed on compatible devices.
to create more natural and complex erators. Google Assistant also works Many IVAs enable third-party ven-
dialogue. Motivated by research indi- with Android and iOS devices. Bixby is dors to link their devices and services
cating that most users prefer to inter- a new IVA for Samsung products. to the intelligent assistant, dramat-
act with human-like programs, simple ically expanding the IVA’s features
chatbots are now integrated in many IVA ECOSYSTEMS or “skills.” For example, Alexa works
phone systems and web applications To understand IVAs’ potential secu- with many smart-home devices from
for customer service, information re- rity and privacy threats, we performed brands including ecobee, Philips Hue,
trieval, marketing, education, enter- cloud-native artifact analysis, packet Nest, Ring, and Leviton. It also inte-
tainment, and other purposes. analysis, voice-command tests, appli- grates with numerous apps to order
IVAs extend chatbot functionality cation analysis, and fi rmware analysis food (for example, Domino’s Pizza and
to Internet of Things (IoT) devices. to better understand IVA ecosystems. Wingstop), stream music and video
Thus, they respond to text and voice As Figure 1 shows, such an ecosystem (Pandora and Spotify), get a ride (Uber
commands to answer questions, play consists of three main components. and Lyft), and check account balances
SEPTEMBER 2017 41CYBERTRUST
IVA
WAV
TXT
Packet analysis Packet analysis
Cloud native-
artifact analysis
Voice-command test
(Wake-up word), what’s the
weather today?
User’s voice
IVA-enabled
Companion applications device
Today’s sunny and the temperature is 71°.
Application analysis User
(executable codes, artifacts) Firmware analysis
Figure 1. An intelligent virtual assistant (IVA) ecosystem has three main components: the cloud-based IVA, IVA-enabled devices, and
companion applications.
and make credit card payments (Cap- check network connectivity, making to the Internet and exploited for ne-
ital One). The Alexa Skills Store (www it possible to detect IVA devices in a farious purposes such as distributed
.alexaskillstore.com) currently lists home network. Firmware image data denial-of-service (DDoS) attacks. For
more than 10,000 voice-activated apps. might also be transferred over unen- example, in October 2016, a DDoS at-
crypted packets, exposing the system tack against the Internet performance
IVA SECURITY to man-in-the-middle attacks and management company Dyn exploited
AND PRIVACY RISKS possible malicious modification of im- vulnerabilities in tens of millions of
Given the large ecosystem of IVA- ages. Even if firmware images aren’t home IoT devices such as webcams
enabled devices and cloud-hosted ser- altered, the ability to obtain them is a and DVRs to infect them with the Mi-
vices from IVA and third-party devel- security concern because it provides rai malware and use them as part of
opers, Figure 2 illustrates four attack unauthorized entities a chance to un- a botnet to temporarily cripple Dyn’s
vectors that can put system security derstand an IVA-enabled device’s in- networks.10
and user privacy at risk. ternal operations.7 Figure 2b shows how a hacker
Most communication between IVA- could compromise an IVA-enabled
Wiretapping an IVA ecosystem enabled devices and the IVA is en- device through its “always on” listen-
Even if companion apps use encrypted crypted using HTTPS. However, var- ing capability, enabling the hacker to
network connections, sniffing the ious machine-learning techniques monitor all voices and sounds within
traffic between the apps and the IVA to classify network traffic can still the device’s range in real time. This
can expose the ecosystem’s commu- reveal payload sizes, data rates, and danger was highlighted by a disturb-
nication mechanisms (left side of Fig- other patterns in encrypted traffic that ing incident in Washington State in
ure 2a). For example, we used packet could be used to identify the device’s April 2015, when parents discovered
interception tools to analyze HTTPS status—for example, idle or in use—or that a stranger had hacked into their
requests and responses and then deter- the user’s behavior such as turning the three-year-old son’s baby monitor by
mine which APIs are used for sending device on or off, talking to the assis- obtaining the companion app’s login
and receiving data to and from the IVA. tant, listening to music, and ordering credentials and was speaking to him
In the case of communication be- products or services.8,9 at night through the device’s speaker
tween IVA-enabled devices and cloud- as well as operating its camera.11 The-
hosted services, our analysis revealed Compromised IVA-enabled devices oretically, an attacker could also re-
that not all network traffic is transmit- Because IVA-enabled devices are part motely control an IVA by talking to the
ted over a secure protocol (right side of of the IoT, devices with security vulner- system through another compromised
Figure 2a). For example, many devices abilities can be compromised like any device in the home, such as a smart
don’t use encrypted connections to other computing system connected speaker or intercom.
42 COMPUTER W W W.CO M P U T E R .O R G /CO M P U T E RPacket sniffing Packet sniffing
Web proxy Cloud
Unveiling Tomorrow 24/7 voice
communication Firmware I have to go to
analysis recording
mechanism a conference
in DC.
Compromised
Who are you? IVA-enabled
IVA-enabled Remotely controlled device
Companion applications User
device speaker
User User
(a) (b)
Unwanted
ordering
WAV WAV
Adversary TXT TXT
Voice Conversation
Home Home
He was
Door driving Unintentional
Malicious voice attack a Lexus in voice record
Stealing car a way she
commands
said was
Adversary IVA-enabled User dangerous
device IVA-enabled
device
(c) (d)
Figure 2. IVA security and privacy risks: (a) wiretapping an IVA ecosystem, (b) compromised IVA-enabled devices, (c) malicious voice
commands, and (d) unintentional voice recording.
Malicious voice commands break into the database—to eavesdrop
Figure 2c depicts a third security and on private conversations. The potential DISCLAIMER
privacy risk associated with IVAs: an for accidental recording means that Certain commercial entities, equip-
attacker who impersonates a user and users don’t necessarily have complete ment, or materials identified in this
issues malicious voice commands to, control over their voice data.12 document were used only to ade-
for example, unlock a smart door to quately describe an experimental pro-
gain unauthorized entry to a home or cedure or concept. Such identification
A
garage or order items online without s virtual assistants become is not intended to imply recommen-
the user’s knowledge. Although some more intelligent and the IVA dation or endorsement by NIST, nor is
IVAs provide a voice-training feature ecosystem of services and de- it intended to imply that the entities,
to prevent such impersonation, it can vices expands, there’s a growing need materials, or equipment are necessarily
be difficult for the system to distin- to understand the security and privacy the best available for the purpose.
guish between similar voices. Thus, a threats from this emerging technol-
malicious person who is able to access ogy. Several recent incidents highlight
an IVA-enabled device might be able to significant vulnerabilities in IVAs.
fool the system into thinking that he Better diagnostic testing can reveal Hearing Its Name on TV,” The Verge, 7
or she is the real owner and carry out such vulnerabilities and lead to more Jan. 2017; www.theverge.com/2017
criminal or mischievous acts. trustworthy systems. /1/7/14200210/amazon-alexa-tech
-news-anchor-order-dollhouse.
Unintentional voice recording REFERENCES 3. K. Opam, “Google’s Super Bowl Ad
Finally, as Figure 2d shows, voices 1. “Gartner Says Worldwide Spending Accidentally Set off a Lot of Google
within range of an IVA-enabled de- on VPA-Enabled Wireless Speakers Homes,” The Verge, 5 Feb. 2017; www
vice can be recorded accidentally and Will Top $2 Billion by 2020,” press .theverge.com/2017/2/5/14517314
transmitted to the cloud, enabling other release, Gartner, 3 Oct. 2016; www /google-home-super-bowl-ad-2017.
parties—including commercial enti- .gartner.com/newsroom/id/3464317. 4. M. Anderson, “How Burger King
ties with legitimate access to the stored 2. A. Liptak, “Amazon’s Alexa Started Revealed the Hackability of Voice
data as well as hackers who might Ordering People Dollhouses after Assistants,” Associated Press, 5 May
SEPTEMBER 2017 43CYBERTRUST
2017; bigstory.ap.org/2d8036 9. C. Gu, S. Zhang, and Y. Sun, “Real-Time
d742504890b2f9edc3f98c77ef. Encrypted Traffic Identification Using HYUNJI CHUNG is a PhD candidate
5. Z. Rodionova, “Burger King Ad Back- Machine Learning,” J. Software, vol. 6, at the Graduate School of Information
fires after Asking Google What’s in a no. 6, 2011, pp. 1009–1016. Security at Korea University and a
Whopper and Is told ‘Cyanide,’” The 10. K. York, “Dyn Statement on 10/21/2016 guest researcher in NIST’s Computer
Independent, 13 Apr. 2017; www DDoS Attack,” blog, 22 Oct. 2016; dyn Security Division. Contact her at hyunji
.independent.co.uk/news/business .com/blog/dyn-statement-on .chung@nist.gov.
/news/burger-king-advert-ask -10212016-ddos-attack.
-google-big-whopper-cyanide-cancer 11. C. Owens, “Stranger Hacks Family’s MICHAELA IORGA is the senior
-causing-wikipedia-page-us-a7681561 Baby Monitor and Talks to Child at security technical lead for cloud
.html. Night,” The San Francisco Globe, 3 computing at NIST and cochair of
6. E. McCallister, T. Grance, and K. Nov. 2016; sfglobe.com/2016/01/06 its Cloud Computing Security and
Scarfone, Guide to Protecting the Con- /stranger-hacks-familys-baby Cloud Computing Forensic Science
fidentiality of Personally Identifiable -monitor-and-talks-to-child-at-night. working groups. Contact her at
Information (PII), Special Publication 12. C. Wood, “Devices Sprout Ears: What michaela.iorga@nist.gov.
800-122, NIST, Apr. 2010. Do Alexa and Siri Mean for Privacy?,”
7. “Exploring the Amazon Echo Dot, The Christian Science Monitor, 14 Jan. JEFF VOAS is an IEEE Fellow and
Part 1: Intercepting Firmware Up- 2017; www.csmonitor.com computer scientist at NIST. Contact
dates,” 2 Jan. 2017; medium.com /Technology/2017/0114/Devices him at j.voas@ieee.org.
/@micaksica/exploring-the-amazon -sprout-ears-What-do-Alexa-and
-echo-dot-part-1-intercepting-firmware -Siri-mean-for-privacy. SANGJIN LEE is a professor in the
-updates-c7e0f9408b59. Graduate School of Information
8. T.T.T. Nguyen and G. Armitage, “A Sur- Security and director of the Digital
vey of Techniques for Internet Traffic Read your subscriptions Forensics Research Center at Korea
Classification Using Machine Learn- through the myCS University. Contact him at sangjin@
publications portal at korea.ac.kr.
ing,” IEEE Comm. Surveys & Tutorials,
http://mycs.computer.org
vol. 10, no. 4, 2008, pp. 56–76.
44 COMPUTER W W W.CO M P U T E R .O R G /CO M P U T E RYou can also read
