Managing Alarming Situations with Mobile Crowdsensing Systems and Wearable Devices
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
DEGREE PROJECT IN INFORMATION AND COMMUNICATION TECHNOLOGY, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2020 Managing Alarming Situations with Mobile Crowdsensing Systems and Wearable Devices VIKTORIYA KUTSAROVA KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
Managing Alarming Situations with Mobile Crowdsensing Systems and Wearable Devices VIKTORIYA KUTSAROVA Master in Computer Science ICT Innovation, Cloud Computing and Services Date: August 18, 2020 Supervisor: Shatha Jaradat Examiner: Mihhail Matskin School of Electrical Engineering and Computer Science Host company: RISE Swedish title: Hantering av farliga situationer med Mobile Crowdsensing Systems och bärbara enheter
iii Abstract Dangerous events such as accidental falls, allergic reactions or even severe panic attacks can occur spontaneously and within seconds. People experienc- ing alarming situations like these often require assistance. On the one hand, wearable devices such as smartphones or smartwatches can be used to detect these situations by utilising the plethora of sensors built into them. On the other hand, mobile crowdsensing systems (MCS) might be used to manage the detection and mitigation of alarming situations. To be able to handle these events, an MCS requires integration with mobile sensory devices, as well as the voluntary participation of people willing to help. This thesis investigates how to incorporate wearables into an MCS. Furthermore, it explores how to utilise the gathered data and the participants in the system to manage alarming situations. The contributions of this thesis are twofold. First, we propose the exten- sion of a mobile crowdsensing system for managing alarming situations that allows integration of wearables. We base our work on CrowdS - an MCS that facilitates the distributed interactions between people and sensory devices. We integrate a commodity smartwatch into CrowdS using different techniques (i.e. Internet and Bluetooth). The smartwatch’s sensors enable the detection of various alarming situations and their transmission to the MCS. The mobile crowdsensing system then relays the data and finds volunteers willing to help. Our solution can be adapted to handle various types of dangerous situations. Moreover, the system can easily be integrated with other types of wearables. Second, to test the usefulness of an MCS without actually deploying it in real life, we create a simulation that models different scenarios that rep- resent dangerous events. It allows us to represent the event visually and to parametrise various factors that influence the effectiveness of the system. The simulation helps to identify how different parameters might affect the outcome of the alarming situation. Our results show that important attributes include but are not limited to the coverage of the system, the number of participants and their density, as well as distribution and means of transportation. We enhance the capabilities of CrowdS by enabling the integration of var- ious Bluetooth wearable devices. Thus we expand CrowdS into a prototype of a system for managing alarming situations. Moreover, through the MCS simulation, we identify essential parameters that need to be considered when building such a system. The simulation is a tool that can also be used to find the optimal configuration of the MCS.
iv Sammanfattning Farliga händelser som till exempel oavsiktliga fall, allergiska reaktioner eller till och med panikångest kan inträffa utan förvarning och inom några sekun- der. Människor som upplever livsfarliga situationer som dessa behöver ofta hjälp. Bärbara enheter som smartphones eller smartwatches användas för att upptäcka dessa situationer genom att använda en mängd sensorer som är in- byggda i dem. Mobile Crowdsensing Systems (MCS) användas för att hantera upptäckten av dessa situationer och hjälpa människor få de hjälp som behövs. För att kunna hantera dessa situationer kräver en MCS integration mellan mo- bila sensoriska enheter, såväl som ett deltagande av människor som är villiga att hjälpa. Denna avhandling undersöker hur man kan integrera wearables i en MCS. Man undersöker även hur man kan samla in data och deltagare i systemet för att hantera farliga situationer. Bidragen i denna avhandling är tvåfaldiga. För det första föreslår vi utök- ning av en MCS för att hantera farliga situationer som möjliggör integrationen av bärbara enheter. Vi baserar vårt arbete på CrowdS - ett MCS som under- lättar distribuerade interaktioner mellan människor och sensoriska apparater. Vi integrerar en smartwatch med CrowdS med hjälp av olika metoder (exem- pelvis. Internet och Bluetooth). Smartwatch-sensorerna möjliggör upptäckt av olika farliga situationer och deras överföring till MCS. MCS vidarebefordrar sedan datan och försöker hitta frivilliga som är villiga att hjälpa. Vår lösning kan anpassas för att hantera olika typer av farliga situationer. Dessutom kan systemet enkelt integreras med andra typer av bärbara enheter. För att testa nyttan av MCS utan att distribuera den i verkliga livet, skapar vi en simulering av olika scenarier som representerar farliga händelser. I si- muleringen kan vi ändra parametrar och faktorer under händelseförloppet för att se hur det påverkar systemets effektivitet. Simuleringen hjälper till att iden- tifiera hur olika parametrar kan påverka resultatet av den farliga situationen. Våra resultat visar att en del viktiga attribut inkluderar men inte är begränsa- de till området som täcks av systemet, antalet deltagare och deras täthet, samt distribution av människor samt tillgång till transportmedel. Vi förbättrar förmågan hos CrowdS genom att möjliggöra integrationen av olika Bluetooth-bärbara enheter. Vi har utvecklat CrowdS till en prototyp av ett system för att hantera farliga situationer. Genom MCS-simuleringen iden- tifierar vi dessutom viktiga parametrar som måste uppmärksamma när man bygger ett sådant system. Simuleringen är ett verktyg som kan användas för att hitta den optimala konfigurationen av MCS.
v Acknowledgement I am incredibly grateful to my examiner prof. Mihhail Matskin for the invalu- able insights and practical suggestions on how to tackle the work during the creation of this master thesis. I am grateful to my supervisor Shatha Jaradat and to Ronja Jösch for the thor- ough feedback and advice on how to improve my work, as well as the relentless support. Special thanks to Mikael Bengtsson for the vast amount of assistance when providing me with the necessary resources to finish this work. I’d like to recognise the effort that I received from Niklas Fürderer from Nec- tarine Health for our great discussions on how their product works. I gratefully acknowledge the support of my friends and family that allowed me to pursue my higher education. Last, but not least, I express my deepest gratitude to Milko Mitropolitsky who is my cornerstone in every aspect of life.
Contents 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Research questions . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.5 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 Thesis contributions . . . . . . . . . . . . . . . . . . . . . . . 4 1.7 Thesis limitations . . . . . . . . . . . . . . . . . . . . . . . . 5 1.8 Research methodology . . . . . . . . . . . . . . . . . . . . . 5 1.9 Ethics and Sustainability . . . . . . . . . . . . . . . . . . . . 6 1.10 Outline of the document . . . . . . . . . . . . . . . . . . . . 6 2 Background 8 2.1 Risk management . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.1 Health risk situations . . . . . . . . . . . . . . . . . . 9 2.1.2 Fall detection and prevention . . . . . . . . . . . . . . 9 2.2 Sensing wearables . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Crowdsourcing and crowdsensing . . . . . . . . . . . . . . . 11 2.3.1 Crowdsourcing . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 Crowdsensing . . . . . . . . . . . . . . . . . . . . . . 12 2.3.3 Mobile crowdsoucring and crowdsensing . . . . . . . 12 2.4 Mobile crowdsensing taxonomy . . . . . . . . . . . . . . . . 13 2.4.1 Sensing scale . . . . . . . . . . . . . . . . . . . . . . 13 2.4.2 User involvement and responsiveness . . . . . . . . . 13 2.4.3 Sampling rate . . . . . . . . . . . . . . . . . . . . . . 14 2.4.4 Network infrastructure . . . . . . . . . . . . . . . . . 14 2.5 MCS for health care, emergency, safety . . . . . . . . . . . . 14 2.5.1 Emergency management and prevention . . . . . . . . 15 2.5.2 Healthcare and wellbeing . . . . . . . . . . . . . . . . 15 vi
CONTENTS vii 2.5.3 Mobile social networks . . . . . . . . . . . . . . . . . 16 2.6 Platform: Crowds . . . . . . . . . . . . . . . . . . . . . . . . 16 3 Design and Implementation 19 3.1 Extending CrowdS Android application . . . . . . . . . . . . 19 3.2 Smartwatch Application . . . . . . . . . . . . . . . . . . . . 23 3.3 Smartwatch integration with CrowdS via Internet . . . . . . . 25 3.4 Smartwatch integration with CrowdS using Bluetooth . . . . . 25 3.5 Extended CrowdS sample scenario . . . . . . . . . . . . . . . 26 3.6 Nectarine’s platform . . . . . . . . . . . . . . . . . . . . . . 29 4 Experiments 31 4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.2 Experiments setup . . . . . . . . . . . . . . . . . . . . . . . . 32 4.3 Parameters selection . . . . . . . . . . . . . . . . . . . . . . 34 5 Results and Discussion 38 5.1 Smartwatch integration with CrowdS . . . . . . . . . . . . . . 38 5.1.1 Communication protocols comparison . . . . . . . . . 38 5.1.2 Comparison between smartwatch-oriented and smartphone- oriented approaches . . . . . . . . . . . . . . . . . . 40 5.2 Simulation experiments . . . . . . . . . . . . . . . . . . . . . 40 5.2.1 Scenario 1: Different density . . . . . . . . . . . . . . 40 5.2.2 Scenario 2: Different transportation methods . . . . . 42 5.2.3 Scenario 3: Different probability of participation . . . 43 5.2.4 Scenario 4: Different distribution of participants . . . 44 5.2.5 Scenario 5: Constantly increasing density . . . . . . . 45 5.2.6 Scenario 6: Different radius of MCS . . . . . . . . . . 46 5.2.7 Discussions . . . . . . . . . . . . . . . . . . . . . . . 48 6 Conclusion and Future Work 49 6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.2.1 Smartwatch application . . . . . . . . . . . . . . . . . 50 6.2.2 CrowdS platform . . . . . . . . . . . . . . . . . . . . 51 6.2.3 Simulation . . . . . . . . . . . . . . . . . . . . . . . 52 6.2.4 Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . 52 6.2.5 Discover opinions regarding participation in risk de- tection system . . . . . . . . . . . . . . . . . . . . . . 52
viii CONTENTS Bibliography 53 A Calculating the number of visitors and density for simulation 64
List of Tables 4.1 Difference between use cases used to derive parameters of an MCS for risky situation detection . . . . . . . . . . . . . . . . 34 4.2 Parameters and default values used in experiments with a sys- tem for handling emergencies . . . . . . . . . . . . . . . . . . 37 5.1 Advantages and disadvantages of integrating smartwatch with CrowdS directly via Internet . . . . . . . . . . . . . . . . . . 39 5.2 Advantages and disadvantages of connecting smartwatch to a smartphone through Bluetooth . . . . . . . . . . . . . . . . . 40 5.3 Parameters used in Scenario 1: Different density . . . . . . . . 41 5.4 Parameters used in Scenario 3: Different probability of par- ticipation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 ix
List of Figures 3.1 CrowdS System Overview. Source: (V. Granfors et al. 2018) . 20 3.2 Device overview. Adapted from source: (V. Granfors et al. 2018) 21 3.3 Extended client side overview. Adapted from source: (V. Gran- fors et al. 2018) . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4 CrowdS platfrom emergency detection dataflow . . . . . . . . 22 3.5 Basic Architecture of Fall Detection System. Source: (Casi- lari and Oviedo-Jiménez 2015) . . . . . . . . . . . . . . . . . 24 3.6 Smartwatch Application Components. . . . . . . . . . . . . . 24 3.7 Galaxy watch application for detecting a risky situation . . . . 26 3.8 CrowdS app: pairing with Bluetooth device . . . . . . . . . . 27 3.9 CrowdS app: Task creation flow . . . . . . . . . . . . . . . . 28 3.10 Scenario of extended CrowdS . . . . . . . . . . . . . . . . . . 29 4.1 Example setup of experiment . . . . . . . . . . . . . . . . . . 32 5.1 Handled emergencies versus not handled emergencies . . . . . 41 5.2 Average time to reach target depending on density . . . . . . . 42 5.3 Average time to reach target depending on means of trans- portation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.4 Handled emergencies versus not handled emergencies . . . . . 44 5.5 Clustered distribution of agents . . . . . . . . . . . . . . . . . 45 5.6 Correlation between density and active participants. Active participants in logarithmic scale. . . . . . . . . . . . . . . . . 46 5.7 Handled emergencies versus not handled emergencies . . . . . 47 5.8 Average time to reach target depending on radius of MCS system 47 A.1 Popular times in Hyde Park during Sunday. Source: Google Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 A.2 Popular times in Hyde Park during Tuesday. Source: Google Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 x
Chapter 1 Introduction This thesis discusses a system for managing alarming situations by connecting people in need with volunteers ready to provide help. In particular, the sys- tem we consider takes advantage of smartwatches and mobile crowdsensing (MCS) technologies. By using the watch’s sensors, the system can identify dangerous events and take action to resolve the arisen situation. The approach for mitigating risks is integrating a smartwatch application with CrowdS (V. Granfors et al. 2018) - an open-source mobile crowdsensing system that en- ables creation and distribution of various computing or human-oriented tasks. This research also provides simulation as a proof of concept how a system for managing risky situation would work and what are the important parameters that would make such a system beneficial to its participants. This chapter dis- cusses the motivation and context of the problem, as well as some additional aspects of the work performed. 1.1 Motivation Dangerous situations can occur within seconds and can cause irreversible damage to a person’s mental and physical health. We can consider a few cat- egories of such situations. First one is the safety category. Any situation or event that can endanger the safety or well-being of a person represents a safety risk. For example, walking alone in the park during the night or going through a part of a city that is with higher criminal activity can yield a safety risk. The second category is health. We consider any event that can directly impact the health of a person - for instance, an accidental fall of a person or a severe panic attack. Especially older people are exposed to much higher health risk compared to younger people. The third type of dangerous situations is the 1
2 CHAPTER 1. INTRODUCTION emergence of a natural disaster. Any natural event such as earthquakes, hurri- canes or floods that cause enormous damage to people and communities fall into this category. The risks from the categories mentioned above are starting to be predictable or at least automatically detectable. This is possible thanks to various sensors that measure either biometric signals such as heart rate, movement, breathing or measure different environmental characteristics such as noise level, lights, etc. In order to automatically track specific events, sensory infrastructure needs to be installed. Such infrastructure can trigger an alarm if a danger- ous situation has occurred or is about to happen. For example, if a person is walking and suddenly falls, sensors can trigger an alarm. Moreover, modern mobile devices contain most of these sensors. We use the term wearables to denote them. The combination of the non-intrusiveness and multiple sensors incorporated in the wearables makes them a suitable candidate for devices that could prevent risky situations. There is an approach that enables the combinations of multiple wearable devices to allow more sophisticated people-centric services. Utilising de- vices carried by people such as smartphones, tablets, smartwatches and other portable devices is the most cost-effective way of creating a sensory infras- tructure. This approach is called mobile crowdsensing. It enables information sharing between sensory devices. Measuring tasks can be performed either by the devices or could involve the user for more specific insights. Appli- cations of crowdsensing are countless - from fostering healthy eating (Gao, Kong, and Tan 2009; Chen et al. 2009; Reddy et al. 2007) to monitoring traf- fic conditions (Singh et al. 2017) and air pollution (Hasenfratz et al. 2012). That is why plenty of researchers have explored the paradigm for facilitating rapid responses and adequate resource allocations during crises. 1.2 Problem statement There are numerous challenges around the combination of sensory devices and mobile crowdsensing systems. Plenty of scientists have investigated ar- chitecture, data and communication with mobile crowdsensing systems that combine multiple heterogeneous devices. Since such platforms rely on hu- man participants, problems like encouragement and motivation of people to participate have also been considered. Researchers have also discussed the process of verifying contributed data. In the context of health and safety risk management, though, there are fewer researches. Most of them focus on using devices carried by people and
CHAPTER 1. INTRODUCTION 3 mobile crowdsensing to monitor the environment or to allocate resources af- ter a natural disaster event has occurred. However, in those works, the user is engaged only with contributing sensed data and not actively reacting to an alarming event. Moreover, the parameters needed for a risk management sys- tem to be valuable and practical are unexplored. Thus in this master thesis, we consider the problem of how to use wearables together with mobile crowdsensing to manage a risky situation. Moreover, we explore how to mitigate a dangerous situation by providing a rapid response from the users within the network. We also explore what parameters we need to make such a platform a viable alternative to other approaches for handling emergencies. 1.3 Research questions Generally, mobile crowdsensing has been explored by researchers and it has proved to be a good way of using distributed expertise. In solving the task of how to use crowdsensing to detect and mitigate the alarming situation, we first examine the integration of smart wearable devices with an existing crowd- sensing platform. Next, we showcase what are the essential parameters that can make this approach a feasible solution in providing risk reduction and mitigation. We can formulate the following three research questions: 1. How to use a smartwatch to monitor a person’s behaviour and environ- ment for risk situations? 2. How can a smartwatch be integrated into a mobile crowdsensing system to enable firing an alarm in case of a risky situation/behaviour? 3. How effective is a mobile crowdsensing system when an emergency oc- curs and what are the parameters that people who are building such a system need to consider? 1.4 Purpose The purpose of this project is to explore how integrating smartwatches with a mobile crowdsensing system can be beneficial for identifying alarming sit- uations and reacting to them promptly. Furthermore, it shows how flexible and extensible such a system can be and how easy it is to add various devices
4 CHAPTER 1. INTRODUCTION with different sensors. The experiments conducted in this thesis can serve to any person who wants to develop and deploy such a system. The results can help identify the essential characteristics of an MCS solution. For example, what density within the system’s radius is needed to provide a specific cer- tainty about how many people will respond to a call for help or how does the different speed of participants affect the time to reach a person. 1.5 Goals The main goals of the project are as follows: 1. Develop a prototype application on a commodity smartwatch that utilises the watch’s Motion API to perform movement detection. 2. Extend an existing crowdsensing system to support Bluetooth integra- tion. 3. Integrate the prototype smartwatch application with the crowdsensing system by using two different channels - directly through the Internet and also via Bluetooth. 4. Implement a parametrised simulation where we mock how the devel- oped prototype would work in case of an emergency and allows the ex- ploration of required properties of the system. 1.6 Thesis contributions This thesis produces three main contributions: 1. We provide an extension to CrowdS platform that allows easy integration of Bluetooth wearable devices into the mobile crowdsensing platform. More concretely, we extend the CrowdS Android client application to enable the incorporation of Bluetooth devices (Kutsarova 2020a). 2. We develop a smartwatch application that utilises its sensor to emulate the detection of alarming situations and mitigate the risk (Kutsarova 2020b). 3. We implement a simulation of a system for managing alarming situa- tion using mobile crowdsensing. Such a simulation provides valuable insights on important metrics that need to be considered before design- ing a mobile crowdsensing based solution.
CHAPTER 1. INTRODUCTION 5 1.7 Thesis limitations Due to time constraints and feasibility, we impose some constraints regard- ing the produced work from this thesis. First of all, even though we discuss wearable devices throughout the whole thesis, we focus mostly on commod- ity smartwatches. They do not require specific hardware and generally sup- port standard OS and communication protocols. Furthermore, smartwatches are both easily accessible and affordable. Second, the smartwatch application that we built is only used as a prototype and in no way aims to build a proper fall detection model. The goal is to explore different communication channels that allow the integration of the smartwatch with the MCS. Third, we do not test our prototype application in real life but instead chose to build a simula- tion. Testing the prototype would have been infeasible as it needs recruitment of volunteers, their coordination, choosing a proper place to perform the test and many other considerations. Finally, we do not perform interviews with people to extract the default values of our simulation parameters. To collect enough information from people, it would have taken too much time and re- sources. We choose to find research public statistical data which give us the base values for our simulation. 1.8 Research methodology There are two primary objectives of this research. The first one is to build a prototype system for managing alarming situations by integrating commodity smartwatch with an MCS. The prototype allows us to explore various alterna- tive components and to evaluate our idea. We explored different protocols for connecting the smartwatch to the MCS as well as different methods for utilis- ing the smartwatch’s capacity. These actions allowed us to perform a qualita- tive analysis and to give a better perspective of the positive and negative sides of each approach. The second objective is to build a simulation that represents the emergence of a risky situation and how the discussed system can handle it. The simulation is performed by feeding quantitative data into a built model of a system for managing alarming situations. The simulation allows us to use parameters for each property that we want to explore. Parameters can easily be changed and adapted according to what hypothesis is being tested. The simu- lation produces quantitative results, and its objective is to gain understanding, which requires a qualitative point of view. Study of relevant literature, as well as the best practices of designing and building simulations, are used in this
6 CHAPTER 1. INTRODUCTION thesis. 1.9 Ethics and Sustainability Wearable devices can contribute to beneficial outcomes in plenty of aspects. They could help people with specific health problems or could serve a specific part of the population. For example, fall detection devices decrease the risks of permanent injuries among the older population, or electronic glasses could benefit people with problematic sight. Some wearable devices can also be used among the entire population. Monitoring various health aspects, location or sounds around a person could improve their health, safety and decrease the risks of various types of incidents. The biggest ethical concerns connected to sensing wearables are privacy, safety and data management. Questions like where are the person’s data stored, who owns it and can it be deleted once collected are topics that need to be considered when implementing a solution using such a device. The idea of crowdsensing is to improve access to knowledge, resources and skills. Utilising a crowdsensing system in combination with wearable de- vices can tremendously improve people’s safety and provide faster and cheaper disaster management. However, crowdsensing faces plenty of ethical challenges. Most notable of those is preserving privacy. Crowdsensing systems collect sensitive personal information and using state-of-the-art techniques of preserving that data and complying with the latest regulations is essential. 1.10 Outline of the document The first chapter introduces the problem and describes the context of the the- sis. The next Section 2 provides a review of existing literature. The focus of the chapter is sensing wearables and crowdsensing systems. The chapter ends with a brief introduction to CrowdS - a mobile crowdsensing platform used as a base for the current work. Chapter 3 describes the design and imple- mentation of the smartwatch application and how it integrates with CrowdS. A discussion is made between integrating the watch directly with CrowdS or connecting the smartwatch with a smartphone via Bluetooth and letting the smartphone handle the communication with CrowdS. The following sections 4 and 5 discuss the simulation of mobile crowdsensing system. The results are then summarised and analysed. Findings of the thesis are described in a brief
CHAPTER 1. INTRODUCTION 7 but succinct manner in chapter 6. In the same chapter, the limitations of this research are discussed, and directions for future work are proposed.
Chapter 2 Background The rise of popularity of wearable devices filled with a plethora of sensors has driven forward personal safety and healthcare. This thesis investigates the pos- sibility of integrating wearable electronics into a more comprehensive system, where all of the devices can share computational power, measuring capabilities and can facilitate help in case a risky situation occurs. The following Chap- ter provides the necessary background knowledge, which the reader should possess to grasp the ideas behind this thesis fully. First, we discuss the definition of risk, what is a risky situation and how such an event can be detected (Section 2.1). The segment provides a more general discussion of existing terminology and approaches. In this work, we try to build a system that can manage risky situations. The importance of health prevention has inspired our platform. We use a very simplified approach for fall detection and try to create an abstraction of a risky situation. Thus we also present current state-of-the-art research for fall detection and prevention (Section 2.1.2). Next, the focus moves towards wearable devices (Section 2.2), more specif- ically smartwatches, and the possibilities of using their embedded sensors to identify risks. A connection between mobile smartwatches and mobile crowd- sensing systems is described. A big component of this work is the integration of a commodity smartwatch into a mobile crowdsensing system. Understand- ing both topics is essential to understanding the solution we propose. After that, we briefly explain the terms crowdsourcing and crowdsensing (Section 2.3). They take a central place in this thesis, as the proposed so- lution is an integration between wearables and such a system. Furthermore, we present a short discussion of mobile crowdsensing taxonomies (Section 2.4). We focus on the MCS categories that are relevant to our work. We also 8
CHAPTER 2. BACKGROUND 9 describe applications of mobile crowdsensing systems such as emergency pre- vention, healthcare and mobile social networks since they are closely related to this thesis. (Section 2.5). Last, we discuss the CrowdS platform (Section 2.6). This platform had been developed by Ville Granfors (2019) and is an example of a working mobile crowdsensing system. The architecture and approaches implemented within the platform are reviewed. Understanding of the CrowdS software is crucial to understanding the current thesis as it extends and enhances the ca- pabilities of CrowdS. Readers with a background in smartwatches and their sensors can skip section 2.2. Readers with knowledge about crowdsensing platforms can go directly to Section 2.6 where CrowdS is described. The platform is the base for detecting a risky situation using mobile devices. 2.1 Risk management According to the Oxford Dictionary, the definition of risk is "the possibility of something bad happening at some time in the future; a situation that could be dangerous or have a bad result" (Definition of risk 2020). For a long time, the use of technology has been explored as an opportunity to detect risky situations and to help to prevent them from happening. Alternatively, at least to give peo- ple more time for reaction. For instance, gamma rays have been examined for detecting dangerous materials at ports (Shurkin 2015). Moreover, simulation technology has been investigated for preventing financial crisis (Simulation technology and financial crisis 2009). 2.1.1 Health risk situations With the advancement of smartphones and wearable devices, the risks con- nected with personal health have become of great interest to the research com- munity (Piwek et al. 2016). The hope is that these devices can detect issues before they even happen. Moreover, instead of just collecting and relaying health data, those devices would be able to mitigate health problems (Woyke 2016). 2.1.2 Fall detection and prevention Fall detection is a clear example of how technology can help detect or even prevent a risky situation from happening with an application, particularly in
10 CHAPTER 2. BACKGROUND health care. Fall can cause unanticipated injuries, and its detection on time is essential to minimising the effects caused by such an event (Xu, Y. Zhou, and Zhu 2018). Fall detection is an active research area and there are various implementations of such a system - starting from dedicated wearables devices (Vilarinho et al. 2015; H. Nguyen et al. 2017; Ngu et al. 2017; Khojasteh et al. 2018; Yacchirema et al. 2018), combination of smartphones and smartwatches (Casilari and Oviedo-Jiménez 2015) to using Commodity Wifi (Wang, Wu, and Ni 2017; Palipana et al. 2018), Deep Learning approaches (Putra et al. 2018), cameras (Tao and Yun 2017) and combinations of the above mentioned approaches (Lu et al. 2019). In general, the advancement of IoT and smart sensor devices have stimulated the development of this area even further (Xu, Y. Zhou, and Zhu 2018). 2.2 Sensing wearables Wearables are lightweight devices worn close to, on or in the body that mon- itor, transmit and analyse data, providing bio-feedback (Düking et al. 2018) such as commercial wrist-watches (Galaxy Watch 2020; Fitbit Official Site 2020), medical patches (Smart patches 2020), pendants (Fall Detector Pen- dant 2020) and others. Plenty of research is being done around wearable tech- nologies and how they could help to detect, assess and mitigate health risks and what are the main concerns around their usage (Lutze and Waldhör 2015; Heikenfeld et al. 2018; Staff 2019). Those devices can continuously monitor meaningful data extracted from the human body, for instance, heart rate, body temperature, breathing, sweat and others (Heikenfeld et al. 2018). All of this information could be used when treating someone with a specific disease or just monitoring their health status daily - either by professionals or caregivers (Oswald and Zhang 2016). The ultimate goal for all wearables is not only to collect and send health data but rather to detect and mitigate health problems and risk situations (Woyke 2016). Anticipation and prevention of "adverse health events" would bring countless benefits to their users and the healthcare system in general. For example, a wrist band that uses pulse oximetry that is a predictor for oxygen levels in the blood is also the most reliable indicator of an overdose (Staff 2019). Furthermore, it is critical to identify an overdose on time for the person suffering to receive the necessary help promptly. It is beyond the present scope to discuss all relevant wearables. The fo- cus of this thesis is commodity smartwatches. Most smartwatches are full of sensors which give them great potential for health risks prevention. To name a few sensors: electrical heart sensor to take ECG readings; an accelerome-
CHAPTER 2. BACKGROUND 11 ter to keep tabs on movement; an optical heart sensor to measure heart rate; a gyroscope to track movement and rotation, and an ambient light sensor to control the brightness of the screen (Caddy 2019). The combination of the discreteness of a smartwatch and the plethora of sensors incorporated in the device makes it a suitable candidate for managing risky situations. 2.3 Crowdsourcing and crowdsensing 2.3.1 Crowdsourcing Jeff Howe and Mark Robinson (Howe 2006) first introduced the term crowd- sourcing. During the last decade, the phrase has earned enormous popularity, especially with the rise of e-commerce, social media and smartphones. According to Jacquelyn White (White 2019), crowdsourcing is the prac- tice of utilising the knowledge of a group of people for a common goal. It is best applied when tackling significant complex problems that can be split into smaller chunks. Crowdsourcing provides the ability to streamline intri- cate processes. It represents outsourcing a task to an undefined (and generally extensive) network of agents via an open call instead of directly assigning the task to a particular agent. Crucial to this technique is the use of an open call format and the vast network of potential agents. According to Phuttharak and Loke (2019), the ultimate goal of crowd- sourcing is to utilise mobile sensing and humans to collect and analyse infor- mation of people and the surrounding environment. The collected data enables crowdsourcing to provide useful information and services to end-users. Crowdsourcing allows companies to use agents located anywhere in the world and allows tapping into an enormous quantity of skills and expertise with little overhead costs. The approach has shown its effectiveness for plenty of tasks, including building the online encyclopedia Wikipedia, the Chess match of Kasparov against the world in 1999, and many more (Howe 2009). A great example of how companies use crowdsourcing is the toy company Lego. "The company allows users to design new products and at the same time, test the demand. Any user can submit a design that other users can vote for. The idea with the most amount of votes gets moved to production, and the creator receives a 1% royalty on the net revenue." (Kearns 2015).
12 CHAPTER 2. BACKGROUND 2.3.2 Crowdsensing The terms crowdsourcing and crowdsensing are often interchanged within the literature even though there is a subtle difference between the two methods. Crowdsourcing is the more general term; thus, crowdsensing can be viewed as a subset of crowdsourcing. The focus of crowdsensing is on using ubiq- uitous devices (e.g. smartphones, smartwatches and even mobile robots) and their sensors to perform particular work (Pilloni 2018) with or without explicit actions from the user. The term crowdsensing was coined by Ganti, Ye, and Lei (2011). It is a new sensing paradigm that allows the contribution of gen- erated/sensed data from mobile devices. This data can later be processed to provide human-centric service delivery. Crowdsensing can take advantage of the active participation of citizens (Capponi et al. 2019). Sensor’s coverage can be improved as well as humans with their intelligence can provide more in-depth context compared to traditional sensing systems. For instance, large-scale data that concerns urban planning can be collected using the crowdsensing technique. Karamshuk et al. (2013) tried to identify the optimal placement of new retail stores by using, among other features, crowdsensed user mobility data (transitions between venues or the incoming flow of mobile users from distant areas). 2.3.3 Mobile crowdsoucring and crowdsensing As Pilloni (Pilloni 2018) summarises, crowdsensing and crowdsourcing, of- ten referred to as mobile crowdsensing and mobile crowdsourcing, provide a new way to collect data collaboratively, by exploiting the pervasive presence of Internet-connected geolocated smart user devices, such as smartphones, smartwatches and tablets (Shu et al. 2017). The main focus of crowdsensing is that devices serve as moving sensors to collect data from different places (Guo et al. 2014). The goal in crowdsourcing (Peng et al. 2016) is to enable users to complete a dedicated task, usually by providing some form of feedback. These techniques are powerful approaches incorporating human wisdom into mobile computations to solve problems while exploiting the advantages of mobility and context-awareness (Phuttharak and Loke 2019). In the rest of this thesis, when no further differentiation is needed, Mobile crowdsensing (MCS) will be used to refer to both mobile crowdsensing and mobile crowdsourcing. The benefits that MCS provides boost its popularity. The approaches men- tioned above allow easy scaling out and giving away small portions of tasks to be completed by remote workers. Also, the remote nature of the work provides excellent flexibility. MCS gives access to unavailable skills and expertise. It
CHAPTER 2. BACKGROUND 13 can also accelerate various processes as group agents can perform a task faster than a single agent (White 2019). These benefits explain why the ideas behind mobile crowdsensing attract more and more research and business. MCS has proven to be useful both for scientific and business purposes (Phuttharak and Loke 2019). Typical application areas include environmental monitoring and disaster management, infrastructure monitoring, community healthcare, transportation and many more. Sensors that enable the develop- ment of applications in those scenarios are accelerometer, gyroscope, GPS, microphone, camera and others. For instance, GasMobile (Hasenfratz et al. 2012), HazeWatch (Sivaraman et al. 2013), and Third-Eye (Liu et al. 2018) rely on active citizen participation to monitor air pollution. Other examples are HealthAware (Gao, Kong, and Tan 2009), MPCS (Chen et al. 2009), and DietSense (Reddy et al. 2007), which try to encourage healthy eating habits by extracting time and location of where specific food has been consumed. To improve recycling, WasteApp (Bonino et al. 2016) has been developed to monitor the content of recycling bins and user behaviour regarding the topic. This unexhaustive list shows the potential that a paradigm like MCS has to provide new perspectives to urban societies (Capponi et al. 2019). 2.4 Mobile crowdsensing taxonomy Boubiche et al. (2019) has devised a taxonomy of mobile crowdsensing ap- plications based on existing literature. In this section, we are going to briefly discuss them and show their connection with the current master thesis. 2.4.1 Sensing scale There are three categories concerning sensing scalability: separate, cluster and community sensing. In separate sensing, the data is collected for personal use only. The cluster sensing is more applicable for a smaller group of users. They share a common interest and share and collect data to achieve their goals. Community sensing is the highest level of scalability. It utilises large call sensing and is mostly used to predict global trends. In this thesis, we explore the cluster sensing category. 2.4.2 User involvement and responsiveness • Opportunistic sensing - in this group, the data is automatically collected, and the user involvement is as little as possible. In the current research,
14 CHAPTER 2. BACKGROUND we use this approach for our smartwatch application. It automatically collects data and fires an alarm when it encounters unusual activity. This category can be considered as mostly implicit sensing. This category is also known a passive crowdsensing and can include even data submitted for purposes different than original intention (Ghermandi and Sinclair 2019). • Participatory sensing - in this category, the user is active in the data collection process. We use this approach once the MCS system has re- ceived the alarming event and is looking for a person within a radius that is ready to provide help. This category is part of the explicit sens- ing paradigm. 2.4.3 Sampling rate In this category, Boubiche et al. (2019) compares continuous sensing to sens- ing executed depending on specific periods or places (context dependant). In the current research, the smartwatch application performs continuous sensing to be able to determine if an uncommon event has occurred. The drawbacks of this type of sensing is that it may exhaust the sensing resources. Once our mobile crowdsensing platform is searching for a candidate to provide support and help, we employ the context-aware sensing category. 2.4.4 Network infrastructure In this category, MCS can either exploit existing infrastructures (access points and GSM) or adopt peer-to-peer infrastructure. There is also a hybrid approach that combines both the groups mentioned above. The platform we base our work on is making use of existing infrastructure. 2.5 MCS for health care, emergency, safety This thesis is limited to perform research in the field of mobile crowdsensing connected to health care, emergency and safety. The applications of mobile crowdsensing are much more than that. However, in the current master the- sis, we are going to focus on the areas mentioned above. The fact that MCS can capture real-time data about the rapid development of a crisis makes the technology a great candidate to simplify rapid response and effective resource allocation.
CHAPTER 2. BACKGROUND 15 Capponi et al. (2019) classify MCS into domain-specific and general-purpose according to their scope. With regards to the application domains of mobile crowdsensing, Boubiche et al. (2019) based on Ganti, Ye, and Lei (2011), or- ganise applications into four categories based on the type of sensed phenom- ena: environmental, infrastructure, social, and behaviour. A brief discussion of the categories important for this research can be found below. 2.5.1 Emergency management and prevention Emergency response applications are critical regarding their time-sensitivity and usually require more sensors. Examples of applications included in this category are those that monitor and response in case of accidents, natural dis- asters such as floods, fires or even nuclear disasters. For instance, Bengts- son et al. (2011) explore the use of mobile phone positioning data to monitor population movements during disasters and outbreaks, which can be used for assisting in coping with disasters, such as earthquakes and floodings. The au- thors of (D.-A. Nguyen et al. 2014) investigate the gas shortage that occurred after Hurricane Sandy in 2012 using crowdsensed data. Another example is Creekwatch - an application developed by the IBM Almaden research cen- tre (Kim et al. 2011). Using crowdsensed data such as the estimation of the amount of water in the river bed, the amount of trash in the river bank, the flow rate, and a picture of the waterway the application allow monitoring the watershed conditions. Citizens movements and shared data are successfully used in CrowdMonitor - a crowdsensing approach in monitoring emergency services, coordinating real and virtual activities and providing an overview of an emergency, overall in unreachable places (Ludwig et al. 2015). 2.5.2 Healthcare and wellbeing An ageing society like ours is going to rely more and more on better approaches to improve health and wellbeing, starting as early as possible. Mobile crowd- sensing helps in this field as well by collecting a huge amount of health data. Applications that range from self-diagnosis to physical activities tracking and also community healthcare fit into this category. The main focus is to allow diagnosis, treating and preventing illnesses, injuries, and providing timely as- sistance.
16 CHAPTER 2. BACKGROUND Community healthcare In (Ferguson et al. 2006), Google researchers estimated illness distribution in the US by taking advantage of health-related search queries. This research was pioneering work in 2006 and started a sparkle of research in the area of combining MSC with healthcare data. Wesolowski et al. (2012) analysed the spread of malaria in Kenya using widespread dispersion of mobile phones. Similarly, the authors in (Haddawy et al. 2015) demonstrated how MCS could help mitigate the problem of tracking cholera outbreaks. Personal healthcare In (Rabbi et al. 2011), authors present an MCS system for measuring mental wellbeing from behavioural indicators in natural everyday settings as primary healthcare, e.g. weight loss. Dong et al. (2012), authors present an MCS sys- tem for measuring intake via automated tracking of wrist motion in healthcare. 2.5.3 Mobile social networks These networks allow people to connect by exploiting mobile devices. The most fundamental aspect of this category is the collaboration between users. They can share the data sensed by a mobile device and establish social rela- tions. Crowdsense@place (Chon et al. 2013) is a system that tries to provide place related information, including the relationship between users and cover- age of the system. This category of mobile crowdsensing applications focuses not only on user behaviour but also on their emotional state and social needs. Rachuri et al. (2010) is a tool suitable for conducting social and psycholog- ical studies. The basic concept is to map user activities to their emotions in order to create a correlation between them. The application collects user emo- tions and also proximity and patterns of conversations using the audio of the microphone. 2.6 Platform: Crowds V. Granfors et al. (2018) have developed a crowdsensing system called CrowdS. The platform connects mobile devices such as smartphones or tablets and al- lows those devices to provide or request data. The data refers to information about the surrounding environment of the users, for example, the user’s loca- tion, traffic conditions, weather, noise level or air pollution. Currently, CrowdS
CHAPTER 2. BACKGROUND 17 supports two types of data: text and numeric. The former type includes tex- tual answers to questions such as "What is the current weather?". The latter is any numeric value corresponding to the needs of the requester, for instance, answers to the question "How many live in Stockholm" or measurements col- lected from the sensors of the participants’ devices. The platform supports both opportunistic and participatory sensing approaches. The proposed system consists of a server, a set of requesters and a set of providers. "The requester creates a task, pays for completion and has access to the results. The provider replies to received tasks, either manually by provid- ing input personally or automatically when gathering sensor data, and collects the promised payment afterwards" as described by (V. Granfors et al. 2018). A task is an act of requesting data. The server has the primary responsibility for allocating the tasks. All devices that are part of the system can be con- sidered as both a requester or a provider. As long as the smartphone or tablet contains the necessary sensor to execute a particular task, it can be recognised as a provider. The server also takes care of the quality control, reputation management and reward management. All of these components are crucial for every crowd- sensing system that would like to provide valuable results. To start with, the data in MCS is very heterogeneous, and also users of the system are generally not experts in the necessary fields. Quality control is the mechanism that en- sures a better quality of the contributed data. How it is accomplished depends on the data. In CrowdS, verifying textual data uses majority voting, while dis- tance formula performs the check for correctness of numeric values. Further- more, reputation management is used to discourage malicious behaviour and to provide high-quality data. In CrowdS, the approach is to classify providers by their willingness to participate in the system and by previous experience. Last but not least, the crowdsensing activities drain battery and computing power of participants’ devices. Therefore, an incentive mechanism is essential to reward the efforts and resources consumption, as well as for keeping participation in the system high. In CrowdS, micropayments mechanism provides payment and enforce a reward management system. For more information regarding the control and management mechanisms in CrowdS, check (V. Granfors et al. 2018). The example below illustrates CrowdS’ capabilities. Imagine an area with- out any specific sensors installed, but with enough people with smart devices, participants of CrowdS. Someone needs to monitor the level of noise in that environment. This person (requester) creates a task in CrowdS such as asking the question "What is the noise level in location X?" and proposes a reward
18 CHAPTER 2. BACKGROUND for each participant that would execute the task. The server identifies the pos- sible providers of the data and allocates the task to them. The people and their devices execute the measurement and submit their result in CrowdS. They re- ceive the reward specified once the task was created. Now the requester can see the noise level in the specified location. The cost of the achieved result is much lower than installing and maintaining a specific infrastructure for mea- suring the noise level. Furthermore, the platform can be successfully used to perform surveillance tasks and emergency management in various geograph- ical areas. All of the above leads to the conclusion that CrowdS is a suitable base platform for developing a solution that could detect and mitigate risky situations.
Chapter 3 Design and Implementation This chapter describes the two components that extend the existing CrowdS system: a smartwatch application and additional Bluetooth support for CrowdS’ smartphone application. The selected smartphone used to perform a demo is Samsung Galaxy S7 Edge, Samsung Exynos Octa 8890 2.60 GHz CPU, 4GB RAM, 32 GB internal storage and 3600 mAh battery capacity with Android version 8.0.0 and Samsung Experience version 9.0. The selected smartwatch was Galaxy Watch 46mm LTE, Exynos 9110 Dual-core 1.15GHz CPU, 1.5GB RAM, 4GB Internal Memory and 472 mAh battery capacity with Tizen Based Wearable OS 4.0. Sensors available on the smartwatch were Accelerometer, Gyro, Barometer, HRM, Ambient Light. 3.1 Extending CrowdS Android application As of the time of writing, CrowdS system includes a server side and a device side - set of smart devices that can both make requests to the server, as well as provide data on request. The server side is the part responsible for task allo- cation as well as quality control and reputation management. The architecture of the system is a typical representative of centralised architecture - there is no direct communication among the smart devices. All of the requests and responses go through the server. 19
20 CHAPTER 3. DESIGN AND IMPLEMENTATION Figure 3.1: CrowdS System Overview. Source: (V. Granfors et al. 2018) In this research, the server side of CrowdS is used without any adaptation. The only changes done are fixing a few minor issues which occurred while testing the system. For more information on how CrowdS performs task al- location, what is reputation and reward management, and how does quality control work, please refer to (V. Granfors et al. 2018). We have extended the device side of CrowdS to fit the current research purpose. First, we briefly describe the existing architecture of the client side, and then we explain the modifications and their purpose. The main components of the client side of CrowdS are communication manager, Sensor task (ST) manager and Human intelligence task (HIT) man- ager as shown in Figure 3.2. The communication manager is responsible for handling all of the communication and redirecting messages to the proper des- tination. The ST manager handles the collection of data from sensors and sending the data to the communication manager. HIT manager takes care of new and expired human intelligence tasks.
CHAPTER 3. DESIGN AND IMPLEMENTATION 21 Figure 3.2: Device overview. Adapted from source: (V. Granfors et al. 2018) For example, think of a user that wants to measure the temperature in a room within a public building. The user creates a sensing task for measuring the temperature. The communication manager relays the task to the server side. The server finds appropriate devices within CrowdS that can execute the task, e.g. devices that can measure temperature in the defined location. After the devices finish with the temperature measuring, the communication manager notifies the requester that its task has completed. User involvement is not necessary in this case which is why this particular type has the name sensor task. On the other hand, human intelligence task could be considered a task that requires the involvement of the user. For instance, if the participants in CrowdS need to say how long is the queue in front of a canteen, they need to actively take out their devices and take a picture of the queue. We have extended the current version of CrowdS’ client side by adding a new Bluetooth manager, in addition to the ST and HIT managers. The new manager is depicted in Figure 3.3 The Bluetooth manager enables the phone to receive information from paired Bluetooth devices. This approach allows smart devices connected to the phone to access CrwodS using the same communication channels as for the ST and HIT managers. Furthermore, such a layer provides tremendous flexibility and can include a diverse set of wearables quickly and straightfor- wardly. Currently, this layer enables a connected device to send data via Blue- tooth. This data represents an alarm event, which indicates that a risky sit-
22 CHAPTER 3. DESIGN AND IMPLEMENTATION Figure 3.3: Extended client side overview. Adapted from source: (V. Granfors et al. 2018) uation has occurred. After the smartphone receives the event, it relays the collected data to the CrowdS server side. Figure 3.4 visualises the standard dataflow in CrowdS platform when using a smartwatch connected to the sys- tem via Bluetooth. Figure 3.4: CrowdS platfrom emergency detection dataflow We have implemented the Bluetooth service using a standard approach for Android OS (Bluetooth overview 2020). This way, we can easily extend the
CHAPTER 3. DESIGN AND IMPLEMENTATION 23 Bluetooth manager and add multiple devices. One constraint of this approach is that the smartwatch and the smartphone ought to be close to each other as Bluetooth protocol implementation has a limited range of transmitting and receiving data (Understanding Bluetooth Range 2020). 3.2 Smartwatch Application The smartwatch used in this research is Galaxy Watch 46mm LTE (Galaxy Watch 2020). The sensors that are being utilised in this thesis are the gy- roscope, accelerometer and GPS. The goal was to create an application that can fire alarms when it detects abnormal behaviour or situation. We have been strongly inspired by research in the field of fall detection and prevention. There is an enormous amount of literature focused on fall detection (Habib et al. 2014; Khojasteh et al. 2018; Casilari and A.Santoyo-Ramón 2018), which provide various models that enable easy and effective fall detection using wear- ables. In the current master thesis, the focus is not on building an accurate and reliable fall detection model. Falls are being simulated by merely tracking the changes in the accelerometer’s Z-axis. Once a certain threshold is reached, a fall is simulated, and the application can notify CrowdS that an emergency occurred. The goal is to showcase how, after a risky situation appears, the smartwatch can connect to our mobile crowdsensing system. Then the MCS takes care of relaying the signal to people nearby who can provide help. Falls are just one example of such a system. Other examples could be detecting dan- gerous situations by using a microphone or identifying dangerous locations for allergic people by using pollen sensors. The architecture of our system is sim- ilar to the standard architecture of a fall detection system, displayed in Figure 3.5.
You can also read