Radar-Based Non-Contact Continuous Identity Authentication - MDPI
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
remote sensing Review Radar-Based Non-Contact Continuous Identity Authentication Shekh Md Mahmudul Islam * , Olga Borić-Lubecke, Yao Zheng and Victor M. Lubecke Department of Electrical Engineering, University of Hawai‘i at Mānoa, Honolulu, HI 96822, USA; olgabl@hawaii.edu (O.B.-L.); yaozheng@hawaii.edu (Y.Z.); lubecke@hawaii.edu (V.M.L.) * Correspondence: shekh@hawaii.edu Received: 11 June 2020; Accepted: 11 July 2020; Published: 15 July 2020 Abstract: Non-contact vital signs monitoring using microwave Doppler radar has shown great promise in healthcare applications. Recently, this unobtrusive form of physiological sensing has also been gaining attention for its potential for continuous identity authentication, which can reduce the vulnerability of traditional one-pass validation authentication systems. Physiological Doppler radar is an attractive approach for continuous identity authentication as it requires neither contact nor line-of-sight and does not give rise to privacy concerns associated with video imaging. This paper presents a review of recent advances in radar-based identity authentication systems. It includes an evaluation of the applicability of different research efforts in authentication using respiratory patterns and heart-based dynamics. It also identifies aspects of future research required to address remaining challenges in applying unobtrusive respiration-based or heart-based identity authentication to practical systems. With the advancement of machine learning and artificial intelligence, radar-based continuous authentication can grow to serve a wide range of valuable functions in society. Keywords: Doppler radar; non-contact measurement; respiration; heartbeat; sensor; authentication 1. Introduction Doppler radar has been used in widespread applications, including weather forecasting, vehicle speed measurement, structural health monitoring, and the monitoring of air and sea traffic [1]. This technology has most recently been recognized for promise in healthcare applications though long term unobtrusive physiological monitoring [2–6]. The fundamental Doppler principle is illustrated in Figure 1a, where a reflected signal undergoes a phase shift due to the subtle movement of the chest surface caused by heartbeat and respiration [4–8]. Doppler radar remote life sensing of humans has been widely reported, with proof of concepts demonstrated for various applications [7,8]. This non-contact and non-invasive form of measurement has several potential advantages in medicine, especially for the monitoring of neonates or infants at risk of sudden infant death syndrome [9], adults with sleep disorders [10], and burn victims [11,12]. In addition, separation of respiratory signatures in a multi-subject environment has also been investigated [13–16]. Moreover, this form of respiration monitoring reduces patient discomfort and distress as electrodes need not be attached to the body. The inherent advantage of this unobtrusive non-contact measurement technique broadens potential applications beyond healthcare to include occupancy sensing [17] and related energy management in smart homes [18,19] and baby monitoring [20]. Remote Sens. 2020, 12, 2279; doi:10.3390/rs12142279 www.mdpi.com/journal/remotesensing
Remote Sens. 2020, 12, 2279 2 of 22 Remote Sens. 2020, 12, x FOR PEER REVIEW 2 of 21 (a) (b) Figure Figure 1. 1. Basic Basic principle principle of of Doppler Doppler radar radar physiological physiological sensing sensing (a), (a), and and non-contact non-contact continuous continuous identity identity authentication concept (b). A radar system typically consists of a transmitterand authentication concept (b). A radar system typically consists of a transmitter andaareceiver. receiver. When When aa transmitted transmitted signal signal of frequency ω of frequency ω is is reflected reflected its its phase changes, φφ(t), phase changes, (t),in indirect direct proportion proportion to to the the subtle subtle motion. motion. Growing interest Growing interestininphysiological physiologicalmotion motionsensing sensingthrough through radar radar has has ledled to to thethe development development of of new front-end architectures [20,21], baseband signal processing new front-end architectures [20,21], baseband signal processing methods [22], and system-level methods [22], and system-level integration [21,23] integration [21,23] to to improve improve detection detection accuracy accuracy and and robustness robustness [24].[24]. A A review review of of recent recent advances advances in Doppler in Doppler radarradar sensors sensors has has been been reported reported by by Li Li et et al. al. [25]. [25]. One One example example is is the the application application of of this this non-invasive technology to monitor infants non-invasive infants for forsudden suddeninfant infantdeath deathsyndrome syndrome(SIDS) (SIDS)[26,27] [26,27]which which is one is oneofofthetheleading leadingcauses causesofofinfant infantmortality. mortality.Moreover, Moreover, Doppler Doppler radar radar has also been implementedimplemented to monitor to monitor the the health health and and behavior behavior of of terrestrial terrestrial and aquatic animals [28–30]. Sleep Sleep monitoring monitoring is is another anotheremerging emergingapplication applicationwhere where radar radar alleviates alleviatesthethe measurement measurement interference interference introduced introduced by theby conventional the conventional use ofuse obtrusive of obtrusivedevices suchsuch devices torsotorso straps and or straps andspirometers or spirometers [31]. [31]. A clinical studystudy A clinical was performed was performed to comparatively to comparatively monitor patients monitor suffering patients sufferingfromfrom sleepsleep apnea apneausing a radar using a radarsensor sensorin conjunction in conjunction with traditional with traditional intrusive intrusive sleep sleepmonitoring monitoring equipment, equipment, wherewhere the theradar radarwaswas foundfoundto provide to provide stand-alone stand-alone detection detection of ofmost most apnea apnea events, events,asaswellwellasascomplementary complementarydetail detailto to facilitate facilitate conventional conventional diagnostics diagnostics [31]. [31]. Furthermore, Furthermore,Food Foodand andDrug Drug Administration Administration(FDA) (FDA) approval approval for for the the first first commercial commercial use use ofof wireless, wireless, non-contact non-contact respiratory respiratory devices devices inin the the Unites Unites States States was was granted granted in in 2009 2009 [32]. [32]. Beyond Beyond healthhealth sensing, sensing, Doppler Dopplerradar radaralsoalsoholds holdsgreat great promise promise to to enhance enhance systemsystem security security and and privacy, privacy, particularly particularly in in the the area of user authentication as illustrated in Figure 1b. Existing Existing system system authentication authentication methodsmethods predominately predominately employ employaaone-off, one-off,interruptive interruptiveapproach, approach,which whichauthenticates authenticates only only atatthe theinitial initiallog-in log-inof ofaasession session[33–36], [33–36],withwithusers usersactively activelyengaging engagingan aninput inputdevice deviceor orbiometric biometric reader. reader. Such Suchdesigns designsare arevulnerable vulnerableto toopen opensession sessionexploitations exploitationsand andmay mayinterfere interferewith withuseruseractivity. activity. There There have have beenbeen manymany studies studies focused focused on on continuity continuity in user authentication. authentication. Several Several havehave explored explored sensing sensing technologies technologies to toacquire acquirecommoncommonphysiological physiological traits, traits, including including fingerprint fingerprint [37],[37], palm palm printprint [38], [38], and and iris [39], iris [39], used used to monitor to monitor and and authenticate authenticate usersthroughout users throughoutaasession.session. Recent advancements advancements in in wearable wearable sensors sensors and and pattern pattern recognition recognition further further enable enable system system architects architects to to collect collect more more subtle subtle physiological physiological patterns, patterns,such suchas asthose thoseassociated associatedwith withelectroencephalogram electroencephalogram(EEG) (EEG)[40], [40],finger-vein finger-vein[41],[41], and and gait gait[42], [42],to toverify verifyusers usersimplicitly implicitlyand andcontinuously. continuously.ComparedComparedto tothese thesecontact-based contact-basedsolutions, solutions, continuous authenticationusing continuous authentication usingnon-contact, non-contact, unobtrusive unobtrusive techniques, techniques, suchsuch as Doppler as Doppler radar, radar, can can further further improve improve system system usability usability and expand and expand the range theofrange of applications applications into domainsinto domains with known with privacy known privacy concernsconcerns [43,44]. [43,44]. For example, For example, variousvarious visible andvisible and thermal-based thermal-based camerascamerasare employedare employed to acquire to acquire face andface gaitand gait features features for userfor user verification verification [45–47]. [45–47]. However, However, image-based image-based approaches approaches suffersuffer from from severalseveral irreconcilable irreconcilable dilemmas, dilemmas, including including a lackaof lack of privacy privacy and degraded and degraded performance performance underundera low alight lowambient light ambient conditions conditions [48,49]. Alternatively, [48,49]. Alternatively, a solution unobtrusive a solution leveraging leveraging radar unobtrusive measurementradar measurement of cardiopulmonary of cardiopulmonary motion can bemotion immune cantobesuch immune to such and deficiencies deficiencies achieve and achieveand consistent consistent reliable and reliable under recognition recognition under privacy-sensitive privacy-sensitive conditions [49–54]. conditions [49–54]. This paper reviews recently reported research efforts in identity authentication based on the use of dedicated microwave Doppler radar or WiFi devices in a bi-static radar configuration for recognition of cardiopulmonary signatures, heart activity patterns, and respiratory features. This paper also critically evaluates the practical adoption of the proposed solutions and discusses future requirements for solving some of the remaining challenges needed for practical application.
Remote Sens. 2020, 12, 2279 3 of 22 This paper reviews recently reported research efforts in identity authentication based on the use of dedicated microwave Doppler radar or WiFi devices in a bi-static radar configuration for recognition of cardiopulmonary signatures, heart activity patterns, and respiratory features. This paper also critically evaluates the practical adoption of the proposed solutions and discusses future requirements for solving some of the remaining challenges needed for practical application. The remainder of this paper is organized as follows: Section 2 describes cardiopulmonary diversity and physiological motion measurement. Section 3 describes research publications on the identification of people from dedicated radar and WiFi-based bi-static radar measured cardiopulmonary patterns and evaluates the applicability and limitations. Section 4 provides concluding remarks. 2. Cardiopulmonary Diversity and Physiological Motion Measurement Doppler radar detects motion that occurs due to physiological events, including heartbeat, arterial pulsation, and breathing [55]. This physiological motion is concentrated in the thorax, where the heart and lung lie, but also includes the abdomen, which moves with respiration, and superficial pulses, which are present at any points in the body [55–58]. Torso deformation, due to respiratory effort, is a complex, three-dimensional pattern and it varies greatly with subject parameters and activity context [59,60]. Respiratory effort motion can come from one of the two primary regions: chest or abdomen, known as thoracic and diaphragmatic breathing, respectively [61]. When the heart contracts to generate the pressure that drives blood flow, it moves within the chest cavity, hitting the chest wall, and creating a measurable displacement at the skin surface that can be detected with Doppler radar [54–57]. Non-contact Doppler radar can operate at frequencies where primarily skin-surface motion is detected [54]. Changes in the volume and shape of the heart during systole move the ribs and soft tissue near the heart, causing the chest to pulse with each heartbeat [54–57]. Thus, Doppler radar systems can sense respiratory-related information as well as cardiac patterns. It has been demonstrated in various clinical investigations that sedentary adult human subjects exhibit a diversity in respiratory pattern while awake, not only in terms of tidal volume and inspiratory and expiratory duration, but also in terms of air flow profile [52,55]. Every individual selects to exhibit one pattern among the infinite number of possible ventilatory variables and air flow profiles [55–57]. These variabilities are non-random and may be explained by either central neural mechanisms or chemical feedback loops [54,55]. In addition, each individual has a different physical size and shape of lungs, as well as different rib cage and abdominal muscle strength that contributes to the variations in breathing patterns [56]. In addition to variations in respiratory features, there is also significant variation in heart-based geometry [54–57]. In general, the human heart contains two upper cavities (atria) and two bottom chambers (ventricles) [62,63]. The successive contraction (systole) and relaxation (diastole) of both atria and ventricles circulate the oxygen-rich blood throughout the whole human body [62,63]. A diagrammatic section of the heart is shown in Figure 2a. The heart drives blood through the lungs and to tissues throughout the body [54]. Cardiac motion consists of contraction and relaxation of both atria and ventricles [58]. In one cardiac cycle, ventricles relax and passively fill with blood to 70% of the total volume from atria through the open mitral valve [58]. At the same time, the atria contract with heart muscles and pump blood. Figure 2b shows the motion of the heart through the phases of the cardiac cycle [57]. The cardiac motion cycle consists of five distinct stages including: (1) ventricular filling (VF), (2) atrial systole (AS), (3) isovolumetric ventricular contraction (IC), (4) ventricular ejection (VE), and (5) isovolumetric ventricular relaxation (IR) [62,63]. These cycles are significantly unique because of the different volumes, surface shape, and moving dynamics (speed, acceleration, etc.), and also deformation of the heart [58,62,63]. These stages or cycles are different for each person because of variations in size, position, anatomy of the heart and chest configuration, and various other factors [63,64]. It has also been demonstrated in various clinical investigations that no two persons have the same cardiac blood circulation [62,63].
Remote Sens. 2020, 12, 2279 4 of 22 Remote Sens. 2020, 12, x FOR PEER REVIEW 4 of 21 Figure Figure 2. Diagrammaticsection 2. Diagrammatic sectionof of the the heart, heart,(a), (a),[55] [55]and andmotion motionof the heart of the through heart the phases through of the of the phases cardiac cycle, (b) [56]. The arrows indicate direction of blood flow. the cardiac cycle, (b) [56]. The arrows indicate direction of blood flow. Thus, cardiopulmonary motion is a unique identity marker for everyone, due to differences in Thus, cardiopulmonary physical motion organ size and shape, andismuscle a unique identity strength [62].marker for everyone, In addition, due to differences cardiopulmonary motion is in physical organ controlled bysize andneural central shape, and muscle mechanisms strength feedback, or chemical [62]. In addition, cardiopulmonary which makes motion it extremely difficult to is controlled by central counterfeit or hide neural a livingmechanisms or chemical individual’s motion feedback, signature [52]. which makes it extremely difficult to counterfeit or hide a living individual’s motion signature [52]. 3. Radar-based Continuous Identity Authentication Research 3. Radar-based Identity Continuous Identity authentication Authentication using microwave Research Doppler radar is gaining attention as it can add an extra layerauthentication Identity of security to the vulnerable using microwave traditional Doppler one-pass radarvalidation is gainingapproach attention (e.g., fingerprint, as it can add an password, and facial/iris) [58]. Pattern recognition and unique identification extra layer of security to the vulnerable traditional one-pass validation approach (e.g., fingerprint, are always challenging for this non-contact technology because of variations in human breathing patterns due to physical password, and facial/iris) [58]. Pattern recognition and unique identification are always challenging activity and emotional stress [55]. As future big data analyses emerge and machine learning algorithms for this non-contact technology because of variations in human breathing patterns due to physical improve, Doppler radar-measured physiological signals can be turned into increasingly useful data activity and emotional stress [55]. As future big data analyses emerge and machine learning and knowledge [58]. In particular, diverse respiratory motion patterns have good potential to be used algorithms improve, Doppler radar-measured physiological signals can be turned into increasingly as biometric identifiers [58,64–72]. useful data Identityandtheftknowledge continues[58]. In everyday to pose particular,challenges diverse for respiratory consumers motion and thepatterns associated havethreatgood potential to be used as biometric identifiers [58,64–72]. is increased as traditional identity authentication systems are targeted [58,69]. Traditional identity Identity theftmethods, authentication continuessuch to pose everydaypassword, as fingerprint, challengesand forfacial consumers and the recognition, all associated threat is require an initial increased spot checkas traditional at the start identity authentication of user session, systems are which potentially targeted conveys [58,69]. personal Traditional information like identity bank authentication account, social methods, securitysuchnumber,as fingerprint, and credit cardpassword, and socialand facial recognition, networking all require account details [72]. Inan initial 2018, spotover check at the start 14 million people ofwere uservictims session,ofwhich identitypotentially fraud in the conveys United personal States [69].information Identity fraud likecan bank be significantly account, reduced social security by implementing number, and credit multi-factor card and socialauthentication networking systems, account which can [72]. details be further In 2018, overenhanced 14 million through people integration were victims of unobtrusive of identitycontinuous fraud in the radar-based identity United States authentication [69]. Identity fraud [58].can be significantly reduced by implementing multi-factor authentication systems, which can bebased In this section, radar-based sensing authentication is categorized in two different ways, further either on enhanced breathing-related through integrationfeatures, or heart-based of unobtrusive features. continuous Breathing identity radar-based motion isauthentication generally periodic, [58]. and respiratory-related features can be extracted from the time domain signature In this section, radar-based sensing authentication is categorized in two different ways, based of the reflected phase signal and rate information extracted by performing an FFT [4–6]. Heartbeat motion is modulated on top either on breathing-related features, or heart-based features. Breathing motion is generally periodic, of respiratory motion and the larger resulting breathing signal is dominant over the heartbeat signal [4]. and respiratory-related features can be extracted from the time domain signature of the reflected This leads to a classical problem in FFT, where the stronger signal at given frequency leaks into other phase signal and rate information extracted by performing an FFT [4–6]. Heartbeat motion is frequencies and can mask a weaker signal at nearby frequencies [4,5]. Generally, the radar captured modulated on top of respiratory motion and the larger resulting breathing signal is dominant over signal is filtered outside the 0.005–0.5-Hz frequency band for extracting respiration information and the 0.8–2 heartbeat Hz forsignal [4]. This extracting leads to a classical heartbeat-related problem information. All theinAll FFT, the where the strongerresearch radar authentication signal at given cited frequency leaksisinto in this paper other focused on frequencies extracting two and can mask separate a weakerfeatures, distinguishable signal at nearby based frequencies on either respiration [4,5]. Generally, the radar captured signal is filtered outside the 0.005–0.5-Hz frequency or heartbeat. Extracting both simultaneously has the potential for stronger authentication; however, band for extracting respiration information such a process and 0.8–2 may increase Hz for extracting computational heartbeat-related complexity. information. Table 1 provides a summaryAll the of All the radar published authentication researchnon-contact work on radar-based cited in this paper isidentity continuous focused on extracting authentication two separate considered in this distinguishable paper. In the features, based next two on eitherdetails subsections, respiration or heartbeat. are provided on theseExtracting two differentbothunique simultaneously has the potential features (breathing and heart,for stronger authentication; however, such a process may increase computational complexity. Table 1 provides a summary of published work on radar-based non-contact continuous identity authentication considered in this paper. In the next two subsections, details are provided on these two different unique features (breathing and heart, respectively), including related identification demonstrations along with associated challenges for further development. There have also been attempts to use Doppler
Remote Sens. 2020, 12, 2279 5 of 22 respectively), including related identification demonstrations along with associated challenges for further development. There have also been attempts to use Doppler modulation of WiFi signals to authenticate people and this research is described in the third subsection. Table 1. Systematic review on radar-based non-contact continuous identity authentication. Reference Year Hardware Identification Features Outcome of Publication (RF Frequency) • Respiration-based • Accuracy: 90% [52] A. Rahman 2.4 GHz # Power • Neural network classifier et al., 2016 Doppler Radar # spectral density • Participants: 3 # Packing density # Inspiratory time • Heart-based dynamics • Accuracy: 98.61% [58] F. Lin et al., 2.4 GHz • Support Vector Machine 2017 Doppler Radar # Cardiac-motion cycle • Participants: 78 # Five points • Respiration-based • Accuracy: 95% [68] A. Rahman 2.4 GHz • K-nearest neighbor et al., 2018 Doppler Radar # Inhale-exhale area ratio • Participants: 6 # Minor component • Accuracy: 100% [70] S. M. M. • Support Vector Machine 2.4 GHz • Respiration-based Islam et al., • Participants: 10 Doppler Radar # FFT-based feature 2019 • Only sedentary breathing • Accuracy: 98.8% (normal) • Respiration-based • Accuracy: 92% (combined) [71] S. M. M. • Support vector machine 2.4 GHz Islam et al., # Exhale Area: Air flow • Mixture of sedentary and after Doppler Radar 2020 short exertion breathing # Breathing depth • Participants: 10 • Respiration-based OSA patient • Accuracy: 93% [72] S. M. M. 2.4 GHz and • OSA patient recognition Islam et al., 24-GHz # Peak power spectral density • Support Vector Machine 2020 Doppler Radar # Linear envelop error • Participants: 6 # Inspiratory duration • Heart-based dynamics • Accuracy: 82% [73] D. 2.4 GHz # Cardiac motion • K-nearest neighbor Rissacher et al., Doppler Radar # Wavelet based time and • Participants: 20 2015 frequency feature • Accuracy: 94.6% [74] K. Shi et al., 24 GHz • Heart based dynamic • Support Vector Machine 2018 Doppler Radar # Heartbeat signal complexity • Participants: 4
Remote Sens. 2020, 12, 2279 6 of 22 Table 1. Cont. Reference Year Hardware Identification Features Outcome of Publication (RF Frequency) • Accuracy: 92.8% [75] T. Okano 24 GHz • Heart based dynamics • Autoregressive analysis et al., 2017 Doppler Radar # Power spectral density • Participants: 11 • Accuracy: 98.5% • Heart based dynamics • Convolutional • Short-time Fourier Transform Neural Network [76] P. Cao et al., 24 GHz 2020 Doppler Radar # Heartbeat • Participants: 10 # Energy • Mixture of normal and # Bandwidth abnormal breathing • Accuracy: 93% [77] J. Zhang WiFi router & • Channel state Information • K-nearest neighbor et al., 2016 Laptop # Gait Pattern • Participants: 16 • Respiration-based • Accuracy: 95% [78] J. Liu et al., WiFi router & Morphological pattern • Deep learning 2020 Laptop • Fuzzy Wavelet based • Participants: 20 3.1. Radar-Based Identity Authentication through Respiration Related Features One of the first attempts at radar-based identity authentication using breathing pattern was reported by a research group at the University of Hawaii, where they integrated a neural network classifier to recognize individual human beings [52]. In this work they extracted three different respiratory features (peak power spectral density, linear envelop error, and packing density) from respiratory motion measurements, as illustrated in Figure 3. A 2.4 GHz quadrature direct conversion. Doppler radar system was used for this experiment, assembled from coaxial components. A data acquisition system recorded the data and post processing was performed in MATLAB (MathWorks, Natick, MA, USA). Three subjects with similar breathing rates were selected and various features were investigated, such as power spectral density, linear envelop error, and packing density, which convey the breathing energy and air flow profile related phenomena. The research concentrated on using the Levenberg–Marquadrt back propagation algorithm to perform classification [52]. Figure 4 illustrates the experimental setup and reported results for training and applying a neural network classifier to recognition of the Doppler radar physiological measurements [52]. The overall classification accuracy was above 90%, which clearly illustrates that the proposed technique can be effective for this application. However, this work was limited to identifying only three participants. Another issue is that the experiment was entirely focused on measuring a single subject at a time.
illustrates the experimental setupback using the Levenberg–Marquadrt and propagation reported results for training algorithm and applying to perform a neural classification [52].network Figure 4 classifier illustratestotherecognition experimental of setup the Doppler radarresults and reported physiological for trainingmeasurements and applying[52]. The network a neural overall classification classifier to accuracy recognition wasof above the 90%, whichradar Doppler clearly illustrates that physiological the proposed [52]. measurements technique The can be overall effective for this application. However, this work was limited to identifying only three classification accuracy was above 90%, which clearly illustrates that the proposed technique can be participants. Another effectiveissue is that for this the experiment application. wasthis However, entirely workfocused on measuring was limited a single to identifying onlysubject three at a time. participants. Remote Sens. 2020, 12, 2279 7 of 22 Another issue is that the experiment was entirely focused on measuring a single subject at a time. Figure 3. Radar-measured respiratory features from 30 second epochs. Pioneering efforts at recognizing Figure subject identity Figure3.3.Radar-measured Radar-measured from radar respiratory respiratory measured features fromrespiratory features 30 second from signalsPioneering 30 epochs. second (a) involved epochs. extraction efforts of three at recognizing Pioneering efforts at different subject features: identity breathing from radar rate (b), measured linear envelop respiratory error signals(c), (a)and packing involved density extraction (d). recognizing subject identity from radar measured respiratory signals (a) involved extraction of threeof Linear three envelop different error shows features: different the peak breathing features: ratedistribution (b), linear breathing differences envelop rate (b), and error linear envelop (c),packing and (c), error density packing illustrates density and packing the(d). differences (d). Linear density envelop Linear in air error envelop flow errorprofile shows shows with the peak the the peak inhale distribution and exhale differences distribution areapacking and episodes. differences Taken anddensity packing from [52]. illustrates density the differences illustrates the in air flow profile differences in air with flowthe inhale profile withandthe exhale inhalearea andepisodes. exhale areaTaken from [52]. episodes. Taken from [52]. (a) (a) (b) (b) Figure 4. Human identification experiment using a radar system. The test set-up is shown, (a), along with the neural network recognition results, (b). From [52]. Subsequent research by the same group reported on continuous authentication based on dynamic segmentation where they used inhale and exhale area ratios of the captured respiratory pattern as unique features for six different participants [68]. Dynamic segmentation evaluates the displacement and identifying points in the range of 30–70% of both exhale and inhale episodes, which defines four boundary points of a trapezium [65]. The ratio of these two areas provides a useful feature which indicates how quickly the next cycle of inhalation begins [68]. Figure 5 illustrates the inhale/exhale area ratio features for two different subjects, which differ significantly. Based on extracted unique features,
pattern as unique features for six different participants [68]. Dynamic segmentation evaluates the displacement and identifying points in the range of 30%–70% of both exhale and inhale episodes, which defines four boundary points of a trapezium [65]. The ratio of these two areas provides a useful feature which indicates how quickly the next cycle of inhalation begins [68]. Figure 5 illustrates the Remote Sens. 2020, 12, 2279 8 of 22 inhale/exhale area ratio features for two different subjects, which differ significantly. Based on extracted unique features, a K-nearest algorithm was integrated to identify each person, which a K-nearest showed algorithm a classification was integrated accuracy to identify of almost eachInperson, 90% [68]. which order to showed increase theaaccuracy classification accuracy of the proposed of almost 90% [68]. In order to increase the accuracy of the proposed method, minor component method, minor component analysis was performed on subject data sets which showed overlapping analysis was performed on subject data sets which showed overlapping inhale/exhale area ratios. inhale/exhale area ratios. For minor-component analysis, a linear demodulation technique was For minor-component analysis, a linear demodulation technique was employed [68]. The variation in employed [68]. The variation in minor component shows the radar cross section and high frequency minor component shows the radar cross section and high frequency component of respiration and component of respiration heart signal modulation. and heart Higuchi signal fractal modulation. dimension analysisHiguchi fractalondimension was performed analysisof was minor components performed the radaroncaptured minor components of the signals to identify radar captured overlapped signals inhale/exhale toratios area identify overlapped of subjects, whichinhale/exhale increased area the ratios of subjects, which increased the classification accuracy to 95% [68]. The classification accuracy to 95% [68]. The proposed method clearly shows efficacy. However, proposed method clearly theshows number efficacy. of subjectsHowever, of tested the wasnumber small andoffurther subjects of tested was investigation small and is required furtherlarger to establish investigation data set functionality. is required to establish larger data set functionality. Figure Figure 5. Respiratory 5. Respiratory pattern pattern classifiers used classifiers used for for subject subjectrecognition. recognition.Dynamically Dynamically segmented segmented inhale/exhale area ratios of two subjects significantly differs, (a), as do signal patterns relating to the inhale/exhale area ratios of two subjects significantly differs, (a), as do signal patterns relating to the dynamics of breathing near the points where the inhale and exhale transition occurs, (b). From [68]. dynamics of breathing near the points where the inhale and exhale transition occurs, (b). From [68]. Another limitation of this approach is the reliance on two different parameters (inhale/exhale area Another ratio limitation and minor of thisanalysis). component approach is theinvestigations Further reliance on alsotwodemonstrated different parameters (inhale/exhale that, as inhale/exhale area ratio ratiobecome and minor component more similar, analysis).may false classification Further investigations occur [70]. also demonstrated To increase performance further, an that, FFT as inhale/exhale ratioextraction based feature become more approachsimilar, false classification was applied may occur with an integrated [70]. To increase support-vector machines performance (SVM) classifier further, an FFTusing a radial based basis extraction feature function [70]. The performance approach was appliedof the with proposed system increased an integrated as the support-vector machines (SVM) classifier using a radial basis function [70]. The performance of the proposedrate, FFT based feature extraction approach contains all breathing dynamics related features (breathing system breathing increased depth, as the FFT inhale basedrate,feature exhale rate and airflow extraction profile). contains approach Figure 6 illustrates the FFTdynamics all breathing based feature related extraction approach used for six different participants. As the data set and number of participants features (breathing rate, breathing depth, inhale rate, exhale rate and airflow profile). Figure 6 was small, continued experimentation remains needed to verify the efficacy of the FFT-based feature illustrates the FFT based feature extraction approach used for six different participants. As the data extraction approach. For further investigation, the feasibility of the FFT-based approach for extracting set and number of participants was small, continued experimentation remains needed to verify the identifying features from radar respiratory traces for sedentary subjects was tested, along with efficacy of the FFT-based measurements featurejust of the subjects extraction approach. after performing For further activities physiological investigation, (walkingtheupstairs) feasibility of the [69]. FFT-based approach It was found that for extracting subject identifying recognition features still worked from but was notradar respiratory as effective traces for sedentary after performing short subjects was tested, exertions as it wasalong with measurements for sedentary of the subjects subjects [71]. Experimental justdemonstrated results after performing physiological that, after short activities (walking exertion, upstairs)segmented the dynamically [69]. It was found exhale areathat subject recognition and breathing stillby depth increased worked more thanbut1.4 was timesnot as for all effective participants, after performing which made short evident the exertions as uniqueness of the residual it was for sedentary heart [71]. subjects volume after expiration Experimental results for recognizing each subject, even after short exertions [71]. They also integrated a machine learning classifier SVM, with a radial basis function kernel which resulted in an accuracy of 98.55% for subjects in a sedentary condition and almost 92% for a combined mixture of conditions (sedentary and after short exertions) [71]. Furthermore, they also investigated identity authentication of patients with obstructive sleep apnea (OSA) symptoms based on extracting respiratory features (peak power spectral
demonstrated that, after short exertion, the dynamically segmented exhale area and breathing depth increased by more than 1.4 times for all participants, which made evident the uniqueness of the residual heart volume after expiration for recognizing each subject, even after short exertions [71]. They also integrated a machine learning classifier SVM, with a radial basis function kernel which resulted in an accuracy of 98.55% for subjects in a sedentary condition and almost 92% for a combined Remote Sens. 2020, 12, 2279 9 of 22 mixture of conditions (sedentary and after short exertions) [71]. Furthermore, they also investigated identity authentication of patients with obstructive sleep apnea (OSA) symptoms based on extracting respiratory features density, packing (peakand density, power linearspectral envelopdensity, error) forpacking density, paradoxical radar captured and linear envelop breathingerror) for patterns, radar captured paradoxical in a small-scale breathing clinical sleep patterns, in three study integrating a small-scale differentclinical machinesleep study integrating learning three classifiers (SVM, different k-nearestmachine neighborlearning (KNN), classifiers and random (SVM, k-nearest forest). neighbor (KNN), Their proposed OSA-based andauthentication random forest). Their method proposed was testedOSA-based and validated authentication for five OSA method was patients withtested andaccuracy, 93.75% validatedusing for five a KNNOSAclassifier patientswhich with 93.75% accuracy, outperformed otherusing a KNN[72]. classifiers classifier which This study wasoutperformed other limited to only classifiers six supine [72]. in subjects This thestudy was controlled limited to onlyofsix environment supine a sleep subjects in the controlled environment of a sleep center. center. Figure6. Figure FFT/SVMbased 6.FFT/SVM basedsubject subjectrecognition recognitionstudy. study. A A2.4-GHz 2.4-GHzradar radarsystem system(a) (a)was wasused usedtotomeasure measure subjects in seated position (b) and FFT-based extracted features up to 10 Hz were used to identify subjects in seated position (b) and FFT-based extracted features up to 10 Hz were used to identify six six different participants (c). From [70]. different participants (c). From [70]. 3.2. Radar-Based Identity Authentication through Heart-Based Features 3.2. Radar-Based Identity Authentication through Heart-based Features: One of the first attempts at recognizing people from their heart-based features (cardiac cycle) One of the first attempts at recognizing people from their heart-based features (cardiac cycle) from Doppler radar was reported on by a research group at Clarkson University [73]. They used a from Doppler radar was reported on by a research group at Clarkson University [73]. They used a 2.4-GHz heterodyne radar system from which cardiac data was extracted, and an ensemble average 2.4-GHz heterodyne radar system from which cardiac data was extracted, and an ensemble average was computed using ECG as time reference [73]. A continuous wavelet transform was integrated to was computed using ECG as time reference [73]. A continuous wavelet transform was integrated to provide time-frequency analysis of the average radar-measured cardiac cycle and a k-nearest neighbor provide time-frequency analysis of the average radar-measured cardiac cycle and a k-nearest algorithm was used to recognize people with an accuracy of 82%. This was the first reported attempt neighbor algorithm was used to recognize people with an accuracy of 82%. This was the first reported for applying cardiac-based features using a cardiac-radar system as biometric identification tool [73]. attempt for applying cardiac-based features using a cardiac-radar system as biometric identification The low classification accuracy occurred as there was overlap in the ensemble average of the cardiac tool [73]. The low classification accuracy occurred as there was overlap in the ensemble average of cycle; therefore, further investigation and experimentation is required to demonstrate efficacy for more the cardiac cycle; therefore, further investigation and experimentation is required to demonstrate reliable recognition of subjects from radar captured signals. efficacy for more reliable recognition of subjects from radar captured signals. A study conducted by a research group at the University of Buffalo [58] proposed a continuous A study conducted by a research group at the University of Buffalo [58] proposed a continuous identity authentication system named “Cardiac Scan”. This system used a 2.4-GHz Doppler radar identity authentication system named “Cardiac Scan”. This system used a 2.4-GHz Doppler radar transceiver with two antennas (one for transmit and another for receive functions) each having a transceiver with two antennas (one for transmit and another for receive functions) each having a beam width of 45◦ . The radar power consumption was 650 mW with a 5V-volt source and 130 mA of beam width of 45°. The radar power consumption was 650 mW with a 5V-volt source and 130 mA of current [58]. The customized Doppler radar was placed in front of the subject at 1 meter [58]. Figure 7 current [58]. The customized Doppler radar was placed in front of the subject at 1 meter [58]. Figure shows the experimental set up of the proposed cardiac scan system, from which five different points 7 shows the experimental set up of the proposed cardiac scan system, from which five different points were extracted from the radar captured respiration patterns which were hypothesized to fully represent were extracted from the radar captured respiration patterns which were hypothesized to fully cardiac motion. Based on the hypothesis the experiment illustrated that these heart-based geometry represent cardiac motion. Based on the hypothesis the experiment illustrated that these heart-based measures differ from person to person due to difference in size, position and anatomy of the heart, chest configuration, and various other factors [58]. From their experimental results, it was also clear that no two subjects had the same heart, tissue, and blood circulation system, as there were significant differences in their cardiac cycle points measured in the radar data set [58]. Figure 8 illustrates the cardiac motion marker for one segment captured from the radar respiration measurement. In this work, the user’s cardiac-motion related features were stored in the system. A SVM with a radial basis
clear that no two subjects had the same heart, tissue, and blood circulation system, as there were heart, chest configuration, and various other factors [58]. From their experimental results, it was also significant differences in their cardiac cycle points measured in the radar data set [58]. Figure 8 clear that no two subjects had the same heart, tissue, and blood circulation system, as there were illustrates the cardiac motion marker for one segment captured from the radar respiration significant differences in their cardiac cycle points measured in the radar data set [58]. Figure 8 measurement. In this work, the user’s cardiac-motion related features were stored in the system. A illustrates the cardiac motion marker for one segment captured from the radar respiration SVM with a radial basis function (RBF) kernel classifier was employed to uniquely identify different Remote Sens. 2020, 12, measurement. In 2279 this work, the user’s cardiac-motion related features were stored in the system. 10 of 22 A participants. A study of a 78 subject data set was reported, and the proposed system achieved an SVM with a radial basis function (RBF) kernel classifier was employed to uniquely identify different accuracy of 98.61% and a 4.42% equal error rate [58]. One of the limitations of the above proposed participants. function (RBF) Akernel study classifier of a 78 subject data settowas reported, and the proposed system achieved an technique is that the complete was studyemployed uniquely was performed withidentify healthydifferent participants. sedentary persons andA study only offora accuracy of 98.61% and a 4.42% equal error rate [58]. One of the limitations of the above proposed single-subject 78 subject datameasurements. set was reported, If subjects have cardiovascular and the proposed diseases, system achieved unique identification an accuracy of 98.61% and may be a 4.42% technique is that the complete study was performed with healthy sedentary persons and only for problematic equal error rate as the [58].cardiac One ofcycle would be of the limitations affected. the aboveFurther studytechnique proposed is also required to complete is that the verify that the study single-subject measurements. If subjects have cardiovascular diseases, unique identification may be proposed heart-based was performed cardiacsedentary with healthy cycle points remain persons andconsistent after subjects measurements. only for single-subject perform varying If degrees subjects problematic as the cardiac cycle would be affected. Further study is also required to verify that the of physiological have cardiovascular activity. diseases, unique identification may be problematic as the cardiac cycle would be proposed heart-based cardiac cycle points remain consistent after subjects perform varying degrees affected. Further study is also required to verify that the proposed heart-based cardiac cycle points of physiological activity. remain consistent after subjects perform varying degrees of physiological activity. Figure 7. Experimental setup for cardiac scan continuous authentication system using microwave Doppler Figure 7.radar. A data acquisition Experimental device and setup for cardiac scanLABVIEW interfaces continuous were used authentication to capture system signals. A using microwave Figuresensor pulse 7. Experimental and Achest setup werefor cardiac scan continuous authentication system using microwave Doppler radar. databelt used acquisition for reference device measurements. and LABVIEW From interfaces [58]. were used to capture signals. Doppler radar. A data acquisition device and LABVIEW interfaces were used to capture signals. A A pulse sensor and chest belt were used for reference measurements. From [58]. pulse sensor and chest belt were used for reference measurements. From [58]. Figure Figure 8. 8. Cardiac Cardiacmotion motionmarker. marker. The Thecardiac cardiacmotion motioncycle cycledefined definedby byfive fivedifferent different points points(red (reddots) dots) within five fivedifferent differentpoints of displacement points and timing of displacement was calculated and timing as a unique was calculated as feature for recognizing a unique feature for Figure 8.From people. recognizingCardiac [58].motion people. From marker. [58]. The cardiac motion cycle defined by five different points (red dots) within five different points of displacement and timing was calculated as a unique feature for In another another reported recognizing In people. reported study, From cardiac [58]. study, measurement cardiac measurement of different persons of different was used persons wastoused uniquely identify to uniquely each using identify a 24-GHz each using continuous wave radar system a 24-GHz continuous employing wave radar system six-port measurement employing six-porttechnology measurement [71]. FigureIn 9another reported represents the study, cardiac hardware setup measurement used for this of differentApersons experiment. six-port was used to uniquely measurement system technology [71]. Figure 9 represents the hardware setup used for this experiment. A six-port identify of consists each using input aports twosystem 24-GHz continuous andoffour wave The radar system signals employingsuperimposed six-port measurement measurement consists twooutput input ports. ports and two fourinput output ports.areThe two input signalsto extract are technology phase [71]. Figure shift information due9 to represents the hardware chest displacement. setup used This particular forfocused work this experiment. on extractingAheartbeat six-port measurement signal system information forconsists of two input each participant, ports as the andposition exact four output and ports. angle ofThethetwo input heart signals in the are thorax, as well as the anatomy of the thorax itself, is a little different for every person due to varying tissue and muscle/fat components [74]. Due to these differences, the radar-captured heartbeat signal involved different propagation and attenuation characteristics. As each person has a different heart position and dimensions, dominant features exist in the heartbeat signal which form a complex and unique pattern. Figure 10 illustrates the heartbeat signal variation for each participant. Initially a 5-second heartbeat signal was used for identifying unique features for each participant. Integrated machine learning
angle of the heart in the thorax, as well as the anatomy of the thorax itself, is a little different for every participant. Integrated machine learning classifiers were also used to recognize people. A quadratic person due to varying tissue and muscle/fat components [74]. Due to these differences, the radar- SVM outperformed captured heartbeat other classifiers, signal involved with an accuracy different propagationofand 74.2%. To increase attenuation the accuracy, characteristics. As eacha 7-second heartbeat segment person was used, has a different heartwhich increased position the classification and dimensions, accuracy dominant features tothe exist in 94.6%. The signal heartbeat study provides a clearwhich indication Remote form Sens. that 12, 2279heart-based a complex 2020, geometry and unique pattern. Figurecan be usedthe 10 illustrates asheartbeat a unique feature signal to for variation identify 11each of 22 people. participant. Initially a 5-second heartbeat signal was used for identifying However, validation for this study only included four different participants. Thus, further unique features for each participant. Integrated machine learning classifiers were also used to recognize people. A quadratic investigation iswere classifiers required also usedfor larger data setsAhaving varying conditions, especially those an involving SVM outperformed othertoclassifiers, recognize people. with quadratic an accuracy SVM outperformed of 74.2%. other To increase the classifiers, accuracy, with a 7-second measurements accuracy of heartbeat made 74.2%.after segment was physical To increase used, which activities. the accuracy, increaseda the 7-second heartbeat classification segment accuracy to was used, 94.6%. Thewhich studyincreased provides athe classification clear indicationaccuracy to 94.6%. geometry that heart-based The study can provides a clear be used as aindication that heart-based unique feature to identifygeometry people. can be used as a unique feature to identify people. However, validation for However, validation for this study only included four different participants. Thus, this study only included further four different is investigation participants. required for Thus, further larger data investigation is required sets having varying for larger conditions, data setsthose especially having varying involving conditions, especially measurements thosephysical made after involving measurements made after physical activities. activities. Figure 9. Experimental setup for unique identification of a human from radar captured respiration patternFigure which Experimental Experimental 9. includes setup setup for six-port for unique unique identification technology.identification From [74]. of of a human from radar captured respiration pattern which includes six-port technology. technology. From [74]. Figure 10. Heartbeat curves recorded by a 24-GHz radar for four subjects. Each signal is periodic but for each subject the pattern is a bit different which serves as a unique feature for recognition of identity. From [74].
Figure 10. Heartbeat curves recorded by a 24-GHz radar for four subjects. Each signal is periodic but for each subject the pattern is a bit different which serves as a unique feature for recognition of identity. From [74]. Remote Sens. 2020, 12, 2279 12 of 22 Another study used a 24-GHz radar system to extract heartbeat related unique features to recognizeAnother eleven different study used participants a 24-GHz [75]. radarAn autoregressive system (AR) model-based to extract heartbeat frequency related unique analysis features to was introduced, recognize elevenwhich is superior different to FFT, participants having [75]. a window length An autoregressive of 100 milliseconds, (AR) model-based frequency from which analysis powerwasspectral density introduced, could which be calculated is superior to FFT, [75]. having Each peak inlength a window this analysis represents the of 100 milliseconds, fromcontraction which power spectral and extraction density of the could heart. Thebe calculated first peak was [75].used Eachas peak in this analysis a reference and thenrepresents the contraction a period of 0.2 s before and extraction and after, 0.4 s, wasof the heart. used as aThe first peakTemplate template. was used as a reference matching andused was then atoperiod detectofall 0.2heartbeats. s before and The after,of average 0.4the s, was used asspectral power a template. Template density was matching used was as aused to detect unique all heartbeats.number identification The average forofeach the power spectral density was used as a unique identification number for each participant. Figure 11 participant. Figure 11 illustrates the power spectral density features extracted from the radar illustrates the power spectral density features extracted from the radar respiration signal and PSD respiration signal and PSD profile for eleven different participants. The success rate was 92.8%. The profile for eleven different participants. The success rate was 92.8%. The proposed method clearly proposed method clearly demonstrates heart-based PSD feature extraction efficacy for recognizing demonstrates heart-based PSD feature extraction efficacy for recognizing people. However, if the people. However, position between if the position the radar and between the radar human subject and changes or human the heartsubject changesgreatly rate fluctuates or thethen heart the rate fluctuates greatly then the proposed method produces false classification. Motion proposed method produces false classification. Motion artifacts and multi-subject scenarios were not artifacts and multi- subject scenarios considered were and not aconsidered remain and remain significant challenge a significant for this approach. challenge for this approach. FigureFigure 11. 11. Measured Measured 24-GHz 24-GHz heartbeatpatterns heartbeat patterns for for autoregressive autoregressivePSD PSDanalysis based analysis subject based subject recognition. Measurement of heartbeats are shown for electrocardiogram (reference) (a), Doppler radar recognition. Measurement of heartbeats are shown for electrocardiogram (reference) (a), Doppler radar (b), PSD of Doppler sensor output (c), and (d) PSD for 15-s Doppler radar for eleven participants [75]. (b), PSD of Doppler sensor output (c), and (d) PSD for 15-s Doppler radar for eleven participants [75]. Recently, another study demonstrated the efficacy of radar-based identity authentication using a Recently, anotherTransform short-time Fourier study demonstrated theperson (STFT) [76]. Each efficacy satof radar-based a 1.5 m distance andidentity authentication physiological using signatures a short-time Fourier Transform (STFT) [76]. Each person sat a 1.5 m distance were recorded for about 6 seconds of the breathing pattern, using a 24-GHz continuous wave radar. and physiological signatures An STFT were was recorded for aboutthe used to characterize 6 seconds of thesignature micro-Doppler breathing pattern, of ten differentusing a 24-GHz participants, continuous followed by basic image transformation methods like translation, rotation, zoom, mirror, wave radar. An STFT was used to characterize the micro-Doppler signature of ten different and cropping. The STFT image was participants, used to represent followed by basicheart-based features for each image transformation differentlike methods subject. A deep convolutional translation, rotation, zoom, mirror, and cropping. The STFT image was used to represent heart-based features micro-Doppler neural network (DCNN) was used to classify subjects based on their radar captured for each different signatures. Figure 12 illustrates the STFT images for four participants which are significantly different subject. A deep convolutional neural network (DCNN) was used to classify subjects based on their for each subject and include unique features for identification. From the spectrogram, three different radar captured micro-Doppler signatures. Figure 12 illustrates the STFT images for four participants heart-based features were extracted, such as the period of the heartbeat, the energy of the heartbeat, which are significantly different for each subject and include unique features for identification. From and the bandwidth of the signal. A deep convolutional neural network was then trained, and the the spectrogram, three different resulting classification heart-based accuracy was almost 98.5%.features were extracted, such as the period of the heartbeat,Tothe energy extract of the heartbeat, heart-based andinformation, or respiratory the bandwidth of the signal.generally data segmentation A deep plays convolutional neural an important network role. was then trained, Segments correspondand theFFT to the resulting window classification size and shouldaccuracy contain was almost at least 98.5%. one full respiration cycle and multiple cardiac cycles [58,70]. The number of segments used for a data set also plays an important role for authentication time and accuracy [58]. Increasing the FFT window size will bring a benefit in resolution as a higher number of samples are included in the operation, but this will also increase the time delay and complexity of authentication and is not generally justified for real-time operation.
You can also read