Wearable Technology and Visual Reality Application for Healthcare Systems

Page created by Victor Mitchell
 
CONTINUE READING
Wearable Technology and Visual Reality Application for Healthcare Systems
electronics
Article
Wearable Technology and Visual Reality Application for
Healthcare Systems
Kuang-Hao Lin * and Bo-Xun Peng

 Department of Electrical Engineering, National Formosa University, Yunlin 632, Taiwan;
 11065131@gm.nfu.edu.tw
 * Correspondence: khlin@nfu.edu.tw

 Abstract: This study developed a virtual reality interactive game with smart wireless wearable
 technology for healthcare of elderly users. The proposed wearable system uses its intelligent and
 wireless features to collect electromyography signals and upload them to a cloud database for further
 analysis. The electromyography signals are then analyzed for the users’ muscle fatigue, health,
 strength, and other physiological conditions. The average slope maximum So and Chan (ASM S & C)
 algorithm is integrated in the proposed system to effectively detect the quantity of electromyography
 peaks, and the accuracy is as high as 95%. The proposed system can promote the health conditions of
 elderly users, and motivate them to acquire new knowledge of science and technology.

 Keywords: wearable; IoT; VR; EMG; healthcare

 1. Introduction
 With the advances of modern medicine, people’s average lifespan has prolonged;
 
  hence, the aging of population structures has become a global issue. Moreover, the changes
Citation: Lin, K.-H.; Peng, B.-X.
 of lifestyles have induced various chronic diseases, increasing the demand for medical
Wearable Technology and Visual
 care. As routine health information can enhance disease prevention and medical care,
Reality Application for Healthcare the application of telecare in routine health care can promote people’s quality of life.
Systems. Electronics 2022, 11, 178. In telecare, the first-aid devices can also collect personal physiological information for
https://doi.org/10.3390/ ambulances or relevant medical units, so that the patients under care can receive instant
electronics11020178 and effective assistance.
 Literatures [1,2] used electrocardiography (ECG) combined with the Internet of Things
Academic Editor: Stefanos Kollias
 (IoT) to design a remote monitoring and telemetry system that can improve the accuracy
Received: 29 November 2021 and reliability of heart health monitoring. Literatures [3,4] used electroencephalography
Accepted: 5 January 2022 (EEG) to design a sleep monitoring system and an epilepsy prediction system. Bhowmick
Published: 7 January 2022 used photoplethysmography (PPG) to design a PPG monitoring system and stored the PPG
Publisher’s Note: MDPI stays neutral
 model parameters in the cloud for clinical diagnosis [5]. Huang used electromyography
with regard to jurisdictional claims in
 (EMG) to design a diet monitoring system, which receives EMGs of the mastication muscles
published maps and institutional affil- through glasses to detect intake-related events [6].
iations. This study combined physiological signal modules (namely, ECG, EEG, PPG, and
 EMG) with innovative educational technologies, such as the Internet, IoT [7,8], and virtual
 reality (VR) [9,10], to develop a cloud system with telecare services [11,12]. Using the VR
 and open-source software (OSS), this study developed an interesting VR digital game that
Copyright: © 2022 by the authors. could stimulate the elderly users to utilize the application for health management [13–15].
Licensee MDPI, Basel, Switzerland. The usage scenario of the proposed VR health game is shown in Figure 1.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).

Electronics 2022, 11, 178. https://doi.org/10.3390/electronics11020178 https://www.mdpi.com/journal/electronics
Wearable Technology and Visual Reality Application for Healthcare Systems
Electronics 2022,11,
Electronics2022, 11,178
 x FOR PEER REVIEW 22 of
 of 17
 17

 Figure 1. Schematic diagram of a system usage scenario.
 Figure 1. Schematic diagram of a system usage scenario.
 Artificial intelligence (AI) has been widely used in the identification and analysis of
 Artificialsignals
 physiological intelligence
 in recent(AI)years.
 has beenRecentwidely used in
 common EMGthe analyses
 identification and analysis
 and studies focused of
 physiological
 on AI application. signals
 Forin recent years.
 example, SugiartoRecent
 [15] common
 analyzed EMG analyses
 the surface EMG and studiesand
 (sEMG) focused
 high-
 on AI application.
 density EMG (HD-EMG) For example, Sugiarto
 of the neck muscle [15](designated
 analyzed the surfaceand
 muscle) EMG used(sEMG) and high-
 Convolutional
 density EMG (HD-EMG) of the neck muscle (designated muscle)
 Neural Network (CNN) for training to correct the end-to-end delay of the VR system and and used Convolutional
 Neural Network
 alleviate negative (CNN)
 effects suchfor training
 as motion to correct
 sickness. the end-to-end
 Sugiarto delay
 [15] also of the VR
 employed system
 three pairsand
 of
 alleviatesEMG
 wireless negative effects
 sensors from such as motion
 Delsys Trignosickness. Sugiarto
 (Delsys, MA, USA)[15] alsosoftware.
 as the employed three pairs
 Pancholi [16]
 of wireless
 analyzed thesEMG
 peak sensors
 averagefrom power Delsys
 (PAP)Trigno
 of EMG (Delsys, MA, USA)
 and adopted Linearas Discriminant
 the software. Analysis
 Pancholi
 [16] analyzed
 (LDA) the peakDiscriminant
 and Quadratic average power (PAP) of
 Analysis EMGfor
 (QDA) andtraining
 adoptedand Linear Discriminant
 prosthetics. The
 Analysisof
 analysis (LDA) and [16]
 Pancholi Quadratic
 has been Discriminant
 done usingAnalysis
 MATLAB (QDA)
 2015afor ontraining
 an i7 core.andRaurale
 prosthetics.
 [17]
 The analysis of Pancholi [16] has been done using MATLAB 2015a
 employed subsequently classified to classify and identify eight kinds of EMG actions. The on an i7 core. Raurale
 [17] employed
 system subsequently
 is also shown to operate classified to classify
 in real-time on anandARM identify
 Cortexeight
 A-53kinds of EMG
 embedded actions.
 processor
 The system
 suitable is also shown
 for housing in anto EMGoperate in real-time
 wearable device.onAdditionally,
 an ARM Cortex Wang A-53 embedded
 [18] modifiedpro- the
 So andsuitable
 cessor Chan algorithm
 for housing to inincrease
 an EMG thewearable
 accuracydevice.
 of ECG peak detection
 Additionally, Wang to [18]
 99.16% from
 modified
 94.61% andChan
 the So and used algorithm
 FPGA for to verification.
 increase the However,
 accuracyitofisECG complicated and nottoapplicable
 peak detection 99.16% from to
 biomedical
 94.61% andVR used games,
 FPGAwhich are characterized
 for verification. However, by low
 it is complexity
 complicated[19,20].
 and not Therefore,
 applicable thisto
 study aims to develop an innovative realization architecture
 biomedical VR games, which are characterized by low complexity [19,20]. Therefore, this with low complexity. The
 proposed
 study aims system specifically
 to develop targets therealization
 an innovative irregularities of EMG peaks,
 architecture with low in order to improve
 complexity. The
 the operational
 proposed system accuracy and smoothness
 specifically of VR games.ofToEMG
 targets the irregularities verify the EMG
 peaks, peaktodetection
 in order improve
 accuracy, this study
 the operational first used
 accuracy andthe automaticof
 smoothness calibration
 VR games. detection
 To verify (ACD)
 the EMG to capture the EMC
 peak detection
 signals,
 accuracy, and setstudy
 this a threshold parameter
 first used to assess the
 the automatic captureddetection
 calibration signals. To further
 (ACD) toimprove
 capture thethe
 EMG peak detection accuracy, this study applied the average slope
 EMC signals, and set a threshold parameter to assess the captured signals. To further im- maximum So and Chan
 (ASM
 prove Sthe& C)EMGalgorithm [21–25]. accuracy,
 peak detection The comparisonthis studyof recent
 applied studies on physiological
 the average slope maximumsignal
 detection systems is as shown in Table 1. The operation sensitivity
 So and Chan (ASM S & C) algorithm [21–25]. The comparison of recent studies on physi- was thus enhanced in
 the proposed VR game.
 ological signal detection systems is as shown in Table 1. The operation sensitivity was
 thus enhanced
 Table 1. Comparison in the proposed
 of Recent VR on
 Studies game.
 Physiological Signal Detection Systems.

 Table 1.Refrence
 Comparison of Recent Studies on Physiological Signal
 Algorithm Detection Systems.
 Complexity Equipment

 Refrence Algorithm Complexity Delsys Trigno
 Equipment
 Sugiarto [15] CNN High
 IMU
 Delsys Trigno
 Sugiarto
 Pancholi[15]
 [16] CNN
 LDA High
 High I7 core
 IMU
 Raurale[16]
 Pancholi [17] Subsequently
 LDA classified Medium
 High ARMI7Cortex
 core A-53
 Wang [17]
 Raurale [18] Enhanced classified
 Subsequently So and Chan Low
 Medium FPGA A-53
 ARM Cortex
 Wang [18]
 Our propose Enhanced So
 ASM and
 S&CChan LowLow FPGA
 nRF52840
 Our propose ASM S&C Low nRF52840
Wearable Technology and Visual Reality Application for Healthcare Systems
Electronics 2022, 11, x FOR PEER REVIEW 3 of 17

Electronics 2022, 11, 178 3 of 17

 The remainder of the paper is organized as follows. Section 2 describes the system
 architecture design and
 The remainder its specifications.
 of the Section
 paper is organized as 3follows.
 discusses the detailed
 Section design
 2 describes themethod-
 system
 ology for EMG peak detection. Section 4 presents the analysis of the experimental results.
 architecture design and its specifications. Section 3 discusses the detailed design methodol-
 Finally,
 ogy for conclusions are drawnSection
 EMG peak detection. in Section 5.
 4 presents the analysis of the experimental results.
 Finally, conclusions are drawn in Section 5.
 2. System Architecture
 2. SystemArchitecture
 Hardware Architecture
 Hardware Architecture
 Figure 2 shows the hardware architecture of the smart wearable biomedical sensing
 systemFigure 2 shows
 developed inthe
 thishardware
 study. The architecture
 sensing end of the
 usessmart wearableanbiomedical
 the Tri-BLE, sensing
 IoT development
 system developed in this study. The sensing end uses the Tri-BLE,
 platform, as the main body, and is provided with a Tri-EMG sensor, a button, and an an IoT development
 platform,light.
 indicator as theThe
 main body,
 human andsignal
 EMG is provided
 is sent with
 via a adisposable
 Tri-EMG electrode
 sensor, a to
 button, and an
 the Tri-EMG
 indicator light. The human EMG signal is sent via a disposable electrode
 sensor, and subsequently exported. The Tri-BLE platform filters the received EMG data, to the Tri-EMG
 sensor,
 and the and subsequently
 threshold of the EMGexported.
 signal The Tri-BLE
 is judged platformtofilters
 according the received
 the relationship EMG data,
 between the
 and the threshold of the EMG signal is judged according to the relationship
 EMG signal and the threshold. The output state is transferred to the central controller and between the
 EMG signal and the threshold. The output state is transferred to the central
 processed. Furthermore, the button controls a calibration mechanism. The threshold of controller and
 processed.
 the EMG signalFurthermore,
 varies withthethe
 button
 human controls
 body aandcalibration mechanism.
 the electrode The
 position, andthreshold of the
 this threshold
 EMG signal varies with the human body and the electrode position, and this
 can be adjusted according to the indicated action after the button is pressed. The indicator threshold can
 be adjusted according to the indicated action after the button is pressed. The indicator light
 light displays the state of the sensing end—the light is red during calibration, and the light
 displays the state of the sensing end—the light is red during calibration, and the light turns
 turns green if the EMG signal exceeds the threshold. The central controller integrates the
 green if the EMG signal exceeds the threshold. The central controller integrates the states of
 states of the left- and right-hand sensing ends, and communicates with the VR device to
 the left- and right-hand sensing ends, and communicates with the VR device to control the
 control the action in the game. Additionally, the system has a mobile app that is linked
 action in the game. Additionally, the system has a mobile app that is linked via Bluetooth.
 via Bluetooth. The muscle conditions of the left and right hands can be recorded, and ac-
 The muscle conditions of the left and right hands can be recorded, and accessed by doctors
 cessed by doctors for future diagnoses.
 for future diagnoses.

 Figure
 Figure 2.
 2. Hardware
 Hardware architecture
 architecture of
 of the
 the smart
 smart wearable biomedical sensing
 wearable biomedical sensing system.
 system.

 Table 2 lists the equipment specifications.
 specifications. Tri-BLE
 Tri-BLE and
 and Tri-EMG
 Tri-EMG ofof TriAnswer
 TriAnswer series
 are used for the biomedical development
 development platform,
 platform, Arduino
 Arduino Uno
 Uno is
 is used
 used as
 as the central
 includes an HTC VIVE VR headset
 controller, and the monitor set includes headset and
 and an app, with the
 associated equipment of HTC-VIVE and Android smartphone, respectively.
 respectively.
 Table 2. Equipment specifications.
 Table 2. Equipment specifications.
 System
 System Requirement
 Requirement Equipment
 Equipment
 Biomedical
 Biomedical TriAnswer
 TriAnswer
 Development
 Development Platform
 Platform (Tri-BLE,
 (Tri-BLE, Tri-EMG)
 Tri-EMG)
 Central controller
 Central controller Arduino
 ArduinoUNO
 UNO
 Virtual reality HTC-VIVE Cosmos
 Monitor Virtual reality HTC-VIVE Cosmos
 Monitor Android Smartphones
 Android Smartphones
Wearable Technology and Visual Reality Application for Healthcare Systems
Electronics 2022, 11, x FOR PEER REVIEW 4 of 17
 Electronics 2022, 11, 178 4 of 17

 3. EMG Peak Detection
 3. EMG
 3.1. System Peak
 FlowDetection
 3.1. Figure
 System 3Flow
 presents the moving average system flowchart. Notably, the human EMG
 signalsFigure 3 presents the
 are complicated andmoving average
 susceptible systeminfluences.
 to external flowchart.This
 Notably,
 studythe human
 thus EMG
 employed
 signals are complicated and susceptible to external influences. This
 the moving average and ACD methods to reduce the number of signal judgment errors.study thus employed
 thethe
 For moving
 moving average andthe
 average, ACD
 EMG methods
 signals to
 arereduce
 importedthe at
 number
 500 Hz.ofThe
 signal judgment
 length of eacherrors.
 data
 For the moving average, the EMG signals are imported at 500 Hz. The length
 is 8 bits, and the system output is the average of 20 data points. Therefore, the first of each data
 in first
 is 8(FIFO)
 out bits, and the system
 buffer which output is the
 can store average
 20 data of 20was
 points data points.
 used. Therefore,
 When the first
 the buffer was in first
 filled
 out (FIFO)
 with 20 databuffer which
 points, can storevalue
 the average 20 data
 waspoints was used.
 calculated as theWhen
 outputtheY(n).
 buffer was filled with
 20 data points, the average value was calculated as the output Y(n).

 X(n)

 FIFO
 Buffer

 No
 n>20

 Yes
 Average
 FIFO Buffer

 Y(n)

 Figure
 Figure3.3.Moving
 Movingaverage
 averagesystem
 systemflowchart.
 flowchart.

 Figure
 Figure44isisthe
 the ACD
 ACD system flowchart.
 flowchart. The
 The ACD
 ACDsystem
 systemadopts
 adopts10,000
 10,000EMG
 EMGsignals
 signalsas
 asthe
 thethreshold
 thresholdcalibration
 calibrationcriteria.
 criteria.First,
 First, the
 the input data are compared
 compared withwiththe
 theeight
 eightdata
 data
 ininthe
 thebuffer.
 buffer.When
 Whenthe theinput
 inputis is larger
 larger than
 than thethe buffer,
 buffer, thethe minimum
 minimum value
 value in the
 in the buffer
 buffer is
 is replaced
 replaced by bythethe
 new new input
 input data.
 data. After
 After thethe system
 system comparesthe
 compares the10,000
 10,000data
 datapoints,
 points,the
 the
 threelargest
 three largestdata
 data points
 points are
 are removed from from the
 the buffer
 buffertotoavoid
 avoidthethesignal
 signalerrors
 errorsresulting
 resultingin
 ina high
 a high threshold.
 threshold. After
 Afterfivefive
 data points
 data are are
 points averaged, this this
 averaged, value is subtracted
 value by thebyinitial
 is subtracted the
 value (O
 initial value ) and multiplied by the scale factor of D to obtain the threshold after
 ini (Oini) and multiplied by the scale factor of D to obtain the threshold after cal- calibration.
 Figure 5 is the average slope maximum So and Chan (ASM S & C) system flowchart.
 ibration.
 The initial_value is calculated first, and if the EMG signal exceeds the initial_value, the ini-
 tial_slope_maxi is being searched. The slope_threshold is then obtained by the initial_slope_maxi.
 If two consecutive EMG slopes exceeds the slope_threshold, the current signal is recorded
 as the starting point of the waveform characteristics, and the peak could be determined.
 Finally, the peak and starting points are used for updating the maxi. The ASM S & C
 algorithm is described in Section 3.2.

 3.2. Algorithm
 EMG signal shows the potential difference when a muscle contracts. According to the
 state of motion, the amplitude and frequency of the electromyogram change accordingly.
 Under ideal conditions, an electromyogram indicates no movement, while obvious fluc-
 tuations occur when the muscles are contracted (as shown in Figure 6). However, EMG
 signals are susceptible to random interference caused by radio waves, electrode noise, and
 other factors [26–28].
Wearable Technology and Visual Reality Application for Healthcare Systems
Y(n)

 No Yes
 n>10,000
Electronics 2022, 11, x FOR PEER REVIEW 5 of 17
Electronics 2022, 11, 178 5 of 17

 Compare Average
 Y(n)

 Scale Factor
 Buffer[0:7]
 No Yes D
 n>10,000

 Compare Initial
 AverageValue
 Oini

 Scale Factor
 Buffer[0:7] threshold
 D

 Figure 4. ACD system flowchart.
 Initial Value
 Figure 5 is the average slope maximum So and Chan (ASM S & C) system flowchart.
 Oini
 The initial_value is calculated first, and if the EMG signal exceeds the initial_value, the ini-
 tial_slope_maxi is being searched. The slope_threshold is then obtained by the ini-
 tial_slope_maxi. If two consecutive EMG slopes exceeds the slope_threshold, the current sig-
 nal is recorded as the starting point of thethreshold
 waveform characteristics, and the peak could
 be determined. Finally, the peak and starting points are used for updating the maxi. The
 ASM S & C algorithm is described in Section 3.2.
 Figure 4. ACD system flowchart.
 Figure 4. ACD system flowchart.

 Figure 5 is the average
 Y(n)
 slope maximum So and Chan (ASM S & C) system flowchart.
 The initial_value is calculated first, and if the EMG signal exceeds the initial_value, the ini-
 tial_slope_maxi is being searched. The slope_threshold is then obtained by the ini-
 tial_slope_maxi.No
 If two consecutive
 maxi
 Yes slopes exceeds the slope_threshold, the current sig-
 EMG
 nal is recorded as the starting point of the waveform characteristics, and the peak could
 be determined. Finally, the peak and starting points are used for updating the maxi. The
 slope_threshold
 ASM SAverage
 & C algorithm is described in Section 3.2.
 (ST)

 Y(n)
 initial_value slope
 No Yes
 maxi
 Get No
 slope > ST
 initial_slope_maxi slope_threshold
 Average
 (ST) Yes

 update Get
 maxi EMG Peak
 initial_value slope
 Figure5.5.ASM
 Figure ASMSS&&CCsystem
 systemflowchart.
 flowchart.

 Get No
 slope > ST
 initial_slope_maxi
 Yes
 update Get
 maxi EMG Peak

 Figure 5. ASM S & C system flowchart.
Wearable Technology and Visual Reality Application for Healthcare Systems
3.2. Algorithm
 EMG signal shows the potential difference when a muscle contracts. According to
 the state of motion, the amplitude and frequency of the electromyogram change accord-
 ingly. Under ideal conditions, an electromyogram indicates no movement, while obvious
 fluctuations occur when the muscles are contracted (as shown in Figure 6). However,
Electronics 2022, 11, 178 6 of 17
 EMG signals are susceptible to random interference caused by radio waves, electrode
 noise, and other factors [26–28].

 (A) (B)
 Figure 6. EMG
 Figurecomparison with and without
 6. EMG comparison with andmovement. (A) No action.
 without movement. (A) No(B)action.
 Action.(B) Action.

 The two Theconsiderations of noise interference
 two considerations and hardware
 of noise interference design allow
 and hardware designtheallow
 moving the moving
 average algorithm to effectively
 average algorithm eliminateeliminate
 to effectively random interference in a largein
 random interference amount
 a largeof data. of data.
 amount
 The moving
 Theaverage
 moving algorithm used in this
 average algorithm usedpaper adopted
 in this paper feedback control ascontrol
 adopted feedback the filtering
 as the filtering
 method. method. After
 After a fixed a fixed
 period sizeperiod sizethe
 is given, is given,
 outputthe output
 is the averageis theof average
 the inputofvalues
 the input in values
 the periodin size.
 the period size. Its
 Its operating operating
 principle principle
 is to remove is tooldest
 the remove the oldest
 element element
 and add the nextand add the
 nextbefore
 input value input value beforethe
 averaging averaging
 output. the Thisoutput.
 outputThis valueoutput value
 divides divides
 the random the noise
 random noise
 evenly andevenly
 makesandit makes
 smaller, it thus
 smaller, thus the
 making making overallthedata
 overall data waveform
 waveform smoother. smoother.
 The moving The average
 moving algorithm
 average algorithm
 is expressed is expressed in the following
 in the following equation:equation:
 ∑ ( P−= )
 h i
 = ∑ P = 0T − 1 X (n − P) (1)
 Y = (1)
 T
 where Y is the output of the moving average; T is the period length; and X(n) is the current
 where Y is the output of the moving average; T is the period length; and X(n) is the current
 input data.
 input data.
 Although the moving average can effectively remove random noise, EMG changes
 Although
 can be easily read the moving
 and compared withaverage can effectively
 the threshold remove random
 signal. However, factors noise,
 such as EMGdif- changes
 ferent users or patch positions can change the EMG amplitude and cause a large gap in such as
 can be easily read and compared with the threshold signal. However, factors
 different
 the threshold. users orthis
 Therefore, patch positions
 paper designedcan an change the EMG
 automatic amplitude
 calibration and cause
 detection (ACD) a large gap
 algorithm to solve the problem of threshold error. The ACD algorithm collects 10,000 (ACD)
 in the threshold. Therefore, this paper designed an automatic calibration detection
 pieces ofalgorithm
 data to settothesolve the problem
 threshold. The ACDof threshold
 algorithm error. The ACD
 equation is asalgorithm
 follows: collects 10,000 pieces
 of data to set the threshold. The ACD algorithm equation is as follows:
 1
 ℎ ℎ = ( 1 )n =× k + (1 − ) × (2)
 " #
 
 threshold =
 kn=1∑ S(n + 3) × D + [(1 − D ) × Oini ] (2)
 where S is the output after comparison, k = m − 3, D is the scale factor, and Oini is the initial
 EMG value.
 where A method
 S is thesimilar
 outputtoafterthe sorting
 comparison,algorithmk=m is −
 used3, Dinisthethecomparison.
 scale factor,Asand long Oini is the
 as the input value of the ACD algorithm is greater than the value in
 initial EMG value. A method similar to the sorting algorithm is used in the comparison. the buffer, it can be
 directly replaced.
 As long asThis the method
 input valuecan find
 of thethe
 ACD largest m data is
 algorithm and sort them
 greater than from the largest
 the value in the buffer, it
 to the smallest. In orderreplaced.
 can be directly to prevent Thisnoise
 methodinfluence,
 can find thethelargest
 largest three
 m datapieces
 andofsort
 data themare from the
 removedlargest
 from the m data.
 to the There
 smallest. Inare
 ordersignal bounces
 to prevent occasionally
 noise influence,inthe thelargest
 process of EMG
 three pieces of data
 are removed
 signal sensing; hence,fromthe top m data.
 thethree There
 of the eightarepieces
 signal of bounces
 data are occasionally
 removed toinprevent
 the process the of EMG
 values ofsignal sensing;
 high signal hence,from
 bounces the top three of
 affecting thethe eight pieces
 average value.of data are removed to prevent the
 Thevalues of high signal
 data averaged bouncesbyfrom
 is multiplied affecting
 the scale factor theDaverage
 to improve value. the accuracy of dis-
 crimination. Finally, Oini is added as the initial compensation of theDthreshold.
 The data averaged is multiplied by the scale factor to improve The thescale
 accuracy of
 factor D discrimination.
 can obtain the best Finally, Oini is added
 parameters after as thethe initial compensation
 numerical of the threshold.
 statistical analysis, as shownThe scale
 in Figurefactor D can
 7. When theobtain
 scale the bestisparameters
 factor 50%, the accuracyafter theofnumerical
 the EMGstatistical analysis,
 peak detection canas shown
 reach 71%.in Figure 7. When the scale factor is 50%, the accuracy of the EMG peak detection can
 reach 71%.
Wearable Technology and Visual Reality Application for Healthcare Systems
Electronics 2022, 11, x FOR PEER REVIEW 7 of 17
Electronics 2022, 11, 178 7 of 17

 80

 60

 Accuracy (%)
 40

 20

 0
 10 20 30 40 50 60 70 80 90 100
 Scale factor D (%)
 Figure
 Figure 7.
 7. Scale
 Scale factor
 factor statistical
 statistical analysis.
 analysis.

 The optimal scale factor in the ACD algorithm varies with the users; therefore, the
 data from each individual user should be statistically analyzed, and the scale factor cannot
 be corrected during real-time hardware
 hardware variations.
 variations. As a result, the accuracy varies by each
 user. In
 In order
 orderto toenhance
 enhancethetheaccuracy
 accuracyinin
 detection,
 detection,thisthis
 study detected
 study the the
 detected EMG peaks,
 EMG and
 peaks,
 usedused
 and the transient peakpeak
 the transient when the muscle
 when put put
 the muscle forthforth
 strength as the
 strength as reference frame
 the reference for the
 frame for
 overall
 the threshold.
 overall ThisThis
 threshold. study adopted
 study thethe
 adopted So So
 andandChan
 Chan (S &
 (S C) algorithm,
 & C) which
 algorithm, which is is
 the R
 the
 wave detection algorithm proposed by So and Chan in 1997 [22], and modified
 R wave detection algorithm proposed by So and Chan in 1997 [22], and modified it into it into the
 average
 the slope
 average maximum
 slope maximumSo and Chan
 So and (ASM
 Chan S & SC)&algorithm.
 (ASM C) algorithm.
 The S & C algorithm is given below
 The S & C algorithm is given below [22]: [22]:
 Y(n) is
 Y(n) is the
 the processed
 processed EMG
 EMG amplitude,
 amplitude, and the slope
 and the slope ofof each
 each point
 point can
 can be
 be found
 found inin the
 the
 time domain
 time domain by by Equation
 Equation (3):
 (3):
 ( )
 slope(n=
 )=−2 ( 
 −2Y (−n 2)
 − 2−) ( 
 − Y (−n 1)
 − 1+) ( 
 + Y (+n1)
 ++ 1) 2 ( 
 + 2Y (+n2)
 + 2) (3)

 The slope_threshold can be
 The slope_threshold obtained
 can by Equation
 be obtained (4): (4):
 by Equation
 ℎ ℎ _ 
 _ ℎ ℎ = threshold_paprameter× (4)
 slope_threshold = 16 × maxi (4)
 16
 First, a sample quantity is set up according to the sample rate of signal reception to
 First,
 find the a sample quantity
 initial_slope_maxi. Theissample
 set uprate
 according
 used intothis
 thepaper
 sample rateHz,
 is 500 of signal reception
 as shown in Al-
 to find the initial_slope_maxi.
 gorithm 1: S & C algorithm. The sample rate used in this paper is 500 Hz, as shown in
 Algorithm 1: S & C algorithm.
 Algorithm
 Algorithm 1:1:SS&&C C Algorithm.
 Algorithm.
 Input:
 Input:
 Y(n)
 Y(n)
 Output:
 Output:
 1:1: slope
 slope = 0= 0
 3:2: initial_slope_maxi
 initial_slope_maxi =0= 0
 4:3: for
 for i: 1= to
 i: = 1 to500500
 do do
 5:4: if i if≥i3>=then
 3 then
 6:5: = −2 =×−2
 slopeslope Y(i×−Y(i
 2) −−2)
 Y(i− −
 Y(i1)−+1)
 Y(i+ +Y(i
 1) + 2 ×+ Y(i
 + 1) 2 ×+Y(i
 2) + 2)
 7:6: if slope > initial_slope_maxi
 if slope then then
 > initial_slope_maxi
 7: initial_slope_maxi = slope
 8: initial_slope_maxi = slope
 8: end if
 9: end if
 9: end if
 10:
 10: end forend if
 11: end for
 11: maxi = initial_slope_maxi
 12: maxi = initial_slope_maxi
 The slope_threshold is determined by using the initial maximum slope. When two
 consecutive slopes are larger
 The slope_threshold than theby
 is determined slope_threshold,
 using the initial thismaximum
 point is marked as thetwo
 slope. When starting
 con-
 secutive slopes are larger than the slope_threshold, this point is marked as the starting point
Wearable Technology and Visual Reality Application for Healthcare Systems
Electronics 2022, 11, x FOR PEER REVIEW 8 of 17

Electronics 2022, 11, 178 8 of 17

 of the signature waveform, and then the peak point of waveform could be determined.
 The
 pointmaxi is signature
 of the then updated, as expressed
 waveform, inthe
 and then Equation (5): of waveform could be determined.
 peak point
 _ 
 The maxi is then updated, as expressed − (5):
 in Equation
 = + (5)
 _ 
 f irst_maxi − maxi
 maxi = + maxi (5)
 f iliter_parameter
 _ = Height of p_point − Height of start_point (6)
 f irst_maxi = Height of p_point − Height of start _point (6)
 For the filiter_parameter which can be set as 2, 4, 8, or 16, this paper selected 16. The
 peak For
 wasthe
 recorded, and the which
 filiter_parameter peak detection
 can be setaccuracy
 as 2, 4, 8, of
 or the algorithm
 16, this was calculated
 paper selected [22].
 16. The peak
 was recorded,
 There and the
 were 100,000 EMGpeak detection
 samples, theaccuracy
 number of the algorithm
 making was 154,
 a fist was calculated
 SP was[22].
 the There
 num-
 were
 ber of100,000 EMG samples,
 correct peaks captured the number ofMP
 successfully, making
 was the a fist was 154,
 number SP waspeaks
 of correct the number
 that wereof
 correct peaks
 missed, and EP captured
 was thesuccessfully, MP waserror.
 position of capture the number of correct
 The accuracy canpeaks that were
 be computed bymissed,
 Equa-
 and (7).
 tion EP was
 The the
 S &position of capture
 C algorithm error.
 captured 154The accuracy
 peaks can be computed
 successfully, by Equation
 but there were (7).
 54 capture
 The S & C algorithm captured
 errors, for an accuracy of 74%. 154 peaks successfully, but there were 54 capture errors, for
 an accuracy of 74%.
 SP
 Accuracy(%) = × 100 (7)
 SP + MP SP+ EP
 Accuracy(%) = × 100 (7)
 Figure 8 simulates the peak detection of + MP
 SPthe S&+ EP
 C algorithm. It was observed that the
 S & CFigure
 algorithm had no the
 8 simulates obvious
 peak waveform
 detection of characteristics in the first
 the S & C algorithm. 500 samples,
 It was observedindi-
 that
 cating
 the S &there were errors
 C algorithm hadinnocapturing the initial_slope_maxi
 obvious waveform and in
 characteristics leading to errors
 the first in the
 500 samples,
 indicating there
 subsequent were errors
 slope_threshold in capturing
 and the initial_slope_maxi and leading to errors in the
 peak detection.
 subsequent slope_threshold and peak detection.
 2
 EMG value
 EMG Peak
 1.9

 1.8
 Voltage (Vp-p)

 1.7

 1.6

 1.5

 1.4

 1.3
 0 10 20 30 40 50 60
 Time(s)

 Figure
 Figure 8.
 8. Peak
 Peak detection
 detection of
 of the
 the SS &
 &CC algorithm.
 algorithm.

 The S & C C algorithm
 algorithm is is mainly
 mainly used
 used for
 for ECG
 ECG detection.
 detection. TheThe main
 main difference
 difference between
 between
 EMG and ECG signalssignals is
 is that
 that an
 an EMG
 EMG signal
 signal has
 has obvious
 obvious waveform
 waveform characteristics
 characteristics only if
 the muscle
 muscle contracts,
 contracts, whereas
 whereas an an ECG
 ECG signal
 signal always
 always hashas waveform
 waveform characteristics.
 characteristics. Hence,
 Hence,
 the initial_slope_maxi capture requirements of the
 the initial_slope_maxi capture requirements of the S & algorithm S & C algorithm are not applicable to
 applicable
 EMG. The
 EMG. initial_slope_maxi in
 The initial_slope_maxi in the
 the SS &
 &C C algorithm
 algorithm is is captured
 captured in in aa preset period of cycle
 In ECG
 time. In ECG with
 with aacycle
 cyclethat
 thatchanges
 changesslightly,
 slightly, accurate initial_slope_maxi can be
 accurate initial_slope_maxi be obtained
 obtained
 from the S & C algorithm after adjusting
 from adjusting thethe cycle
 cycle based
 based on the ECG signals signals of different
 persons. However,
 However,for forEMG
 EMGsignals
 signalswith irregular
 with irregular changes,
 changes,detection
 detectionbased on a on
 based fixed cycle
 a fixed
 is not feasible. In order to solve this problem, the capture requirements
 cycle is not feasible. In order to solve this problem, the capture requirements of the maxi- of the maximum
 slope slope
 mum were changed in thisinpaper,
 were changed as shown
 this paper, in Algorithm
 as shown 2: ASM
 in Algorithm 2: SASM
 & C Salgorithm.
 & C algorithm.
Wearable Technology and Visual Reality Application for Healthcare Systems
Electronics 2022, 11, 178 9 of 17

 Algorithm 2: ASM S & C Algorithm.
 Input:
 Y(n)
 Output:
 1: initial_value = avg(sum(Y(1:n))) × (λ + 1)
 2: slope = 0
 3: initial_slope_maxi = 0
 4: count = 0
 5: en = 0
 6: for i:= 1 to length(Y) do
 7: if I ≥ 3 && Y(i) > initial_value then
 8: initial_slope_maxi = −2 × Y(I — 2)−Y(I — 1)+Y(I + 1)+2 × Y(I + 2)
 9: count = count + i
 10: break
 11: end if
 12: end for
 13: for i:= count to length(Y) do
 14: slope = −2 × Y(I — 2) — Y(I — 1) + Y(I + 1) + 2 × Y(I + 2)
 15: if slope > initial_slope_maxi then
 16: en = 1
 17: end if
 18: if en == 1 then
 19: if slope < initial_slope_maxi then
 20: initial_slope_maxi = old_slope
 21: break
 22: end if
 23: end if
 24: end for
 25: maxi = initial_slope_maxi

 As the initial value of the EMG signal was not zero, the initial maximum slope capture
 requirements were changed in this study by using the average of n data and multiplying it
 by reduction ratio λ as the initial_value. The calculation approach could be expressed as
 Equation (8) in which n is set as 1000 samples. Each sample was 3.3 ms, and 1000 pieces of
 data (about 3.3 s) were captured for analysis during the initial setting of the initial_value, so
 as to capture stable initial_value data. When the detected signal exceeds this initial_value,
 it means that muscles have contracted and there are waveform characteristics. After
 determining the initial_slope_maxi, the subsequent slope_threshold calculation and peak
 detection can be performed.
 n
 ( λ + 1)
 initial_value = × ∑ Y (n) (8)
 n 1

 λ in the initial_value calculation is the gathered statistics parameter. In order to avoid the
 error signal exceeding the average initial value, the initial_value calculation was multiplied
 by λ + 1 to evade the error signal. As only the error signal needed to be prevented, the
 accuracy was enhanced and stabilized after increasing the initial_value by 1%. An excessive
 λ value could increase the initial_value and result in the failure to capture the peak and
 a reduction of accuracy. Λ was thus selected as 5% of the stable zone after the statistical
 analysis, as shown in Figure 9.
Electronics 2022, 11, x FOR PEER REVIEW 10 of 17

Electronics 2022, 11, x FOR PEER REVIEW 10 of 17
Electronics 2022, 11, 178 10 of 17

 100

 100
 95

 95
 90

 (%) (%)
 90

 Accuracy
 85
 Accuracy
 85
 80

 80
 75

 75
 70
 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%
 70 λ
 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%
 Figure 9. λ statistical analysis. λ

 Figure 9.
 Figure statistical analysis.
 9. λλ statistical analysis.
 Figure 10 illustrates a simulation of the peak detection by the ASM S & C algorithm,
 which employs
 Figure the initial_value
 10 illustrates
 illustrates calculation topeak
 solvedetection
 the initialby
 detection error in this study.
 Figure 10 aasimulation
 simulation ofthe
 of the peak detection theASM
 by the ASM SS&&CCalgorithm,
 algorithm,
 The initial_slope_maxi
 which employs
 employs the was captured when
 initial_valuecalculation
 theinitial_value the
 calculationto detected
 tosolve
 solvethe signal
 theinitial exceeded
 initialdetection the
 detectionerror EMG
 errorininthis initial
 thisstudy.
 study.
 which
 value.
 The The ASM S & Cwas
 initial_slope_maxi captured 154 when
 captured peaks the
 successfully
 detected and there
 signal were eight
 exceeded the capture
 EMG er-
 initial
 The initial_slope_maxi was captured when the detected signal exceeded the EMG initial
 rors,
 value. leading
 The to an
 ASM S accuracy
 & C of 95.06%.
 captured 154 peaks successfully and there were eight capture errors,
 value. The ASM S & C captured 154 peaks successfully and there were eight capture er-
 leading
 rors, to antoaccuracy
 leading of 95.06%.
 an accuracy of 95.06%.
 2
 EMG value
 2 EMG Peak
 1.9 EMG value
 EMG Peak
 1.9
 1.8
 (Vp-p)

 1.8
 1.7
 (Vp-p)
 Voltage

 1.7
 1.6
 Voltage

 1.6
 1.5

 1.5
 1.4

 1.4
 1.3
 0 10 20 30 40 50 60
 1.3 Time(s)
 0 10 20 30 40 50 60
 Figure
 Figure 10. Peak
 Peak detection
 detection of
 of the
 the ASM & CTime(s)
 ASM SS & algorithm.
 Figure 10. Peak detection of the ASM S & C algorithm.
 Table 3 compares the accuracy of the three algorithms. Equation (7) was used for the the
 calculation of accuracy.
 accuracy. While the S & C and ASM S & C algorithms could capture allthe
 While the S & C and ASM S & C algorithms could capture all of of
 Table 3 compares the accuracy of the three algorithms. Equation (7) was used for the
 correct peaks, incorrect
 the correct peaks, values
 incorrect werewere
 values captured as well.
 captured The ASM
 as well. S & CSalgorithm
 The ASM improved
 & C algorithm im-
 calculation of accuracy. While the S & C and ASM S & C algorithms could capture all of
 the accuracy
 proved of the S of
 the accuracy &C thealgorithm in capturing
 S & C algorithm the EMG’s
 in capturing theinitial
 EMG’s maximum slope, reduced
 initial maximum slope,
 the correct peaks, incorrect values were captured as well. The ASM S & C algorithm im-
 the subsequent
 reduced error rate,error
 the subsequent and resulted
 rate, andinresulted
 a final accuracy
 in a finalof accuracy
 95.06%. Therefore,
 of 95.06%.the accuracy
 Therefore,
 proved the accuracy of the S & C algorithm in capturing the EMG’s initial maximum slope,
 of the
 the ASM Sof
 accuracy & the
 C method
 ASM S proposed
 & C method in this paper was
 proposed higher
 in this paper than
 wasboth the Sthan
 higher & Cboth
 and the
 ACD S
 reduced the subsequent error rate, and resulted in a final accuracy of 95.06%. Therefore,
 methods, by 20.98% and 23.71%, respectively.
 & C and ACD methods, by 20.98% and 23.71%, respectively.
 the accuracy of the ASM S & C method proposed in this paper was higher than both the S
 &Table 3. Algorithm
 C and accuracy
 ACD methods, bycomparison.
 20.98% and 23.71%, respectively.
 Table 3. Algorithm accuracy comparison.
 TableAlgorithm
 3. Algorithm SP (Data)
 Algorithm accuracy comparison.
 SP (Data)
 EP (Data)
 EP (Data)
 MP (Data)
 MP (Data)
 Accuracy (%)
 Accuracy (%)
 ACD ACD 132 132 31 31 22 22 71.35
 71.35
 Algorithm SP (Data) EP (Data) MP (Data) Accuracy (%)
 S &SC& C [22]
 [22]
 ACD 154 154
 132
 54
 54 31 0
 0 22 74.08
 74.08
 71.35
 ASM ASM
 S& S & C 154 9 0 95.06
 S& CC[22] 154 154 9 54 0 0 95.06
 74.08
 ASM S & C 154 9 0 95.06
Electronics 2022, 11, x FOR PEER REVIEW 11 of 17
 Electronics 2022, 11, x FOR PEER REVIEW 11 of 17
 Electronics 2022, 11, 178 11 of 17

 4. Experimental Results
 4.Electromyography
 4.1. Experimental Results
 4. Experimental Results
 4.1.In
 4.1. Electromyography
 muscle contractions, the signal generated by the potential difference between both
 Electromyography
 ends isIn called
 In musclethecontractions,
 muscle EMG signal.the
 contractions, Regarding
 the the non-invasive
 signalgenerated
 signal generated bythe
 by measurement
 thepotential
 potential method
 difference
 difference usedboth
 between
 between in
 both
 this paper,
 ends is disposable
 called the EMGelectrodes
 signal. were attached
 Regarding the to the skin
 non-invasive to observe
 measurement
 ends is called the EMG signal. Regarding the non-invasive measurement method used in muscle activities.
 method used in
 The
 thisexperiment
 thispaper, data
 paper,disposable were
 disposable the muscle
 electrodes
 electrodes states
 were
 were of
 attached
 attached making
 to the
 to the a fist
 skinskin once per second
 to observe
 to observe muscle within
 muscle one
 activities.
 activities. The
 minute,
 experimentand the
 The experiment dataexperiment
 data the
 were were platform
 the muscle
 muscle was
 states the of
 states
 of makingdesigned
 making sensing
 a fist oncea fist endper
 peronce
 secondof within
 this system.
 second within
 one Theone
 minute,
 disposable
 minute, electrodes
 and the were attached
 experiment to
 platform the flexor
 was thedigitorum
 designed superficialis
 sensing end
 and the experiment platform was the designed sensing end of this system. The disposable muscle
 of this (FDS)
 system. andThe
 the flexor digitorum
 disposable
 electrodes were profundus
 electrodes
 attachedwere
 to themuscle
 attached (FDP),
 flexortodigitorum as shown
 the flexor digitorumin Figure
 superficialis 11. (FDS)
 superficialis
 muscle muscle (FDS)
 and the and
 flexor
 the flexor digitorum profundus muscle (FDP),
 digitorum profundus muscle (FDP), as shown in Figure 11. as shown in Figure 11.

 Figure 11. Electrode position stereogram.
 Figure11.
 Figure 11.Electrode
 Electrodeposition
 positionstereogram.
 stereogram.
 The TriAnswer platform was strapped to the user’s wrist to capture EMG signals via
 a patch. The
 The
 The capturedplatform
 TriAnswer
 TriAnswer EMG
 platformsignals
 was were
 wasstrapped transmitted
 strapped toto the
 the to the
 user’s
 user’s cloud
 wrist
 wrist totovia the EMG
 capture
 capture Bluetooth
 EMG inter-
 signals
 signals viavia
 a
 face and
 a patch.
 patch. displayed
 TheThe
 capturedin
 captureda smartphone
 EMG EMG signals
 signals app.
 werewere transmitted
 transmitted to the
 to the cloudcloud
 via via
 the the Bluetooth
 Bluetooth inter-
 interface
 face
 and The
 andoriginalinsignal
 displayed
 displayed in aand the app.
 smartphone
 a smartphone filtered
 app.signal are shown in Figure 12. Apparently, the
 curve Thewas smoothened
 Theoriginal
 originalsignal
 signalafter
 andfiltering.
 and thefiltered
 the In Figure
 filtered 12,
 signalare
 signal EMG
 areshown
 shown signals are 12.
 inFigure
 in Figure obviously cleaner
 12. Apparently,
 Apparently, the
 the
 after
 curvefiltration,
 curve was which could
 wassmoothened
 smoothened afterimprove
 after the
 Indetection
 filtering.
 filtering. 12,ability
 In Figure
 Figure 12, EMG
 EMG of the ASM
 signals
 signals S & obviously
 are C algorithm.
 are obviously cleaner cleaner
 after
 after filtration,
 filtration, whichwhich
 could could
 improve improve the detection
 the detection abilityability
 of the of
 ASMtheSASM&CS & C algorithm.
 algorithm.
 Voltage (Vp-p)

 3 Raw data
 Voltage (Vp-p)

 Raw data
 2 3

 1 2

 0 1
 0 2 4 6 8 10 12 14 16 18 20
 0
 0 2 4 6 8 Time(sec)
 10 12 14 16 18 20
 Time(sec)
 Voltage (Vp-p)

 3 After filter
 Voltage (Vp-p)

 After filter
 2 3

 1 2

 0 1
 0 2 4 6 8 10 12 14 16 18 20
 0
 0 2 4 6 8 Time(sec)
 10 12 14 16 18 20
 Time(sec)
 Figure
 Figure12.
 12.EMG
 EMGfiltering
 filteringcross
 crossreference.
 reference.
 Figure 12. EMG filtering cross reference.
 Figure 13 shows the spectrum analysis of Figure 12. Figure 13a presents the spectrum
 distribution of the EMG signals captured. Figure 13b displays the spectrum distribution of
Electronics 2022, 11, x FOR PEER REVIEW 12 of 17
Electronics 2022, 11, x FOR PEER REVIEW 12 of 17

Electronics 2022, 11, 178 12 of 17
 Figure 13
 Figure 13 shows
 shows the
 the spectrum
 spectrum analysis
 analysis of
 of Figure
 Figure 12.
 12. Figure
 Figure 13a
 13a presents
 presents the
 the spectrum
 spectrum
 distribution of the EMG signals captured. Figure 13b displays the spectrum distribution
 distribution of the EMG signals captured. Figure 13b displays the spectrum distribution
 of the
 of the EMG
 EMG signals
 signals after
 after moving
 moving average.
 average. As
 As seen,
 seen, there
 there are
 are obviously
 obviously fewer
 fewer noises
 noises in
 in the
 the
 the EMG signals
 spectrum after below
 distribution moving60 average.
 Hz after As seen,
 the movingthere are obviously fewer noises in the
 average.
 spectrum distribution below 60 Hz after the moving average.
 spectrum distribution below 60 Hz after the moving average.

 Amplitude
 Amplitude
 Amplitude
 Amplitude

 Figure 13.
 Figure 13. Spectrum analysis
 analysis of the
 the EMG signals.
 signals.
 Figure 13. Spectrum
 Spectrum analysis of
 of the EMG
 EMG signals. (a) presents the spectrum distribution of the EMG
 signals captured. (b) displays the spectrum distribution of the EMG signals after moving average
 The proposed
 The proposed system
 system controls
 controls the the actions
 actions ofof the
 the VR
 VR games
 games by by making
 making aa fist.fist. When
 When an an
 EMG
 EMGThe peak
 peak is detected,
 is detected,
 proposed systemthe output
 the output is
 controlsisthe 1; otherwise,
 1; otherwise,
 actions of thethe
 theVRpeak
 peak detection
 detection
 games is 0. The
 is 0. The
 by making experiment
 experiment
 a fist. When an
 data
 EMGare
 data are
 peaktheissame
 the same as the
 detected,
 as thethe
 data for the
 output
 data for the
 is 1;moving average.
 otherwise,
 moving The detection
 the peak
 average. The ASM SS &
 ASM & isC 0.
 C algorithm is used
 The experiment
 algorithm is used
 to
 to determine
 data
 determine the
 are the same initial_value first,
 as the data first,
 the initial_value for the which
 moving
 which is then used
 average.
 is then used toto
 Thefind
 ASM
 find the initial_slope_maxi,
 theSinitial_slope_maxi,
 & C algorithm is as as
 usedwell
 wellto
 as
 as the
 determineslope_threshold.
 the slope_threshold. Whenever
 the initial_value two
 first, which
 Whenever consecutive
 is then usedEMG
 two consecutive EMG
 to findslopes exceed the slope_threshold,
 the initial_slope_maxi,
 slopes as well as
 exceed the slope_threshold,
 the
 the peak can be
 be identified
 slope_threshold.
 the peak can identified
 Whenever and the
 and the maxi
 twomaxi can be
 consecutive
 can be updated.
 updated.
 EMG slopesThe peak
 The peak
 exceedpulse is generated based
 slope_threshold,
 the is
 pulse generated based
 the
 on
 onpeakthe peak,
 thecan and the
 be identified
 peak, and the VR VRandgame is
 the maxi
 game controlled by
 can be updated.
 is controlled the peak pulse.
 The pulse.
 by the peak Figure
 peak pulse
 Figure 14 shows
 is generated
 14 shows the the
 basedASM
 ASM on
 SSthe
 & peak,
 & C algorithm
 C algorithm detecting
 and thedetecting
 VR gamethe the EMG signal
 is controlled
 EMG signal and
 by and generating
 the peak the peak
 pulse. the
 generating peak
 Figure pulse.
 14pulse.
 showsThe The blue line
 the blue
 ASM line is
 S & isC
 the EMG
 algorithm value, the
 detecting pink
 the dotted
 EMG line
 signal is the
 and initial value
 generating of
 the the EMG,
 peak and
 pulse.
 the EMG value, the pink dotted line is the initial value of the EMG, and the red dotted line the
 The red
 blue dotted
 line isline
 the
 is
 is the peak
 EMG
 the peak pulse
 value,pulse generated
 the pink after
 dottedafter
 generated line isthethe
 the peak
 peak is detected.
 initial
 is detected. TheEMG,
 value of the
 The peak pulse
 peak pulse
 and the is used
 is useddotted
 red for control-
 for control-
 line is
 ling
 the the
 peak actions
 pulse of the
 generated VR
 ling the actions of the VR figure. figure.
 after the peak is detected. The peak pulse is used for controlling
 the actions of the VR figure.

 EMG value
 3.5 EMG value
 3.5 initial value
 initial value
 peak pulse
 peak pulse
 3
 3

 2.5
 (Vp-p)

 2.5
 Voltage(Vp-p)

 2
 2
 Voltage

 1.5
 1.5

 1
 1

 0.5
 0.5

 0
 00 2 4 6 8 10 12 14 16 18 20
 0 2 4 6 8 10 12 14 16 18 20
 Time(s)
 Time(s)
 Figure 14.
 Figure 14.
 Figure Peak pulse
 Peak pulse
 14. Peak generated
 pulse generated with
 generated with the
 with the EMG
 the EMG signal.
 EMG signal.
 signal.

 4.2. Virtual Reality
 VR can make the users feel like they are at the scene, making it feasible for training and
 fitness uses. In this study, VR was combined with health care to make health management
 more interesting. This system, which employs an HTC VIVE VR headset, differs from other
 systems that it uses EMG for game control. As shown in Figure 15, the serial port needs
 to be selected to connect with the control terminal before the game could start. When the
4.2. Virtual Reality
 VR can make the users feel like they are at the scene, making it feasible for training
 and fitness uses. In this study, VR was combined with health care to make health man-
Electronics 2022, 11, 178 agement more interesting. This system, which employs an HTC VIVE VR headset, 13 differs
 of 17
 from other systems that it uses EMG for game control. As shown in Figure 15, the serial
 port needs to be selected to connect with the control terminal before the game could start.
 Whenstarts,
 game the game starts, theisdirection
 the direction is by
 controlled controlled by the
 the left and lefthands
 right and right hands
 to evade to evadeand
 obstacles, ob-
 stacles, and the speed increases as the game progresses. The game ends when
 the speed increases as the game progresses. The game ends when the player collides with the player
 collides
 an with
 obstacle. Theanfront-viewing
 obstacle. The angle
 front-viewing angle
 displays the displays
 current timetheandcurrent
 the besttime and
 time. Thethe best
 game
 time. The
 picture game picture
 is shown in Figureis shown
 16. in Figure 16.

 Figure15.
 Figure 15.Game
 Gamesetting.
 setting.

 Figure16.
 Figure 16.Game
 Gamepicture.
 picture.

 Figure17
 Figure 17isisthe
 theVR VRsystem
 systemflowchart.
 flowchart.The TheBluetooth
 BluetoothCOMCOMportportneeds
 needsto
 tobe
 beconnected
 connected
 atthe
 at thebeginning
 beginning of of the
 the game.
 game. When
 When the the game
 game starts
 starts running,
 running, the
 the EMG
 EMG state
 state is
 is detected.
 detected.
 Whenthe
 When theleft-hand
 left-handEMG EMGsignal
 signaltrigger
 triggerisisdetected,
 detected,the
 thegame
 game figure
 figurewould
 wouldwalk
 walk toto the
 the left;
 left;
 whenthe
 when theright-hand
 right-handEMG EMG signal
 signal trigger
 trigger is
 is detected,
 detected, the
 the figure
 figure would
 would dodge
 dodge toto the
 the right.
 right.
 Thescreen
 The screenisisupdated
 updatedperiodically.
 periodically.When
 Whenthethegame
 gamefigure
 figureencounters
 encountersananobstacle,
 obstacle,whether
 whether
 the playtime exceeds the record is determined; if yes, the best time would be updated and
 displayed; if not, the best time would remain and the game would end.
Electronics 2022, 11, x FOR PEER REVIEW 14 of 17

Electronics 2022, 11, 178 14 of 17
 the playtime exceeds the record is determined; if yes, the best time would be updated and
 displayed; if not, the best time would remain and the game would end.

 No

 No No
 Update Print Time

 Yes Collision Yes
 Start Comport Instantiate Detection
 Time END

 Yes Save and
 Time
 Print Time

 Figure 17. VR system flowchart.
 Figure 17. VR system flowchart.

 4.3. The
 4.3. App
 The App
 The proposed
 The proposed system
 systememploys
 employsMITMITApp AppInventor
 Inventor to to
 design
 designa mobile
 a mobile appapp
 for conven-
 for con-
 ient observation
 venient observation andandrecording of the
 recording user.user.
 of the The The
 app app
 communicates
 communicates withwiththe sensing end
 the sensing
 via Bluetooth,
 end and is
 via Bluetooth, andworkable without
 is workable beingbeing
 without connected to a VR
 connected to headset. The start
 a VR headset. Themenu
 start
 displays
 menu the connection,
 displays settings,
 the connection, and related
 settings, app information
 and related (see Figure
 app information 18A). The
 (see Figure user’s
 18A). The
 name, name,
 user’s gender,gender,
 and age areage
 and shown on the on
 are shown connection page. The
 the connection user
 page. Thecould
 userchoose to con-
 could choose
 nect
 to the smartphone
 connect the smartphoneto either
 to both
 eitherhands
 both or one or
 hands hand.
 oneWhen
 hand.the Whencorresponding sensing
 the corresponding
 end is connected,
 sensing the current
 end is connected, EMG diagram
 the current EMG diagramis displayed instantly,
 is displayed so that
 instantly, the users
 so that can
 the users
 check
 can theirtheir
 check muscle conditions.
 muscle Figure
 conditions. 18B shows
 Figure the app’s
 18B shows state state
 the app’s after the
 afterright-hand sens-
 the right-hand
 sensing
 ing end isend is connected.
 connected. Thecould
 The data data could
 be savedbe saved
 duringduring the connection,
 the connection, and theand EMGthesignals
 EMG
 signals
 could be could be automatically
 automatically saved saved
 in the in the micro
 micro database
 database by clicking
 by clicking the save
 the save button.
 button. On
 On the
 the
 otherother
 hand,hand,
 datadata saving
 saving couldcould be stopped
 be stopped immediately
 immediately by clicking
 by clicking the the
 stopstop button
 button or
 or the
 the “Connection Off” button. The previously-accessed EMG
 “Connection Off” button. The previously-accessed EMG data could be queried ondata could be queried on the
 “View
 “View Data”
 Data” page.
 page. Figure
 Figure 18C
 18C shows
 shows thethe EMG
 EMG data
 data found
 found using
 using “View
 “View Data”.
 Data”. The user
 information could be modified in the setting interface (Figure 18D). In addition to EMG
 signals, the app could collect biomedical data (ECG, EEG, and blood oxygen), making it
 highly
 highly suitable
 suitable for
 for long-term
 long-term collection
 collection and
 and observation.
 observation.
 On the menu displayed when the users open the app, the main operation includes
 the “Connect” and “Set” options. “View Data” or “Scan” could be selected in the connect
 setting. “View Data” is for EMG signal record queries, by which the saved EMG signal
 data could be observed. “Scan” is to view the list of nearby Bluetooth devices, through
 which the wearable system with intelligent and wireless features could be selected and
 the corresponding Bluetooth connection could be established. The current physiological
 condition could be observed after the connection is verified, the data could be saved
 by clicking “Save” while in the connected state, and the Bluetooth connection could be
 disconnected by clicking “Disconnect”. The user data could be input in the Set part, and
 “Enter” could then be chosen to complete the setting, or the user could choose “Clear” to
 clear the user data.
 The operation response time of this system is shown in Table 4. There are 320 pixels in
 one EMG image and it takes 0.1 s for one pixel, so totaling 30 s to draw the whole EMG
 image and the sample rate of the hardware capturing EMG signals is 300 Hz.
 Table 4. The system operation response time.

 Event Time
 EMG image update 30 (s)
 Hardware sampling 3.3 (ms)

 (A) (B)
could be automatically saved in the micro database by clicking the save button. On the
 other hand, data saving could be stopped immediately by clicking the stop button or the
 “Connection Off” button. The previously-accessed EMG data could be queried on the
 “View Data” page. Figure 18C shows the EMG data found using “View Data”. The user
 information could be modified in the setting interface (Figure 18D). In addition to EMG
 Electronics 2022, 11, 178 signals, the app could collect biomedical data (ECG, EEG, and blood oxygen), making it 15 of 17

 highly suitable for long-term collection and observation.

Electronics 2022, 11, x FOR PEER REVIEW 15 of 17

 (A) (B)

 (C) (D)

 Figure 18. Mobile
 Figure 18. app usage
 Mobile apppictures. (A) Home(A)
 usage pictures. page, (B) page,
 Home Connection interface, (C)
 (B) Connection Reading
 interface, (C)the
 Reading the
 EMG data,
 EMG(D) User(D)
 data, data settings.
 User data settings.

 On5.the
 Conclusions
 menu displayed when the users open the app, the main operation includes
 the “Connect” and
 This “Set”
 study options. an
 developed “View Data” wireless
 intelligent or “Scan” could besystem
 wearable selected in the connect
 in combination with IoT
 setting.and
 “View Data” is forgames
 VR interactive EMGinsignal
 orderrecord queries,
 to allow bymonitor
 users to which the
 theirsaved EMG signal
 physiological conditions
 data could be observed.
 anytime “Scan” is and
 through wearable to view the list ofThis
 IoT devices. nearby Bluetooth
 system combined devices,
 EMGthrough
 with VR, so as
 to enhance users’ enjoyment during exercise and rehabilitation. The EMG
 which the wearable system with intelligent and wireless features could be selected and peak detection
 accuracy of the
 the corresponding ASM Sconnection
 Bluetooth & C algorithm
 could proposed in this paper
 be established. was higher
 The current than that of the
 physiological
 S &could
 condition C andbeACD algorithms
 observed by connection
 after the 20.98% andis23.71%, respectively.
 verified, A mobile
 the data could appby
 be saved was also
 clicking “Save” while in the connected state, and the Bluetooth connection could be dis-
 connected by clicking “Disconnect”. The user data could be input in the Set part, and “En-
 ter” could then be chosen to complete the setting, or the user could choose “Clear” to clear
 the user data.
You can also read