Non-Contact Respiration Measurement during Exercise Tolerance Test by Using Kinect Sensor - MDPI
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
sports
Article
Non-Contact Respiration Measurement during
Exercise Tolerance Test by Using Kinect Sensor
Hirooki Aoki 1, * and Hidetoshi Nakamura 2
1 Chitose Institute of Technology, Chitose 066-8655, Japan
2 Department of Respiratory Medicine, Saitama Medical University, Iruma-gun 350-0495, Japan;
hnakamur@saitama-med.ac.jp
* Correspondence: h-aokil@photon.chitose.ac.jp; Tel.: +81-123-25-1005
Received: 1 January 2018; Accepted: 7 March 2018; Published: 13 March 2018
Abstract: This study aims to assess non-contact respiration measurement during the exercise stress
test using an upright bicycle ergometer and to evaluate the ventilation threshold value. We propose
the tracking of the chest and abdomen by applying the motion capture function of the Kinect V2
sensor to cope with an increase in physical exercise accompanied by an increase in exercise intensity.
In the proposed method, the region enclosed by the four joints corresponding to the left and right
shoulders and the right and left hip extracted using the Kinect sensor is set as the region of interest.
The region is updated in response to changes in body movements. By extracting the signal of
the pedaling frequency component from the time series data of the volume in the region, only
the volume change due to respiration was extracted. The point at which the increased rate of the
volume change elevates is estimated as the ventilation threshold. The assessment of the efficacy
of the proposed method by comparative analysis using an expiration gas analyzer confirmed that
non-contact respiration evaluation is possible with an exercise intensity of about 160 W. Furthermore,
the ventilation threshold estimated by the proposed method is ±10 W of the estimated value by
expiratory gas analyzer.
Keywords: non-contact respiration measurement; motion capture; Kinect sensor
1. Introduction
In rehabilitation institutes, the anaerobic threshold is typically evaluated by the exercise tolerance
test to assess the systemic aerobic capacity [1]. Although an expiration gas analyzer is conventionally
used for the assessment of the anaerobic threshold (AT), it is expensive. In addition, convenient
assessment is difficult while using the expiration gas analyzer, because it requires mounting a gas mask
on the face of participants, which poses significant problems, particularly in children and the elderly
who usually have dfficulty wearing a mask. Thus, the development of a simple and cost-effective
method to assess the AT is imperative for performing safe and effective exercise.
The VT, an equivalent index of the AT, is evaluated from the change in minute ventilation (VE)
obtained by the ramp load test using a bicycle ergometer [2]. The generation of anaerobic energy
stimulates the production of the lactic acid, which when buffered with bicarbonate ion increases the
ventilation volume by producing carbon dioxide [3]. Caiozzeo [4] demonstrated that the VT could
be evaluated not only from the increased amount of carbon dioxide in expiration but also from the
increased VE in the ramp load test.
Previously, we investigated a non-contact measurement with respiration in the rest position by
using pattern light projection [5], which elucidated that the volume change in the chest significantly
correlated with the change in ventilation. In addition, this method established non-contact respiration
measurement during pedaling with a recumbent bicycle ergometer [2]. The application of Caiozzeo’s
VT calculation by VE elucidated that the VT could be calculated even by the non-contact respiration
Sports 2018, 6, 23; doi:10.3390/sports6010023 www.mdpi.com/journal/sports
Sports 2018, 6, 23 2 of 11
measurement [6]. In this method, the increase in the volume change because of the increase in the
exercise intensity in the ramp load test is evaluated in the non-contact measurement, and the point
at which the volume change rate increases is estimated as the VT, also referred as QVT. The result of
this method suggests the feasibility of the systemic aerobic capacity assessment using the non-contact
respiration measurement.
In our previous study mentioned above, although we used a recumbent bicycle ergometer, upright
bicycle ergometers are more prevalent [7]. Hence, improving the versatility of the proposed method
necessitates non-contact respiration measurement using an upright bicycle ergometer. In upright
bicycle ergometers, the body motion by pedaling is higher than those in the recumbent types.
However, non-contact respiration measurement by pedaling using an upright bicycle ergometer
is considered challenging. Hence, one study proposed a method to facilitate stable non-contact
respiration measurement by reducing the impact of the pedaling motion [8].
In our previous studies, we used a sensor based on the active stereo method, called fiber-grating
vision sensor (FG sensor), which was developed independently by us and is not sold in general [9].
In 2010, Microsoft launched the Kinect sensor as a non-contact controller of the game console.
The Kinect sensor is a three-dimensional vision sensor based on the active stereo in which the pattern
light—called ‘light coding’—is projected onto an object, and the three-dimensional shape of the object
is reconstructed by image analysis of the pattern light distribution. The Kinect sensor is cost-effective
and it facilitates real-time processing of three-dimensional data measured by our own program because
it supports USB connection to a PC. For this reason, Kinect sensors are used in a wide range of research
fields for various purposes.
In another study of ours, we investigated the respiration measurement using the Kinect sensor
as an alternative to the FG sensor in the ramp load test. The findings revealed that the quantitative
respiration measurement in the resting posture is possible with the Kinect but not the FG sensor [10].
As mentioned above, since the change in the posture by pedaling is significant in the test using
the upright bicycle ergometer, reducing the impact of the body movement on the assessment of
signal is essential. Hence, we proposed a method to consider the part of the body (i.e., chest and
abdomen) where the respiratory motion was likely to appear as a region of interest (ROI). Consequently,
we illustrated that the accuracy of the quasi tidal volume (QTV) fluctuation can be enhanced by filtering
the three-dimensional shape change in the ROI [11]. However, even this method did not facilitate the
complete elimination of the effect of a large change in the posture and was, thus, considered difficult
to apply at a high exercise intensity of ≥100 W.
In 2014, a new version of the Kinect sensor (Kinect v2 sensor) was released, in which the
measurement method was revised from active stereo to time-of-flight (TOF). The principle of TOF
states that the distance is calculated from the time until the light projected from the light source
is reflected by an object and reaches a light-receiving sensor. Typically, light-receiving sensors are
distributed in a two-dimensional array and the distance distribution, in which the distance image can
be acquired. Apparently, data obtained from the distance image facilitates three-dimensional shape
reconstruction of the object. Although the measurement accuracy of the Kinect v2 sensor is almost
equal to that of the previous version, its motion capture function has been enhanced for its easy use.
By tracking the thoracoabdominal activity using the motion capture function of the Kinect
v2 sensor, this study aims to propose a non-contact respiration measurement during the exercise
tolerance test corresponding to an increase in the body movement with an increasing exercise intensity.
Additionally, it assesses the validity of the proposed method by comparative experiment using the
expiratory gas analyzer.
Sports 2018, 6, 23 3 of 11
Sports 2018, 6, x FOR PEER REVIEW 3 of 11
2. Materials and Methods
2.1. Configuration of
2.1. Configuration of the
the Measurement
Measurement System
System
Figure
Figure 1a
1a shows
shows thethe measurement
measurement systemsystem in in this
this study.
study. The
The Kinect
Kinect sensor
sensor (Kinect
(Kinect v2v2 sensor,
sensor,
Microsoft Corporation, Redmond, WA, USA) is placed facing the test subject
Microsoft Corporation, Redmond, WA, USA) is placed facing the test subject sitting on the saddle ofsitting on the saddle
of
thethe upright
upright bicycle
bicycle ergometer.
ergometer. The
The testsubject
test subjectisisasked
askedtotopedal
pedalthethebicycle
bicycle at
at aa constant
constant rotation
rotation
frequency, which was set at
frequency, which was set at 1 Hz. 1 Hz.
As
As described earlier, the
described earlier, the Kinect
Kinect v2v2 sensor can acquire
sensor can acquire the
the distance
distance image
image by by the
the TOF
TOF method,
method,
which
which isis more
morerecently
recentlyknown
known as as a depth
a depth image.
image. The Kinect
The Kinect v2 sensor
v2 sensor is connected
is connected to the
to the general-
general-purpose laptop PC with a USB port. Then, a sequence of depth image
purpose laptop PC with a USB port. Then, a sequence of depth image obtained by the Kinect v2 sensor obtained by the
Kinect v2 sensor is processed by the PC. The Kinect v2 sensor primarily comprises
is processed by the PC. The Kinect v2 sensor primarily comprises a color camera (RGB camera), an a color camera
(RGB camera),
infrared cameraan(IRinfrared
camera),camera
and (IR camera), projector
an infrared and an infrared projectoras(IR
(IR projector), projector),
shown as shown
in Figure in
1b [12].
Figure 1b [12]. Table 1 shows the specification of the Kinect. The field of view is 60 ◦ vertical and 70◦
Table 1 shows the specification of the Kinect. The field of view is 60° vertical and 70° horizontal. In
horizontal.
addition, theInresolution
addition,ofthetheresolution
RGB image ofobtained
the RGBby image obtained
the RGB camera bywas
the1080
RGBpcamera was 1080
(1920 × 1080 p
pixels).
(1920 × 1080the
In contrast, pixels). In contrast,
resolution the resolution
of the depth of the depth
image obtained by theimage
infraredobtained by the comprising
depth sensor, infrared depththe
sensor, comprising the IR camera and the IR projector, was 512 × 424 pixels.
IR camera and the IR projector, was 512 × 424 pixels. Notably, each pixel of the depth image Notably, eachhaspixel
13-
of
bitthe depth image
information, and has 13-bit resolution
the depth information, is 1and
mm; the
in depth resolution
the depth is 1 mm; in the
image measurement, depth
the limit image
depth
measurement,
range is 0.5–4.5them.limit depth range is 0.5–4.5 m.
(a) (b)
Figure
Figure 1.
1. (a)
(a) System
System configuration
configuration and
and (b)
(b) Kinect v2 sensor.
Kinect v2 sensor.
Table 1.
Table 1. Specifications
Specifications of
of the
the Kinect
Kinect v2
v2 sensor.
sensor
Item Specification
Item Specification
Resolution of RGB camera 1920 × 1080 pixel
Resolution of RGB camera
Frame rate of RGB camera 1920 ×30 1080
fpspixel
Frame rate of RGB camera 30 fps
Resolution of IR camera 512 × 422 pixel
Resolution of IR camera 512 × 422 pixel
Frame rate of rate
Frame IR camera
of IR camera 30fps
30 fps
Depth range
Depth range 0.5–4.5mm
0.5–4.5
Field ofField
viewof view 70◦ (vertical),
70° (vertical), 60 ◦ (horizontal)
60° (horizontal)
The Kinect
The Kinectv2v2sensor
sensor is characterized
is characterized by aby a markerless
markerless motionmotion
capturecapture
functionfunction (also
(also called called
skeleton
skeleton tracking), which can estimate and track the coordinates of 25 joints of the human
tracking), which can estimate and track the coordinates of 25 joints of the human body (Figure 2). body
(Figure 2).
Sports 2018, 6, 23 4 of 11
Sports 2018, 6, x FOR PEER REVIEW 4 of 11
Sports 2018, 6, x FOR PEER REVIEW 4 of 11
Figure 2. The 25 joints
The extracted by the markerless motion capture function ofofthe Kinect v2 sensor.
Figure
Figure 2.2.
The 2525 joints
joints extracted
extracted byby the
the markerless
markerless motioncapture
motion capturefunction
functionof theKinect
the Kinectv2
v2sensor.
sensor.
2.2. Non-Contact
2.2.Non-Contact
Respiration
Non-ContactRespiration
Measurement
RespirationMeasurement
Measurement
2.2.
In this study, we measured breathing of participants who pedaled on the upright bicycle
In this study,
In study,wewe
measured
measured breathing of participants
breathing who pedaled
of participants on the upright
who pedaled on the bicycle
uprightergometer
bicycle
ergometer without contact in four steps as explained below.
without contact
ergometer withoutin four steps
contact as explained
in four steps as below.
explained below.
2.2.1. Setting
2.2.1.Setting
Settingof of the
ofthe
theROIROI
ROI
2.2.1.
In the
Inthe measurement
themeasurement
measurementsystem, system,
system,thethe horizontal
thehorizontal distance
horizontaldistance between
distancebetween
betweenthe the Kinect
theKinect
Kinectv2v2 sensor
v2sensor
sensorandand
andthethe saddle
thesaddle
saddle
In
tip
tip of the
of thebicycle
bicycleergometer
ergometer waswas
0.70.7
m. m.
In In the
the depth depth
image image acquired
acquired by theby the Kinect
Kinect v2 v2 sensor,
sensor, we imagedwe
tip of the bicycle ergometer was 0.7 m. In the depth image acquired by the Kinect v2 sensor, we
imaged
the upper the upper body of subjects (Figure 3a). Then, using the joint extraction function of the Kinect
imaged thebody
upper of body
subjects (Figure 3a).
of subjects Then,
(Figure 3a).using
Then,theusing
joint the
extraction function function
joint extraction of the Kinect v2Kinect
of the sensor,
v2
we sensor,
extracted we four
extracted
joints four
as joints asright
follows: follows: right shoulder,
shoulder, left left shoulder,
shoulder, right right
hip, and hip,
left and
hip. left
A hip. A
region
v2 sensor, we extracted four joints as follows: right shoulder, left shoulder, right hip, and left hip. A
region
surroundedsurrounded
by these by thesejoints
four was
joints was set as the ROI (Figure 3b). We anticipated changes in the
region surrounded by four
these four joints set as set
was the ROI
as the(Figure 3b). We
ROI (Figure 3b).anticipated changes
We anticipated in thein
changes body
the
body
surface surface
because because of respiration
of respiration becausebecause the
the body body
surfacesurface corresponding
corresponding to
to thetoROIthe ROI
was waswas the
the chest chest
wall.
body surface because of respiration because the body surface corresponding the ROI the chest
wall.
Since Since
the the coordinates
coordinates of the of the
four fourwere
joints joints wereby
shifted shifted
the bodyby movement,
the body movement,
the ROI was theupdated
ROI was in
wall. Since the coordinates of the four joints were shifted by the body movement, the ROI was
updated
every frame. in every frame.
By using theByBy using
depth the
informationdepth information from the ROI, we performed respiration
updated in every frame. using the depth from the ROI,from
information we performed
the ROI, we respiration
performed assessment
respiration by
assessment
the following byprocedure.
the following procedure.
assessment by the following procedure.
(a) (b)
(a) (b)
Figure 3. (a) Depth image and (b) setting of ROI (region of interest).
Figure
Figure3.3.(a)
(a)Depth
Depthimage
imageand
and(b)
(b)setting
settingof
ofROI
ROI(region
(regionof
ofinterest).
interest).Sports 2018, 6, 23 5 of 11
2.2.2.Sports
Volume2018, 6, x FOR PEER REVIEW
Calculation 5 of 11
BasedVolume
2.2.2. on the principle
Calculation of triangulation, the depth of each pixel of the depth image can be converted
to the three-dimensional coordinates of the vertex (X, Y, and Z) by the Equation (1)
Based on the principle of triangulation, the depth of each pixel of the depth image can be
converted to the three-dimensional coordinates of the vertex (X, Y, and Z) by the Equation (1)
( x p − ph /2) tan(θh /2)
X= z
( x −pp /2 / 2) tan(θ h / 2) p
X = p hh zp
( pv /2 − y p ) ptan h /(2θv /2) (1)
Y= zp
( p / 2 v− y p ) tan(θ v / 2)
p /2 (1)
Y= v zp
Z = zp pv / 2
Z = zp
where xp is the horizontal pixel coordinate on the depth image, yp is the vertical pixel coordinate on
whereimage,
the depth xp is thezphorizontal
is the depth pixel coordinate
value at a pixel oncoordinate
the depth image, (xp andypyis p ),the vertical
ph is pixel
the total coordinate
pixel numberon of
the depth
horizontal image,pzvp is the depth
direction, total pixelvaluenumber
at a pixel coordinate
of vertical (xp and θyhp),isphthe
direction, is the total pixel
horizontal number
angle of viewof
of thehorizontal
IR camera, direction,
and θ v ispvthe
is the total pixelangle
horizontal number of vertical
of view of the direction,
IR camera. θhWeis the horizontal
resampled theangle of view
coordinates
of the IR camera, and θ
of the vertices by applying the Delaunay triangulation with linear interpolation [13] and calculated the
v is the horizontal angle of view of the IR camera. We resampled the
volumecoordinates
of the bodyof the vertices
surface in bytheapplying
ROI. the Delaunay triangulation with linear interpolation [13] and
calculated the volume of the body surface in the ROI.
2.2.3. Extraction of the Respiration Component
2.2.3. Extraction of the Respiration Component
Next, we calculated the change in the volume in the ROI between frames. Changes in the volume
Next, we
were primarily calculated
caused by thethe pedal
changemotion
in the volume in the ROI
of subjects. between frames.
In addition, Changes
the volume in the volume
change slightly
were primarily caused by the pedal motion of subjects. In addition, the volume change slightly
contained the component of the motion of the chest–abdominal wall caused by respiration. In the
contained the component of the motion of the chest–abdominal wall caused by respiration. In the
exercise tolerance test, subjects were asked to pedal at a constant rotation frequency, which, in this
exercise tolerance test, subjects were asked to pedal at a constant rotation frequency, which, in this
study, was set at 1 Hz (60 rpm). Figure 4a shows the frequency spectrum of the volume change.
study, was set at 1 Hz (60 rpm). Figure 4a shows the frequency spectrum of the volume change. Two
Two peaks emerged at a 1-Hz neighborhood and at lower-frequency band than 1 Hz. The peak at
peaks emerged at a 1-Hz neighborhood and at lower-frequency band than 1 Hz. The peak at about 1
about 1 Hz is attributed to the pedaling motion of subjects. Here, we term the frequency reflected by
Hz is attributed to the pedaling motion of subjects. Here, we term the frequency reflected by the peak
the peak as ‘pedaling
as ‘pedaling motion motion frequency’.
frequency’. However, However, as theaspeak the peak at a lower
at a lower frequency
frequency band band
than 1than
Hz 1wasHz
was caused
caused by by the
therespiratory
respiratorymotion,motion,it itwas was termed
termed ‘respiration
‘respiration frequency’.
frequency’.
Figure Figure 4b shows that the respiration waveform was extractedby
4b shows that the respiration waveform was extracted byapplying
applying the the FFT
FFT bandpass
bandpass
filterfilter
to the waveform of the volume change between frames. In
to the waveform of the volume change between frames. In this study, the bandwidth of this study, the bandwidth of the
the FFT
FFT
bandpass
bandpassfilterfilter
waswasset between
set between 0.1 0.1
andand 0.70.7Hz. Hz.Notably,
Notably, it it
isisessential
essentialtotoapply
applythethe FFT
FFT filter after
after the
the
end ofend the
of measurement,
the measurement, andand it isitnot possible
is not possible to to
extract
extractthe thebreathing
breathingwaveform
waveform in in real time.
time. Hence,
Hence,
real-time processing
real-time processing of the
of thebreathing
breathing waveform
waveform extraction
extractionwas wasattained
attained by by applying
applying aa digital
digitalfilter.
filter.
The designed
The designed digital
digital filter
filter comprised
comprised a least-squares
a least-squares linear-phaseFIR
linear-phase FIRfilter
filterofofthe
the order
order 90. Figure
Figure 55
shows shows the characteristics
the characteristics of the
of the digital
digital filter.
filter.
↓ Component of respiration
↓ Component of pedaling motion
Spectrum intensity
0 1 2 3 4 5 6
Frequency [Hz]
(a) (b)
Figure 4. (a) Frequency spectrum of the measurement waveform and (b) extracted respiration waveform.
Figure 4. (a) Frequency spectrum of the measurement waveform and (b) extracted respiration waveform.Sports 2018, 6, x FOR PEER REVIEW 6 of 11
Sports 2018,6,6,23
Sports2018, x FOR PEER REVIEW 66ofof11
11
振幅応答 (dB) インパルス応答
振幅応答 (dB) インパルス応答
0
0.04
0
-10 0.04
-10 0.03
-20
[dB]
0.03
Amplitude
-20
[dB]
-30
Amplitude
0.02
振幅 (dB)
Amplitude
振幅
-30
0.02
振幅 (dB)
-40
Amplitude
振幅
-40 0.01
-50
0.01
-50
-60
0
-60
0
-70
-70 -0.01
0 2 4 6 8 10 12 14 0 0.5 1 1.5 2 2.5 3
周波数 (Hz) -0.01 時間 (秒)
0 2 4 Time
6 [s]8
周波数 (Hz)
10 12 14 0 0.5 1 Time [s]
1.5
時間 (秒)
2 2.5 3
Time [s] Time [s]
(a) (b)
(a) (b)
Figure 5. (a) Amplitude response and (b) impulse response of the designed digital filter.
Figure5.5.(a)
Figure (a)Amplitude
Amplituderesponse
responseand
and(b)
(b)impulse
impulseresponse
responseof
ofthe
thedesigned
designeddigital
digitalfilter.
filter.
2.2.4. Calculation of the QTV
2.2.4. Calculation of the QTV
2.2.4. Calculation of the QTV
The respiration waveform calculated above represents the volume change between frames of the
The respiration waveform calculated above represents the volume change between frames of the
ROI. The
In the extracted
respiration respiratory
waveform waveform,
calculated abovethe positivethe
represents and negative
volume are between
change reversed frames
between of
ROI. In the extracted respiratory waveform, the positive and negative are reversed between
exhalation
the ROI. Inandtheinhalation,
extracted thereby allowing
respiratory the separation
waveform, of exhalation
the positive and inspiration
and negative are reversedby detecting
between
exhalation and inhalation, thereby allowing the separation of exhalation and inspiration by detecting
the zero-crossing
exhalation of the waveform.
and inhalation, Integrating
thereby allowing the area ofofthe
the separation waveform
exhalation andofinspiration
one expiration or one
by detecting
the zero-crossing of the waveform. Integrating the area of the waveform of one expiration or one
inspiration
the facilitates
zero-crossing calculating
of the waveform.the Integrating
amount equivalent
the area to
of the
the TV, definedofasone
waveform QTV; the QTVor
expiration is one
not
inspiration facilitates calculating the amount equivalent to the TV, defined as QTV; the QTV is not
equal as thefacilitates
inspiration absolute amount of TV,
calculating the but the rate
amount of change
equivalent to in
thethe
TV,QTV withas
defined time
QTV;is equal to that
the QTV of
is not
equal as the absolute amount of TV, but the rate of change in the QTV with time is equal to that of
the TV.
equal as We
the defined
absoluteQTV multiplied
amount bythe
of TV, but breathing rate (number
rate of change of breaths
in the QTV per is
with time minute) asthat
equal to quasi VE
of the
the TV. We defined QTV multiplied by breathing rate (number of breaths per minute) as quasi VE
(QVE).
TV. We defined QTV multiplied by breathing rate (number of breaths per minute) as quasi VE (QVE).
(QVE).
2.3. Participants
2.3. Participants
2.3. Participants
The validity
The validity ofof the
the method
method explained
explained above
above waswas determined
determined through
through experiments
experimentsin inthis
thisstudy.
study.
We The validity of the method explained above was determined through tolerance
experiments in this study.
We conducted non-contact respiratory measurements during the exercise tolerance test on six male
conducted non-contact respiratory measurements during the exercise test on six male
We conducted non-contact
subjects respiratory measurements duringallthe exercise tolerance totest onasix male
subjects (Subject
(Subject A, A,B,B,C,C,D,
D,E,E,and
andF).F).
During
During thethe
assessment,
assessment, subjects
all were
subjects asked
were asked wear
to wear T-shirt
a T-
subjects
that (Subject A, B, C, D, E, and F). During the assessment, all subjects were asked to wear a T-
shirt remained
that remained in ininclose contact
in close with
contact their
with body.
their body.Wrinkles
Wrinklesandand twists
twistsofofthe
thematerial
materialcloth
cloth might
might
shirt
affect that remained
the measured in
measuredvalue; in close contact
value;however,
however,since with their
sincethere body.
thereare
are Wrinkles and twists of the material cloth might
affect the nono means
means to to validate
validate thisthis assumption,
assumption, we did
we did not
affect
not the
consider measured
its impactvalue; however,
in this since
experiment. there
Figureare no means
6 shows to validate
a subject this
pedaling assumption,
the bicycle we did not
ergometer.
consider its impact in this experiment. Figure 6 shows a subject pedaling the bicycle ergometer. This
consider its
This impact in thisbyexperiment. FigureEthical
6 shows a subject pedaling the bicycle (approval
ergometer.code:This
studystudy was
was approved approved the Institutional
by the Institutional Ethical Review Review
BoardBoard
of our of our university
university (approval code: 13-
study
13-044 was
(12 approved
September by the
2013)), Institutional
and all Ethical
experiments Review
were Board
conducted of
in our university
accordance with(approval
the code:
Declaration 13-
of
044 (12 September 2013)), and all experiments were conducted in accordance with the Declaration of
044 (12
Helsinki. September 2013)), and all experiments were conducted in accordance with the Declaration of
Helsinki. We Weobtained
obtained written informed
written consent
informed from allfrom
consent participants after explaining
all participants after the experimental
explaining the
Helsinki.
procedures, We obtained written informed consent from all participants after explaining the
experimentalrisks, and benefits.
procedures, risks, and benefits.
experimental procedures, risks, and benefits.
(a) (b)
(a) (b)
Figure 6. A subject pedaling an upright bicycle ergometer. (a) Side direction;
direction; (b) Back
Back direction.
Figure6.6.A
Figure Asubject
subjectpedaling
pedalingan
anupright
uprightbicycle
bicycleergometer.
ergometer.(a)
(a)Side
Side direction;(b)
(b) Backdirection.
direction.Sports 2018, 6, 23 7 of 11
Sports 2018, 6, x FOR PEER REVIEW 7 of 11
Procedures
2.4. Procedures
The validity
validity of
of the
theassessment
assessmentusing
usingananupright bicycle
upright ergometer
bicycle ergometer(75XLII; Konami
(75XLII; Sports
Konami LifeLife
Sports Co.,
Ltd., Ltd.,
Co., Tokyo, Japan)Japan)
Tokyo, was examined by comparing
was examined with thewith
by comparing simultaneous assessment
the simultaneous using an expiration
assessment using an
expiration
gas analyzergas(Aeromonitor
analyzer (Aeromonitor AE-280S;
AE-280S; Minato Minato
Medical Medical
Science Co.,Science Co., Ltd.,
Ltd., Tokyo, Tokyo,
Japan). TheJapan). The
ergometric
ergometric
intensity wasintensity was thenautomatically
then increased increased automatically
to 20 W/min to ramp
20 W/min
loadramp
from load from
0 W. All 0 W. All
subjects subjects
continued
continued
pedaling forpedaling for 8 min.ifHowever,
8 min. However, the fatigueifreached
the fatigue reached
the limit, the limit,
exercise was exercise was stopped
stopped halfway. halfway.
Furthermore,
Furthermore, the examinee
the examinee pedaling pedaling
rate was guidedrate
bywas guided by sound
a metronome a metronome
to keepsound
aroundto60keep around 60 rpm.
rpm.
3. Results
3. Results
Figure 77 shows
Figure shows the
the results
results of
of the
the simultaneous
simultaneous measurement
measurement as as follows:
follows: yellow
yellow squares,
squares, the
the QVE
QVE
obtained by
obtained bythetheproposed
proposedmethod;
method;blueblue triangles,
triangles, thethe
VEVE obtained
obtained by by
thethe expiration
expiration gas gas analyzer;
analyzer; the
horizontal axis, the normalized value of QVE/VE. With the minimum and the maximum as as
the horizontal axis, the normalized value of QVE/VE. With the minimum and the maximum a
a standard,
standard, we we calculated
calculated the
the normalizedvalue
normalized valuesosothat
thatthe
thevalue
valuechanged
changedfrom
from0 0toto1.1.
(a) (b)
(c) (d)
(e) (f)
Figure
Figure 7.
7. Results
Results of
of the
the simultaneous
simultaneous measurement
measurement withwith the
the proposed
proposed method
method and
and expiratory
expiratory gas
gas
analyzer. (a) Subject A; (b) Subject B; (c) Subject C; (d) Subject D; (e) Subject E; and (f) Subject F.
analyzer. (a) Subject A; (b) Subject B; (c) Subject C; (d) Subject D; (e) Subject E; and (f) Subject F.Sports 2018, 6, 23 8 of 11
Sports 2018, 6, x FOR PEER REVIEW 8 of 11
Among the six subjects, Subjects A, B, C, and D rode the bicycling ergometer up to an exercise
intensity of 160 W. W. In
In contrast,
contrast, Subjects
Subjects E and
and FF were
were unable
unable toto continue
continue pedaling
pedaling with the exercise
intensity around 140 140 WW because
because ofof fatigue
fatigue and
and stopped
stopped exercising
exercising at at 140
140 W.
W.
In all results, compared to the VE, the QVE was highly variable
all results, compared to the VE, the QVE was highly variable (Figure (Figure 7). However, we observed
7). However, we
that the VE and QVE changed with a similar tendency in each subject. Figure
observed that the VE and QVE changed with a similar tendency in each subject. Figure 8 shows 8 shows scatter diagrams
scatter
of the VE and
diagrams QVE.
of the Furthermore,
VE and the linear regression
QVE. Furthermore, analysis revealed
the linear regression analysisthat the correlation
revealed coefficient
that the correlation
was ≥ 0.79 in any of our subjects. For each subject, the bias and the 95% confidence interval
coefficient was ≥0.79 in any of our subjects. For each subject, the bias and the 95% confidence interval (95% CI) of
the
(95%difference
CI) of thebetween VEbetween
difference and QVE VEasand
well as VT
QVE estimated
as well as VT by the expiration
estimated gas analyzer
by the expiration gasand QVT
analyzer
estimated by the method
and QVT estimated by theproposed in our previous
method proposed in ourworks
previous[6] are
worksshown in Table
[6] are shown2. in Table 2.
1 1
0.8 0.8
Normalized VE
Normalized VE
0.6 0.6
0.4 0.4
R=0.8981 R=0.8361
pSports 2018, 6, 23 9 of 11
However, as the exercise intensity increased, we observed the tendency for the VE and QVE to
Sports 2018, 6, x FOR PEER REVIEW 9 of 11
diverge, which may warrant further investigation. For example, with Subject D, the VE and QVE
started
started diverging
diverging at at an
an exercise
exerciseintensity
intensityof of≥ 120 W.
≥120 W. AtAt this
this time,
time, Subject
Subject D D was
was in in aa posture
posture in in which
which
the
the left and right inclinations were significant (Figure 9a). Although the ROI was correctly set at
left and right inclinations were significant (Figure 9a). Although the ROI was correctly set at this
this
time,
time, the large left and right movements presumably hindered the detection of respiration, which is
the large left and right movements presumably hindered the detection of respiration, which is
moving in the depth direction. In addition, the ROI could not be set
moving in the depth direction. In addition, the ROI could not be set correctly because the subject wascorrectly because the subject was
too
too close
close toto the
the sensor,
sensor, which
which restricted
restricted the the precise
precise respiration
respiration measurement
measurement (Figure (Figure 9b).9b). This
This may
may
be
be attributed
attributed to to the
the fact
fact that
that distance
distance information
information is is not
not measured
measured because because the the upper
upper body body was was set
set at
at
aa position closer than the lower limit (50 cm) of the sensing range of the
position closer than the lower limit (50 cm) of the sensing range of the Kinect v2. Thus, further study Kinect v2. Thus, further study
is
is necessary
necessary to to elucidate
elucidate the conditions at
the conditions which the
at which the accuracy
accuracy of of non-contact
non-contact respiration
respiration measurement
measurement
deteriorates.
deteriorates. As As aa countermeasure,
countermeasure,two twoapproaches
approachesare areconceivable.
conceivable.The Thefirst
firstapproach
approachis istotosetseta
alonger
longerdistance
distancebetween
betweenthe thesubject
subjectand andthe thesensor.
sensor.Another
Anotherapproach
approach is is to
to place
place thethe backrest
backrest on on the
the
ergometer’s seat chair. When subjects pedal with their back in contact
ergometer’s seat chair. When subjects pedal with their back in contact with the backrest, it is possible with the backrest, it is possible
to
to prevent
prevent measurement
measurement from from being
being impossible
impossible due dueto toaaforward-tilting
forward-tiltingposture. posture.
The
The correlation coefficient between the VE and QVE ininthis
correlation coefficient between the VE and QVE this study
study was was ≥0.79
≥0.79 (Figure
(Figure 8). An8).
An investigation
investigation by by optoelectronic
optoelectronic plethysmography
plethysmography (OEP)is isa awell-known
(OEP) well-knownstudy study on on the
the correlation
correlation
between
between the volume change in the chest and exhalation flow during exercise. Apparently, OEP
the volume change in the chest and exhalation flow during exercise. Apparently, OEP is is an
an
application of motion capture using markers that enables strict volume
application of motion capture using markers that enables strict volume measurement than markerless measurement than markerless
motion
motion capture.
capture. A A previous
previous studystudy on on OEP
OEP revealed
revealed thatthat the
the correlation
correlation coefficient
coefficient with with spirometer
spirometer
was
was >0.75, which corroborates our results [14]. In our method, estimation of the thoracic region
>0.75, which corroborates our results [14]. In our method, estimation of the thoracic region was was
not
not strict;
strict; however,
however, we we obtained
obtainedaareasonable
reasonablecorrelation
correlationby bynoise
noisereduction
reductionprocessing.
processing.
In
In addition,
addition, aa difference
difference magnitude
magnitude of of±±1010 W W between
between the the QVT
QVT estimated
estimated by by the
the proposed
proposed
method and the VT estimated by the expiratory gas analyzer suggests
method and the VT estimated by the expiratory gas analyzer suggests that the proposed method that the proposed method might
allow
mightVT calculation
allow withoutwithout
VT calculation contact in the lamp
contact load
in the test using
lamp load test a general
using upright
a general bicycle
upright ergometer.
bicycle
As shown in Table 2, the difference between VE and QVE is rather
ergometer. As shown in Table 2, the difference between VE and QVE is rather large, but the trends large, but the trends of changes
in
of VE and QVE
changes in VEare andsimilar,
QVE are as reflected
similar, asbyreflected
the correlation coefficient.coefficient.
by the correlation Consequently, VT and QVT
Consequently, VT
may show close values. In this experiment, because the distribution
and QVT may show close values. In this experiment, because the distribution of the subjects’ VT was of the subjects’ VT was biased to
approximately 100 W, additional tests are required for subjects
biased to approximately 100 W, additional tests are required for subjects with more diverse VT. with more diverse VT. However, the
primary
However, objective
the primary of thisobjective
researchof was
thistoresearch
constructwas a method
to constructto conveniently
a method to calculate the VTcalculate
conveniently from an
engineering perspective, which has been significantly attained
the VT from an engineering perspective, which has been significantly attained in this study. in this study.
This
This study
study has has some
some limitations.
limitations. First,
First, the
the sample
sample size size of
of this
this study
study isis small,
small, andand all
all participants
participants
were
were males. Thus, it
males. Thus, it is
is necessary
necessary to to conduct
conduct extensive
extensive studies
studies targeting
targeting aa variety
variety of of subjects
subjects fromfrom
different age groups, genders, and athletic abilities. In addition,
different age groups, genders, and athletic abilities. In addition, the structural and statistical the structural and statistical
examination
examination is is required
required for formore
moresubjects.
subjects.From Fromthe theperspective
perspective ofof motion
motion analysis,
analysis, a comparison
a comparison by
by introducing OEP is considered to be effective for assessing the
introducing OEP is considered to be effective for assessing the validity and reliability of the proposed validity and reliability of the
proposed
method. method.
(a) (b)
Figure 9.
Figure 9. (a)
(a) Lateral
Lateral bending
bending position
position and
and (b)
(b) forward-bending
forward-bending position.
position.
5. Conclusions
By tracking the thoracoabdominal region using the motion capture function of the Kinect v2
sensor, we proposed a non-contact respiration measurement during the exercise tolerance test
corresponding to an increased body movement with increasing exercise intensity. In this study, weSports 2018, 6, 23 10 of 11
5. Conclusions
By tracking the thoracoabdominal region using the motion capture function of the Kinect v2 sensor,
we proposed a non-contact respiration measurement during the exercise tolerance test corresponding
to an increased body movement with increasing exercise intensity. In this study, we designed a digital
filter for bandpass filter processing in real time. As a result of investigating the correlation between the
QVE and the VE by simultaneous measurement using an expiration gas analyzer, a high correlation
coefficient up to about 160 W of the exercise intensity was obtained. The study clarifies that posture
during the pedaling motion affects the measurement accuracy of the proposed method. Nevertheless,
elucidation of the condition that deteriorates the precise measurement warrants further study.
QVT estimated from QVE showed a difference of ±10 W from VT calculated by expiratory gas
analyzer. In this experiment, the number of subjects, as well as difference in VT among these subjects,
was small. Thus, the examination of the correlation between QVT and VT was impossible. Therefore,
in this experiment, the feasibility of the proposed method could not be completely demonstrated.
However, QVT estimated by the proposed method shows a value close to VT. In addition, our VT
estimation method based on the knowledge of Caiozzeo shows a robust estimation accuracy in our
previous research. Taken together, these results suggest that VT can be estimated by non-contact
sensing for the unstable posture of the subject who pedals with an upright bicycle ergometer.
Acknowledgments: This work was supported by Grant-in-Aid for Scientific Research(C) from Japan Society for
the Promotion of Science, grant number JP17K09627.
Author Contributions: H.N. conceived, designed, and performed the experiments; H.A. developed the
measurement system, performed the experiments, and wrote the manuscript.
Conflicts of Interest: The authors declare no conflict of interest. The founding sponsors had no role in the design
of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the
decision to publish the results.
References
1. Wasserman, K.; McIloy, M. Detecting the threshold of anaerobic metabolism in cardiac patients during
exercise. Am. J. Cardiol. 1964, 14, 844–852. [CrossRef]
2. Robergs, R.; Roberts, S. Fundamental Principles of Exercise Physiology: For Fitness, Performance, and Health;
McGraw-Hill Higher Education: New York, NY, USA, 2000; pp. 176–177, ISBN 0801679079, 9780801679070.
3. Hirakoba, K.; Maruyama, A.; Inaki, M.; Misaka, K. Effect of Endurance Training on Excessive CO2 Expiration
due to Lactate Production in Exercise. Eur. J. Appl. Physiol. Occup. Physiol. 1992, 64, 73–77. [CrossRef]
[PubMed]
4. Caiozzo, V.J.; Davis, J.A.; Ellis, J.F.; Azus, J.L.; Vandagriff, R.; Prietto, C.A.; McMaster, W.C. A Comparison of
gas exchange indices used to detect the anaerobic threshold. J. Appl. Physiol. 1982, 53, 1184–1189. [CrossRef]
[PubMed]
5. Mizobe, Y.; Aoki, H.; Koshiji, K. Basic Study on Relationship between Respiratory Flow Volume and Volume
Change of Thorax Surface. In Proceedings of the 6th International Special Topic Conference on Information
Technology Applications in Biomedicine, Tokyo, Japan, 8–11 November 2007. [CrossRef]
6. Aoki, H.; Ichimura, S.; Kiyooka, S.; Koshiji, K. Calculation of Ventilation Threshold Using Noncontact
Respirometry. In Proceedings of the 30th Annual International Conference of the IEEE Engineering in
Medicine and Biology Society, Vancouver, BC, Canada, 20–25 August 2008; pp. 2273–2276. [CrossRef]
7. Aoki, H.; Ichimura, S.; Kiyooka, S.; Koshiji, K. Non-contact Measurement Method of Respiratory Movement
under Pedal Stroke Motion. In Proceedings of the 29th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society, Lyon, France, 22–26 August 2007; pp. 374–377. [CrossRef]
8. Aoki, H.; Sakaguchi, M.; Fujimoto, H.; Tsuzuki, K.; Nakamura, H. Noncontact Respiration Measurement
under Pedaling Motion with Upright Bicycle Ergometer Using Dot-matrix Pattern Light Projection.
In Proceedings of the IEEE Region 10 Conference, TENCON 2010, Fukuoka, Japan, 21–24 November 2010.
[CrossRef]Sports 2018, 6, 23 11 of 11
9. Nakazawa, K.; Nakajima, M.; Kobayashi, H. Development of 3-D shape measurement system using fiber
grating. Syst. Comput. Jpn. 1987, 18, 11–17. [CrossRef]
10. Aoki, H.; Miyazaki, M.; Nakamura, H.; Furukawa, R.; Sagawa, R.; Kawasaki, H. Non-contact respiration
measurement using structured light 3-D sensor. In Proceedings of the SICE Annual Conference (SICE), Akita,
Japan, 20–23 August 2012; pp. 614–618, ISBN 978-1-4673-2259-1.
11. Aoki, H.; Nakamura, H.; Fumoto, K.; Nakahara, K.; Teraoka, M. Basic Study on non-contact respiration
measurement during exercise tolerance test by using Kinect Sensor. In Proceedings of the 2015 IEEE/SICE
International Symposium on System Integration (SII), Nagoya, Japan, 11–13 December 2015. [CrossRef]
12. Microsoft Research: Human Interface Guidelines v2.0. 2014, pp. 6–8. Available online: http://download.
microsoft.com/download/6/7/6/676611B4-1982-47A4-A42E-4CF84E1095A8/KinectHIG.2.0.pdf (accessed
on 31 December 2017).
13. Lee, D.T.; Schachter, B.J. Two Algorithms for constructing a delaunay triangulation. Int. J. Comput. Inf. Sci.
1980, 9, 219–242. [CrossRef]
14. Parreira, V.F.; Vieira, D.S.; Myrrha, M.A.; Pessoa, I.M.; Lage, S.M.; Britto, R.R. Optoelectronic
plethysmography: A review of the literature. Rev. Bras. Fisioter. 2012, 16, 439–453. [CrossRef] [PubMed]
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).You can also read
