The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments - Niels Henze
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
The Effect of Offset Correction and Cursor on
Mid-Air Pointing in Real and Virtual Environments
Sven Mayer, Valentin Schwind, Robin Schweigert, Niels Henze
University of Stuttgart, Stuttgart, Germany, {firstname.lastname}@vis.uni-stuttgart.de
ABSTRACT Beyond today’s comment input devices, resent systems use the
Pointing at remote objects to direct others’ attention is a fun- whole body as an input. Here we see mid-air pointing as one
damental human ability. Previous work explored methods for emerging input technique, and others have also been developed
remote pointing to select targets. Absolute pointing techniques out of the early work by Bolt [10]. Plaumann et al. [53]
that cast a ray from the user to a target are affected by humans’ inspired their investigation of mid-air pointing through smart
limited pointing accuracy. Recent work suggests that accuracy environments such as smart home, while others facilitated
can be improved by compensating systematic offsets between mid-air pointing to interact with large high resolution displays
targets a user aims at and rays cast from the user to the target. (LHRDs) [38, 60]. Findings in the domain of LHRDs can also
In this paper, we investigate mid-air pointing in the real world be adopted to improve the interaction with public displays, and
and virtual reality. Through a pointing study, we model the other work such as Winkler et al. [61] used mid-air pointing
offsets to improve pointing accuracy and show that being in a to enrich the input space for a personal projector phone. Mid-
virtual environment affects how users point at targets. In the air pointing has been proposed as one possible interaction
second study, we validate the developed model and analyze the technique for virtual content; for instance Argelaguet et al. [3]
effect of compensating systematic offsets. We show that the used mid-air pointing in a CAVE environment, using pointing
provided model can significantly improve pointing accuracy as one collaborative tool to interact within a collaborative
when no cursor is provided. We further show that a cursor virtual environments [62]. Beyond simple mid-air pointing
improves pointing accuracy but also increases the selection actions, a vast number of research projects investigated mid-air
time. gesture sets e.g., [34, 37, 39].
ACM Classification Keywords
Already Bolt’s seminal work [10] demonstrated the potential
of mid-air pointing to select remote targets. A large body of
H.5.m. Information Interfaces and Presentation (e.g. HCI):
work investigated selecting remote physical and virtual targets.
Miscellaneous
Previous work proposed relative and absolute input devices to
Author Keywords enable remote pointing [7, 36, 42]. Early work was typically
Mid-air pointing; ray casting; modeling; offset correction; limited by the accuracy of the tracking technology. Absolute
cursor; virtual environment. ray casting techniques enable users to use the same pointing
gestures they use for communicating with other people but
INTRODUCTION require tracking a user’s hands or controllers with high preci-
From early childhood on, humans have used mid-air pointing sion. The recent revival of virtual reality (VR) has increased
to direct others’ attention [8]. Developing the skill to use the need for fast and precise methods to point at objects in
referential gestures has been described as a pivotal change in three dimensions. Current VR devices such as the HTC Vive
infants’ communicative competence and the foundation for and the Oculus Rift are delivered with controllers that enable
engaging in conversations [8, 11]. Consequently, pointing a user to select virtual objects.
plays an important role in human-computer interaction (HCI). Although pointing in three dimensions to communicate with
Today’s graphical user interfaces (GUIs) are essentially built other humans is a fundamental human skill, work in experi-
around the user’s ability to point at objects. Over the last mental Psychology shows that humans’ pointing accuracy is
decades, the effort went into building, evaluating, and refining limited [19]. Recent work not only describes systematic errors
pointing methods for GUIs to enable fast and precise input [57]. when humans point at distant objects but also provides a first
Today the input is mostly limited to mice, touchpads, and step towards modeling the error and compensating for syste-
touchscreens. matic inaccuracies [40]. Mayer et al. [40] asked participants to
Permission to make digital or hard copies of all or part of this work for personal or point at crosshairs on a projection screen, measured the accu-
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
racy of different ray casting methods and provided a model to
on the first page. Copyrights for components of this work owned by others than the compensate the systematic offset for real-world (RW) mid-air
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or pointing. While the work by Mayer et al. is promising and the
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org. authors conclude that they can improve pointing accuracy by
CHI 2018, April 21–26, 2018, Montreal, QC, Canada 37.3%, the achieved accuracy is too low for precise selection,
© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. and the model has not been validated. Furthermore, it remains
ISBN 978-1-4503-5620-6/18/04. . . $15.00
DOI: https://doi.org/10.1145/3173574.3174227
unclear if the model can be generalized to other contexts such Tracking Techniques
as virtual reality and how it compares to a cursor that likely The user needs to be tracked to enable interaction from a
also improves pointing accuracy. distance. Related work presents two approaches for tracking.
Either the user interacts with a controller or the user’s body is
In this paper, we investigate the possible use of freehand mid-
tracked by surrounding equipment.
air pointing in the real and virtual environment. Further, we
extend existing correction models and investigate the impact More and more computerized systems using a controller such
of visual feedback on humans’ pointing performance. The- as mice, keyboards or 3D input devices (e.g. Zhai et al. [63])
refore, we present two studies that investigate how humans as the primary interaction device are now hitting the consumer
point at targets in the real and virtual environment. In the market. In the domain of LHRDs most prototypes use control-
first study, participants pointed at targets inside and outside lers to overcome the distance between display and user [51].
VR. The results show that participants point differently while We see the same trend in the game console market. Here even
they are in VR. We argue that this is likely an effect caused body movement focused game consoles like the Nintendo Wii
by the VR glasses and the limited field of view. Using the [41], use a controller to recognize the body movement of the
collected data we developed models to compensate systematic player. Even the latest technical innovation of augmented rea-
offsets, which we validate in a second study. We show that the lity (AR) glasses, the Microsoft Hololense is shipped with a
developed models can significantly improve pointing accuracy. controller. Also VR glasses such as the Oculus Rift and the
We further show that a cursor can enhance mid-air pointing HTC Vive offer a controller for interaction with the VR scene.
accuracy but thereby increases the selection time. Third party technologies even provide the ability to track all
ten fingers using gloves.
RELATED WORK In contrast to controller and wearable systems passive systems
Previous work investigating mid-air pointing focused on the can deliver the same richness of interaction without equipping
influences of psychology and physiology on pointing gestures, the user. Nickel and Stiefelhagen [49] used RGB cameras
tracking techniques, mid-air ray cast techniques, offset com- with skin color tracking to approximate the pointing direction.
pensation, and limb visualization in VR. In the following, we While the LEAP Motion has been adopted to provide finger
discuss these topics. orientation to current VR glasses the overall detectable range
is still limited. The limited range is mostly due to stereo vision
reconstruction using two infrared cameras. To overcome the
Psychology and Physiology limited tracking possibilities most research prototypes simu-
It has been shown that children in early childhood begin to late a perfect tracking using six-degree-of-freedom (6DOF)
express themselves with pointing gestures [25]. Pointing is lin- technologies, also known as motion capture systems. These
ked to learning others’ intentions and has a substantial impact passive tracking systems have widely been used over the last
on developing a theory of mind [12] as well as in associating decade, for instance, by Kranstedt et al. [32] or Vogel and
verbal declarations [5]. Kendon [28] differentiates pointing Balakrishnan [59, 60].
gestures using the index finger, open hand, or thumb. While
thumb and the open hand are used when the object being
indicated is not primary focus or topic of the discourse, the
extended index finger is used when a specific person, object, Mir-Air Ray Casting Techniques
or location is meant [28]. Pointing requires a fine level of In the following, we present absolute mid-air pointing ray cas-
dexterity and motor control over intrinsic oscillations of the ting techniques [60]. Mid-air pointing ray casting techniques
own body (tremor) as a result of involuntary, approximately can further be classified by the origin of the ray. Argelaguet
rhythmic, and roughly sinusoidal movements [18]. Further- et al. [3] distinguish between eye-rooted and hand-rooted
more, both Christakos and Lal [13] and Riviere et al. [54] techniques.
concluded that the hands move at 8 to 12 Hz oscillations, and
Basmajian and De Luca [6] stated that the oscillation is less Two eye-rooted ray casting approaches are widely used; the
than 13 Hz. Further, Morrison and Keogh [47] conducted a eye orientation and the eye position as root of the ray. a) Using
frequency analysis for pointing with the hand and index finger the eye orientation as a ray cast is refered to as gaze ray cas-
and found dominant frequency peaks between 2 and 4 and bet- ting [49] and is implemented similar to pointing tasks using
ween 8 and 12 Hz. They also found that oscillations increased eye-tracking [35]. However, eye orientation ray casting re-
when participants attempted to reduce the tremor by exerting quires special equipment and extra eye calibration. To avoid
extra equipment and calibration, Nickel and Stiefelhagen [49]
greater control over the hand. Hand tremor was already descri-
proposed using the orientation of the head; we refer to this
bed as an issue for HCI in an interaction scenario by Olsen and
technique as head ray cast (HRC). b) On the other hand are ray
Nielsen [50] while using a laser pointer for selection tasks.
casting techniques which use the eye position as root of the
Ocular dominance is known to influence mid-air pointing [29]. ray. The most common technique eye-finger ray cast (EFRC),
Human ocular dominance can tested with, e.g., a test by Mi- was specified in 1997 by Pierce et al. [52]. However, today
les [43] and by Porta [16]. Plaumann et al. [53] confirmed EFRC, actually uses the “Cyclops Eye”, which is the position
these results using a high precision motion tracking system. between the eye, as root [32]. Kranstedt et al. [32] suggest
Further, they concluded that handedness also has an influence that EFRC is defined by using the cyclops eye as root and the
on how humans point to distant targets. index fingertip as the direction.
Hand-root methods use the hand as the origin for the ray [45, a gender-related difference and, for example, recommended
46]. Corradini and Cohen [15] identified index finger ray cast avoiding gender swapping in VR by using non-realistic or
(IFRC) as the most common hand-rooted method. On the other androgyny avatars. Furthermore, research comparing input
hand, Nickel and Stiefelhagen [49] purposed and investigated methods in VR and real world found that VR is still limited.
an elbow-rooted ray casting method. We refer to this method For example, Knierim et al. [30] compared the typing perfor-
as forearm ray cast (FRC). mance of users in the real and virtual world. Their results
show that the typing performance of users in the virtual world
Offset Compensation is limited and depends on their experience of the users.
Foley et al. [19] found a distance-dependent trend to overreach
targets using pointing with the index finger. This finding was Summary
confirmed by Mayer et al. [40]. In their work, the authors
A substantial body of research has investigated the selection
describe systematic errors of absolute pointing and present
of distant targets. Previous work has shown that interaction
a polynomial offset model for compensation. Akkil and Iso-
without a controller is hard to implement, however it has also
koski [1] conducted a study to compare different pointing
been shown that carrying no controller has its advantages. In
techniques including eye gaze for compensation. Their results
this paper, we focus only on absolute mid-air pointing without
indicate that overlaying gaze information on an egocentric
using a controller. Mayer et al. [40] presented a systematic
view increases the accuracy and confidence while pointing.
offset between the ray cast and the target for the RW. Howe-
On the other hand, Jota et al. [27] recommended using EFRC
ver, they have not tested how effective the model is in a real
to reduce the parallax influence.
selection task. Further, the model has not been applied to a
Visual Feedback real selection task, thus the impact on task completion time
Wong and Gutwin [62] investigated different ways to visualize (TCT) is unknown. Due to the rise of AR and VR availability
the pointing direction for VR. Their results suggest that a red it also would be interesting to see how the model performs in
line in the pointing direction is optimal for direction visualiza- different environments.
tion. However, this is hard to realize in the RW. As a second To address these open questions, we replicate the work by
option Wong and Gutwin [62] propose projecting a cursor on Mayer et al. and extend it by also determining offset models
the object a user interacts with. In their implementation they for VR. We then apply the models in a real selection task to
used a red dot as cursor visualization. In an LHRD scenario ensure the external validity of the developed models. Since
Jiang et al. [26] used a red circle to visualize the cursors’ posi- previous work did not apply and validate their model, we
tion on a large display. Both “dot” and “circle” visualization investigate how the model performs in RW and VR regarding
can be realized in the RW using camera projector systems offset and TCT. Further, as related work suggested using a
as provided by Benko et al. [9] and Gugenheimer et al. [23]. cursor for precise input, we investigate the effect of displaying
Kopper et al. [31] encoded the uncertainty of the position by a cursor and how a cursor affects offset and TCT.
mapping the amount of jitter to the circle size. Lastly, Nancel
et al. [48] as well as Olsen et al. [50] used a red crosshair
DATA COLLECTION STUDY
for their selection task. Furthermore, Cockburn et al. [14]
We conducted the first study to record labeled body postures
investigated the effect of selection targets at a distance with
while participants performed mid-air pointing gestures. Our
and without visual feedback. They found that visual feedback
goal was to compare RW and VR. Thus, participants were as-
improves selection accuracy. However, visual feedback might
ked to perform mid-air pointing gestures in both environments.
also influence the immersion in VR as Argelaguet and An-
Differences between the two environments would suggest that
dujar [2] showed that tracking technology, latency, and jitter
to correct the systematic error, separate models are needed.
influence the overall input performance.
As Mayer et al. [40] showed that angular models are sufficient
Limb Visualization for all pointing distances, we only investigate standing in 2 m
As related work suggests using a finger and the forearm to distance to the target. Further, as presenting feedback might
indicate directions, it is necessary to visualize the arm and change the users behavior we did not present any feedback
the hands to make mid-air pointing in VR feasible. Previous to the user to record natural mid-air pointing gestures. Mo-
work found that the brain is able to accept virtual limbs [17] reover, to build a precise model we needed to present targets
and bodies [56] as part of the own body. Rendering the own without an area. This is in line with Mayer et al. [40]. No
body in VR avoids fundamental limitations of human propri- target area means that the target becomes a single coordinate
oception as the brain encodes limb positions primarily using on the projection canvas. This allowed us to build a model
vision [21, 22]. However, the illusion of body-ownership is af- without possible misinterpretation by participants, as pointing
fected by the visual appearance of the avatar. For example, Lin on a target with an area might convey the message of pointing
and Jörg [33] found that human-like hand models increased onto the center or somewhere on the target area.
the illusion of body ownership and led to behavioral changes
compared to more abstract representations. Similar findings Study Design
were presented by Argelaguet et al. [4], who found that the We used a within-subject design with a single independent
appearance of avatars’ hands in VR influences the user’s sense variable (IV): E NVIRONMENT. The IV E NVIRONMENT has
of agency. However, the illusion of body ownership increa- two levels: RealWorld and VirtualReality. We replicated the
ses with human-like virtual hands. Schwind et al. [55] found setup of Mayer et al. [40], and also used 35 targets in a 7 × 5
These measurements also have been used to calculate the
perfect ray-casts for all 4 mid-air ray casting techniques:
Index finger ray cast (IFRC): Using the finger tip marker
plus a user-specific marker placement measurement we cal-
culate the true finger tip position. Additionally we used the
finger tip markers orientation to determine the direction of
the ray.
Head ray cast (HRC): We used the Cyclops Eye ray cast
as proposed by Kranstedt et al. [32]. Therefore, in the VR
Figure 1. One participant pointing at a target in RW.
condition, we used the HMDs maskers to calculate the posi-
tion of the bridge of the nose and its forward direction. On
the other hand, in the RW condition, we used a marker on
grid. Participants had to point 6 times on each target per con- the head of the participant plus head measurements to also
dition resulting in a total of 420 pointing tasks. The order of determine the bridge of the nose and the forward direction
the targets was randomized while the order of E NVIRONMENT of the head.
was counter-balanced.
Eye-finger ray cast (EFRC): The root for the ray cast, Cy-
Apparatus clops Eye calculated the same way for the HRC. The finger
As apparatus, we used a PC running Windows 10 connected tips position was optioned in the same way as for the IFRC
to a projector, a head-mounted display (HMD), and a marker- and used as the direction vector.
based 6DOF motion capture system namely an OptiTrack
system. As HMD we used an HTC Vive. To guarantee a Forearm ray cast (FRC): We calculated the center or the
smoothly running VR experience we used a NVIDIA GeForce forearm by approximating the forearm with a frustum of a
GTX 1080. The tracking system delivers the absolute position cone. This was achieved using the position and orientation
of the markers attached to the participant at 30 FPS. We cali- of the forearm marker plus additional measurements.
brated the system as suggested by the manufacturer resulting in The 35 presented targets were arranged in a 7 × 5 (column ×
millimeter accuracy. The software to interact with the tracking row) grid. The targets were either projected on a projection
system provides a full-body tracking by attaching a number screen (269.4 cm × 136.2 m) or presented in VR on the same
of markers. However, as the software is closed source and sized the virtual projection screen. The spacing of the target
approximates the position of body parts, especially the finger- grid was 44.9 cm × 34. cm.
tip, we did not use OptiTrack’s commercial full-body tracking
implementation. Instead, we used 7 rigid bodies to track the Both VR scene and RW projection were implemented using
body without any approximations. We tracked the head/HMD, Unity version 5.6. The projector mounted in the study room
both shoulders, the right upper arm, the right lower arm, the projected the targets rendered in the VR scene to avoid align-
hand root, and the index finger as shown in Figure 2. We used ments issues. We therefore designed the VR scene to replicate
markers with a diameter of 15.9 mm and 19. mm to ensure a the real room the participants were standing in, see Figure 3a.
stable tacking. We 3D printed custom mounts1 to determine To ensure a precise representation of the room in VR we used
the pose of the right arm, the hand, the index finger, and the a professional laser measurement tool (accuracy ±1.5 mm).
HTC Vive. As depicted in Figure 2 the index finger marker is We recreated the room in VR to avoid any interference on
wrapped around the finger and the upper as well as the forearm the pointing performance and to keep the results comparable.
marker are wrapped around the arm. As humans use their hand as a reference point for mid-air
pointing, it is important to represent them accurately in VR.
To perfectly represent the participant in VR we took the follo-
wing measurements: We took the precise length of the index
finger, hand, lower and upper arm and measured the diameter
of the finger, hand, wrist, elbow, lower arm, upper arm and
head. Further, we took measurements of both shoulder and eye
position in relation to the reflective shoulder markers. Lastly
we measured the position of the lower and upper arm markers
in relation to the elbow. The tracked positions of the marker
combined with these 14 participant specific measurements ena-
bled us to precisely determine the position and orientation of
the upper body, the arm, and the index finger. We adjusted the
avatar’s dimensions as well as the underlying bone structure
to precisely represent the participant’s real dimensions.
1 3D models of the custom mounts used in our study: github.com/ Figure 2. The seven rigid body markers used for body tracking in our
interactionlab/htc-vive-marker-mount study.
sessions each with 105 gestures. Between the sessions, we as-
ked them to fill a raw NASA-Task Load Index (raw TLX) [24]
to check for fatigue effects. We randomized the target order
and counter-balanced the order of E NVIRONMENT.
Participants
We recruited participants from our university’s volunteer pool.
In total, 20 participants took part in the study (4 female, 16
male). The age of the participant was between 17 and 30
(M = 22.1, SD = 3.1). The body height was between 156 cm
and 190 cm (M = 175.2, SD = 9.9). As Plaumann et al. [53]
showed a strong influence of handedness we only recruited
right-handed participants who had no locomotor coordination
(a) Replicated Study Room problems. We used the Miles [43] and Porta test [16] to screen
participants for eye-dominance. 10 participants had right-eye
dominance, 6 had left-eye dominance, and 4 were unclear.
Results
We collected a total amount of 8, 400 mid-air pointing postures.
For all of them, we calculated the following four different
ray casting methods (M ETHOD): eye-finger ray cast (EFRC),
(b) Hand (c) Hand and Body
index finger ray cast (IFRC), forearm ray cast (FRC), and head
Figure 3. The VR scene we used in our study.
ray cast (HRC).
Fatigue effect
First, we analyzed the raw TLX score to determine if potential
Therefore, we used the additional 14 participant specific mea- workload or fatigue effects had to be considered in the further
surements to ensure a precise visualization of the user’s arm analysis. The mean raw TLX score was M = 35.42 (SD =
and hand. Furthermore, Schwind et al. [55] showed an effect 10.46) after the first, M = 35.38 (SD = 12.31) after the second,
of hand representation on the feeling of eeriness of the partici- M = 35.46 (SD = 15.37) after the third, and M = 36. (SD =
pants. Thus, we used the same human androgynous hands2 as 16.15) after the last session. We conducted a one-way repeated
these caused the smallest gender-related effect on participants’ measures analysis of variance (RM-ANOVA). As the analysis
acceptance. The hand representation is shown in Figures 3b did not reveal a significant effect, F3,57 = .047, p = .986, we
and 3c. assume that the effect of participants’ fatigue or workload was
Procedure negligible.
We followed the instructions and procedure that Mayer et Preprocessing
al. [40] used to record their ground truth data. After welco- To determine the cast rays for each mid-air pointing postures,
ming a participant, we explained the procedure of the study we used the samples between 100 ms and 900 ms to counteract
and asked them to fill an informed consent as well as a de- possible hand tremor and possible movements at the beginning
mographic questionnaire. Afterward, we took 14 participant and end of the pointing phase. We further defined the offset as
specific measurements to have a perfect representation of the the distance between the position where the ray cast intersects
arm and hand in VR. We asked them to stand at a specific with the projection screen and the position of the target. We
position in the room (in RW the point was marked on the floor then filtered the mid-air pointing postures to remove outliers
and in VR the point was indicated by a red dot on the floor, see using two times the standard deviation as an upper bound.
Figure 3a) which was centered 2 m away from the projection Related work has shown that the head is the origin of human
screen. From this point, participants were asked to aim at the pointing. However, the participants were of different sizes so
targets using their dominant hand. to compensate for different heights we aligned the heads of
To compensate for natural hand tremor, described as an is- the participants to build one universal model.
sue by Olsen and Nielsen [50], participants had to hold the Accuracy of Ray Casts
pointing position for one second. To ensure this time span, Table 1 shows the average offsets for E NVIRONMENT and
participants had to click with the non-dominant hand on the M ETHOD respectively. The average offset is 9.33 cm for
button of a remote control when they started to hold a gesture. EFRC, 28.09 cm for IFRC, 65. cm for FRC and 42.46 cm for
The target disappeared after one second. We instructed the par- HRC. We performed four one-way RM-ANOVAs to determine
ticipant to point as they would naturally do in other situations. if the variance within one ray casting method is different in the
We intentionally did not restrict participants body pose to re- RealWorld compared to the VirtualReality. We found a statisti-
cord a range of pointing postures. In total the participates had cally significant difference for EFRC, F1,19 = 5.845, p = .026,
to perform 420 mid-air pointing gestures. We split these into 4 FRC, F1,19 = 33.13, p < .001, and HRC, F1,19 = 31.48,
2 Source for human androgynous hands we used in out study: github. p < .001. However, we found no statistically significant diffe-
com/valentin-schwind/selfpresence rence for IFRC, F1,19 = .447, p = .512.
2 2 2 2
y in m
y in m
y in m
y in m
1 1 1 1
Target Target Target Target
VR VR VR VR
RW RW RW RW
0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4
x in m x in m x in m x in m
(a) Eye-finger ray cast (EFRC) (b) Index finger ray cast (IFRC) (c) Forearm ray cast (FRC) (d) Head ray cast (HRC)
Figure 4. The average offset for each of the four ray cast techniques.
EFRC IFRC FRC HRC
E NVIRONMENT M SD M SD M SD M SD
Distance RealWorld 10.21 5.56 29.14 19.24 60.34 20.16 47.15 23.71
Distance VirtualReality 8.45 4.54 27.03 18.95 69.66 25.42 37.77 19.19
Distance after correction RealWorld 8.02 4.54 15.17 9.15 27.01 12.36 46.96 23.71
Distance after correction VirtualReality 8.41 4.56 12.90 7.94 30.18 13.83 37.81 19.26
Table 1. Overall offsets between interact and target. Distance are reported in cm.
Modeling Model Discussion
As Mayer et al. [40] we built models to compensate the sys- We found statistically significant differences between Real-
tematic error. Therefore we first define α pitch as the vertical World and VirtualReality for EFRC, FRC, and HRC but not
deviation angle and αyaw as the horizontal deviation angle for IFRC. We assume this is due to the limited field of view
each between the ray cast and the body. Further ∆ pitch and (FoV) of the HMD. As depicted in Figure 4d VR caused more
∆yaw are the two correction angles respectively. head movement than RW. More head movements reduce the
offset between head ray and actual target, resulting in a lower
We used the following four functions also used by Mayer et HRC offset for VR. Furthermore, this also reduces the offset
al. [40]. The first function f1 (αω ) is a one-dimensional second
for EFRC as here the head is used to calculate the cyclops eye
degree polynomial function (parabola) to predict the correction
for the ray. The already reduced offset limits the possibility
∆ω . For ω we use pitch or yaw to predict ∆ pitch and ∆yaw . For
for an offset correction. Thus we only achieved a reduction of
the rest of the models, we are using α pitch and αyaw to predict
.5% for the VR EFRC model.
∆ω . The functions f2 (α pitch , αyaw ) and f3 (α pitch , αyaw ) are
complete two-dimensional polynomial functions, where f2 is As our new model fits best using a two-dimensional polyno-
of degree 1 and f3 of degree 2. The function f4 (α pitch , αyaw ) mial and fits the best for offset correction for the RW, we
is the function which performed best for Mayer et al. [40] to confirmed the offset correction model presented by Mayer et
compensate the offset: al. [40] in the RW. We also showed that the same polynomial
also reduced the offset the best for VR even though we found
f4 (α p , αy ) = x14 α p4 + x13 αy4 + x12 α p3 αy + x11 α p αy3 + a significant difference between RealWorld and VirtualReality.
x10 α p3 + x9 αy3 + x8 α p2 αy2 + x7 α p2 αy + x6 α p αy2 + However, we could not confirm that IFRC outperforms EFRC
the remaining error after correction for RW. We found that the
x5 α p2 + x4 αy2 + x3 α p αy + x2 α p + x1 αy + x0 (1) offset for IFRC is 89.2 % larger than EFRC while Mayer et
al. [40] found that EFRC is 4.9 % larger than IFRC (for 2 m
While x0 to x14 are the 15 parameters to fit. We used a nonli- standing). However, before correction, they also reported that
near least-squares solver to fit out data. EFRC outperforms IFRC.
Since we found that three ray cast M ETHODS are significantly Overall Mayer et al. [40] reported errors before correction 4.8
different for RealWorld and VirtualReality, we fit models inde- times larger for EFRC, 1.9 for IFRC and 3.7 for FRC than the
pendently for each E NVIRONMENT. For a first evaluation of errors of the presented study. We believe this is due to their
the models, we used leave-one-out cross-validation (LOOCV). different tracking method. While Mayer et al. used one marker
We found that f4 performed best with an overall correction for each position and a post process labeling step, we used
of 29.3%. We achieved the best correction with FRC (55.9%) at least three markers per position of interest (e.g. fingertip).
than IFRC with 50.1% then EFRC with 10.9% then HRC with This enabled us to monitor participants’ movements in real
.2%. However, the remaining offset was the smallest with time which was necessary for the VR visualization, and also
EFRC (8.2 cm) then IFRC with 14. cm then FRC with 28.6 cm contributed towards a more stable and precise tracking.
and the biggest when using HRC with a remaining error of
42.3 cm. The average improvement results using LOOCV are While the offsets reported by Mayer et al. [40] are larger than
reported in Table 1. the offsets we found, the overall direction is the same. They
reported that the intersect is shifted to the upper left for IFRC
and FRC while EFRC is shifted to the lower right. As depicted
in Figure 5 we can confirm these findings for VR as well as
RW. As we also investigated HRC here, we see a different
trend. The offsets are shifted towards the center of the grid.
Our HRC method is only derived from the head movement.
Thus the eye movements are neglected in our implementation.
The difference of the eye ray and the head ray could explain
the effect of a shift towards the center as participants always
focus on the target with their eyes. This can be confirmed
with findings from the field of neurophysiology which studied
the coordination of eye, head, and body movements in detail. (a) The RW scene.
Here, John S. Stahl [58] found that “head movements are
orchestrated to control eye eccentricity”. Further, Freedman
and Sparks [20] found that humans even rotate their head to
focus on the target while at the same time minimizing the
effort put on ocular muscles. However, another factor could
again be the limited FoV of the HMD.
EVALUATION
To validate the developed models and investigate the effect
on users’ performance we conducted a second study. As eye-
finger ray cast (EFRC) resulted in the lowest offset, we tested
the effect offset correction on participants’ performance using (b) The VR scene.
EFRC. We were interested in testing the models in the real Figure 5. The RW and VR scenes used in our evaluation study while the
green cursor is visible.
world as well as in VR. In contrast to our first study, we also
investigated how the model performs when visual feedback is
presented to the participant. Again we used targets without
a target size to evaluate the models’ performance. We used Procedure
E NVIRONMENT (with levels RealWorld and VirtualWorld), After welcoming a participant, we explained the procedure of
C ORRECTION (Yes and No), and C URSOR (Yes and No) as IVs. the study and asked him/her to fill an informed consent as well
As dependent variables (DVs) we measured pointing precision, as a demographic questionnaire. Afterward, we took 14 mea-
the TCT, and again used raw TLX questionnaires. We use the surements of the participant to have a perfect representation in
distance between a target’s center and the intersection of the VR. Participants had to press the button of a remote control
ray cast with the projection screen as accuracy. TCT is the with their non-dominant hand when they were confident that
time between the appearance of the target and the selection by they wanted to continue with the next target. However, after
the participants, as confirmed by a button on a remote control the button press, we implemented a random .5 sec to 1. sec
pressed with the non-dominant hand. delay to ensure that participants did not move their arm before
the next target appeared, to counteract possible false starts. As
Study Design in the first study, we asked participants to stand at a specific
We employed a 2 × 2 × 2 factorial design for the second position in the room centered 2 m away from the projection
study. However, the conditions No C URSOR with or without screen and point at the targets using their dominant hand. We
correction were the same for the participant for both Real- further instructed them to point as they would naturally do in
World and VirtualWorld, so the correction could not be noticed other situations, but as quickly and accurately as possible. We
by the participant during the study. Therefore we were able intentionally did not restrict their body pose to record a range
to reduce the number of conditions to 6 while internally ap- of pointing postures. After each condition we let participants
plying the correction or not to get all 8 conditions. With 2 fill a raw TLX questionnaire. All targets were randomized.
repetitions per condition, we managed to keep the trials ma- C ORRECTION and C URSOR were randomized within E NVI -
nageable for the participant and the time reasonable. Thus RONMENT while E NVIRONMENT was counter-balanced.
we had 6 conditions × 35 targets × 2 respiration = 420 trails,
which the participants completed in approximately one hour. Participants
We recruited new participants from our university’s self-
Apparatus volunteer pool. In total, 16 participants took part in the study
The overall setup was the same as in the first study. We (1 female, 15 male), aged between 19 and 26 (M = 22.7,
used the same tracking system, optical markers, 35 targets, SD = 1.8). The body height was between 156 cm and 181 cm
HMD, projector, and software. However, the Unity scene was (M = 170.4, SD = 7.1). All of them were right-handed, and
adjusted to support our model if needed as well to support the none had locomotor coordination problems. We again used
visual feedback C URSOR. The visual feedback C URSOR was the Miles [43] and Porta test [16] to screen participants for
represented by a green crosshair as suggested by Olsen and eye-dominance. Ten had right-eye dominance, 1 left-eye do-
Nielsen [50]. minance, and 5 were unclear.16 or C URSOR × E NVIRONMENT, F1,15 = 3.79, p = .070. Ho-
No Correction
14 Correction wever, there was a significant interaction between C OR -
RECTION × C URSOR , F1,15 = 4.592, p = .048, but not bet-
12
Remaining offset (cm)
ween C ORRECTION × C URSOR × E NVIRONMENT, F1,15 =
10 2.03, p = .175. In summary, using the correction models sig-
8 nificantly increases participants’ pointing accuracy in the real
and in the virtual world. However, the accuracy depends on
6 using a cursor, see Figure 6 and Table 3.
4 In the following we will estimate target sizes to fit at least 90%
2 of the mid-air pointing actions for all conditions independently.
For simplicity we only fit a squared target shape. For No-
0
RW VR RW VR Cursor in RW the sides of the target need to be 17.6 cm wide,
No Cursor Cursor in VR 18.8 cm and with Cursor for RW and VR respectively
Figure 6. Remaining offset between interact and target for C OR - 4.1 cm and 4.5 cm. With correction the size for the four squared
RECTION × C URSOR × E NVIRONMENT . targets could be respectively 6.9 %, 11.6 %, 6.5 %, and 8.9 %
smaller and still fit 90 % of the pointing actions. The estimated
Results
target sizes are optimal for a target in 2 m distance from the
human.
In the following, we present the results of our correction mo-
dels applied on eye-finger ray cast (EFRC) for RealWorld and Task completion time (TCT)
VirtualWorld. We conducted a three-way RM-ANOVA with We found no significant effects of C ORRECTION, F1,15 =
the independent within-subject variables C URSOR (with the .158, p = .697, or E NVIRONMENT, F1,15 = .004, p = .956 on
levels Yes and No) vs. C ORRECTION (Yes and No) vs. E NVI - the TCT. However, there was a significant effect of C URSOR,
RONMENT (RealWorld and VirtualWorld). Since all factors F1,15 = 7.834, p = .013 on TCT. Furthermore, we found signi-
had only two levels, no pairwise post-hoc comparisons were ficant interaction effects between C URSOR × E NVIRONMENT,
conducted. We used the distance between the ray cast using F1,15 = 15.61, p < .001. No interaction effects were found
eye-finger ray cast (EFRC) and the target as accuracy me- between C ORRECTION × C URSOR, F1,15 = .067, p = .799,
asure and TCT as an indicator of the participants’ pointing between C ORRECTION × E NVIRONMENT, F1,15 = 1.291, p =
performance. .274, or C ORRECTION × C URSOR × E NVIRONMENT, F1,15 =
Fatigue effect 1.163, p = .298. Since the participants received no feedback
First, we again analyzed the raw NASA-Task Load Index about their accuracy when using the correction models, the
(raw TLX) score to determine if potential workload or fatigue correction model did not affect the TCT in the real as well as
effects had to be considered in the further analysis. The mean in the virtual environment. However, presenting a cursor in-
raw TLX score was M = 36.25 (SD = 11.37) after the first, creased the time for pointing since the participants used more
M = 38.28 (SD = 13.46) after the second, M = 39.84 (SD = time to adjust, see Table 2.
15.10) after the third, M = 37.81 (SD = 16.32) after the fourth,
M = 39.74 (SD = 15.73) after the fifth, M = 37.76 (SD = Discussion
17.68) after the last session. We conducted a one-way RM- In our second study, we investigated the effect of the develo-
ANOVA. As the analysis did not reveal a significant effect, ped models in a real-time setup. As we validated the models
F5,15 = .654, p = .659, we again assume that the effect of for all ray casting techniques only using LOOCV, our eva-
participants’ fatigue or workload was negligible. luation study ensured the external validity of the presented
models by inviting 16 new participants. We investigated par-
Accuracy ticipants’ performance with and without correction models
We found a significant effect of C ORRECTION, F1,15 = (C ORRECTION) as well as the effect of displaying a cursor
5.321, p = .027, C URSOR, F1,15 = 131.9, p < .001, and E NVI - as pointing indicator on our model (C URSOR). The effect of
RONMENT, F1,15 = 1.3, p = .027 on the participants’ pointing model and cursor were tested for both real and virtual environ-
accuracy. There were no significant interaction effects bet- ments (E NVIRONMENT). As also found in the first study, we
ween C ORRECTION × E NVIRONMENT, F1,15 = .983, p = .36 found statistically significant differences between RealWorld
No Cursor With Cursor No Cursor With Cursor
Correction M SD M SD Correction M SD M SD
RealWorld False 1.48 .43 1.83 .43 RealWorld False 7.08 3.26 1.14 .89
RealWorld True 1.48 .43 1.89 .73 RealWorld True 5.92 3.29 1.13 .96
VirtualWorld False 1.64 .61 1.76 .56 VirtualWorld False 6.37 3.42 1.30 .85
VirtualWorld True 1.64 .61 1.67 .45 VirtualWorld True 5.76 3.26 1.20 .76
Table 2. Overall TCT to select a the target. TCTs are reported in se- Table 3. Remaining offset interact and target. Distances are reported in
conds. cm.and VirtualReality. This supports our choice of building inde- As the pointing accuracy may be affected by the HMD we
pendent models for RealWorld and VirtualReality, as we found envision as next step a study using HMDs with a variety of
no significant effect of raw TLX over time. Thus, we again FoVs to understand the impact of a limited FoV. In the presen-
assume that the effect of participants’ fatigue or workload was ted paper we investigated real-world (RW) and virtual reality
negligible. (VR) which are representing the edges of the Milgram conti-
nuum [44], in the next steps, we will also investigate pointing
Our analysis revealed that the offset between the eye-finger in augmented reality (AR) and mixed reality.
ray cast and the target can be significantly decreased in real
and virtual environments when using the proposed models.
FUTURE WORK
While the models overall improvement without a cursor was
13.1 %, the improvement for VirtualReality was 9.5 % and In comparison to Mayer et al. [40] we used a marker set
for RealWorld 16.3 %. However, the accuracy depends on which allowed us to online track the limbs of the participant.
whether a cursor was displayed or not. With a cursor, the We expect that this also contributes towards a more stable
average improvement was 4.5 %. The interaction effect of and precise tracking. In the future the potential influence of
C ORRECTION and C URSOR on the accuracy can be explained the marker placement should be investigated to determine a
by a realignment of the user’s arm while presenting visual universal marker placement. This would contribute towards
feedback (the cursor) and applying the correcting models. The models which could be applied by everyone who follows the
increased precision is marginally compensated by the user marker placement conventions. This is especially important
while moving the arm to the target. This is the case in both when future technologies are used for tracking the user without
environments, which is supported by the lacking significant attaching markers but retaining the same precision. On the
effect of the three-way interaction between C ORRECTION, other hand, this would be also important if the model is applied
C URSOR, and E NVIRONMENT. to already existing less precise tracking technologies like the
Microsoft Kinect skeleton tracking.
While the accuracy clearly increased when using a cursor
which is in line with Cockburn et al. [14], analysis of the In both studies the target had no actual size. This was done to
TCT revealed that the cursor also increased the time to select build a precise model where there was no room left for the par-
a target. However, C ORRECTION and E NVIRONMENT did ticipant to interpret the actual target position. We estimate that
not significantly affect the TCT. Furthermore, the interaction the target size can on average be 8.5 % smaller when applying
effect of C URSOR and E NVIRONMENT on the TCT was sig- our new correction models. Future work should investigate
nificant. Having a cursor in the real world is potentially less how a target size influences the models’ performance.
relevant than having a cursor in VR. We assume that this is Incorporating the findings by Plaumann et al. [53] could result
caused by novelty effects and the users’ higher attention to the in more accurate models and improve pointing accuracy. Ho-
task while being in VR. The second study shows that the de- wever, today we cannot determine eye and ocular dominance
veloped models have a positive effect on the mid-air pointing of a user by just observing the user’s behavior. Hence, incorpo-
accuracy without a negative effect on the time to select a target. rating eye and ocular dominance would result in user depended
While displaying a cursor also had a positive effect on pointing models and limit the use cases, e.g. these user-dependent mo-
accuracy, the it also increases the TCT. We, therefore, present dels are not useful for public display scenarios.
the following design considerations for mid-air pointing in
both real and virtual environments: ACKNOWLEDGEMENTS
1. Always apply the model to correct systematic mid-air poin- This work was financially supported by the German Research
ting error. Foundation (DFG) within Cluster of Excellence in Simulation
Technology (EXC 310/2) at the University of Stuttgart and
2. For high precise mid-air selection, a cursor should additio- through project C04 of SFB/Transregio 161.
nally be displayed.
3. For fast mid-air selections, a cursor should not be displayed. REFERENCES
1. Deepak Akkil and Poika Isokoski. 2016. Accuracy of
CONCLUSION Interpreting Pointing Gestures in Egocentric View. In
In this paper, we built mid-air pointing offset compensation Proceedings of the 2016 ACM International Joint
models for real and virtual environments based on pointing Conference on Pervasive and Ubiquitous Computing
gestures of 20 participants. We built models for four different (UbiComp ’16). ACM, New York, NY, USA, 262–273.
ray casting techniques and used cross-validation (CV) to show DOI:http://dx.doi.org/10.1145/2971648.2971687
that we achieve the smallest remaining offset when using 2. Ferran Argelaguet and Carlos Andujar. 2013. A survey of
eye-finger ray cast (EFRC). In a second study, we further 3D object selection techniques for virtual environments.
investigated EFRC in a selection task. We confirm findings of Computers & Graphics 37, 3 (2013), 121 – 136. DOI:
previous work that using a cursor improves mid-air pointing http://dx.doi.org/10.1016/j.cag.2012.12.003
precision. We show that the accuracy of mid-air pointing
without a cursor can be improved through correction models 3. Ferran Argelaguet, Carlos Andujar, and Ramon Trueba.
for both real and virtual environments by 13.1%. Further, we 2008. Overcoming Eye-hand Visibility Mismatch in 3D
show that using a cursor a correction model can reduces the Pointing Selection. In Proceedings of the 2008 ACM
remaining pointing error by 4.5%. Symposium on Virtual Reality Software and Technology(VRST ’08). ACM, New York, NY, USA, 43–46. DOI: and evaluation of spatial target acquisition with and
http://dx.doi.org/10.1145/1450579.1450588 without visual feedback. International Journal of
Human-Computer Studies 69, 6 (2011), 401 – 414. DOI:
4. Ferran Argelaguet, Ludovic Hoyet, Michaël Trico, and
http://dx.doi.org/10.1016/j.ijhcs.2011.02.005
Anatole Lécuyer. 2016. The role of interaction in virtual
embodiment: Effects of the virtual hand representation. 15. Andrea Corradini and Philip R. Cohen. 2002. Multimodal
In Proceedings of the 2016 IEEE Virtual Reality speech-gesture interface for handfree painting on a virtual
Conference (IEEE/VR ’16). Greenville, SC, USA, 3–10. paper using partial recurrent neural networks as gesture
DOI:http://dx.doi.org/10.1109/VR.2016.7504682 recognizer. (2002). DOI:
http://dx.doi.org/10.1109/IJCNN.2002.1007499
5. Tiziana Aureli, Maria Spinelli, Mirco Fasolo,
Maria Concetta Garito, Paola Perucchini, and Laura 16. Giambattista Della Porta. 1593. De refractione Optices
D’Odorico. 2017. The Pointing-Vocal Coupling Parte: Libri Novem... Ex officina Horatii Salviani, apud
Progression in the First Half of the Second Year of Life. Jo. Jacobum Carlinum, & Antonium Pacem.
Infancy (2017), 18. DOI:
http://dx.doi.org/10.1111/infa.12181
17. H. Henrik Ehrsson, Charles Spence, and Richard E.
Passingham. 2004. That’s My Hand! Activity in
6. John V. Basmajian and Carlo J. De Luca. 1985. Muscles Premotor Cortex Reflects Feeling of Ownership of a
alive: their functions revealed by electromyography. Limb. Science 305, 5685 (2004), 875–877. DOI:
Williams & Wilkins, Baltimore, London, Sydney. http://dx.doi.org/10.1126/science.1097011
http://books.google.de/books?id=H9pqAAAAMAAJ
18. Rodger J. Elble and William C. Koller. 1990. Tremor.
7. Scott Bateman, Regan L. Mandryk, Carl Gutwin, and Johns Hopkins University Press, Baltimore.
Robert Xiao. 2013. Analysis and comparison of target https://google.com/books?id=-LdrAAAAMAAJ
assistance techniques for relative ray-cast pointing.
International Journal of Human-Computer Studies 71, 5 19. J. M. Foley and Richard Held. 1972. Visually directed
(2013), 511–532. DOI: pointing as a function of target distance, direction, and
http://dx.doi.org/10.1016/j.ijhcs.2012.12.006 available cues. Perception & Psychophysics 12, 3 (01
May 1972), 263–268. DOI:
8. Elizabeth Bates. 1979. The Emergence of Symbols: http://dx.doi.org/10.3758/BF03207201
Cognition and Communication in Infancy. Academic
Press. 404 pages. 20. Edward G. Freedman and David L. Sparks. 2000.
https://books.google.de/books?id=_45-AAAAMAAJ Coordination of the eyes and head: movement kinematics.
Experimental Brain Research 131, 1 (01 Mar 2000),
9. Hrvoje Benko, Andrew D. Wilson, and Federico Zannier. 22–32. DOI:http://dx.doi.org/10.1007/s002219900296
2014. Dyadic Projected Spatial Augmented Reality. In
Proceedings of the 27th Annual ACM Symposium on User 21. Michael Steven Anthony Graziano. 1999. Where is my
Interface Software and Technology (UIST ’14). ACM, arm? The relative role of vision and proprioception in the
New York, NY, USA, 645–655. DOI: neuronal representation of limb position. Proceedings of
http://dx.doi.org/10.1145/2642918.2647402 the National Academy of Sciences 96, 18 (1999),
10418–10421. DOI:
10. Richard A. Bolt. 1980. Put-that-there: Voice and gesture http://dx.doi.org/10.1073/pnas.96.18.10418
at the graphics interface. In Proceedings of the 7th annual
conference on Computer graphics and interactive 22. Michael S. A. Graziano, Dylan F. Cooke, and Charlotte
techniques (SIGGRAPH ’80). ACM, New York, NY, S. R. Taylor. 2000. Coding the Location of the Arm by
USA, 262–270. DOI: Sight. Science 290 (2000), 1782–1786.
http://dx.doi.org/10.1145/800250.807503 http://www.jstor.org/stable/3081775
11. Jerome S. Bruner. 1975. The ontogenesis of speech acts. 23. Jan Gugenheimer, Pascal Knierim, Christian Winkler,
Journal of Child Language 2, 1 (1975), 1–19. DOI: Julian Seifert, and Enrico Rukzio. 2015. UbiBeam:
http://dx.doi.org/10.1017/S0305000900000866 Exploring the Interaction Space for Home Deployed
Projector-Camera Systems. In IFIP TC 13 International
12. Luigia Camaioni, Paola Perucchini, Francesca Conference on Human-Computer Interaction (INTERACT
Bellagamba, and Cristina Colonnesi. 2004. The Role of 2015). Springer International Publishing, Cham, 350–366.
Declarative Pointing in Developing a Theory of Mind. DOI:http://dx.doi.org/10.1007/978-3-319-22698-9_23
Infancy 5, 3 (2004), 291–308. DOI:
http://dx.doi.org/10.1207/s15327078in0503_3 24. Sandra G. Hart. 2006. NASA-Task Load Index
(NASA-TLX); 20 years later. In Proceedings of the
13. C. N. Christakos and S. Lal. 1980. Lumped and Human Factors and Ergonomic Society annual meeting,
population stochastic models of skeletal muscle: Vol. 50. SAGE Publications, SAGE Publications, Los
Implications and predictions. Biological Cybernetics 36, Angeles, CA, USA, 904–908. DOI:
2 (01 Feb 1980), 73–85. DOI: http://dx.doi.org/10.1177/154193120605000909
http://dx.doi.org/10.1007/BF00361076
25. John B. Haviland. 2003. How to point in Zinacantán.
14. Andy Cockburn, Philip Quinn, Carl Gutwin, Gonzalo
Psychology Press, Mahwah, New Jersey, USA, 139–169.
Ramos, and Julian Looser. 2011. Air pointing: Design26. Hao Jiang, Eyal Ofek, Neema Moraveji, and Yuanchun 2016 CHI Conference Extended Abstracts on Human
Shi. 2006. Direct Pointer: Direct Manipulation for Factors in Computing Systems (CHI EA ’16). ACM, New
Large-display Interaction Using Handheld Cameras. In York, NY, USA, 1706–1712. DOI:
Proceedings of the SIGCHI Conference on Human http://dx.doi.org/10.1145/2851581.2892479
Factors in Computing Systems (CHI ’06). ACM, New
36. I. Scott MacKenzie and Shaidah Jusoh. 2001. An
York, NY, USA, 1107–1110. DOI:
Evaluation of Two Input Devices for Remote Pointing. In
http://dx.doi.org/10.1145/1124772.1124937
Engineering for Human-Computer Interaction: 8th IFIP
27. Ricardo Jota, Miguel A. Nacenta, Joaquim A. Jorge, International Conference (EHCI’01). Springer Berlin
Sheelagh Carpendale, and Saul Greenberg. 2010. A Heidelberg, Berlin, Heidelberg, 235–250. DOI:
Comparison of Ray Pointing Techniques for Very Large http://dx.doi.org/10.1007/3-540-45348-2_21
Displays. In Proceedings of Graphics Interface 2010 (GI
’10). Canadian Information Processing Society, Toronto, 37. Ville Mäkelä, Tomi Heimonen, Matti Luhtala, and
Ont., Canada, Canada, 269–276. Markku Turunen. 2014. Information Wall: Evaluation of
http://dl.acm.org/citation.cfm?id=1839214.1839261
a Gesture-controlled Public Display. In Proceedings of
the 13th International Conference on Mobile and
28. Adam Kendon. 2004. Gesture: Visible Action as Ubiquitous Multimedia (MUM ’14). ACM, New York,
Utterance. Cambridge University Press, Cambridge. IX, NY, USA, 228–231. DOI:
400 S. pages. http://dx.doi.org/10.1145/2677972.2677998
https://books.google.de/books?id=WY2bBQAAQBAJ
38. Anders Markussen, Sebastian Boring, Mikkel R.
29. Aarlenne Z. Khan and J. Douglas Crawford. 2003. Jakobsen, and Kasper Hornbæk. 2016. Off-Limits:
Coordinating one hand with two eyes: optimizing for Interacting Beyond the Boundaries of Large Displays. In
field of view in a pointing task. Vision Research 43, 4 Proceedings of the 2016 CHI Conference on Human
(2003), 409 – 417. DOI: Factors in Computing Systems (CHI ’16). ACM, New
http://dx.doi.org/10.1016/S0042-6989(02)00569-2 York, NY, USA, 5862–5873. DOI:
30. Pascal Knierim, Valentin Schwind, Anna Feit, Florian http://dx.doi.org/10.1145/2858036.2858083
Nieuwenhuizen, and Niels Henze. 2018. Physical 39. Anders Markussen, Mikkel Rønne Jakobsen, and Kasper
Keyboards in Virtual Reality: Analysis of Typing Hornbæk. 2014. Vulture: A Mid-air Word-gesture
Performance and Effects of Avatar Hands. In Proceedings Keyboard. In Proceedings of the 32Nd Annual ACM
of the 2018 CHI Conference on Human Factors in Conference on Human Factors in Computing Systems
Computing Systems (CHI ’18). ACM, New York, NY, (CHI ’14). ACM, New York, NY, USA, 1073–1082. DOI:
USA. DOI:http://dx.doi.org/10.1145/3173574.3173919 http://dx.doi.org/10.1145/2556288.2556964
31. Regis Kopper, Doug A. Bowman, Mara G. Silva, and 40. Sven Mayer, Katrin Wolf, Stefan Schneegass, and Niels
Ryan P. McMahan. 2010. A Human Motor Behavior Henze. 2015. Modeling Distant Pointing for
Model for Distal Pointing Tasks. International Journal of Compensating Systematic Displacements. In Proceedings
Human-Computer Studies 68, 10 (Oct. 2010), 603–615. of the 33rd Annual ACM Conference on Human Factors
DOI:http://dx.doi.org/10.1016/j.ijhcs.2010.05.001 in Computing Systems (CHI ’15). ACM, New York, NY,
32. Alfred Kranstedt, Andy Lücking, Thies Pfeiffer, Hannes USA, 4165–4168. DOI:
Rieser, and Marc Staudacher. 2006. Measuring and http://dx.doi.org/10.1145/2702123.2702332
Reconstructing Pointing in Visual Contexts. In 41. Victoria McArthur, Steven J. Castellucci, and I. Scott
Proceedings of the 10th Workshop on the Semantics and MacKenzie. 2009. An Empirical Comparison of
Pragmatics of Dialogue (brandial’06). Universitätsverlag "Wiimote" Gun Attachments for Pointing Tasks. In
Potsdam, Potsdam, 82–89. http://opus.kobv.de/ubp/ Proceedings of the 1st ACM SIGCHI Symposium on
volltexte/2006/1048/pdf/brandial06_proceedings.pdf Engineering Interactive Computing Systems (EICS ’09).
33. Lorraine Lin and Sophie Jörg. 2016. Need a hand? How ACM, New York, NY, USA, 203–208. DOI:
Appearance Affects the Virtual Hand Illusion. In http://dx.doi.org/10.1145/1570433.1570471
Proceedings of the ACM Symposium on Applied 42. Samuel Navas Medrano, Max Pfeiffer, and Christian
Perception (SAP ’16). ACM Press, New York, New York, Kray. 2017. Enabling Remote Deictic Communication
USA, 69–76. DOI: with Mobile Devices: An Elicitation Study. In
http://dx.doi.org/10.1145/2931002.2931006
Proceedings of the 19th International Conference on
34. Lars Lischke, Pascal Knierim, and Hermann Klinke. Human-Computer Interaction with Mobile Devices and
2015. Mid-Air Gestures for Window Management on Services (MobileHCI ’17). ACM, New York, NY, USA,
Large Displays. In Mensch und Computer 2015 - Article 19, 13 pages. DOI:
Tagungsband. De Gruyter Oldenbourg, Berlin, 439–442. http://dx.doi.org/10.1145/3098279.3098544
DOI:http://dx.doi.org/10.1515/9783110443929-072 43. Walter R. Miles. 1930. Ocular Dominance in Human
35. Lars Lischke, Valentin Schwind, Kai Friedrich, Albrecht Adults. The journal of general psychology 3, 3 (1930),
Schmidt, and Niels Henze. 2016. MAGIC-Pointing on 412–430.
Large High-Resolution Displays. In Proceedings of theYou can also read
