ARPP: An Augmented Reality 3D Ping-Pong Game System on Android Mobile Platform
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
N2.4.pdf WOCC 2014
ARPP: An Augmented Reality 3D Ping-Pong Game
System on Android Mobile Platform
Xin Gao, Jie Tian, Xiaoyuan Liang, Guiling Wang
Department of Computer Science
New Jersey Institute of Technology
Email: {xg54,jt66,xl367,gwang}@njit.edu
Abstract—Ping-pong is a very popular physical game all over
the world. People need to play it in some fixed physical locations.
Based on Augmented Reality (AR) technology, we provide a more
interesting and convenient way for people to play ping-pong
game on smartphones. In this paper, we propose an Augmented
Reality 3D Ping-Pong game system (ARPP) for two players on
Android platform through Wi-Fi Direct. The game is rendered
when two players aim their phones’ camera at a specific marker.
The players can view the virtual table tennis scenario through the
screen of their smartphones. They move their phones to control
paddles to play ping-pong game. The experiment results show
that the proposed game system can work effectively and provide
winner results on two Android mobile devices.
Keywords—Augmented Reality; 3D game; ping-pong; table
tennis; Android platform; smartphone; tablet; Wi-Fi Direct
I. I NTRODUCTION
In recent years, smartphones are widely used in our daily
life. They usually have high-resolution touch-screens, video
cameras, web browsers, GPS and high-speed data access-
ing through Wi-Fi and mobile network[1]. Several operating
systems can be run in smartphones, such as iOS, Android
and Windows operating system. Android is listed as the
most popular smartphone operating system in the world and
occupied 79.3% of the smartphone market in 2013[2]. Android
operating system is an open source, Linux-based operating sys-
tem which is not only for smartphones but also for tablets[3].
Fig. 1: Ping-Pong Game Scene for Both Two Players.
There are over five million Android applications in on-line
stores by February in 2014[4]. At the same time, thousands of
new android apps appear online every month. People can use
smartphones almost at any time anywhere, such as commuting,
doing exercises, staying at home or working. When people devices’ cameras to find a marker in the real world and then
stay at office, it is very possible to feel tired and want to draws a basic ping-pong table. Based on devices’ relative
find something interesting to relax. A large number of people locations to the marker, the two players can move their phones
would like to do physical leisure activities, like going to a to manipulate their own virtual paddles so that they can
gym. While in most conditions, gyms are far away from our hit the ball back and forth. In order to show the proposed
places. To buy a ping-pong table is expensive, and the table game system is efficient, we implement the proposed system
occupies a large place. Thus we develop an application on in an application on Android platform. Despite their limited
smartphones to simulate the game. computation power, we think handheld devices are the best
Augmented Reality (AR) is a new technology that provides choice if we want to provide a truly immersive Augmented
ways to blend virtual images into the video stream of a mobile Reality experience. The reason is that most of these devices
device’s camera in real time and superimposes a computer- have a camera on their back, just behind the screen, which is
generated image on a user’s view of the real world. Our project the only way to achieve see-through effect at mass-market cost.
is to design a two-player 3D Ping-Pong in an Augmented We strongly believe that Augmented Reality is a promising
Reality world over WiFi. Both players use their devices as idea, which can enhance the players’ gaming experience by
paddles to reflect the vitual ball. The game system uses the providing exciting new ways to control their actions, through
978-1-4799-5249-6/14/$31.00 ©2014 IEEE
N2.4.pdf WOCC 2014 2
positions and 3D movement. The two players need to print the marker first. Marker’s size
The rest of this paper is organized as follows: We present is same as the size of a smartphone. It is preferably located at
the game system design, architecture and implementation in the center of the paper so that there is a contrast for the marker
Section II. Then we introduce the user interface of the game border. After that, the players need to stand a few meters apart
and describe the game play in Section III. Section IV analyzes and face to each other. Then they can connect to each other’s
some recent related work. Finally, Section V concludes the device, and start the game to play AR ping-pong. They look at
paper and discusses the future work. the devices’ screens, where the ping-pong table is drawn and
move their phones to control the virtual paddles. The rotation
II. G AME S YSTEM D ESIGN
of the marker also matters. The players need to make sure
A. System Scenario that one of the devices views the marker with its shaded at
The purpose of our research project is to design and imple- the corner in the top left and the other one at the same corner
ment a game system for 3D AR ping-pong game, which needs in the bottom right. They play the game along the Y axis. The
two separate devices over Wi-Fi Direct for two players. Based X axis is left-and-right and the Z axis is up-and-down. X axis
on AR environment, the players may need to use their phones’ shows the scores of the two players. It always shows the score
cameras to view the world. The game system is rendered at on the player’s right side. Z axis is responsible for the height
the location of a specific marker, which is a required printout of ball and paddles.
as shown in Fig. 2. The two players use their mobile devices We add the supplemental ARMarker Library for Marker
to control their own virtual paddles to play ping-pong game. Detection to draw the virtual ping-pong table over a marker,
Players can play the game based on the same marker, or on using the CameraMarker opposed to the BasicMarker. The
two markers with the same image on them. It means they main difference is where the coordinate (0,0,0) is located. For
can play the game standing on the opposite position of the the CameraMarker, it is the center of the detected marker.
same marker or sitting in front of different tables not far away Sometimes you may lose track of your paddle if you move
from each other based on two markers. In this situation, the your device too fast or you rotate your device too often,
positions of markers on different tables are different. One of because the camera needs time to do the position calculation.
them is upside down. In other words, one player needs to stand However, it is still also playable and does not affect the results.
on the opposite side of the marker different from where the When a game is created, the device begins to try to get the
other player stands on. best possible GeoLocation while waiting for a client to join.
Likewise, when a client joins their device, it tries to get the
best possible GeoLocation after a certain duration time plus
the time that it takes to start the game. Once the game starts,
it takes the best position of Player 1 and the best position
of Player 2 as the end points, and then generates a virtual
ping-pong table at a GeoLocation at the line’s midpoint. Then
the game starts and the players use their devices to hit the
Fig. 2: The Marker. virtual ball. One player aims his/her camera towards the other
player and bounce the ball off the screen. The ping-pong table
stays stationary fixed at that GeoLocation as the players move
We use two libraries in our project: DroidAR and ARMark- around it.
er. DroidAR is an Augmented Reality library developed by We choose to keep the coordinate (0,0,0) at the detected
Simon Heinen from Bitstars.com[5], which takes a major part marker location. In this time, we attempt to display the paddle.
of the camera work and image processing work. It covers a lot It means that we created a 3D cube to act as a paddle
of common AR tasks, such as marker recognition, geolocation with its position being offset of the camera’s starting raycast
and landmarks, placing or collecting objects, drawing different position. Moreover, we try to compare the difference between
shapes, simplifying the conversion from world to virtual the start position of the ray mentioned earlier and the provided
coordinates and handling camera functionality. The ARMarker function of “get ground position”. The library returns the (x,y)
library for marker recognition is also required in our project. coordinate of the position that the camera is looking at. The
Basically, some parts of the libraries are written in C/C++ latter one is stable, but it was too awkward to play ping-
since Java would be too slow for what the libraries are doing. pong game. Thus to help smooth out the jitteriness of the
The DroidAR library is a bridge between the true native code coordinates, we attempt to buffer the z coordinate over 10
of C/C++ and the Java layer, and exposes the functionality to update cycles or 100 ms approximately.
your project. Then we import the two libraries into Android
IDE and make a little change on them. Now we build our B. Wi-Fi Direct
own system and do the following two steps: 1) Set DroidAR Wi-Fi Direct (Wi-Fi P2P), is a Wi-Fi standard. It enables
as a library of ARMarker; 2) Set DroidAR and ARMarker as mobile devices to connect with each other easily without
libraries for ARPong. We then implement our game system in requiring a wireless access point and to communicate at typical
Java on Android OS version 4.2. Wi-Fi speeds for everything from file transfer to Internet
N2.4.pdf WOCC 2014 3
connectivity[6]. It allows two or more Android 4.0 devices ZƵŶƉƉ
to connect with each other in an Ad Hoc network. Among
>ŝƐƚ
them, one device takes the role of being the server. Also, WiFi ƌĞĂƚĞ
ĞǀŝĐĞƐ
tethering and WiFi hotspot are disabled while Wi-Fi Direct
is active. Some devices need WiFi to be turned off manually ^ƚĂƌƚ^ĞƌǀĞƌ ^ĞůĞĐƚŽŶĞ
before starting Wi-Fi Direct, otherwise the regular WiFi would ĞǀŝĐĞ
take priority and disable the Wi-Fi Direct. Some file sharing tĂŝƚĨŽƌ
WĞĞƌƐ
applications also use Wi-Fi Direct. It really simplifies the ZĞƋƵĞƐƚƚŽ:ŽŝŶ ͲĚũƵƐƚƉĂĚĚůĞ
networking process. ŽŶĨŝƌŵƚŚĞ ďĂƐĞĚŽŶĐĂŵĞƌĂ
We use the WiFiDirectDemo provided by Android Devel- ZĞƋƵĞƐƚ ͲDŽǀĞďĂůů
ͲhƉĚĂƚĞƐĐŽƌĞ
opers, which comes with the Android SDK. However, the
demo is unidirectional and the connection only opens to pass ^ƚĂƌƚ
a file and then immediately closes. We convert it from using 'ĂŵĞ
a Socket to using a DatagramSocket so we could pass a ZĞŐŝƐƚ ŚĂŶŐĞĂůů ^ĐŽƌĞZĞĂĐŚĞƐ
byte array instead. We also have both the client and the DĂƌŬĞƌ ŝƌĞĐƚŝŽŶ >ŝŵŝƚ
server listening on a port so we could have bidirectional
ƌĂǁ
communication, where the bidirectional sockets transfering a
tŽƌůĚ
serialized object. The communication needs to be on separate ^LJŶĐĂůů DĂŝŶ
threads. Once the listening starts, it creates the socket and WŽƐŝƚŝŽŶΘ ^ĐƌĞĞŶ
ŝƌĞĐƚŝŽŶ
loops until it disconnects all the intercepting objects sent to
that port. Sending an object is also done on an AsyncTask Fig. 3: System Architecture.
to prevent the application from hanging/freezing. Our project
uses the initial setup from the demo, but implements a multi-
step simple handshake. Once peers are connected, the client
starts the game. When the start button is clicked, it sends a When the application is running, it shows the players a Wi-
message to the server device with its IP address and starts Fi Direct peer list provided by the WiFiDirectDemo. Some
the game on the server. When the server receives the request, mobile devices need to have their WiFi turned off before
it sends a reply back confirming the connection, which then turning on Wi-Fi Direct and some do not need to enable it.
starts the game on the client. This is to ensure that the socket is You can set it by following the steps: you need to click the
not null and that both the client and server have the appropriate search/discovery button, the application displays that you need
destination address. The socket connection is persistent and to enable it first. Then you can setup from the list shown by
WiFiDirectDemo will close it after transferring. the system. Some devices with the correct API level for Wi-Fi
Direct don’t have access to Wi-Fi Direct. For instance, one of
C. System Architecture the test devices, a Droid Razr, does not have this capability.
We design the system architecture as shown in Fig. 3. If one Once the Wi-Fi Direct is enabled and active, some devices
player runs the application, he or she needs to select the device automatically detect peers, as shown in Fig. 4. Otherwise both
from the list and make a connection through Wi-Fi Direct. devices need to click the search/discover button again.
After the other player confirms the connection request, they
start the game. The game needs to register the marker and to
draw a virtual ping-pong environment. Then the loop starts.
Each time after they hit the ball, the scores of one side are
updated and the virtual AR world is drawn again. If the ball’s
direction is changed, it synchronizes the updated position and
direction to the virtual world and a new loop begins.
Fig. 4: The Screen Shot Through One Tablet.
III. E XPERIMENTAL P ERFORMANCE AND U SER
I NTERFACE
In this section, we present the experimental performance If everything goes well, we can see our peer’s device
and user interface of our game system. The overall ping- information, as shown in Fig. 5. Since the screen of two
pong game scene for two players is shown in Fig. 1. In our players’ devices are similar, we only show the server side.
experiment, testing on real Android’s tablets and smartphones The following is the procedure to run our game application:
are introduced. The tests show that on Android smartphones • Click the device you want to connect to and click the
our system runs great. In order to get a large screen to show connect button once it shows up. The other device needs
our results and observe the process more clear, we install the to confirm the connection. It is shown in Fig. 5.
application on two Nexus 7 Android tablets with Android 4.2 • Once both devices are connected, it lists some informa-
on both of them. tion about the devices. It is provided by WiFiDirectDemo.
N2.4.pdf WOCC 2014 4
Fig. 7: The Screen Shot Through One Tablet.
Fig. 5: The Screen Shot Through One Tablet.
Fig. 8: The Screen Shot Through One Tablet.
IV. R ELATED W ORK
Fig. 6: The Screen Shot Through One Tablet. Augmented reality is a live, copy, view of a physical,
real-world environment whose elements are augmented by
computer-generated sensory input such as sound, video, graph-
ics or GPS data. Virtual reality replaces the real world with
• After the connection is established, the client device has a simulated one. With the help of advanced AR technology,
a start button shown in Fig. 7. the information about the surrounding real world of the user
• Click that and both devices will open the ARPP activity. becomes interactive and digitally manipulable[7]. There are
Before the game starts, we need to do the following some recent advances in Augmented Reality. The displays for
confirmations: viewing the merged virtual and real environments can be clas-
sified as head mounted, handheld and projective. Besides, there
– Make sure the camera can clearly see the marker you are four main ways of interaction in AR applications: tangible
printed out earlier. AR interfaces, collaborative AR interfaces, hybrid AR inter-
– Make sure the area is clear and you have good faces, and emerging multimodal interfaces. New applications
lighting for best results. Shadows can interfere with can also be grouped into three areas: mobile, collaborative,
the marker recognition. and commercial applications. Among these three, the most
• The ARPP starts. You can see the device to recognize the related one to our work should be the collaborative application,
marker correctly when you see the green pong table snap which can benefit from having multiple people simultaneously
to the maker’s position. The blue cube is your paddle and view, discuss, and interact with the virtual 3D models[8, 9].
the white cube is the ball. Players can also change the Augmented reality allows gamers to experience digital game
size of ping-pong table. If the player slides upwards the play in a real world environment.
screen, the table will turn to be large, and vice versa. We In [10], it proposed a AR display-based game environment
can see the vitual ping-pong game environment in Fig. 9. with some small robots, which are needed to be one part
• Adjust your phone accordingly to move the blue cube of the experimental system. Similarly, Knoerlein et al.[11]
along your side of the table to reflect the ball in the other implemented an collaborative AR ping-pong game with a
direction. If the player hits the ball successfully, the score haptic interface. When hitting the virtual ping-pong ball, they
representing his result on the right side will increase by can receive force feedback through optically tracked ping-
1. As shown in Fig. 9, the player with blue paddle gets pong racket handles attached to SensAble PHANTOM haptic
18 scores and another player gets 21. devices. In addition, [12] developed a new calibration method
• Once a player reaches the desired points (Here we set to align the haptic and world coordinate systems. It integrates
the number of desired points as 50), the end screen will all elements into an AR haptics ping-pong game. However, the
show up. Then the players can choose whether to play players need to use virtual bats collocated with haptic devices,
again or to quit, which is shown in Fig. 8. which is not convenient for users to play. And these items are
not that common in our daily life.
From the results, it can be observed that the game system There is another networked table tennis-like game[13] that
can provide a user-friendly screen for two players to play is played with a real paddle and ball, augmented with a large-
through wifi connection . scale video conference. It is a physical interactive game and
N2.4.pdf WOCC 2014 5
Direct service to display game results. We also conduct several
experiments to evaluate our application. The experimental
results show that our developed system can work effectively
on android devices.
In the game system, there are still several improvements
that can be made in the future. Friendly user interface is very
important for an application. Most of people decide whether to
use an application only by its user interface. Thus, the paddles
and ball need to be looked more smooth and real. Moreover,
although the game system can show fantastic virtual game
scenario, the improvement of accuracy and sensitivity of the
system is still needed. In the real world, when we play ping-
pong, we always move a lot. Thus the table and paddles need
to change more quickly and exactly in our future work.
R EFERENCES
[1] “Smartphones,” http://en.wikipedia.org/wiki/Smartphone,
April 2012.
[2] “Apple cedes market share in smartphone operating
system market as android surges and windows phone
Fig. 9: The Screen Shot Through One Tablet. gains, according to idc,” https://www.idc.com/getdoc.jsp?
containerId=prUS24257413, 2014.
[3] T.-M. Grønli, J. Hansen, and G. Ghinea, “Android vs
can be played by three players in three different locations. We windows mobile vs java me: a comparative study of mo-
have also found a wonderful game named AR Pong 3D[14], bile development environments,” in Proceedings of the
whose interface is similar to ours. It is a remake of the classic 3rd International Conference on PErvasive Technologies
arcade game from the 80’s with beautiful 3D graphics for Related to Assistive Environments. ACM, 2010, p. 45.
Windows 8. It is an online computer game that require the [4] “Number of available android applications,” http://
players to play against the computer and get to 10 scores first www.appbrain.com/stats/number-of-android-apps, 2014.
to win the game. It means we can only play with the computer [Online]. Available: http://www.appbrain.com/stats/
using a mouse. Sometimes it is not that interesting and we will number-of-android-apps
get tired and bored after sitting in front of the computer for a [5] “DroidAR,” https://github.com/bitstars/droidar.
relatively long time. [6] “Wi-Fi Direct,” http://en.wikipedia.org/wiki/Wi-Fi
There are some other kinds of AR games. For example, Oda Direct, 2014.
et al.[15] developed an AR racing game created by modifying [7] “Augmented Reality,” https://en.wikipedia.org/wiki/
an existing racing game. While the player needs to wear a Augmented reality/, 2014.
tracked video see-through head-worn display to control the [8] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier,
car with a passive tangible controller. Woodward et al.[16] and B. MacIntyre, “Recent advances in augmented reali-
presented a computer system CamBall for natural interaction ty,” Computer Graphics and Applications, IEEE, vol. 21,
in an augmented virtual environment, which enabled people no. 6, pp. 34–47, 2001.
to play table tennis over Internet/LAN with real rackets. [9] J. Carmigniani, B. Furht, M. Anisetti, P. Ceravolo,
Undeniably, they all make a significant improvement in AR E. Damiani, and M. Ivkovic, “Augmented reality tech-
game implementation. However, they all need assistant tools to nologies, systems and applications,” Multimedia Tools
implement the AR games. Devices other than mobile devices and Applications, vol. 51, no. 1, pp. 341–377, 2011.
in these conditions are not portable. Thus the game cannot be [10] M. Kojima, M. TOMITA, M. INAMI, M. SUGIMO-
played at any time anywhere as the users want. That’s why TO, A. NAKAMURA, and N. Hideaki, “Augmented
we propose our game system. coliseum: An augmented game environment with small
vehicles,” in First IEEE International Workshop on Hor-
V. C ONCLUSION AND F UTURE W ORK izontal Interactive Human-Computer Systems (TABLE-
In this paper, we have presented a 3D Ping-Pong game TOP’06). IEEE Computer Society, 2006, pp. 3–8.
system based on Augmented Reality technology. In this game, [11] B. Knoerlein, G. Székely, and M. Harders, “Visuo-
we combine the advantages of both WiFi-supported game haptic collaborative augmented reality ping-pong,” in
and traditional smartphone online games. We implement the Proceedings of the international conference on Advances
game system through an application on Android platform. in computer entertainment technology. ACM, 2007, pp.
To illustrate the effectiveness of the system, we adopt Wi-Fi 91–94.
N2.4.pdf WOCC 2014 6
[12] G. Bianchi, B. Knoerlein, G. Székely, and M. Harders,
“High precision augmented reality haptics,” in Proc.
EuroHaptics, vol. 6, 2006, pp. 169–178.
[13] F. Mueller and M. R. Gibbs, “A physical three-way inter-
active game based on table tennis,” in Proceedings of the
4th Australasian conference on Interactive entertainment.
RMIT University, 2007, p. 18.
[14] “Ar Pong 3d,” http://apps.microsoft.
com/windows/en-us/app/ar-pong-3d/
7eb354e5-7b9c-4034-9569-48b907ea08ea/, 2013.
[15] O. Oda, L. J. Lister, S. White, and S. Feiner, “Developing
an augmented reality racing game,” in Proceedings of the
2nd international conference on INtelligent TEchnologies
for interactive enterTAINment. ICST (Institute for
Computer Sciences, Social-Informatics and Telecommu-
nications Engineering), 2008, p. 2.
[16] C. Woodward, P. Honkamaa, J. Jäppinen, and E.-P.
Pyökkimies, “Camball: augmented networked table ten-
nis played with real rackets,” in Proceedings of the 2004
ACM SIGCHI International Conference on Advances in
computer entertainment technology. ACM, 2004, pp.
275–276.
You can also read
