Improving Wiimote Gesture Recognition With User Feedback

Denis Lalanne
Matthias Schwaller
Jan Aurel Kühni
Project status: 


Through the last years it has become increasingly popular to extend the way applications handle user input by creating devices that can measure movement and acceleration a device is undergoing. For a long time such devices were quite expensive and complicated to include in an application. Lastly it was the gaming industry that pushed the production of equipment containing accelerometers and other sensors. In November of 2006 Nintendo released a new controller called wiimote, which is capable of tracking movement of a device in three-dimensional space. This development has been pushed ahead recently by Microsoft releasing a well affordable camera capable of tracking objects only by using an infrared emitter.

Problem statement

Today, many modern devices make use of accelerometers or gyroscopes to allow users to control an application by performing movements with their hands or other parts of their body. When using such devices, operators often lack understanding on how the system interprets the input generated by the measuring sensors and are therefore not able to handle the device well.

This bachelor thesis tries to examine how a powerful gesture recognition can be implemented into a game application that exposes how user input is interpreted in order to increase human performance. The core of the project is a gesture recognition algorithm and an elaborate feedback method that helps the person using the application to identify his errors. The feedback is supposed to support the user when learning and performing new gestures. At the end of the project, user evaluation will show whether the implemented feedback can reduce learning times and increase the users performance in a game like application.


Since the gesture feedback is done within a game, the application has to be created first by using the OGRE rendering engine. It was chosen to create a space shooter like game, that features different sections in which the user has to perform specific gestures to trigger special abilities and collect points.

The device used to capture these gestures is the Nintendo wiimote. Gestures will be recorded by reading a stream of accelerometer values from the controller. For the wiimote to correctly pair with the computer and get the acceleration values into the game, a library needs to be integrated into the project that handles the connection to the wiimote.

Once the game is implemented and information can be read from the wiimote, different gesture recognition approaches and feedback methods are evaluated. In relation to the ability to produce a meaningful feedback, one of these gesture recognition methods is chosen and implemented into the game. In a next step the feedback is programmed and tested.

The last part of the project will focus on user evaluation and data analysis. Several persons will be asked to play the game with different gesture sets and feedback types activated. From the gathered data it will be possible to evaluate whether the implemented feedback helps people to perform better in the game or not.