Gesture recognition

Teachers: 
Denis Lalanne

Introduction

In recent years, there has been a tremendous effort in research toward novel devices and techniques to improve the naturalness of interaction with computers. Gesture recognition is among the most promising attempt in this direction. Gestures are expressive, meaningful body motions involving physical movements of the fingers, hands, arms, etc., with the intent of conveying meaningful information or interacting with the environment.

First attempts to solve this problem resulted in mechanical devices that directly measure hand and/or arm joint angles and spatial position. This group is best represented by the so-called glove-based devices. Glove-based gestural interfaces require wearing a cumbersome device, and generally carrying a load of cables. This hinders the ease and naturalness with which the user can interact.

The problem of having to wear gloves can be overcome by using video-based interaction techniques. Numerous approaches have been applied to the problem of visual interpretation of gestures for HCI. Images of hands, geometric moments, contours, silhouettes, and 3D hand skeleton models are a few examples. In recent years, however, there has been an interest in incorporating the dynamic characteristics of gestures, which carry as many information as postures.

The major goal of this seminar is to brush a state-of-the-art in gesture recognition, both at the recognition, dialog and application levels.

 

Goal

Read articles corresponding to the select theme (defined with the organizer), synthesize them in an article and present them to the participants of the seminar. The article must be authored in Latex following ACM Strict.

 

Participants

If you are willing to participate to this seminar, please contact the organizer. Max 6 students will be accepted.

 

Agenda

  • October 2nd, 14h: Seminar Kick-off
    • Introduction to Gesture, acquisition and recognition (Denis Lalanne)
      
  • October 30th, 10h15: 
    
    • Présentations d'un chercheur invité (Frédéric Kaplan);
  • December 4th, 10h15:
    • « how to make a good scientific presentation » (Rolf Ingold) [PDF]
  • December 18th, 9h15:
  • February 19th, 9h15:

 

Themes & References

 

1. Surveys

  • Sushmita Mitra, Tinku Acharya, “Gesture Recognition: A Survey”, IEEE Transactions on systems, man, and cybernetics - part c: applications and reviews, vol. 37, no. 3, May 2007. Download PDF

  • Ying Wu, Thomas S. Huang: Vision-Based Gesture Recognition: A Review. Book on Gesture-Based Communication in Human-Computer Interaction, Springer, 103-115 (1999). Download PDF

 

2. Finger tracking & Pointing Gesture

  • J.L. Crowley, F. Berard, and J. Coutaz, “Finger Tracking As an Input Device for Augmented Reality,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, pp. 195-200, June 1995. Download PDF

  • M. Fukumoto, Y. Suenaga, and K. Mase, “Finger-Pointer”: Pointing Interface by Image Processing,” Computers and Graphics, vol. 18, no. 5, pp. 633-642, 1994. Download PDF

  • Yepeng Guan; Mingen Zheng, "Real-time 3D pointing gesture recognition for natural HCI," Intelligent Control and Automation, 2008. WCICA 2008. 7th World Congress on , vol., no., pp.2433-2436, 25-27 June 2008. Link to IEEExplore (download PDF through the UniFR network)

  • Kehl, R.; Van Gool, L., "Real-time pointing gesture recognition for an immersive environment," Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on , vol., no., pp. 577-582, 17-19 May 2004. Download PDF

3.Static hand gesture (posture)

  • V. Athitsos and S. Sclaroff. Estimating 3D hand pose from a cluttered image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 2003. Download PDF

  • T. Starner and A. Pentland, “Visual Recognition of American Sign Language using Hidden Markov Models,” Proceedings of the International Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, 1995. Download PDF

4. Dynamic hand gesture

  • J. Davis and M. Shah, “Recognizing Hand Gestures,” Proc. European Conf. Computer Vision, Stockholm, Sweden, pp. 331-340, 1994. Download PDF

  • K. Ishibuchi, H. Takemura, and F. Kishino, “Real Time Hand Gesture Recognition Using 3D Prediction Model,” Proc. 1993 Int’l Conf. Systems, Man, and Cybernetics, Le Touquet, France, pp. 324- 328, Oct. 17-20, 1993. Link to IEEExplore (download PDF through the UniFR network)

  • Vladimir Pavlovic, Rajeev Sharma, Thomas S. Huang: Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review. IEEE Trans. Pattern Anal. Mach. Intell. 19(7): 677-695 (1997). Download PDF

5. Arm and body gesture recognition

  • Y. Azoz, L. Devi, R. Sharma, "Reliable Tracking of Human Arm Dynamics by Multiple Cue Integration and Constraint Fusion," Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, pp. 905, 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'98), 1998. Link to IEEExplore (download PDF through the UniFR network)

  • Wren, C.R.; Azarbayejani, A.; Darrell, T.; Pentland, A.P., "Pfinder: real-time tracking of the human body," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.19, no.7, pp.780-785, Jul 1997. Download PDF

  • A.D. Wilson and A.F. Bobick, “Recovering the Temporal Structure of Natural Gestures,” Proc. Int’l Conf. Automatic Face and Gesture Recognition, Killington, Vt., pp. 66-71, Oct. 1996. Download PDF

6. Applications

  • T. Baudel and M. Baudouin-Lafon, “Charade: Remote Control of Objects Using Free-Hand Gestures,” Comm. ACM, vol. 36, no. 7, pp. 28-35, 1993. Download PDF

  • W.T. Freeman and C.D. Weissman, “Television Control by Hand Gestures,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, pp. 179-183, June 1995. Download PDF

  • R. Kjeldsen, J. Kender, "Toward the use of gesture in traditional user interfaces," Automatic Face and Gesture Recognition, IEEE International Conference on, pp. 151, Second IEEE International Conference on Automatic Face and Gesture Recognition (FG '96), 1996. Link to IEEExplore (download PDF through the UniFR network)

  • Yang Liu, Yunde Jia, "A Robust Hand Tracking and Gesture Recognition Method for Wearable Visual Interfaces and Its Applications", Proceedings of the Third International Conference on Image and Graphics (ICIG’04), 2004. Link to IEEExplore (download PDF through the UniFR network)

7. Devices based gesture recognition

  • Thomas Schlomer, Benjamin Poppinga, Niels Henze, Susanne Boll, "Gesture Recognition with a Wii Controller", Proceedings of the 2nd international Conference on Tangible and Embedded interaction, 2008. Download PDF

  • Deller, M.; Ebert, A.; Bender, M.; Hagen, H., "Flexible Gesture Recognition for Immersive Virtual Environments," Information Visualization, 2006. IV 2006. Tenth International Conference on , vol., no., pp.563-568, 5-7 July 2006. Link to IEEExplore (download PDF through the UniFR network)
  • Fröhlich, M. & I. Wachsmuth (1998): Gesture recognition of the upper limbs - from signal to symbol. In I. Wachsmuth and M. Fröhlich (eds.): Gesture and Sign Language in Human-Computer Interaction (pp. 173-184). Berlin: Springer-Verlag (LNAI 1371). Download PDF
  • Wei Du, Hua Li, "Vision based gesture recognition system with single camera", 5th International Conference on Signal Processing Proceedings, 2000. Link to IEEExplore (download PDF through the UniFR network)

    8. Multimodal gestures

  • Koons, D. B., Sparrell, C. J., and Thorisson, K. R. 1993. Integrating simultaneous input from speech, gaze, and hand gestures. In intelligent Multimedia interfaces, M. T. Maybury, Ed. American Association for Artificial Intelligence, Menlo Park, CA, 257-276. (ask Bruno Dumas in room B440 for the book)

  • Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. 1997. QuickSet: multimodal interaction for distributed applications. In Proceedings of the Fifth ACM international Conference on Multimedia (Seattle, Washington, United States, November 09 - 13, 1997). MULTIMEDIA '97. ACM, New York, NY, 31-40. Download PDF
  • Holzapfel, H., Nickel, K., and Stiefelhagen, R. 2004. Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3D pointing gestures. In Proceedings of the 6th international Conference on Multimodal interfaces (State College, PA, USA, October 13 - 15, 2004). ICMI '04. ACM, New York, NY, 175-18. Download PDF

  • Bolt, R. A. 1980. “Put-that-there”: Voice and gesture at the graphics interface. In Proceedings of the 7th Annual Conference on Computer Graphics and interactive Techniques (Seattle, Washington, United States, July 14 - 18, 1980). SIGGRAPH '80. ACM, New York, NY, 262-270. Download PDF

Date: 
2009, Fall