Multimodal, Natural and Tangible Interaction

Laurence Nigay

Université Grenoble Alpes, France

Laurence Nigay has a PhD in Computer Science and is a Professor Exceptional class in Computer Science at Université Grenoble Alpes (UGA). Her research focuses on the fundamental and conceptual aspects of interaction, novel interaction techniques as well as engineering of Human-Computer Interaction. She is the director of the Engineering Human-Computer Interaction (EHCI) research group (40 members) of the Grenoble Informatics Laboratory (LIG). Laurence Nigay has received several scientific awards (including the CNRS Bronze medal in 2002 and the UGA gold medal in 2003 and again in 2005) for excellence in her research. She has published more than 220 articles in journals, conferences, and books. Research axes: Interaction techniques, Multimodality, Augmented Reality, Mobility.

Course Description

This course is dedicated to interfaces that go beyond the mouse, keyboard and touch, focusing on tangible objects, gestural and natural languages as interfaces for interaction. This course will present different interaction modalities and how to combine (fusion and fission) them to create more effective technologies. It will cover the CARE and CASE design spaces for multimodality among other concepts. The topics include:

  1. Introduction
    - Multimodality: definition, benefit and -mobile or not- application domains
    - Human perception/action: Human modalities and multisensory perception (ICS, Interactive cognitive sub-systems theory)
    - Challenges
  2. Application domains
    - Multimodal applications illustrated by videos
  3. Design space: Characterization of a modality and combination of modalities
    - Underlying concepts: MSM framework and Pipe-Line model
    - Selection of one or several modalities: Actors & Criteria
    - Characterization of an interaction modality
    - Combination of interaction modalities: CARE properties
    - Heuristics, rules of thumb

Date

November 2nd, 2018