What is Gesture-Based Computing?

It is already common to interact with a new class of devices entirely by using natural gestures. The Microsoft Surface, the iPhone and iPod Touch, the Nintendo Wii, and other gesture-based systems accept input in the form of taps, swipes, and other ways of touching, hand and arm motions, or body movement. These are the first in a growing array of alternative input devices that allow computers to recognize and interpret natural physical gestures as a means of control. We are seeing a gradual shift towards interfaces that adapt to — or are built for — humans and human movements. Gesture-based computing allows users to engage in virtual activities with motion and movement similar to what they would use in the real world, manipulating content intuitively. The idea that natural, comfortable motions can be used to control computers is opening the way to a host of input devices that look and feel very different from the keyboard and mouse — and that enable our devices to infer meaning from the movements and gestures we make.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - alan alan Jan 27, 2010

(1) How might this technology be relevant to the educational sector you know best?

  • Probably the best argument for this relevance is to place an iPhone -- a gesture-based device -- into the hands of a two-year-old. She will instantly understand how to use it, finding the games or just discovering new things. It is an amazing thing to see -- and it is easy to replicate with any child. It is the intuitive nature of it that makes it relevant. No manual will ever do that. - Larry Larry Aug 31, 2010
  • Enabling us to move beyond the mouse and the keyboard will provide enormous potential for many fields in higher education, particularly in areas where the need to practice skills and apply expertise before trying things in the real world - has real implications. For example, being able to practice medical procedures that require movement-based attributes is one area - however, to current work in natural movement-based technologies need to reach a point where tactile interaction is more mainstream and haptics and other sensory-based technologies are incorporated. A lot of work is being done in this are but currently quite expensive and for education, mainly in medical fields. - caroline.steel caroline.steel Sep 2, 2010
  • The uptake of the iPhone with mid-mature consumers has demonstrated that a truly intuitive interface can make a big difference to uptake and diffusion. In higher education particularly, where we there are many talented academics who are short on time and enthusiasm to learn how to use new technologies, gesture-based devices that feel more natural and intuitive could be key to increasing the rapidity of technology adoption and use accross higher education. - caroline.steel caroline.steel Sep 2, 2010

(2) What themes are missing from the above description that you think are important?

(3) What do you see as the potential impact of this technology on teaching, learning, or creative expression?

(4) Do you have or know of a project working in this area?

  • A project out of MIT called "Gesture-based computing on the cheap" - see http://web.mit.edu/newsoffice/2010/gesture-computing-0520.html - shirley.reushle shirley.reushle Aug 28, 2010
  • Training ENT registrars to become expert surgeons: the role of fidelity in surgical simulation. Gregor Kennedy, Stephen O'Leary, Kalikow Yukawa, Denny Oetomo, Ed Kazmierczak. Garnett Passe and Rodney Williams Memorial Foundation. - caroline.steel caroline.steel Sep 2, 2010
  • Haptic Tele-Rehabilitation: Latency implications for system stability and clinical communication. Denny Oetomo, Gregor Kennedy, Ed Kazmierczak, Mary Galea, Dragan Nesic. Institute for a Broadband Enabled Society Seed Funding. - caroline.steel caroline.steel Sep 2, 2010

Please share information about related projects in our Horizon Project sharing form.