Imaginary Interfaces

TechNewsDaily, June 8, 2010

Edited by Andy Ross

Researchers are experimenting with a new interface system for mobile devices to replace screen and keyboard with hand gestures.

The Hasso Plattner Institute (HPI) in Potsdam, Germany, is hosting a project called Imaginary Interfaces (II) that uses a chest-mounted computer and camera to detect hand movements. Thus users conjure up their own imaginary set of graphical interfaces.

An II could allow people to use gestures during phone calls, much as they do in face-to-face conversations, without the need for clunky hardware.

"We definitely envision a system like this replacing all input for mobile devices," said Sean Gustafson, a research student at the HPI and lead author of an upcoming study on the II.

Many attempts to advance beyond keyboards and mice have focused on gestures. But previous gesture-based interfaces have relied on a physical screen as visual reference.

With II, there is nothing to see; short-term visual memory instead serves as the reference. Users rely on memory to "touch" these virtual elements.

"People are able to interact spatially without seeing what it is that they create," said Patrick Baudisch, a professor of computer science at the HPI and Gustafson's teacher.

In generating the VR interface, the II device combines a camera and a computer to see and interpret gestures.

At present, the device is about 5 cm square and attaches to clothing on the chest. Its makers envision shrinking it down to the size of a button.

An LED ring around the camera beams out IR light. The camera sees the light reflected by nearby hands but not the distant background.

To operate an II, a user has two basic commands. Making out an L shape opens up a 2D interaction plane. A pinching gesture at a point in the plane can then press a virtual button at that point.

Other more sophisticated II techniques are in the works. "We are exploring how users can sketch interfaces, then use them," said Baudisch.

The II study will be presented at the Symposium on User Interface Software and Technology, ACM, New York, October 2010.

AR  Wow, great! I didn't know about this, but it's a major element in chapter 0001 of my new book! Read it, guys, and be astounded at my prophetic powers.