TECH

Hand gestures expand keyboard commands

It is possible to use the built-in camera of notebooks and other computers to upgrade the commands passed to the machine by the keyboard.

This is what Nalin Chhibber and his colleagues at the University of Waterloo in Canada have demonstrated in a technology they call Typealike, something like "the same as typing".

The system works through a common webcam, equipped with a simple mirror. The program recognizes the user's hands next to or near the keyboard and generates commands based on different hand positions.

The user can, for example, place their right hand with their thumb pointing up next to the keyboard, and the program will recognize this as a signal to turn up the music volume.

Different gestures and different combinations of gestures can be programmed to perform a wide range of operations, including for specific programs such as word processors and spreadsheets.

Beyond keyboards...The idea is to facilitate human-computer interaction, making the user experience faster by eliminating the need for keyboard shortcuts or mouse work.

The mirror is needed to redirect the camera, normally facing

forward, making it capable of capturing images of the keyboard and its surroundings. The rest is up to the program created to interpret the images.

"It's a neural network, so you need to show the algorithm examples of what you're trying to detect," explained Professor Fabrice Matulic. "Some people will make slightly different gestures and the hands will vary in size, so you have to collect a lot of data from different people with different lighting conditions."

The team is now working on other applications for their Typealike program, such as in virtual reality, where it could eliminate the need for manual controllers.

Article: Typealike: Near-Keyboard Hand Postures for Expanded Laptop Interaction