the

HandVu
vision-based hand gesture interface
overview | screenshots | requirements | download | installation | documentation | people

News

Overview

This software collection implements a vision-based hand gesture interface. Called HandVu, it detects the hand in a standard posture, then tracks it and recognizes key postures - all in real-time and without the need for camera or user calibration. The output is accessible through library calls, a client-server infrastructure in a custom format and as OpenSound Control packets, allowing control of music applications that use this format.
The software package consists of the main HandVu library and several applications that demonstrate video capture with OpenCV's highgui, DirectShow, and soon the ARtoolkit. If your application includes video processing, you would just make a few library calls and hand gesture recognition results will be available to you. If your application does not use camera input already, you can run one of the sample programs on the side and interface to your application with a networking interface. Functionality to capture screenshots and snapshots of the tracked object is availabe, allowing data collection for research or other purposes.

Screenshots

Videos are all in WMV format for now: The first one shows mouse-based interaction with a head-worn camera. The second one focuses on the hand tracking aspect in both indoor and outdoor scenes. The third video demonstrates interaction with virtual objects and two-handed interaction with help of the ARToolKit.

still images
interacting with virtual objects and ARToolKit
mouse and button interface with hand gestures
rubber band with-two handed interaction
gesturing to our wearable computer

Requirements

Download

One source package for all platforms can be downloaded here:
--> handvu-beta3.tar.gz
It includes documentation, but not the Release Notes.
A binary installer for Windows is available as well:
--> HandVu_Beta3_Setup.exe

Older versions

handvu-beta2.tar.gz (needs OpenCV 0.9.6)
Beta2: A Windows binary with installer (and un-installer;-) is available as well:
HandVu_beta2.msi (needs OpenCV 0.9.6)
You might also want to get the DllTester.exe for Windows.

Installation

The following documentation is also included in both the binary and source distribution.

Windows binary installer

Execute the downloaded MSI file. Then, double-click on either the HandVu_OpenCV icon or the HandVu icon on the desktop. For troubleshooting, see the Windows Installation documentation.

Source package

Please see the documentation for Installation on Linux or for Installation on Windows.

Documentation

Quick Start Guide for Linux

  1. Connect a camera to your computer. Make sure you can run some of OpenCV's example applications.
  2. Run hvOpenCV and perform the closed posture at about the location shown below. After recognition of your hand, tracking will start and a number of key postures will be recognized.

Quick Start Guide for Windows

  1. Connect a camera to your computer.
  2. Run the desktop shortcut to HandVu_OpenCV or to HandVu. The live video will appear in the application window.
  3. Perform the closed posture at about the location shown below. After recognition of your hand, tracking will start and a number of key postures will be recognized.
Tijmen Kuipers kindly wrote an Installation Guide in Dutch

For more details, please have a look at the Application Documentation, which is included in the download package as well. Also, please make use of the Yahoo HandVu Group discuss installation or development problems.

Publications

Please see here.

Coming soon:

- tracking with higher frame rates on Linux
- more recognized postures (picking posture)
- high-precision pointing, tracking (same spot on hand across recognitions)

People

Currently involved with the HandVu project are: Mathias Kölsch, Matthew Turk, Tobias Höllerer, Stephen DiVerdi.
We would also like to thank our patient beta-testers Dan, Kai, Remy, and Sebastian.