Log in / Register
Home arrow Computer Science arrow Virtual, Augmented and Mixed Reality
< Prev   CONTENTS   Next >

3 Design of the Hand Gesture Interface

According to the need of e-learning and e-business, we design an efficient and lowcost human computer interaction interface. Our proposed hand gesture interface is designed to recognize two hands movements, hand poses (open hand and closed hand), and single hand rotations. There is no need to extract individual finger movements in this case. Also, we properly allocate the stereo camera to prevent it from dealing with complex situations. Moreover, we carefully design the gestures of applications so that hand overlapping is not necessary.

As shown in Figure 1(a), the stereo camera is placed on top of the computer screen and tilts down to capture the hands that are placed right above the table. This arrangement prevents from capturing human heads in the view. Also, the reflection of ambient light on hand is mostly uniform, which reduces the recognition error caused by shadow changes. In addition, it is suitable for long time operation because users' hands are well supported by the table. Users' hands are free to move in horizontal (x), vertical (y) and depth (z) direction, and rotate in yaw, pitch and roll.

Fig. 1. The design of hand gesture interaction. (a) hardware configuration, (b) left camera view, (c) right camera view.

One of our contributions to hand gesture interaction is that the system is capable of recognizing single hand grasping-moving-rotating (SH-GMR) activity. Compared with traditional two hand "steering wheel" [8] gesture for rotating a virtual object, a hand gesture interface with single hand rotation integrated is able to fully control an object [9, 10]. Figure 2 illustrates the SH-GMR activity by an example. SH-GMR contains three major actions: preparing, grasping, and moving and rotating.

A human hand changes its shape from open-handed status (a hand with fingers stretched) to a grasping posture such that an object is captured and fully controlled. Moving and rotating action may occur simultaneously or independently. Keeping the same grasping gesture, the hand shifts or rotates so that the virtual object is shifted and rotated correspondingly. The hand changes its shape back to the open-handed posture, thus releasing the virtual object from being controlled. Compared with the traditional "steering wheel" gestures for object rotating, this method naturally maps hand gestures in the real world to the 3D virtual space.

Fig. 2. Illustration of the SH-GMR activity, (a) initial posture, (b) grasping action, (c) moving and rotating actions

In our design, all of the icons, objects, and shapes are treated as physical objects and can be interacted with very natural hand gestures. Users manipulate objects by common sense, not by memorizing a bunch of hand gestures. Only two poses (open and closed) are needed to be discriminated, which allows a wide tolerance range for users' real postures.

Figure 3 shows the diagram of the whole system that we design. The input sensor is the calibrated stereo camera. Hand parameters, including positions, status, rotation angles, etc, are extracted from the hand gesture interface module for each frame. The VR environment in the e-learning module reacts to the gesture input with dynamic information.

Fig. 3. The system diagram of the e-learning and e-business using the proposed hand gesture interface

Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
Business & Finance
Computer Science
Language & Literature
Political science