As robots become more integrated into humans' everyday lives, it becomes essential
for them to perceive and understand people's intentions and actions. People will
want a more natural way to interact with robots, closer to the way humans do it
among themselves. When humans communicate, they use a rich set of gestures and
body language in addition to speech, which significantly enhances their ability to get
thoughts, feelings, and intentions across. Hence, robots strongly need the ability to
understand human gestures and to interpret body language.
One of the most expressive components of body language are hand gestures, which
is why they are so important at the interface between humans and robots. This thesis
presents an extensible, vision-based communication system that is able to interpret
2D dynamic hand gestures.
The system consists of three primary modules: hand segmentation, feature extraction,
and gesture recognition. Hand segmentation uses both motion and skin-color
detection to separate the hand from cluttered backgrounds. The feature extraction
module gathers hand trajectory information and encapsulates it into a form that is
used by the gesture recognizer. This last module identifies the gesture and translates
it into a command that the robot can understand.
Unlike other existing work, this hand gesture recognition system operates in realtime
without the use of special hardware. Results demonstrate that the system is
robust in realistic, complex environments. Moreover, this system does not require
a priori training and allows the hand great flexibility of motion without the use of
gloves or special markers.