 |
 |
 |
 |
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Creation of Non-Manual Signals
Although legibility is the foremost concern in depicting signs in ASL,
a believable human model is also very important to usable software and subtly adds
redundancy that enhances legibility.
This year, we are in the process of developing algorithims
to add realistic body motion to Paula outside of the mandatory arm, hand and wrist
motions of signs. These motions, referred to as non-manual signals,
include, for example, the unconscious motion of one arm while the other
arm moves in a particular direction in a sign.
Essentially, non-manual signals make Paula more passable as a human being to the viewer,
as opposed to her appearing as an expressionless robot. This is similar to the way simulated
inflection and timbre lends credibility to computer voice synthesizers for hearing people.
A Motion Study
For the sake of software expandability, we determined it would be much more efficient to
develop a set of rules about non-manual signals which can be applied to any sign which is in
Paula's vocabulary than to add the motions individually to every sign. Since motion
capture did not lend itself well to our needs, we studied a number of ASL phrases
on specially prepared video, measuring the position of significant
body points at regular frame intervals on an X,Y,Z coordinate plot:
|
|
|
Video Measurements (front view)
|
Video Measurements (side view)
|
With this data, we could then use statistics sofware to calculate
the correlations and trends of motion between various points of the body during different signs.
Since some of the correlations that we discovered proved to be more complex and situation dependent than
we initally anticipated, we have coded non-manual signals into some of Paula's phrases by hand in order to better understand certain
behaviors. An example phrase with and without the addition of non-manual signals can be downloaded here in DivX format:
The second step was to develop a geometric characterization of the observed motions.
This involved running correlations between the positions of the signer's wrists
compared to other landmarks on the head, shoulders and spine. The position of
a signer's hands is one of the most noticeable aspects of signing, and is in fact
one of the phonemic classes of the language. This is was it was important to
pose correlation study in terms of the wrists.
What became apparent was that there was no general correlation between wrist
position and other body landmarks. However, when the wrists moved outside of
a certain space in front of the signer, the correlations became quite pronounced.
This space, bounded vertically and horizontally by the shoulders, was dubbed,
'the zone of comfort.'
The establishment of the zone of comfort facilitated four generalizations.
These a) fingerspelling, b) two-handed raise, c) one-handed lateral, and
d) two-handed asymmetry. Each of these involves positions outside the ‘zone of comfort’.
For fingerspelling, the dominant hand rises above the ‘zone of
comfort’. This requires the additional rotation of the dominant shoulder upward,
and the non-dominant downward to achieve natural movement. For a
two-handed raise, both hands rise above the ‘zone of comfort’ which
requires rotation of both shoulders upward. The one-handed signal occurs
when one hand is outside the width of the ‘zone of comfort’, which
produces a twist in the body. It requires rotations in the waist, spine
and shoulders in the direction of the hand position. In the case of
two-handed asymmetry, both hands are on one side of the midline of the body.
This situation contains a similar, but more pronounced twist in the waist,
spine, and shoulders. At present, these generalizations are being used to
automate non-manual signals of the current animations.
Results
These generalizations have been implemented in the same animation system currently
being used to generate ASL sentences. When viewing previous animations without
non-manual signals, both Deaf and hearing viewers commented that the animations
appeared somewhat mechanical. When reviewing our initial animations using the
four generalizations, viewers were strongly positive in their comments. They
remarked on a more life-like quality of the motion, as well as making it
easier to understand the signing.
Conclusions and future work
Many nonfacial NMS for declaractive can be modeled by observing the path
created by a signer's wrists. This is a great savings for creating computer
animations depicting ASL sentences. Since the NMS can be generated procedurally,
there is no need for a human animator to create them nor for a computer database
to store them explicitly.
Although this approach holds great promise, there is one type of NMS that was not
completely modeled by this approach. This NMS, called "facing", determines the
body orentiation toward the object of the sentence. For example, if a signer
fingerspells the name "Bob", and Bob is the object of the sentence, the signer's
body takes a different orientation than when "Bob" is the subject of the sentence.
Next steps would include an integration of ASL sentence syntax with the
animation geometry to model this NMS.
Our Undergraduate Team
Our undergraduate research team consists of Lesley Carhart, a senior studying Network Technology, and Linsday Semler, a senior studying Computer Graphics. Both intend to graduate from DePaul in 2006.
|
|
|
Lindsay, Lesley, and "Paula"
|
Demonstrating Preliminary Animation
|
|
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
| |
|
|
|
|
|
|
 |
 |
 |
 |
 |
 |
 |
Copyright
2002-2006, DePaul University CTI
Original pages by MaryJo Davidson.
Revised by Jerry Schnepp, 2002-2006.
This page by Lesley Carhart and Lindsay Semler.
|
