Home Page
Link to Project Information Link to Demo Link to Team Info Link to Publications Send us Email


Computer Generated Fingerspelling
Abstract

Fingerspelling is a vital component of American Sign Language (ASL) which is the preferred form of communication among the Deaf.

Achieving fingerspelling fluency relies heavily on the visual comprehension of the manual representation of letters. Of particular importance are the transitions between letters.

We are presenting the development of a Fingerspelling learning tool that places emphasis on realism through the principles of 3D animation with an interface that is accessible to all skill levels.


What is FingerSpelling?
Fingerspelling is an important component of American Sign Language (ASL) and is a necessary skill for complete communication [BATTISON 78]. It is useful for spelling proper nouns, technical terms, acronyms, initialized signs, loan signs and words from foreign languages.

When fingerspelling, people use their dominant hand to make handshapes, one corresponding to each letter of the word. Fluency in sign language includes the ability to produce and recognize fingerspelling at a rate of four characters per second [WOLFE 04].

Fingerspelling is usually the first skill learned in ASL. However, it is the last skill mastered. This due in part to the high rate of symbol of recognition required [LAKE 04]. Most ASL signs use one or two hand symbols. In contrast, fingerspelling uses one symbol for each letter in a word.


A Spelling Example



Learning Tool Features
1. Three levels of difficulty

To accommodate different fingerspelling skill levels, users should have the opportunity to experience different recognition challenges. Users have the option to start at the beginner level and practice their way up to more challenging intermediate or advanced levels. Each level includes its own adjustable speed control for an enhanced learning experience.

2. Quick Reference Chart

Since symbol recognition is of great importance it is necessary for those learning fingerspelling to be able to distinguish between hand shapes. For the beginner and intermediate users, our application will display static 3D images of the manual alphabet.

3. Review and self-quiz

Users need opportunities to test their recognition skills. The design should offer multiple choice and fill-in-the-blank practice sessions and quizzes, each unique to the level of difficulty. Feedback should display the number of correctly and incorrectly identified words as well as the missed words with the number of identification tries. Giving the user relevant information during practice sessions and quizzes will allow them to analyze their own progress and areas of improvement [IKEDA 99].
  •   Letter Identification: Implemented at the beginner level, this portion of the program displays a letter of the manual alphabet. Users guess the letter and receive feedback on their response.

  •   Multiple Choice: At the intermediate level, users choose from a category (i.e. food, animals…), view a fingerspelled word, and select from four possible answers. To challenge the user, the possible answers presented are of similar length and contain the same first and/or last letter. The category gives a context to the fingerspelled word and assists the user in selecting the correct response. At the advanced level, the context is removed and the fingerspelled word is chosen from a list of 3,000 words.
  •   Word Identification: The user is presented with a random word spelled by the model. The user is left to type in what was spelled based solely on their recognition skills.
Our Approach
To solve the problem of realistically displaying fingerspelling, we have developed an approach that displays the movement from one letter to the next. We have created a working prototype which implements this technique as part of an interactive learning tool for practicing fingerspelling recognition.

We have implemented the core display engine which uses video clips created in a commercial animation package and stored as AVI files. Each AVI file contains a single transition from one letter to another for a total of 272 clips, This accounts for every possible transition (aa, ab, …, yz, zz) as well as a neutral position.
Avoiding Geometry Collisions
In 3D animation achieving realism has several obstacles; one major barrier in the creation of the fingerspelling animations was the amount of fingers that would pass through one another.

In an effort to eradicate these collisions the team held a ‘Fingerspelling Collision Derby’, in which each member of the team analyzed a pair of animated letters for collisions. The results were passed on to the animator who then went through the necessary letter combinations and added keyframes and subtly exaggerated the movement of the fingers in an effort to achieve realism.
Contributors

Nora Alba, Stacey Billups, Rosalee Wolfe, Cynthia Dwyer, Mary Jo Davidson, Karen Alkoby, John McDonald, Ryan Tsang, Jorge Toro, Jeffrey Young, Glenn Lancaster, Peter Schmitt, Jade Mansueto, Jerry Schnepp and Brent Shiver.

 

Research supported by a grant from The Computing Research Association Committee on the Status of Women in Computing Research (CRAW) under a program entitled Collaborative Research Experience for women (CREW). Funding for this grant was provided by the National Science Foundation's partnership for advanced computational infrastructure's education, outreach and training program (EOT-PACI) and by USENIX.



Link to Project Information Link to Demo Link to Team Info Link to Publications Send us Email

Copyright 2002-2005, DePaul University CTI
Original pages by MaryJo Davidson.
This page by Jeff Young and Nora Alba, 2005.