Project 1-375 Morocco

Partnerships for enhanced engagement in research (PEER) SCIENCE
Cycle 1 (2011 Deadline)

Assistive technology for improving literacy among the deaf and hard of hearing

PI: Abdelhadi Soudi, Ecole National de l'Industrie Minerale

US Partner: Corinne Vinopol, Institute for Disabilities Research and Training, Inc.

Project Dates: June 2012 - February 2014

Project Overview

Moroccan Sign Language (MSL) is a poorly resourced language with communication problems compounded by a severe lack of interpreters. It is also worth noting that Sign Language research in developing countries (including Morocco) is still in its infancy, so standardization of communication tools has rarely been attempted. The high rate of illiteracy among the deaf community and the lack of sign language interpreters often deprives members of this community of critical information, with serious consequences to their welfare, safety, health, opportunities, and rights.

Dr. Soudi with sign language interpreter.

Dr. Soudi and his team.

The U.S. partner, Dr. Vinopol, during her visit to Dr. Soudi’s lab in Morocco.

This project will be carried out in collaboration with the U.S.-based Institute for Disabilities Research and Training, Inc., which is currently developing a translation engine that provides reciprocal translation between American Sign Language (ASL) and English. The project is aimed at creating a robust assistive technology that would be of accessible cost for deaf individuals in Morocco and their families, service providers, educators, or businesses that employ them or have them as customers. The technology should be able to function as both an instructional tool to improve the literacy of deaf children and adults and as a real-time translation device between MSL and standard written Arabic. This translation technology will accommodate a variety of input and output options, including input for standard Arabic text (typing, scanning, screen text transfer); input for MSL (a sensor-enabled glove capable of reading the finger and hand movements of sign language and camera integration); output for standard Moroccan text (standard Arabic text); and output for MSL (sign graphics, sign video clips). By collaborating on this PEER-supported project, the researchers hope to determine whether the gesture capture technology currently under development for ASL is applicable to MSL and whether the English-to-ASL/ASL-to-English translation engine and hardware currently under development are relevant for Standard Arabic-to-MSL/MSL-to-Standard Arabic. The results obtained should benefit both sides in their efforts to use technologies to assist marginalized disabled communities in both Morocco and the United States.

Final Summary of Project Activities

As part of the engine development, the PI and team created a database of 3,000 MSL signs and 8,500 Arabic words. The work included, among other elements, drawing sign graphics and videotaping sign clips, incorporating Natural Language processing tools into the engine and testing the technology at deaf schools in different parts of Morocco. In addition to the development of the translation engine, the research team published several proceeding papers and the project was featured in Odyssey, an award-winning children’s science magazine.

The team has established ongoing connections with several Deaf associations in different regions in the country The research team also developed a set of useful educational resources for the Moroccan Deaf community and their parents: a quiz-maker with automatic Arabic-to-MSL translation, introductory instruction in MSL for parents and service providers, stories and interactive instructional games for Deaf students in both Arabic and MSL.

The researchers have also begun investigating a new research direction in using 3D avatars to produce signs in the translation engine. As part of the project, the Moroccan PI taught the software engineer and the Standard Arabic linguist Natural Language Processing techniques. The project’s sign language interpreter and linguist and the deaf signers and graphic artist explored many issues on the mechanics and linguistics of Moroccan Sign Language that were previously unexplored, and the results of this work will be disseminated to sign language linguists and educators in deaf schools.

The PI and team organized a project awareness day in Rabat to present the outcomes of their project which was attended by over 350 deaf people, 20 organizations and the USAID Morocco Mission Director Dana Mansuri. The government ministry in charge of the disabled is coordinating the formal integration of the technology into mainstream educational courses.

Publications

Soudi, A., Vinopol, C. 2016. Creating instructional materials with sign language graphics through technology. Qatar Foundation Annual Research Conference Proceedings 2016: ICTPP3257 http://dx.doi.org/10.5339/qfarc.2016.ICTPP3257.

Soudi, A., Vinopol, C. 2016. A digital Moroccan sign language STEM thesaurus. In Proceedings of the 7th Workshop on the Representation and Processing of Sign Languages: Corpus Mining. Language Resources and Evaluation Conference (LREC) Portorož, Slovenia, May 28, 2016.

Soudi, A. Vinopol, C. and Er-rady, A. 2014. Virtual reality technologies for deaf education, In Proceedings of the international conference Building International Networks for Enhancement of Research in Jordan, Amman, April 3- 5, 2014.

Soudi, A. Vinopol, C. and Er-rady, A. 2013. Assistive technologies for improved literacy among the deaf and hard of hearing. In Proceedings of Qatar Foundation Annual Research Conference, November 24-25, 2013, Doha, Qatar.

Soudi, A. Vinopol, C. and Er-rady, A. 2013. Standard Arabic-to-Moroccan sign language machine translation: does translation from a morphologically rich language reflect the interpreters' allocation of high cognitive resources? In Proceedings of The Fourth Conference of the Scandinavian Association for Language and Cognition (SALC IV), University of Eastern Finland, Finland, June 12-14, 2013.

Soudi, A. 2013. Robust technology facilitating interaction between the hearing culture and the deaf culture. In Proceedings of the session, Interculturalism, Migration and Multiculturalism of the conference "At the Crossroad between Africa, Asia and Europe: Challenges and Perspectives," Venice, Italy, May 29-June 1, 2013.

Back to PEER Cycle 1 Grant Recipients