Silent Communication

Telephoning silently without disturbing any bystanders? Speaking in a foreign tongue? Giving people a voice who lost it due to illness or accidents? Computer Scientists at the Karlsruhe Institute of Technology (KIT) are working on a groundbreaking technology that opens up a variety of possibilities. “Speech Recognition by Electromyography” is the official labeling for a new approach that allows the recognition of speech from the electrical activity of the facial muscles. The application scenarios are manifold and could change human communication habits in the future.
The prototype was presented at the Annual Meeting of the American Association for the Advancement of Science (AAAS) from February 17 – 21 in Washington, DC.

Professor Tanja Schultz is head of the Cognitive System Laboratory at KIT and responsible for the project: “I came up with the idea a couple of years ago when I was sitting on a train. I grew annoyed by a fellow passenger who was loudly speaking into a cell phone and started thinking of ways to change this. Silent speech interfaces seem to be a great solution for this.” Schultz and her team came up with a prototype that captures the electrical potentials of the articulatory muscles with surface electrodes to recognize spoken speech. This allows to recognize and transmit silently uttered speech. The technology is based on Electromyography, i.e. the capturing and recording of electrical potentials that arise from muscle activities.

Speech is produced by the contraction of muscles that move our articulatory apparatus. The electrical potentials are captured by surface electrodes attached to the skin. The analysis and processing of these signals by suitable pattern matching algorithms allow to reconstruct the corresponding movement of the articulatory muscles and to deduct what has been said. The recognized speech is output as text or synthesized as an acoustic signal. Since electromyography records the muscle activity rather than acoustic signals, speech can be recognized even if it is uttered silently, without any sound production.

The accuracy of the system is very encouraging: “The prototype currently can recognize more than 2000 words vocabulary and give up to 90% accuracy”, states Schultz and believes that in five, maybe ten years, this will be a usable, everyday technology.

The computer scientists at KIT envision several practical applications for Silent Speech Recognition:

1) Silent Telephony: Silent speech recognition allows for silent communication without disturbing any bystanders.
2) Transmitting Confidential Information: The system allows for seamless switches between silent and audibly spoken speech and thus enables to safely and securely transmit confidential information such as passwords and PINs.
3) Robust communication in adverse environments: Since electromyography relies on signals captured directly at the human body, the signal is not corrupted by noisy and adverse conditions.
4) Speaking in a foreign tongue: By feeding the output of silent speech recognition into a component that translates from one language to another, native speakers can silently utter a sentence in their language, and the receivers hear the translated sentence in their language. It appears as if the native speaker produced speech in a foreign language. This technology is developed in cooperation with Prof. Alex Waibel’s lab, also at the Institute for Anthropomatics.
5) Help for disabled persons: The system can assist people who have lost their voices due to illness or accidents.

The Cognitive Systems Laboratory is part of the newly founded Institute for Anthropomatics, which centers around the vision of establishing intelligent and human-centered systems in our everyday life. The term “anthropomatics” was coined by informatics professors in Karlsruhe about 10 years ago and refers to the science of the relationship between humans and machines. It is aimed at enabling machines to communicate, interact, and act autonomously like human beings.