Deep-sea test on Titanic wreck: New technology could greatly improve video communication

The Corona pandemic has given video communication an enormous boost in popularity - but sometimes poor transmission quality, dropouts and disconnections in meetings or calls try the patience of the participants. Researchers at the Karlsruhe Institute of Technology (KIT) and Carnegie Mellon University (CMU) have developed a method that allows video conferences to be transmitted over very low bandwidths and thus used even under extreme conditions. They have now tested it during a dive to the wreck of the Titanic at a depth of 4000 meters in the North Atlantic.

"Transmitting data from a depth of four kilometers through salt water without loss is extremely difficult," says Professor Alex Waibel, who conducts research on language translation at KIT and CMU, describing the challenge. This is because the natural conditions only allowed transmission from the submersible to the ocean surface to the mother ship using sonar, since radio communication does not work in salt water. The researchers have developed synthetic methods to reconstruct video from text. The sound recording is first converted to text in the submarine, then transmitted up by sonar sound pulse, where it is reconstructed from the text as video. "A synthetic voice can then be heard in the video, which is transferred to the voice of the person speaking so that it sounds like the voice of the person speaking. In addition, the video synthesis is controlled so that the speaker's lips move in sync with the sound in the process," explains Waibel, who has been researching speech recognition, speech processing and speech translation for decades.


Method allows video conferencing with very low bandwidth

In the submarine, the researchers used a powerful laptop that first converts the speech of different speakers in the dialog into text. Selected text fragments can then be sent to the surface via sonar. There, the text is then transformed back into video. A new feature is the conversion of a synthetic neutral voice into the individual voices of the respective speakers and the video synthesis, which lip-synchronously synthesizes the video of the respective speakers in the dialogue. The method allows video conferences to be transmitted over a low bandwidth: "In the future, this will facilitate communication in spoken language," says Waibel. But it is also suitable for synthesizing videos in another language or for lip-syncing videos.


Technology based on pioneering work on language translation at KIT

The technology Waibel tested on the wreck of the Titanic builds on his decades of pioneering work in language translation. Among other things, Waibel developed the "Lecture Translator", which is in use at KIT and automatically records the lecturer's talk in lectures and translates the speech signals simultaneously into written English. Students can thus follow the lecture via laptop, smartphone, or tablet.




As "The Research University in the Helmholtz Association", KIT creates and imparts knowledge for society and the environment. Its goal is to make significant contributions to global challenges in the fields of energy, mobility, and information. To this end, about 9,800 employees cooperate on a broad disciplinary basis in natural sciences, engineering, economics, and humanities and social sciences. KIT prepares its 22,300 students for responsible tasks in society, industry, and science through research-oriented university studies. KIT's innovative activities bridge the gap between knowledge and application for the benefit of society, economic prosperity, and the preservation of our natural resources. KIT is one of the German universities of excellence.

mex, 07/25/2022