The Yandex team announced the launch of a very convenient feature for users – videos in English on many popular platforms, including YouTube, Vimeo, can be watched with automatic voice acting in Russian.
In the summer, Yandex began testing machine translation technology for video and offered users a special playlist for demonstration. Today, the new function is open to mass use. As the developers promise, now users will be able to watch almost any English-language video with Russian voice acting.
Russian voice-over translation is offered in two versions – male and female voice. Translation works in Yandex Browser for computers (Windows, macOS, Linux) and Android, as well as in the Yandex application for Android and iOS. To watch a video with Russian voice acting, you need to press a button in the player and wait a little. The translation process takes a few minutes. When everything is ready, you will receive a notification.
Translation takes place in several stages. First, Yandex recognizes speech, turns it into text, and uses neural networks to break it down into sentences. Then he determines the gender of the speaker, translates sentences into Russian and synthesizes the voice. Then the translation is combined with the video sequence using neural networks. As the developers note, this is not a trivial task at all, because sentences in Russian are longer than in English.