All browsers are very similar to each other (especially those that run on the same engine). However, the Yandex web browser stands out from all others mainly by the off-screen video translation function – the browser dubs the original audio track in the video, but in Russian, and plays it over the original video. Fortunately, this option can also be used in other web browsers thanks to a special script hosted on GitHub. In this article, we consider how to install it, what it is in general and what are restrictions.
How to set voice-over translation of Yandex video
Almost any program is supported: not only Chrome, Edge and others on the Chromium engine, but also Firefox with Safari.
- Install the Tampermonkey extension (it’s paid for Safari, you can replace it with the UserScripts utility) – this is a script manager that will allow you to run the desired algorithm for offscreen video translation.
- Download a special script – by clicking the “Install” button, the tab will close.
You can check if it is installed in the extension menu by clicking the “Control Panel” button. Just in case, the script file is attached to this article (at the very bottom) – it can be installed manually if something happens to GitHub.
Now, when you open a video on YouTube, a red circle indicator will appear on top of the Tampermonkey extension icon, and a widget will appear above the video player offering to perform a voice-over.
If you expand the advanced settings, you can enable automatic translation of videos and adjust the volume of the translated voice acting.
Briefly: how Yandex voiceover works
Before testing this technology, it may seem that it is simple to implement and does not work well: voice translation has long been in the same Google Translate, and its machine speech leaves much to be desired. But in practice, everything is quite the opposite – this is a combination of very complex algorithms, and the output is really excellent.
Voice-over and voice acting are carried out in several stages:
- Speech recognition and processing
At this stage, the algorithms try to process the speech – there may be more than one speaker in the video, and if you simply convert all the words you hear into text and then translate it, you will get porridge. The neural network tries to group the words into semantic segments, place punctuation marks (this is very important even in a video with one presenter) and clean up the garbage.
If the video has manually added subtitles, then the neural network uses them, and not the original audio track.
- Determining the gender of the speaker
Yandex technology can voice videos with a female voice (Alice) and a male one. To do this, the algorithms must determine the gender of the people present in the video.
- Gender-sensitive machine translation
In fact, at this stage, a standard translation of an already finished text takes place, but with a nuance: the algorithms take into account the gender of the speakers in order to choose the correct pronouns, as well as the correct forms of verbs and adjectives.
- Speech synthesis taking into account the difference of languages
The translated speech is on average 10-30% longer than the English one. Because of this, there can be a noticeable out of sync between what is happening on the screen and the voice acting. Smart Yandex technologies try first of all to reduce unnecessary pauses between phrases and words, and only then, if this is not enough, they speed up the Russian voice acting.
Although some of these steps happen in parallel, the whole technology is very complex and resource intensive, which takes time before the voice-over is ready.
What are the limitations compared to Yandex.Browser
Although the script in this article uses Yandex technology, the unofficial voice-over works somewhat worse in other browsers due to certain limitations.
- Popular videos don’t translate instantly
In Yandex.Browser, voice-overs for popular videos with a large number of views happen instantly (apparently, the company stores the finished voice-overs for these videos on its servers, and does not do it every time anew). When using the script, even popular videos go through all stages of voice-over, so you have to wait – about 1 minute for every 10 minutes of video.
- Only YouTube supported
Yandex.Browser can translate public videos on many popular platforms, but the script only works with YouTube videos (also public ones).
- Not receiving notification when video is ready
Since voice-over does not happen instantly, Yandex.Browser sends users a notification that it is ready – after requesting a translation, you can even close the tab. The script does not know how to do this, you have to pause the video and wait until everything is ready.
- The original audio track is not muted
Yandex.Browser automatically mutes the original sound (but doesn’t remove it completely). In the case of using the script, the volume of the source video will have to be lowered manually, otherwise the audio tracks will overlap each other.
- Translation subtitles are not available
In addition to the translated voice acting, Yandex.Browser is also able to show subtitles in the original language – when you click on a word, its translation is displayed inside the sentence. This function does not exist in the script.
- Sometimes there are bugs
When using the script, it happens that the inscription about waiting for the translation freezes – you have to reload the page several times in order for the voiceover to become available (but it appears immediately, you don’t have to wait again). There is no such problem in Yandex.Browser.
Despite all the limitations, the script performs its function – re-voicing videos in Russian, and does it very well. Like the technology in Yandex.Browser, it supports not only translation from English, but also from German, French, and Spanish.
Source: Trash Box
Donald-43Westbrook, a distinguished contributor at worldstockmarket, is celebrated for his exceptional prowess in article writing. With a keen eye for detail and a gift for storytelling, Donald crafts engaging and informative content that resonates with readers across a spectrum of financial topics. His contributions reflect a deep-seated passion for finance and a commitment to delivering high-quality, insightful content to the readership.
