Pixel smartphones automatically recognize the music around them. How it works (almost without listening)

Starting with the Pixel 2 and Pixel 2 XL (since 2017), Google-branded smartphones come with an exclusive Now Playing feature. We are talking about background music recognition, which listens to everything that is playing around, and displays the title of the song almost in real time. Using a unique feature is reminiscent of using the Shazam service and analogues, but to get the result you don’t need to first launch it, and then wait a long and tedious time to get the result – this is its main advantage over competitors. That is why it is interesting to understand how the function works “under the hood”. The engineers really did their best.

It is worth noting the existence of Ambient Music Mod right away – a recompiled application that will add the “Now Playing” function to smartphones from other manufacturers. However, its installation cannot be directly recommended: firstly, the chip will still not work fully at 100%; secondly, it is also not necessary to be sure that there are no “extra” lines in the program code. Basically, at your own risk.

How the Now Running implementation differs from the competition

→ it replaces Shazam, but works completely differently

At one time, Shazam turned out to be a real discovery (it was not for nothing that Apple bought the service). To recognize a song that is playing somewhere nearby, just launch the application and click on the treasured button – “shame”. Google engineers went even further and gave owners of Pixel smartphones access to the song title without additional steps. Many people think that for this, a pure Android device constantly listens to everything that happens around, but this is not so. In practice, using a signal processor, the device filters extraneous sounds and uses “hot” keywords (a couple of lines from the text of the composition, for example), which are taken as a search query, as a trigger for the Now Playing function. According to the description, everything is nowhere simpler – in fact, a little differently.

▶︎ Fact: the operation of the signal processor is also tied to the activation of the smart voice “Assistant” from Google. It seems that the smartphone is constantly listening to everything that happens around, hoping to wait for the phrase “Hey Google”. However, in this way any device would be discharged quickly enough, which is why a signal processor is used, waiting for the keywords.

How the Now Playing feature works on Pixel smartphones

→ the processing algorithm is not quite similar to other services

While the “Now Playing” function is working (by the way, it is disabled by default), the smartphone does not constantly listen to ambient sounds – to save battery power, the signal processor reads the keywords about once a minute (if the track has changed in the last 30 seconds, the data may not be up to date ). In this case, the algorithm uses a whole fan of a wide variety of conditions: the device, for example, should not be used at the specified moment for calls or listening to music. Some naively believe that it is after this that the mechanism for determining the name of the composition is launched – they say, the function “listens” to what is currently playing, and in real time tries to find matches anywhere. It’s not quite right.

In fact, if all conditions are met, a fragment of the track that is playing right now is recorded, lasting about 8 seconds. It turns into a so-called digital fingerprint, suitable for comparison with the database. Unlike the same Shazam, the database for comparison on Pixel smartphones is stored locally. Actually, this is why the Now Playing function can be used even without an Internet connection, which distinguishes it from any analogues. When the comparison is successful, the owner of a Google-branded smartphone sees the title of the song and can click on it to get any additional information.

With which database does the function compare what is being played?

→ Is it really possible to fit everything into a local database?!

After the previous paragraph, a logical question should arise about the possibility of storing a huge database of musical compositions in the local storage of a smartphone. After all, for example, Apple Music now has more than 90 million tracks, and Spotify definitely has at least 80 million. Shazam stores over 15 million tracks online for comparison at the click of a button. In the latter case, we are not talking about specific files, but only about data for searching for digital prints. However, this does not solve the problem – if you store all this offline, then at least several gigabytes will be additionally occupied in the smartphone’s storage.

Google uses a completely different rather “cunning” principle. The smartphone’s local library stores a limited number of songs, the list of which adapts to the current location of the owner – by and large, the necessary database is simply “cached”. For example, in the case of the United States, the device contains a list for defining up to 70,000 songs – it takes only 250 megabytes. The list is regularly updated when the gadget connects to Wi-Fi, and is overwritten when the geolocation changes globally. This is more than enough for the correct use of the Now Playing function – at least, there were no complaints about the insufficient amount of tracks on the network.

▶︎ Fact: At the time of the launch of the Now Playing feature in 2017, there were about 10,000 songs stored in the local database of Pixel smartphones. However, as practice shows, then such a number turned out to be more than enough to impress the owners of Google branded gadgets, tech journalists and simply interested enthusiasts.

Why it is very difficult to transfer the function to other gadgets

→ it will only be possible to fully use it on Pixel

To use the Now Playing feature, the signal processor of Pixel smartphones is actively used. In no other way did Google manage to implement the process of automatically detecting offline musical compositions with minimal energy consumption. Therefore, it is very difficult to transfer it to any other device running Android – each device uses its own hardware, and many hardware solutions are not at all suitable for using the described feature. That is why the developers of the same Ambient Music Mod (a redesigned background track recognition engine for third-party Android devices) emphasize that they cannot even guarantee the launch of the function on Android smartphones from other manufacturers. There is no mention of full-fledged work.