Continuous low-power music recognition
research.google.com
research.google.com
It feels more like yet another solution looking for a problem. Worse, it screams like a foot in the door for telling users it's okay for microphones around you to always be listening, ala Amazon Echo. It also weakly smells like it'll immediately be used to send packets of what songs were listened to and similar frequency info off-device to be collected by the Google Big Data machine to be sold to RIAA members, as yet another way of extracting ad dollars from Android.
I view it differently. Percent of battery life isn't an interesting metric in a vacuum. It's whether you easily/regularly get the phone back to a charger without it feeling burdensome. That is to say, does the phone last all your waking hours without threatening to turn off.
I own one of these fancy new Pixel 2's. I find that the phone is usually above 30% when I'm returning to bed at night. The music detection has proved useful at times, and a single percent of battery isn't going to make me sweat.
However, in a year or two, when the battery starts losing performance, you may be right. My old phone, a 5X, often dipped close to 5% by the time I got to bed. I had to go out of my way to find a charger, which was frustrating. Sacrificing yet another bit of battery for this feature would only make that worse.
I agree that the database is a major limitation, although for me it acts as a first pass to see if the song I'm listening to is popular enough to be recognized.
I would also bet the database is probably not much bigger then the shazam app itself.
I frankly just find it a well balanced solution both in terms of UX and in terms of engineering.
It takes me ~4 "seconds" (saying "one-one thousand" aloud) to do it when typing in a password (sorry, don't have a stopwatch handy). Just did it 10 times just to test. It'd be even faster with pin/fingerprint/face unlocking, or with no locking at all.
I also used Shazam and Google's "what song is this" feature for somewhere less than 5 minutes on the initial few seconds of 10 songs and my battery indicator didn't move a single percentage during my testing. Shazam scored 10/10, Google missed one (a song from an esoteric Chicago band called Terminal Bliss - all of the songs I chose were intentionally fairly obtuse to try to find missing info in the databases).
But I can say with fair assurances Google's not shipping their entire database on phones, just whatever's most popular up to some (probably size driven) limit; any given CD releases with about ten songs, any given year sees tens of thousands of CD releases (about 75,000 albums in 2010 alone). Worse, the entire feature's use case is better suited towards less popular music since you're less likely to know the song's name if it's new or esoteric - so you can almost immediately guess that those 70k songs in that database is just ~30 years of various genre chart topping hits.
All you've done is cement my convictions that this is absolutely a solution in search of a problem.
Ah, but did you test this in your quiet office or home, sitting at your desk with both hands free?
Let's try a real-world environment. Something like, say, driving in your car, in moderate traffic, with both hands on the wheel while listening to your radio. That ~4 seconds becomes a barrier-to-entry just large enough to abandon the idea of finding out what the name is to that song you are listening to.
Because, lets be honest, wondering what the song is isn't that important. But it is nice. Which is why this feature has some proponents.
I don't know what you're trying to say here. Shazam has millions of songs in their database, according to the original paper the offline mobile recognition system by google just has something over 70k finger prints. That's the big benefit of using an online system, you don't have to have all the fingerprints stored locally and also don't have to stick to the low power constraint.
very bold anecdata claim.
I would love to see Shazam numbers as to the actual median frequency per month.
I love this feature on my Pixel 2. I glance at my phone at least 2-3 a day just for this feature. To me it adds another layer of separation from being tied to me phone. Previously pulling up Shazam would bait me into engaging with notifications I didn't need to engage with.
I do agree with the privacy concerns though.
Maybe I am missing something, but this seems to be much less respectful of my privacy than an app that only listens to the ambient sound around me when I explicitly give my consent (by opening/activating the app).
Not that they have a reasonable point of view, of course.
Germany has much more stringent privacy regulations than the US does - maybe that's the difference.
If the whole process happens entirely on-device, then this reveals absolutely no information of any kind to anyone, correct?
Sure, if. Which I can verify, but I don't want to have to. I don't want to have to verify that a corporation isn't fucking me at every turn.
And it's possible that vulnerability will be discovered that lets people stream the audio somewhere, or listen to key words and send just those portions. I don't know enough about Android to know whether it's more plausible for that to happen, or if that sort of vulnerability would likely grant an attacker access to my mic even without this feature.
I don't think it's unreasonable for me to expect my device to not always be listening to me.
Wouldn't that be a risk even if this feature didn't exist? How do you know your phone isn't currently listening for music/speech and sending that data to the cloud without your knowledge?
> And it's possible that vulnerability will be discovered
Again though, that's already true regardless of whether this feature exists or not. Is there any reason to believe this feature is more likely to have a vulnerability than any other feature on your device? Why would you trust this code less than, for example, the code for your WiFi driver?
The great thing is that you don't have to! There's plenty of other people that are happy to do so and report their findings. The app would be outed as fast as other software that has attempted to send personal data without disclosure (Windows telemetry, Samsung TVs, etc.). I doubt Echo could ever get away with sending everything to Amazon's servers. And if you don't trust that others would do that, then you just don't have to use it, or use alternatives like Shazam.
I know the recognition is done on-device for some percentage of tracks, but it’s been unclear to me what happens to the running tally of times/locations/tracks identified afterward.
[0] http://www.androidpolice.com/2017/10/26/add-track-id-history...
A case of nominative determinism?
> pronounced "BEH-awe-t"
[ed: i actually think I'd have plenty of room for 70k songs in high quality ogg vorbis vbr or equivalent - that'd frankly be more interesting... ]
Hard to compete with that if you aren't integrated at an OS level.
You say less power to get us to buy more?
How about saving all that energy and using it for real world building.
> Since everything runs locally on the device without sending either audio or fingerprints to a server, the privacy of the user is respected and the whole system can run in airplane mode.
This is far more respectful of user privacy than we usually see from Google. I, for one, am impressed.