Scientists Reconstructed a Pink Floyd Song from Brain Activity
wsj.com
wsj.com
https://journals.plos.org/plosbiology/article?id=10.1371/jou...
Can you identify which song has been reconstructed? I could not.
The basis for song "reconstruction" claims is the song-excerpt identification analysis (e.g. Fig 4) that divides the song into 38 5-second clips and then ranks how closely each identified excerpt is to the expected excerpt--effectively, how well does the reconstruction match 38 excerpts to 38 expected excerpts. This is not reconstructing the song, just matching puzzle parts.
A nice piece of neural science, completely misrepresented in the media.
Not correct for the Dutch state news, at least. They make the same "reconstruction" claims while showing the audio fragments that don't use any input data (for 1 and 29 patients), while mentioning that it's not very accurate but at any rate still recognizable. https://nos.nl/artikel/2486771-computer-herkent-pink-floyd-n...
Based on earlier neural decoding news items I was also expecting something completely overblown, but in this case I think the reporting is quite legit, and you might just be having a knee-jerk contrarian reaction.
Pretty cool stuff even if it's early.
my expectations were low based on this comment, so I found the reconstructions much better than expected.
Impossible to say if I'd be able to ID them without any context...
...but that leads immediately to the thought that a useful metric for these things might well be whether Shazam and similar "landmark identification" tools can.
Especially those which themselves are based on perceptual encoding models of some kind... the more human-perceptual-system based, the more interesting.
I.e.: if our machine models of audio perception, can consume the output of machine models of the output of those models, and reliably track that back to the originals,
the "fidelity" of the the lossy "encoding" ("recording") is becomes less interesting.
Especially from a behavioralist/instrumentalist view.
Which is the view underneath commercialization and e.g. surveillance.
Does the <data blob> sound/look like the original? Who cares, if the machine knows it's <thing>, we're good.
Vaguely related, in the long term it will be interesting to learn if the internal modeling (sic) of music in different people differs significantly as a function of e.g. musical training, proficiency, and the like. This would imply you could tell how good a musician someone is or might be, by inspecting the qualities of their internal model.
Which then makes me wonder about grokking and eidetic memory for audio. Anecdote: Hollywood composer (and sometime pop musician and collaborator with Amy Mann) Jon Brion used to have a regular night at bar in LA, which we attended ~25 years ago. He was regularly joined by other then-famous musicians like Rickie Lee Jones. Part of his shtick was to pick up suggestions from the audience of free-associate musically, performing fragments of various popular music (e.g. 80s pop).
It was truly remarkable, even uncanny and unsettling, how good his ability was to instantly riff out not just recognizable or faithful covers, but inflect them with a large number of the idiosyncratic micro-details of the original performance. At the time I understood this as the rare intersection of something like eidetic memory for music, and, a caricaturist's talent for emphasizing exactly those features which make the thing unique.
Which I suspect are exactly the sorts of landmarks systems like Shazam tend to abstract out; and what I anticipate work like this will reveal are exploited by our own perceptual memory systems.
Which, incidentally, seem to be distinct from "episodic memory" viz. the well established manner in which people far gone to various degraded memory and dementia conditions, can be "brought back" to themselves through flawless recall and performance/singing of songs they knew, even when they are otherwise "lost." C.f. https://www.openculture.com/2013/05/the_strange_day_when_bug... as well
Uncannily, this reminds me of a popular German cartoon from 1819 titled "The Thinkers Club" (Der Denker-Club) [0] which had the slogan: How long will be thinking allowed?" (Wie lange möchte uns noch das Denken wohl noch erlaubt bleiben?)
It was meant to satirize the clamp down on liberal and nationalist ideas by restricting the freedom of assembly and press as a way to follow through the European Restauration after the Napoleonic Wars.
In the current asymmetry of giant tech corporations (in bed with government) I find the idea of further breaking any privacy walls whatsoever by trojan horsing it with mental health and social connection and declaring it a new frontier just totally backwards. I think as with liberal and nationalistic ideas back in the day we should value privacy not try just to let it go for ultimately what? Dissolve ourselves into AGI superseding Gaia: Le monde, c'est moi! But maybe I'm the old-fashioned one, here.
Missed an opportunity to use Brain Damage instead.
Part 2 has a recognizable melody and it includes the iconic chorus "We don't need no education". If you've grown up in western culture, you've probably been exposed to Part 2 through cultural osmosis even if you're not a fan of Pink Floyd or classic rock.
“Hello? (Hello? Hello? Hello?) Is there anybody in there? Just nod if you can hear me Is there anyone home?”
I'd like to see them do this experiment with Brain Damage/Eclipse, climax of Echoes, or Celestial Voices.
I wonder if hidden in that is some good prior for human-cognition-related activities, i.e. extend the token space to add human language tokens, train on that and see if it trains significantly faster than a randomly initialized model.
(This comes from a thread I started on a different submission, see https://news.ycombinator.com/item?id=37142315 and https://news.ycombinator.com/item?id=37143776)
It would be interesting if they picked a hummable tune like Peter Cottontail.
curl https://archive.is/e94Iv
curl: (35) LibreSSL/3.3.6: error:1404B410:SSL routines:ST_CONNECT:sslv3 alert handshake failure
Firefox says Error code: SSL_ERROR_NO_CYPHER_OVERLAPhttps://www.science.org/content/article/hear-classic-pink-fl...