Fully convolutional naturalistic video reconstruction from brain activity

Yes, none of that.

This is just picking up the imprint of a direct perception; it is essentially equivalent to just photographing the reflection of the TV in your eye.

This isn't "decoding" any internal information actually present anywhere in the brain. It is showing, at best, that visual input "lands somewhere" in the brain -- but is not showing any of the brain's processing, nor any of the impact of the visual.

It's just, essentially, looking at the terminal node of the eye.

The model can't guess your thoughts unless it is trained with your brain activity (fMRI, EEG). After training, you feed it more brain imaging and see what comes out.

I don't know if this could be applied to, say a prisoner in Guantanamo, strapped to a fMRI and forced to look at images while the model trains on his brain activity. The subjects in these experiments are cooperative, while a uncooperative subject could try to blur his sight, meditate, think of loud songs, a movie, food, etc., to dissociate their vision from their thoughts.

One thing I don't understand is that the dataset already comes with a "train dataset" (120 recordings) and "test dataset" ("Recordings of 22 repetitions of 7 narrative test set videos"). Why are they not just setting aside a randomly selected subset of the training data for testing?

I'm unclear on whether the testing data contains some of the same video clips that were shown in the training data, repeated literally? It seems it would be best to test the model on a previously unseen image, to ensure that it really learnt the mapping and is not just "memorizing" the responses.

That's pretty impressive. This new work seems like a step forward because it's getting the information from slightly further downstream - the visual cortex which is what the LGN feeds into. But it's still not magic because it was already known that there's a mapping between points in the 2D field of view and points in the 2D layers of the visual cortex. That mapping is already known to be very simple for the V1 layer which is one of the 3 layers of the visual cortex that this technique uses. It also uses the V2 and V3 layers that seem to me to have a higher level representation and doesn't perform as well on them.

Still, this seems like a step towards reading people's imaginations, and they hint at that in the conclusion with "While admittedly the promise of algorithms that reconstruct internally generated or externally induced percepts is yet to be fully achieved".

Not that I know what I'm talking about. Feel free to point out any minunderstandings I've shown.

This is the picture: https://www.biorxiv.org/content/biorxiv/early/2021/02/03/202...

Edit: I just realized from the other pictures that they're frames from Doctor Who, making the resulting aliens somewhat fitting.

That is not probe, but a prosthetic replacement for their exobrain. The Ood are generally good in Doctor Who, it's humans who are horrible abductors for them. If that image is from "Pond Life" mini series as suggested by the title on the first two frames on Figure 2, that's the butler of the eponymous family.

I suggest you to just watch the relevant episodes as an exposure therapy:

https://en.wikipedia.org/wiki/Ood#Appearances

If I'm not mistaken the source material actually has a probe wielding alien of some kind, you may be seeing that

We are there already.

You are looking at an example of mind-reading.

But it's an early version of mind-reading that requires expensive fMRI machines to get data from the brain and uses deep learning models that don't work that well... yet. There are other early examples of mind-reading, e.g., Neuralink implants.

From here on, it's all about improving the technology -- making it better, faster, cheaper, smaller, etc.

Given how the likes of Facebook are investing into mind reading tech... Not far enough.

Here's an example video:

https://twitter.com/lynnle_ai/status/1357770339279917057

Appears to be the latter. However, a computer is also "just" a technological curiosity once we know how transistors work.

I think there's a really interesting topic here. What does it mean to understand? Many phenomena that humans can work with in practice are beyond individual understanding.

For instance, I doubt any single human fully groks the design and manufacture of modern CPUs, for instance, there are mathematical proofs that can only be verified by machine, etc.

We frequently say we understand some physical phenomenon when we have math that explains it. But complex systems seem difficult to describe in less than a simulation and all of those numeric details can't fit in one human head.

I suspect that as we explore complex phenomena like brains, ecosystems, economies and weather systems, that we'll have to get used to a 'proxy' understanding where we can work with them only through the use of models that don't quite fit in human brains.