How Animals Map 3D Spaces Surprises Brain Researchers
quantamagazine.org
quantamagazine.org
Their earlier work involved studying rat and bat navigation in unnatural environments - a "2D" space which was really a 3D space constrained to be along a single level. This forced or even allowed the rat brains to dumb things down to a situation where they had no need or opportunity to consider things outside that unnaturally limited space and so their brains naturally optimized for that simplistic situation.
You have dumbed down their environment to the point where you have established a minimal ability that they need to master in order to function and navigate that environment.
Then when you discover that their memory encodes navigational information from this dumbed-down, unnatural environment in a regularized grid, this should be no surprise. They are minimizing the energy required to thrive in that environment.
Once you allow them to navigate in a more natural environment you should expect to see this regular grid disappear since their options for reaching a destination are no longer constrained to a single path along most of the route to their destination. Their brains will have to incorporate many more clues to guide their route selection and those clues could be time-varying as in the case where they are tracking the source of an odor as they search for food so they will need to monitor air flow and direction, odor intensity, potential obstacles between themselves and the source, alternate routes which present themselves as they move through space, threat detection from predators or bait traps, etc. In short, there so many variables that will need to be evaluated once you remove the flat plane constraints that led to their grid discovery that it should be no surprise to discover that a more complex environment uses different optimizations, some of which are encoded in a regularized notation and others, probably situational events on the path-picking decision tree, are more random.
I just thought it was funny that the researchers were surprised to see such a large difference in how rats handle complexity when we know going in that rats are pretty intelligent. Those rats probably know each researcher, have favorites, remember routes, and get bored with those tasks like we would be. It becomes muscle memory for them so they go through the motions to get the treat they know is waiting.
In their case, they started with a generalized "2D" simple case and concluded after study that a real-world scenario would look very similar or would be a neat permutation of their original grid-optimized discovery. They found instead that they had discovered that the rat brains optimized their navigation to the difficulty of the task and that once you add enough difficulty, the brains spread the computational resources in a different optimization in order to handle the newfound complexity. Patterns were still detectable but they did not follow the original hexagonal grid optimization discovered in the initial tests.
This is, as you say, a process of learning one small piece of a bigger puzzle. In this case, they found that the complexity of the puzzle is higher than the initial guess they made after collating and analyzing all the data from the simple experiments. Useful things were learned but I was attempting to note that they have apparently made assumptions based on a very simple case that turned out not to be entirely accurate and that they appear to be surprised by that instead of accepting that the simplicity of the tasks used to generate the data may have biased the results. No doubt they will continue to add complexity and learn new things, and win more awards for their pioneering work.
That's a post hoc rationalization. All those complicating factors can exist in 2D too. Are you predicting that the 2D hexagonal structure will also be lost in a 2D environment with odors/etc., and this conclusion about it being 2D vs 3D is false?
Very true. Everything I typed is in fact a post hoc rationalization. Sadly enough, I had to look that up since Latin is not my first language. LOL. Have a chuckle at my expense. I promise I don't mind at all.
> Are you predicting that the 2D hexagonal structure will also be lost in a 2D environment with odors/etc., and this conclusion about it being 2D vs 3D is false?
I'm no rat expert and I don't play one at my desk.
I am saying that the discovery that navigating in a true 3D situation, a much more complex situation than navigation of a planar structure with a floor and ceiling and no opportunity to take the stairs to the next level, involves an order of magnitude more complexity and that as a result they should not have expected to see the grid structure preserved and echoed in the brain as a spherical 3D hexagonal grid. There were indications that the grid stored contextual information along with positional information and that changing one of those things warped the grid in ways they didn't understand (2D case).
This in itself should've been a clue that adding another dimension or degree of freedom of movement could have its own very different footprint.
"Bird-brain" may be an insult to people, but I'd bet their 3-D processing capabilities blow away an RTX 3090 - especially when considering power use and heat.
We have a lot left to learn from them.
having done research in this area, i’m fairly confident in my assertion, but am open to any new information/evidence you may provide.
the rest of your statement provides one evolutionary mechanism by which gibbons became good brachiators, not evidence that they’re currently not.
Real world environments are rarely really 3D, and a 2D surface is perhaps the sweet spot in complexity/degrees of freedom for most practical cases? This would explain why 3D mazes are so much more disorienting than 2D mazes, for example.
Your crumpled space-filling surface makes me think of manifolds.
Machine-learning researchers often speak loosely of e.g. the "manifold"* of possible images of naturalistic scenes embedded within the space of all possible images. The networks are presumably learning a lower-dimensional representation of the world than the dimension of all possible combinations of sense impressions. If you have a 100 pixel by 100 pixel image and each pixel can have 256 intensity levels, then that's 256 to the power of ten thousand possible distinct images. If each image is a point in an abstract space of all possible images, then that space has 256 to the ten thousand dimensions. But the vast, vast, vast majority of the volume of that space corresponds to images that just look like static to humans. So the thinking is that we internally represent images as some learned non-linear transformation to a much lower dimensional set of features that actually correspond to stuff we experience / see.
A really simple canonical example is the Swiss Roll dataset. You only need two parameters (numbers) to specify it fully, but you can embed it (nonlinearly) in a 3D space. http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html
In terms of neuroscience, there are competing ideas (as always). There's a body of work that tries to show that as we learn, the brain encodes our high-dimensional sense-impressions in the lowest possible (most efficient) internal representations. However, some recent work seems to imply that instead the brain uses as high a dimension as possible, but up to some limit that demarcates the transition between being differentiable and not differentiable. They found a beautiful power-law: https://www.biorxiv.org/content/10.1101/374090v1.full
* When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. In contrast, in the computer graphics and meshing literature people speak of non-manifold geometry which is isolated points and lines that you can't triangulate with 2d or 3d elements. I.e. 1d or 0d elements.
So they mean "subset" when they say "manifold?"
E.g. Generate 3d points that are on a 2d plane +- some small random offset normal to the plane. If the points are isolated, without each having a local neighbourhood, it's not technically a manifold. But, you could describe the data-set as lying on or near a plane to within some tolerance.
So they're trying to describe something that is much more specific / strongly constrained than an arbitrary subset, but it doesn't meet the very stringent (and frankly idealized) requirements of a manifold. I wonder whether there is a math-object to describe this?
EDIT> Maybe it's just a matter of saying "close to some lower-dimensional manifold", rather than "on a lower-dimensional manifold."
Here is how I would phrase it: there is a probability distribution in the higher-dimensional space that expresses P(this image | given that it's a natural image). The level sets of the probability distribution are manifolds.
Not at all! All of this is far more advanced than my knowledge on the topic, and very interesting! Thanks for sharing
That idea of representing in the highest dimensionality possible, with some constraint is also very interesting. In that case perhaps the 3D space is being represented in a higher dimensional form that makes it more convenient for some neural processing purpose (e.g. just like we use homogenous coordinates) The 2D-2D case is then just a happy coincidence where the highest representation that makes sense maps 1 to 1 with the actual data.
Going back to the image example, a 100x100 pixel image is 2D in that it can be shown on a screen, or printed on a page, or laid flat on a plane. But the (abstract) space of all possible images is #intensities_per_pixel to the 100x100 = 10000 power.
It's abstract in that each point in this space is not a location in the external world, but specifies one particular image. If you're familiar with phase spaces or configuration spaces from physics, it's like that.
The other thing is that we don't seem to have direct access to a 2d or 3d position-tracker sense. So instead, we have to build up some internal representation for navigation, based on our senses which as outlined for images are much higher-dimensional than the physically allowable positions in the world they're sensing. Robotics SLAM is one approach.
Then finally, there's the dimensionality of the neural representation itself. Let's say that your internal navigation map is represented by the firing-pattern of a population of neurons (i.e. more than one). Define a time-step, say one millisecond. For simplicity, consider each neuron to just be have an "activity" in the range 0.0 to 1.0 per time-step by e.g. counting the number of spikes emitted per time-step and dividing by some number to get things in a nice range. So now you can represent the activity of a neuron by one number per timestep. If you have a population of 100 neurons, then that's a 100 number string. But ... you can also think of it as a point in a 100 dimensional space. Each point is one particular pattern of neural activations. The entire space is all of the possible patterns of neural activations. Again, this is not a space in the sense of physical positions in the world or in the brain. It's an abstract space. But all the vector space math works.
SO we're perceiving 3D by means of retinas that take way more than 3 measurements at an instant, and maybe our brain is finding correlations so that it can represent these in a distributed way in << input_dimensions, but > world_dimensions.
> But the grid cells’ firing wasn’t entirely random either. Instead, there was local order: For each grid cell, the places where it fired weren’t arranged in a perfect periodic lattice, but the distances between them were too regular to be merely a matter of chance. Rather than the neat stack of oranges, the researchers were seeing something similar but less orderly, more like marbles filling a box. “They’re always stuck in some local minimum, such that there is not a lattice,” Ulanovsky said. “On the other hand, the local distances there are fixed, because all the [marbles] are sort of touching their neighbors.”