Hacker News Reader

This seems rather optimistic to me. There are days when I count myself lucky to be able to debug my own code. And it's maybe about seven orders of magnitude less complex. And has comments. And unit tests.

I'd be willing to bet that once we've achieved the ability to scan and simulate brains at high fidelity, we'll still be far, far, far away from understanding how their spaghetti code creates emergent behaviour. We'll have created a hyper-detailed index of our incomprehension. Even augmented by AI debuggers, comprehension will take a long long time.

Of course IAMNAMSWABRIAJ (I am not a mad scientist with a brain in a jar), so YMMV.

> Of course IAMNAMSWABRIAJ (I am not a mad scientist with a brain in a jar), so YMMV.

How can you be so sure of that?

Because he stated he's not a scientist with a jarred brain in his possession (to his knowledge/current memory state), not that he has his own brain in a jar, which, while possible, is most unlikely.

Yes, I'm fun at parties.

I'm not sure if it was intentional or not, but there is a clever meta joke hidden in your comment that made me actually laugh. Kudos.

Really appreciate the feedback. Always good to know when something works as intended.

Yep, I'm definitely making no positive statements as to the absolute location of my brain, or its state of empicklement.

This depends on availability of debug/test/research environment for brain images.

There are 20M sw developers on this planet. If 100k of them had daily available dev environment for brain images, then things would progress extremely fast.

Well, training a neural network is not significantly different from how you train a brain. You don’t need to know the inputs as long as it produces the right outputs.

> Execute it step by step, take lots of measures of all parameters, save/reload state, test every possible input and variation.

This assumes that simulation can be done faster than real time. I think it will be the other way around: the brain is the fastest hardware implementation and our simulations will be much slower, like https://en.wikipedia.org/wiki/SoftPC

It also assumes simulation will be numerically stable and not quickly unsable like simulation of weather. We still can't make reliable weather forecasts more than 7 days ahead in areas like Northern Europe.

It's the fastest we currently have but pretty unlikely to be the fastest allowed by the laws of physics. Evolution isn't quite that perfect - e.g. the fastest flying animals are nowhere near the top flying speed that can be achieved. Why would the smartest animal be at the very limit of what's possible in terms of speed of thinking or anything else?

The brain is pretty much guaranteed to be inefficient. It needs living tissue for one, and we can completely dispense with anything that's not actually involved in computation.

Just like we can make a walking robot without being the least concerned about the details of how bones grow and are maintained -- on the scales needed for walking a bone is a static chunk of material that can be abstracted away without loss.

C elegans is a small nematode composed of 959 cells and 302 neurons, where the location, connectivity, and developmental origin/fate of every cell is known.

We still can't simulate it.

Part of the problem is that the physical diffusion of chemicals (e.g., neuromodulators) may matter and this is 'dispensed with' in most connectivity-based models.

Neurons rarely produce identical response to the same stimuli, and their past history (on scales of milliseconds to days) accounts for much of this variability. In larger brains, the electric fields produced by activity in a bundle of nerve fibers may "ephaptically couple" nearby neurons...without actually making contact with them[0].

In short, we have no idea what can be thrown out.

[0] This sounds crazy but data from several labs--including mine--suggests it's probably happening.

> C elegans is a small nematode [...] We still can't simulate it.

This for some reason struck me as profoundly disappointing. I have a couple neuroscientist friends, so I tend to hear a lot about their work and about interesting things happening in the field, but of course I'm a rank layperson myself. I guess I expected/hoped that we'd be able to do more with simpler creatures.

If we can't simulate C elegans, are there less complex organism we can simulate accurately? What's the limit of complexity before it breaks down?

c. elegans is pretty much the only one we fully mapped. (Possibly some fish larvae, too? Recall fuzzy)

But given that we can't even fully simulate animals with exactly zero neurons (Trichoplax), I'd say the current limit is "we can't". It's literally the world's simplest animal, and we're far from understanding how it works.

So, probably no brain uploads by 2031 ;)

I don't think there's anything we can simulate "completely", in the sense that a fire-and-forget model would subsequently go onto have a typical life.

The stomatogastric ganglion might be the closest. It is a network of three dozen neurons in the crustacean stomach. Like the worm, the wiring diagram is completely known and the physiology is easier to measure. Despite being very simple, it can generate intricate patterns of activity in the stomach muscles that let the crab/lobster/etc eat. Scholarpedia has the diagram and some references (http://www.scholarpedia.org/article/Stomatogastric_ganglion) Eve Marder, who has done a lot of pioneering work on this circuit, wrote a book (Lessons From the Lobster) that I'm looking forward to reading.

Don't be disappointed! A lot of media coverage tends to present new results as "we're almost there." In most cases, I think that's nonsense, but it's also exciting to think how many things there are left to discover and how fascinatingly complex the world is.

> We still can't simulate it.

Interesting. Can you give a rough estimate of how much effort has been put into studying it (wall time, researcher-years, money) and how much progress has been made?

Also, is there any estimate of how similar C. elegans neurons are to those of other species, such as humans?

I’m not sure how to put a reasonable number on it, especially the simulation part, but C. elegans is a very common model organism. It’s maybe not as well-known as mice or rats, but probably in the top ten most-studied organisms. Here’s a nice review (https://www.nature.com/articles/nrg2105); a quick glance at the WormBook might also give you a sense for the breadth and depth of what’s been done[0]. http://www.wormbook.org/

Neurotransmission in C. elegans is unusual. They use a different set of neurotransmitters; this isn’t that odd—-insects also use a slightly different set than humans, and their role even flips in many animals (including mammals) during development. The weirder part is what those neurotransmitters do. In other animals, neurons produce stereotyped all-or-none “spikes” of electrical activity. Until quite recently, it was unclear whether C elegans neurons did too. This News and Views (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3951993/#R29) does a nice job describing plateau potentials and the reasons that C. elegans neurons might differ (namely, they’re very small). A few years later Cori Bargmann’s group discovered that the AWA neuron fires something more akin to a “classical” spike—-sometimes. It also uses calcium instead of sodium. https://www.cell.com/cell/fulltext/S0092-8674(18)31034-1

This might complicate simulations a little bit, but these differences are also understood pretty well, and the much smaller nervous system more than offsets them.

[0] I work at the polar opposite end of neuroscience—-large animal neurophys—-but I’ve always been a little jealous of how friendly and tight-knit the C elegans community seems. They have a lot of great open resources.

May I ask you on your opinion on non-faithful simulations (as a layman with only superficial understanding of the topic)? Would not some heuristic or discrete signaling enough for approximating the actual working? Are non-direct effects as you have written in a previous comment significantly modify the responses?

Simulations and modelling are great! They are a powerful way to generate and explore hypotheses.

Neuroscience has a lot of success this way. The properties of cones, the cells that detect colored light, were accurately modeled from behavioral experiments (e.g., people matching paint chips) in the 1800s, even though we didn't have the technology to measure them until the 1960s-1980s. The Hodgkin-Huxely model of action potential generation from the 1950s is still incredibly useful and predicted aspects of ion channel structure that took decades to confirm. David Robinson measured the physical forces produced by eye movements and used that to predict, and then reverse engineer, huge aspects of the "oculomotor plant". Real neurons have incredibly complicated behaviors, and yet artificial neural network models, where those are reduced down to a sigmoid or ReLu, have been very informative, first in the 1980s and then again today.

On the other hand, attempts to produce highly realistic simulations haven't really panned out. The Blue Brain Project has spent tons of time, money, and compute on very detailed simulations, but I think the consensus is that we have not learned a ton from these efforts. One of the most interesting outcomes (IMO) is actually the atlas that was built to build the model. There are probably many reasons for this difference, ranging from technical things like uncertainty propagation to very human expectations about what a model "should" be able to do.

In the specific context of C elegans, there's some data showing that diffusing peptides are essential for certain worm behaviors (e.g., Chen et al, 2013: https://www.sciencedirect.com/science/article/pii/S089662731...). The other mechanisms I mentioned are certainly there too. How much they matter is still up in the air: even for very simple organisms, we're still at the stage of figuring out what we don't know!

Thank you for the really informative answer! It was a pleasure to read!

Do you happen to have any reference you could share about this re [0]?

Of course!

The observation that one neuron can alter the activity of a nearby one is old as dirt. Emil du Bois-Reymond observed it in the late 19th century, but I don't know of anyone trying to quantify it until Katz and Schmitt (1940) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1393925/ and Angelique Arvanitaki (1941) https://journals.physiology.org/doi/abs/10.1152/jn.1942.5.2...., who named it. There are some other reports in squid (Ramon & Moore, 1978) https://pubmed.ncbi.nlm.nih.gov/206154/, rat cerebellum (Korn and Axelrad, 1980) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC350252/, and others. This review by Anastassiou et al. (2011) might be a good place to start https://www.nature.com/articles/nn.2727.pdf?origin=ppub or this Scientific American article about a paper by my grad school neighbors (https://www.scientificamerican.com/article/brain-electric-fi...)

In parallel, people have asked whether external electric fields can be used to alter neurons' activity, which is even older: a Roman physician in 46 reportedly cured headaches by applying a live electric fish to patients' heads. The idea of using electricity to improve mental function has waxed and waned ever since, with the most recent peak around ~2015 or so. Terzuolo and Bullock collected some of the first data on this using crayfish axons in 1956 (https://www.pnas.org/content/42/9/687) and subsequent experiments by Deans et al. (2007), Radman et al. (2007-9), Ozen et al. (2010), and Frolich and McCormick (2010) found similar results using in vitro and small animal experiments. In parallel, people went absolutely wild with human studies of transcranial electrical stimulation (TES), a family of techniques including tDCS (w/ direct current) and tACS (alternating current). While some of the results have been exciting, they have not always been reproducible (Horvath et al, 2015ab) and some work suggested that the previous work relied on fields much stronger than those achievable in humans (Voroslakos et al, 2018).

Together with some awesome collaborators, I set up a non-human primate model that let us test tES under conditions that closely match those found in humans: like macaques (and unlike rodents), we have big, convoluted brains in thick bony skulls and comparatively sparse neural networks. We found that tDCS could affect neural circuits (i.e., LFP oscillations) and behavior (Krause et al., 2017) https://www.cell.com/current-biology/pdfExtended/S0960-9822(... and single neurons, even in deep brain areas (Krause, Vieira, et al. 2019) https://www.pnas.org/content/116/12/5747.abstract [0] The fields we used were much weaker than those produced by some parts of the brain itself (~0.3 - 1 V/m vs ~4-8+ V/m), so it suggests that ephaptic mechanisms are probably pretty common.

I'm pretty confident in those results, but--to bring things back to the original topic--our recent experiments suggest that getting tES to do exactly what you want, when and where you want it, will take some cleverness and a lot of simplifying assumptions tend not to hold up.

[0] The missing full references above are in these two articles' bibliographies.

> We still can't simulate it

302 neurons seems very easy to simulate, even if the connectivity graph were orders of magnitude more complex.

Simulating correctly... that is another thing, I'm sure.

anything that's not actually involved in computation.

This doesn't seem like a very easy problem to solve.

Human synapses top out at <100 Hz and the human brain has <10^14 of them. Single silicon chips are >10^10 transistors, operating at >10^9 Hz. Naively, a high end GPU is capable of more state transitions than the human brain by a factor of 1000. That figure for the brain also includes memory; the GPU doesn't. The human brain runs on impressively little power and is basically self-manufacturing, but it's WAY less compact or intricate than a $2000 processor.

The capabilities of the brain are in how it's all wired up. That's exactly what you don't want if you're trying to coopt it to do something else. The brain has giant chunks devoted to extremely specialized purposes: https://en.wikipedia.org/wiki/Fusiform_face_area#/media/File...

How do you turn that into a workhorse? It would be incredibly difficult. It's like looking at a factory floor and saying oh, look at all that power- lets turn it into a racecar! You can't just grab a ton of unrelated systems and expect them to work together on a task for you.

You're making the implicit assumption that synapses === binary bits, and that synapses are the only thing important to the brains computation. I would be surprised if either of those things were the case.

I don’t think a bit transition is in any way comparable to the “event transmission” to a potentially extremely large number of interconnected other neurons.

An actor-based system would be a better model, and I’m not sure if we have something like that in hardware. I do agree that sometime in the future it will be possible to overcome the biological limit, as cells are most definitely not at an optimum (probably not even at a local one), like duplicated pathways and the like, but it is no way trivial.

John von Neumann had a great paper on the topic, at least his thoughts about it. It is a really great read, even though both technological and biological advances may make it outdated, I think he did see a few things clearly into the future.

In the context of the story we're responding to, it does mention that they can be simulated at at least 100x speed at the time of writing.

Your comment reminded me of a clever and well-written short story called "Understand" by Ted Chiang.

> We could fuzz it and see if we can crash a brain.

Sadly, this we already know. Torture, fear, depression, regret; we have a wide selection to choose from if we want to "crash a brain".

Ted Chiang’s “life cycle of software objects” is also similar to the OP. Basically about how an AI (not strictly an upload) would probably be subjected to all sorts of horrible shit if it was widely available.

My first thought was that this reminded me of an epileptic seizure brought on by "fuzzing" (sensory overload)

I think that's pretty plausible.

I don't mean it quite like that.

Think for instance of a song that got stuck in your head. It probably hits some parts of it just right. What if we could fine tune that? What if we take a brain simulator, a synthesizer, and write a GA that keeps on trying to create a sound that hits some maximum?

It's possible that we could make something that would get it stuck in your head, or tune it until it's almost a drug in musical form.

I've no experience with it, but I imagine it's like heroine or DMT or something like that. Wouldn't that come close to something that "hits some maximum"?

What you're talking about is getting pretty close to a Basilisk - https://en.wikipedia.org/wiki/David_Langford#Basilisks

BLIT is available online http://www.infinityplus.co.uk/stories/blit.htm it's a fun short read.

"fun"...

I don't recall what basilisks are, only that I burned it from memory for probably a good reason.

Brains still operate as brains after severe trauma. They just don't necessarily operate well as humans in a society. Though I guess you could say making a brain destroy itself (suicide) is "crashing it" too

> Brains still operate as brains after severe trauma

Well, except when they don't, but since a brain functioning as a brain is part of the operating requirements for the body that lets the brain operate at all, when they don't, they ultimately fail entirely in short order.

So, assuming that a brain generally operates as a brain after severe trauma is a pretty serious case of survivorship bias.

Brains do have plenty of “backup systems”. Like emotions, stress etc do have an effect on the whole body, spinal reflexes are not affected as much and you will likely not fall over just because of extreme stress or whatever. There are similarly many many more “primitive” systems in place, like you will continue to take breaths even at “shutdown”.

From the title "lena" and the reference to compression algorithms made with MMAlcevedo, it's clear that the story is trying to draw parallels to image processing. In which case being able to store images has come decades before realistic 3D rendering, photoshop, or even computer vision. For example, the sprites from some early video games look like they were modeled in 3D, but were actually images based off of photographs of clay models. I think (with suspension of disbelief that sinulating consciousness is possible) it is realistic to think that being able to capture consciousness would come before being able to understand and manipulate it.

It sounds like, in this world, a lot of the value of a simulated brain is in the as-yet-indescribable complexity of human cognition. If you debug a brain to remove the parts of it that are uncooperative, you likely have to remove the parts of it that have opinions of any sort about the task on which it's working, which seems like it would defeat the value of using a brain at all. If you're giving a task to a simulated brain, it's because it's beyond the reach of what you can efficiently ask a program to do, and you want the subconscious reactions, development of instinct, and deep unplanned reasoning that you get out of asking an educated and experienced human to think about a task. You can likely tweak a simulated brain into cooperation, sure, but you'd have very few guarantees of not breaking those mechanisms while you're at it.

If you can describe the task to be performed well enough that you don't need the je-ne-sais-quoi of a human brain to perform it, you may as well just have a regular computer program do it. (We already have very efficient systems that involve extracting limited amounts of creativity and insight from human brains and forming them into repeatable tasks that can be run on computers - that's what the entire software industry is about.)

Simulation and models are not real. Maybe some "attacks" could be developed against a simulated mind, but are they due to the mind itself or the underlying infrastructure? Just because you can simulate a warp drive in software doesn't mean you can build a FTL ship.

The way I understand the story is that you have a scan of the relevant physical structure of the brain, plus the knowledge of how to simulate every component precisely enough. You may not know how different parts interact with each other, but that doesn't prevent correct functioning.

Just like you can have somebody assemble a complex device by just putting together pieces and following instructions. You could for instance assemble a working analog TV without understanding how it works. It's enough to have the required parts, and a wiring plan. Once you have a working device then you can poke at it and try and figure out what different parts of it do.

In the case of a warp drive we care about a physical result (FTL travel), not a computational result.

We already have emulators and virtual machines for lots of old hardware and software. If I play a Super Nintendo game on my laptop, it's accurately emulating an SNES. The software doesn't care that the original hardware is long gone. The computational result is the same (or close enough to not matter for my purposes). If brain emulations are possible, then running old snapshots in deceptive virtual environments is possible. That would allow for all of the "attacks" described in this piece of fiction.

There are many bugs emulator developers (game console ans otherwise) have faced because of undocumented or emergent properties of the original hardware. Some games required those properties to function.

Yes sometimes emulators have bugs, but they do a good enough job that most people aren't willing to go to the trouble of using the original hardware. Also emulators can unlock new capabilities such as better graphics.[1]

Human brains are far more resilient than software, so my guess is that emulated brains won't have brittle corner-case bugs like emulated software. People today do all kinds of crazy stuff to their brains and remain functioning: drugs, sleep deprivation, getting hit in the head, fasting, aging, etc. If subtle changes to our brains could cause our minds to stop working, we'd know by now.

1. https://en.wikipedia.org/wiki/Pixel-art_scaling_algorithms

"Execute it step by step,"

These are not imperative programs or well organized data. They are NN's we can't fathom how to debug them just yet.

Also, they should tag 100 years onto the timeline, I don't think we're going to be truly making useful images soon.

Cooperation would quickly become unnecessary because you either start from a cooperative state every time, or you quickly figure out how to tweak the brain state into cooperation.

What starts out as mere science will easily be repurposed by its financial backers to do this in real time to non-consenting subjects in Guantanamo Bay and then in your local area.

I think it’s possible that we’ll be able to run large simulations on models whose mechanics we can’t really understand very well. It’s not a given we’ll be able to step through a sequence of states. Even more so if it involves quantum computation.

Many of the things you describe could still happen with Monte-Carlo type methods, providing statistical understanding but not full reverse engineering.

>> how to best create discord, or how to best destroy a human being by just talking to them.

In some cases therapists do this already. Techniques have intended effects which may differ from actual effects. The dead never get to understand or explain what went wrong.

> Within a decade we'd have a manual on how to craft the best propaganda, how to best create discord, or how to best destroy a human being by just talking to them.

It seems like we’re close to that already.

They teach that stuff at the School of the Americas in Fort Benning, Georgia. (Now called WhinSec to try to get away from its past)

Sounds like trained networks to efficiently manipulate uploaded brains would be a thing in your scenario.

Could use ML to reduce manual 'debug' overhead, spooky stuff.