Hacker News Reader

Comment by eyvindn | Hacker News Reader

One of the authors here.

Feel free to ask any questions!

We encourage you to play with the attached Colab with which you can train models from scratch in <30min.

Very cool. I've only just skimmed the article (and quickly), but I was wondering if instead of generating pictures one could apply this to generating a CA (or parameters for a CA) that does certain types of computations? This would allow us to evolve or find CAs that could perform certain types of multi-stage computations in a highly parallel fashion and could be readily implemented in FPGAs (for example).

Oh, sounds like I just found a fun project to work on.

We have a lot of future applications and directions in mind, including the feasibility of applying this to distributed computing and consensus. In particular, the idea of training a model generating CA parameters instead of having a CA fit a specific pattern is of great interest to us and the synthetic biological field. Feel free to get in touch.

This is a wonderful idea. Would you be interested in collaborating on this project?

Dang! I started working mid-last year on practically exactly this as a side project (was going to call the paper "Towards the horse", if you get the reference :) ). Congratulation for making it work, I am really psyched. Can you remember what triggered the motivation to work on this? Somehow I remember there being something on reaction-diffusion equations on HN around that time. Had you tried to use a single Laplace operator instead of two Sobel filters? My approach is to model the reaction-diffusion as a sum of 1x1-convolution (reaction) and a depthwise 3x3 convolution with a fixed kernel multiplied by a learnable constant (diffusion). However, for this to work, a single seed pixel obviously will not work. Any thoughts?

Diffusion-reaction systems were an inspiration in general. Using a Laplace operator (or the discretised equivalent for our 2D grid) might have trouble learning to generate these patterns - the Laplacian wouldn’t always provide unique information as to where a cell might be based on its neighbours. It’s possible the network would learn to exploit the hidden channels to bypass this directional invariance. Starting from a single pixel in such a setting would indeed need some mechanism to break the symmetry (stochastic updates as used here, for instance).

Now having scrolled down to the bottom of the papaer, I have seen a section on the genesis of the idea.

I just see a website that doesn't seem to anything particularly interesting with a CA, is there a paper that explains what's going on (I might have missed the link, but I did look).

Congratulations, it's really cool. It's late here now, I will play with it tomorrow. I actually have a lot of questions!

- How well the CA rules "compress" the image against a typical image compression algorithm?

- Have you tried 3D? Animations?

- What happens if you insert small noise to the learned parameters? It gets destroyed or mutates the image to something similar?

- Is it practical with more complex images, for example, a human face?

1) This is often the first question many people familiar with NNs ask and rightly so. Compression was not one of our goals with this article, and it would look like a terrible compression algorithm if that were its purpose. In fact, the model displayed here is about 8.3k parameters, although the WebGL model is quantized (more info on this in the last section), and each model learns to encode an image consisting of 44x44x3 = 5808 integers. We made no attempt to minimize this number. The key thing to bear in mind is that all the cells have the exact same rule and the image generation starts from a single one of them - meaning they have to learn to communicate locally with their neighbours to self-organize in the correct pattern. This is a very non-trivial task and the majority of the model’s parameters are likely going towards this communication protocol and growth behavior.

2) We have not tried 3D. As for animations, some of our earliest experiments suggested one could achieve “animations” by applying the loss at key-points to have the model learn to iterate through these points across several time steps.

3) One could argue the WebGL implementation does this to some extent by quantizing the learned weights we take from the Tensorflow training code. The model remains very resilient and worked out of the box in almost all cases. Moreover, if one tried to inject explicit noise to the CA in a given location, some models would have no problems adapting to it, while others would fail miserably. Some early experiments yielded some remarkably resistant models, able to resist while being subject to continuous globally occurring noise. We suspect explicitly training them while introducing noise would allow us to drive the model towards more consistently resistant behaviors.

4) One of the main obstacles to larger patterns at the moment is memory usage during a forward/backward pass. There are optimization and tricks we plan to employ to generate larger and more complex patterns, which may be discussed in a follow up thread.

Dave Ackley, who developed the Moveable Feast Machine, had some interesting thoughts about moving from 2D to 3D grids of cells:

https://news.ycombinator.com/item?id=21131468

DonHopkins 4 months ago | parent | favorite | on: Wolfram Rule 30 Prizes

Very beautiful and artistically rendered! Those would make great fireworks and weapons in Minecraft! From a different engineering perspective, Dave Ackley had some interesting things to say about the difficulties of going from 2D to 3D, which I quoted in an earlier discussion about visual programming:

https://news.ycombinator.com/item?id=18497585

David Ackley, who developed the two-dimensional CA-like "Moveable Feast Machine" architecture for "Robust First Computing", touched on moving from 2D to 3D in his retirement talk:

https://youtu.be/YtzKgTxtVH8?t=3780

"Well 3D is the number one question. And my answer is, depending on what mood I'm in, we need to crawl before we fly."

"Or I say, I need to actually preserve one dimension to build the thing and fix it. Imagine if you had a three-dimensional computer, how you can actually fix something in the middle of it? It's going to be a bit of a challenge."

"So fundamentally, I'm just keeping the third dimension in my back pocket, to do other engineering. I think it would be relatively easy to imaging taking a 2D model like this, and having a finite number of layers of it, sort of a 2.1D model, where there would be a little local communication up and down, and then it was indefinitely scalable in two dimensions."

"And I think that might in fact be quite powerful. Beyond that you think about things like what about wrap-around torus connectivity rooowaaah, non-euclidian dwooraaah, aaah uuh, they say you can do that if you want, but you have to respect indefinite scalability. Our world is 3D, and you can make little tricks to make toruses embedded in a thing, but it has other consequences."

Here's more stuff about the Moveable Feast Machine:

https://news.ycombinator.com/item?id=15560845

https://news.ycombinator.com/item?id=14236973

The most amazing mind blowing demo is Robust-first Computing: Distributed City Generation:

https://www.youtube.com/watch?v=XkSXERxucPc

And a paper about how that works:

https://www.cs.unm.edu/~ackley/papers/paper_tsmall1_11_24.pd...

Plus there's a lot more here:

https://movablefeastmachine.org/

Now he's working on a hardware implementation of indefinitely scalable robust first computing:

https://www.youtube.com/channel/UC1M91QuLZfCzHjBMEKvIc-A

I'm wondering these things too. You say in the paper that your net is quite small - I wonder if you could first extend it to regenerative sprite animations for some fun procedural NPCs in a roguelike? If you can efficiently add or change a pixel or two per frame in a sprite+collision mask, imagine battle sequences where things get hacked off, maybe regrow if you wait too long, etc.

Nice work @eyvindn! A few questions:

1. Am I just imagining things, or do the results depend on the speed?

2. I'm able to erase the shape very easily at max speed, despite setting it to persist. Have you studied how much shape loss is required for the shape to vanish? (ex video https://youtu.be/zMQkTyzdphc)

1. The results do not depend on speed in principle. The difference between 60FPS and “max” can be several orders of magnitude depending on your hardware, so it’s likely you may see instabilities that exist on much longer horizons at higher speeds.

Interactions will play out very differently at different speeds because you are interacting with a sped-up/slowed down version of the CA.

2. We haven’t done any rigorous studies of regenerative capability, although this is certainly on our to-do list. From empirically playing with them, models seem to be more susceptible to damage to the centre of their bodies than to limbs, likely as a result of “growing” outwards.

This is because the center point of your models do not expect a foreground/background boundary. By putting a hole at the center, the boundaries seek to stabilize the system by bubbling outward. The best way to put it is that there is no stable state where a doughnut hole exists at the center. An interference pattern ripples outward and destroys the system.

Hi, very intetesting work. Just posted a comment with some related work from my group. Please check it out and let me know if you need any info. Best, Stefano Nichele

Thank you. We will take a look at this. Currently we are working through a backlog of communication.

Have you looked at other applications of this approach for tasks like semantic segmentation or depth estimation?

Yes. Several other applications (both with and without an underlying 2D grid) are in the pipeline which we hope to publicise soon.

Reply on news.ycombinator.com