Infinite Stable Diffusion Videos
orbdog.com
orbdog.com
The videos are generated from random https://lexica.art prompts, with linear interpolation between two random seeds for each video, held at the same prompt, looped with ffmpeg filter_complex reverse/concat. Music from various creative commons / free sources.
Source code at https://github.com/lwneal/duckrabbit/
Hosted on a single $7 node at https://www.digitalocean.com
Could you explain more exactly which Droplet/configuration you are using on DigitalOcean?
From the code [1] I take it that you're using polar interpolation on two random points around the original noise right?
This is very cool stuff, thanks for sharing!
[1]: https://github.com/lwneal/duckrabbit/blob/cb375ec7c6067bf805...
breaking it out for others curious:
- modified img2img to accept 2 prompts and 2 seeds (and a slerpradius and output video path)
- uses imutil.Video to create a video
- stochastic encode with noise, then decode, then add a frame to the video, for all n_iter
one thing i dont understand is why OP keeps calling `.half()` on their models. googled it and it seems to be a newish pytorch feature for "half precision" but couldnt get a clear answer on why you would want that. anyone care to share?
Sound permanently on after hitting next button once on Chromium 104.
Next button does move to next video, but also enables sound. Additionally, this breaks internal state, and the button still shows the muted symbol (despite the sound now being on). Hitting the muted sound button switches the sound button symbol to unmuted, sound still on as before. Hitting it again to mute the sound doesn't work, and doesn't change the symbol back to muted.
If all engineering is, is altering CAD models to conform to local government regulations, then conceptually surely we can procedurally generate a whole fucking city, adding in sewers and power and other infrastructure as it goes.
The reality is much closer to how stable diffusion and other AI-generated art is now - technically and creatively impressive but often nonsensical and broken.
And, as another commenter noted, also for cities design. There are actually a few versions of this: I worked on one called Delve https://www.sidewalklabs.com/products/delve
Typically they use constraints optimization or wave function collapse under the hood, on top of some sort of procedural generation pipeline. A few others are Spacemaker, Hypar, and Testfit.
My bet is that this will happen in 8-9 years from now, but it's just a guess.
I think it's hard to challenge the fact that it WILL happen, at some point in our lifetimes.
If other people find this to be a helpful I'm interested to hear about it.
Either you think the idea is too flawed to consider, or you can consider it and conclude we might as well assume the world is real, because if it wasn't what would we even do, but i don't understand how you can hold both positions.
What I mean to suggest is that, though this is often presented as a scientific hypothesis about reality (eg, sometimes people say things like, "there's a 20% chance the universe is a simulation"), it is unfalsifiable, and therefor not a scientific hypothesis. That doesn't necessarily mean it can't be approached with other ways of knowing, and the argument I present is essentially a spiritual one.
I reject the dichotomy.
This universe is the one we have. Engaging with it wholeheartedly is the right way to go.
That doesn't mean it cannot be a simulated one. If it is a simulation, we might exploit that fact. Finding glitches can lead us to a deeper understanding of things around us. Gaining root on the machine our universe runs on is the ultimate sandbox escape. In my book that thought is invigorating and worth engaging with wholeheartedly.
If this is a useful framing for you, I'm here to understand, but I currently don't see it. To me this seems like solipsism and radical skepticism with a science fiction twist, and I personally find those ideas to inhibit my ability to understand and engage with the world.
And that would be in some sense sensible and perfectly correct.
> We would only recognize it as a "glitch" if we had a specification to compare it to,
Nah. I disagree. If you are playing a game and you fall through a wall you can call that a glitch without seeing the game's specification. It is because that experience is not in line with the rest of the system.
Imagine if we find that a specific reaction between high-energy particles goes measurably differently in one bounded patch of the universe. We find this randomly. Then an army of physicist descends on it. They measure a lot. Perform experiments. They characterise the effect to a high degree. Map out the precise boundaries. Map out what exactly happens on the boundary, and so on and so on. ... Is this just physics? Yes! Would this effect feel like a "glitch"? Also yes!
> I don't understand what "gaining root" would mean either.
Transcending the laws of our Universe. Being able to change them. Rewriting the very fabric of reality. Becoming god or gods. Peaking behind the curtain and finding a brand new playground beyond it. Being able to change PI to be actually 42.
> To me this seems like solipsism
I think you are hang up on some specific flavour of how the universe could be a simulation. Maybe try to think of other possible ways?
Your suggestion that this may lead to us "being gods" certainly does not change my view that this is solipsistic and that the conclusions it leads one to are silly at best and dangerous at worst. The idea that you can become God doesn't seem compatible with a healthy life where you accept your limitations and find a way to live vibrantly despite them. It sounds like something a supervillain says at the beginning of a Marvel movie.
Hey fellow human being. You are being incredibly insulting here. I can assure you that I have both a wholesome and healthy life and believe that maybe we live in a simulation. If the two things seems incompatible to you that is a limitation of your cognition.
Can we discuss ideas without insinuating that the other is living an unhealthy or less vibrant life? Thank you very much.
You talk about this idea being “solipsistic”. I had to look up the word. Merriam-Webster defines solipsism as “a theory holding that the self can know nothing but its own modifications and that the self is the only existent thing”.
I think you are still only thinking about “brains in a jar”. The idea where a brain or brains is attached to some machine which feeds them with simulated sensory input.
That idea is indeed solipsistic. By the very definition. It is also not particularly interesting to be frank and not the kind of “simulated universe” I am thinking about.
The kind I find more interesting is where the laws of physics are executed on a high fidelity simulation. The difference here is that here the brains are not hooked up to the machine, but part of the simulation. An emergent phenomenon inside it. Same way as that flower there, or a neutron star yonder.
This is fully compatible with living a wholesome life. You play board games with your friends, make hot love and cold gazpacho. Walk barefoot under moonless skies on lush meadows etc etc. It just might be a simulated person in a simulated universe, there is nothing wrong with that. Doesn’t make it any less vibrant or feel the grass any less cold.
You might also ask what does it matter if physics is simulated, how would we even know? If the simulation is “perfect” then we might never know. So far we found that the laws of physics are the same no matter where we look. If we would find certain kind of anomalies those could indicate certain kind of simulation systems.
Let me tell you an example: Imagine that you are designing a simulation to run a universe. You have a mindbogingly lot of compute at your disposal but not infinite amount. You might want to shard your simulation to clusters you can compute in paralel, and you want to minimise the information transfer between these clusters. Having gravity (which clumps stuff together) and speed of light (which keeps speeds bounded) helps with this. This is so far not predictive. I just wrote up a bunch of speculations. To make it interesting you need something we can test. Here comes the prediction: if I were coding a universe simulation distributed this way I would hate the boundary interactions between clusters. Would probably be super hard to make it seamless with the rest of the simulation. Therefore if we live in such a simulation I would expect anomalies in the finer details of the laws of physics at the boundaries of clusters. Lets asume that the clusters are rougly aligned with star systems. We could send a probe to the nearest one which keeps making detailed measurements on-board while it travels. If it encounters a boundary where physics “skips a beat” then we might see that in the measurements. We know it can’t be some huge stuff. We see the light of other stars, we observe matter which traveled interstellar distances. But maybe it would be enough to disrupt the internal oscillation of an atomic clock as it is crossing a cluster boundary.
It is important to understand that I am not saying that we for sure live in a clustered simulation of a universe. What I am illustrating here is that this kind of thinking can lead to theories about how the universe might be, and those theories can lead to testable experiments. There are people much smarter than me, and they will think of thousand times more intersting ones. I am sure about that. Do you still feel this solipsistic?
I find the idea you expressed - specifically that godhood might be attainable and that it's a goal one could work toward - to be extreme and dangerous, and I have first- and second-hand experiences where I've observed similar ideas to be harmful. (It would be fair to ask that I share these experiences, but I am not comfortable doing so; I realize that's unfair to you because it denies you the opportunity to interrogate them, and I apologize for that, but they are personal and painful and not all of them are mine to share.) I have no judgement of you, but I have a distaste for this idea. The rest of the ideas you're expressing, about life being just as fulfilling in a simulation, seem to be an ecstatic and joyful response to nihilism, and if that is working for you, that's awesome. But surely you can imagine a circumstance in which the idea "godhood is attainable" causes harm.
I think you're thinking that I just haven't thought this through properly and that if you just explained it better I would agree with you. But I have given this a lot of thought. I first encountered these ideas about 15 years ago, and your elaboration isn't really presenting anything new to me. We agree on what this thought experiment means and what the implications would be, we just interpret the results of the experiment quite differently. I do find it to be solipsistic, with some small concessions, like "okay, I admit that other minds exist as much as my mind does, so instead I hold that all of us are fictions." The idea that reality might be so malleable that we can "hack" it is, in my mind, inherently solipsistic, a natural consequence of centering one's ability to manipulate the universe as the most real thing.
But please change the music!
This is an honest question, I haven't seen any example of anything else so I got to wonder whether the models they are using are specialized for sci-fi and fantasy "air brush/digital" style? Why?
The models, processes, and collective knowledge will just develop more over time to create MUCH better visuals, videos, and temporally coherent animations.
This is like a new form of art canvas, and we're all getting used to the basics of using the "paints" and "brushes" for it. In a few months/years, some of us will master the skill and produce fantastic artpieces.
But for know, the stunning stuff tends to be fantasy, sci-fi, impressionistic. Photos of people that are not a portrait are pretty often anatomically impossible. Getting hands, arms and stuff right seems quite difficult for these networks.
Funny enough, I've seen underwater pictures that to me looked quite believable, but to they expert are ridiculous. Lot's of impossible stuff going on. Human brains are ready to fill in a lot of detail.
Another question: Do the people who run the software claim copyright on the results even though these are (mostly) produced by the software? It sounds like that when you write "some us will [...] produce fantastic artpieces." I guess it's also legally the case but wonder whether that's also how people experimenting with it understand it.
Btw it simply isn't true that the AI generators are "only good for fantasy and sci-fi". I guess you've been seeing a biased selection. They can do pretty much anything. MidJourney is for sure more fine-tuned towards artsy stuff though.
My fun has been with two games: 1) making unusual art from prose using the art styles of famous painters, 2) playing "AI Pictionary" with friends (can you produce image X; example: a person eating ramen with chopsticks that are light sabers).
It's also because they are basically nonsense (fantasy) so the results in the style look more plausible.
I tried using a few AI generators to get some basic placeholder images for products and it's utterly shit, it's clear that networks dont understand what they are generating. Like I tried to generate bycicles and I would get components sticking to ground, floating components, stupid proportions, visual artifacts.