How a 64k intro is made (2017) | Hacker News Reader

How a 64k intro is made (2017)

lofibucket.com

218 pointsaragonite46 comments

I've toyed with doing writeups of some of my demos, but one aspect I've struggled with is how the explanation of how it works is often severely departed from the process of creating it.

The writeup here appears to be fairly natural, and maybe this is the true process for something as large as 64k, but I mostly do 192 byte demos, the largest I've ever done is 512 and 1024 byte. I think there is a difference at this size where the size very much takes charge of the creative process - and that can be quite interesting because you end up doing a lot of experimentation and accidentation and discover things you wouldn't normally be pushed to and can pivot the entire demo. However when deconstructing the result in order to explain it, this juts gets lost and I've struggled to figure out how to convey this duality. To only post a "how it works" would omit the part where all magic happens, yet that's hard to explain on it's own, and hard to combine with the "how it works".

Would be cool to see a full video of the process from start to finish, if you’re up for it.

That way people can get a better impression of it.

I've actually tried that on Twitch, it was fun for another reason: viewer involvement. But as a format I found it didn't really work well because for my most impressive demos 90% of the time I'm just staring intensely at tiny fragments of code and trying to rearrange and run it in my brain - this is not interesting to watch believe me, I suppose one could condense the video to only show the ends of those segments and just talk out-loud about the concrete thought that has coalesced at the end of it and the actions that are now being taken.

> I suppose one could condense the video to only show the ends of those segments and just talk out-loud about the concrete thought that has coalesced at the end of it and the actions that are now being taken.

I like that idea.

This is the same for any creative process. Any painter could describe how they’re reasoning about a composition and what-not. But ultimately each individual needs to digest technique and make it their own.

Writeups like this are great though because it can at least share the tools used and give folks some place to start.

Maybe a "why it works" post would be useful, as in "this part of the demo does x because I was trying to do y but ran into this issue, which caused me to experiment with z and then a side effect inspired me to switch abc to def", etc. Personally I think explaining the creative process is more interesting than explaining the end result.

> Personally I think explaining the creative process is more interesting than explaining the end result.

Thanks, I think this is right, to focus on the process instead (and commenters indigochill and lawn) which I think are describing the same type of writeup.

Perhaps my error was starting with "how it works", whereas I probably should focus on the story of the process and add the "how it works" where it makes sense in the story. I also don't really try to record the process at the moment, it's just from memory.

I know nothing about the subject but an idea is to use a "let's make a demo" story to, as you say, capture the important journey instead of the end result.

It takes more effort than a simple "how it works" explanation but if done correctly I think it should explain how the demo works while also capturing the magic of the creation process.

How long does it take you to make a demo? Might be cool to do a live streaming, or just record the process even if not live.

Sometimes I do it in one sitting, sometimes that's quite long, like 12 hours or more! Other times I will revisit it over days or months - You'd be surprised how much thought and transformation can go into 192 characters. It depends how much potential room for improvement it "feels" like there is left, you get a sense for it, sometimes you just know it's done, others you're never quite sure when to stop trying to push harder.

I've commented in a sibling my experience live streaming which was interesting. Another Demoscener who has given this far more concerted effort on Twitch is KilledByAPixel, I think they are recorded somewhere maybe on youtube. My two attempts are lost to time.

As an example (yes shameless plug) I wrote this one recently over a couple of long evenings, I'd estimate 8 hours maybe:

https://www.dwitter.net/d/31805 (runs fastest in Firefox)

But there are some familiar micro-raymarching techniques I've already developed and reused in this. The bit that took most of the time in this one was figuring out how to fit binary search marching that was required to support large differences in marching distances and textures without producing excessive moire effects.

Although a different context, I always find the writeups of cybersecurity CTFs that go through the "What I was thinking, what surprises I encountered, how I pivoted" process both more enjoyable and more enlightening than writeups that simply explain the solution as if it was known from the beginning.

At the same time there might be some editing since there might have been approaches tried that didn't go anywhere, and whether that's interesting/relevant to the reader is probably a judgement call from someone who knows the domain and whether those dead ends might have been natural things to try in that specific context.

> the explanation of how it works is often severely departed from the process of creating it

This description rings true for a whole swath of creative endeavors, including ones like demos that appear technical on the surface: optimizing an algorithm, conducting scientific research, building an engine, constructing a mathematical proof, and so on.

High-quality 64k intros usually need a film-like production process nowadays, except it is harder because you have to fit it into 64k later, it typically means procedurally generating everything as well as using the appropriate tools (runtime packers, compiler options, etc...). It is, I think, the most demanding demoscene category.

There is much less commitment in a tiny intro. You don't have days of preparation, storytelling, art direction, etc... to stick to. If, when optimizing your intro, you stumble on something better, you are better off throwing everything away and build on that instead, "everything" is just 192 bytes after all. I didn't know 192b was a thing BTW, I am more accustomed to powers of two, like 256b.

It's 194 bytes I keep getting it wrong :D yeah it's a strange size, the short version is SMS (140 bytes) => twitter => dwitter => unicode packing of ascii with JS decoder (which could not be undone because the limit was defined as unicode points) yields 194 bytes.

> If, when optimizing your intro, you stumble on something better, you are better off throwing everything away and build on that instead,

Exactly this, but because of the extreme size limitations you often end up forced to iterate out of frustration trying to fit something, and then something else weird but interesting will often pop out of that process that you wouldn't have directly thought of.

I straddle the tech/art gap as a tech artist in 3D VFX & Animation. I've done work as visual as branding and identity, and as technical as network performance and a back-end web development. I'm also a technical writer for a commercial rendering engine, which is pretty different, but I've some experience reading and writing about these things.

Technical folks often approach process write-ups as they would an incident report, or a tutorial. Rarely a pleasant read! A little prep before you start yields major improvement: First, decide exactly what you're saying, and to whom. For a big one, having a "story" part, a prosaic technical write-up, and a reference section can fit multiple people's needs. Then, consider exactly what your audience wants and needs to know about your message: try not to be overly broad. Make that into a simple narrative and story arc, and expand it into an outline. It sounds like boilerplate, but it's more like scaffolding. Correctly applied, it steers you towards a concise, compelling piece without endless dickering and waffling about what to say next. Start with the broadest, least specialized audience-- adding details is easier than editing them out.

If you write your demo in a recording editor, then when you're done, you can go back and watch the playback of all your edits and add commentary.

Interesting idea, thanks!

It's probably good to know that the focus in the demoscene has shifted away from 64k more towards 4k/1k intros and 'regular' demos. The amount of work required to do a high quality 64k is so huge that, especially now that the demoscene is not as big as it used to be, a lot of people don't think it warrants the time investment anymore.

(Btw, the best way to prove me wrong is to write a 64k intro about it)

Actually a bunch of people that do still like them... it's just that we don't have the infinite time we had at our disposal in our teens that makes us not ever be finished :P

Isn't that the same issue really? The demoscene seems to be in decline because there's hardly any new blood coming in, the teens who would have time to commit to a big production today just aren't interested.

Teens have been so accustomed to immediate gratification and short form content they don't possess the willpower to focus on something truly difficult. You can't watch a few shorts and bust out a 4k demo over the weekend the same way you can "make minecraft in 24 hours".

Maybe that's part of it, but I think not having any personal experience with the constraints of early computing is also a factor. The demoscene does try to keep that spirit going on modern platforms with artificial constraints like "Windows exe limited to 64kb" but I think it's harder for someone coming in fresh to appreciate the appeal of that handicap. They definitely won't appreciate the actual retro computer targets like the C64 and Amiga, why would they, those systems came out before they were born and unlike early game consoles they have almost no lasting appeal outside of nostalgia (sorry Amiga fans).

Teens addicted to only instant gratification were never the ones to produce really good stuff in the demoscene anyhow, they will put countless hours into other arenas.

Also back in the mid 1990s maybe the demoscene _was_ instant gratification to get started? You could start doing graphics with "mov eax,13h / int 10 / mov ax,a0000h / mov es,ax / mov di,0 / (loop with stosb).. ".

To get started today with graphics you either setup openGL/d3d with their weird API's with decades of legacy, use something third party (and learn linking/libraries) or write a 3000 LoC hello-vulkan (Or use Unity,etc but then up until 5 years ago you'd get shouted out of the scene due to elitistic attitudes).

Now for instant gratification there's Shader Toy.

Sure you can. I made a 4k demo during a demo party over a weekend. I think people in this thread might be conflating demos with great demos. The quality bar is entirely in your head. Go and watch any live or recorded party and the good demos are only a small percentage. Most are quite average.

I don't think that the time investment (per-se) is the biggest issue for the demoscene, 64k is the most time consuming category in many ways but also becoming a niche _again_ today.

It'd be easy to write an essay on it, but pirates games and cracktros not being as visible in the days of cheap games, copy/lan parties as focal points of computer usage in the 80s/90s, lan party consolidation and separation of demoscene from lan parties, _lack_ of funding for commercial graphics programming (games) before y2k in Scandinavia compared to available talent, inflexibility of early 3d accelerators were all factors in making the demoscene a thing and later it's decline.

So far I've made a demo that maybe could have been 8k, but it was 32k (fits in 64k category but didn't contain enough content to use more than half).

I know I can't make anything that competes with .the .product or Chaos Theory (at least not yet), but it just feels like a good limit. 64k is plenty of room for code as long as you don't write a huge amount. You can write all sorts of procedural generation; you can use OpenGL and C++; you can compress it generically with xz (at least on Linux) without worrying about ultimate size efficiency. But it's still small enough that you can't include raw texture or sound data - you're forced to generate them with code. It's still code-based art, not "download a bunch of assets from the internet"-based art that might as well be a video file. A lot of >64k demos spend a lot of the size on content files, which doesn't feel like what a demo should be.

To write a 4k you have to actually make it small, but to write a 64k you just have to not make it big.

With 4k most people use pre-made startup (crinkler, win32 minimal window, glSetup, audio often 4klang) and without the audio you are at something like 300-900 bytes(was a while ago since i last fiddled), then add the music (1kb-2kb or so) and then you know what you have to work with in terms of shader code (usually a single pixel-uber-shader for 1-2kb compressed)

Making them is for most more about reaching the upper limit in size and then tweaking until you're happy (more experienced people might have an budget idea from the get-go and might overshoot slightly with their idea and then cut).

64kb can (and often did in the 90s before Jizz and Stash, fr-08 was mostly an well produced evolution) include raw images,textures and samples, and raw data can still be an viable escape hatch today. The killer for 64kb is that you can spend almost infinite time on writing generators and other tools.

Of course you can include smallish pieces of data, but you can't make the demo mostly data. Look at some unlimited size demos. "Sisyphus Unchained" (Andromeda Software Development) is a 220MB zip file. Uncompressed, it's got a 60MB OpenCV DLL for some reason, a 37MB 3D model of a crab, 68MB of JPEGs and PNGs, 120MB of "MVX" which appears to be the animated silhouette from the 4:15 mark, stored as 442 separately compressed frames. The music is a single 11MB MP3 file.

Is that even a demo? I suppose. The EXE file seems custom made for this. It doesn't feel like a demo, though. It just feels like they made a music video in some tool like Blender with no constraints, and writing their own playback tool doesn't "really" make that into a demo.

By contrast, anything you can do in 64k that looks demoey is demoey.

The winning entry, "Rainmaker" by "Byte[censored] & Doomsday" was 450 megabytes. 2 different versions are included in the download, making it 900 megabytes, but that doesn't count. It's packed in one EXE, without as easy visibility as the other one which had all its resources in separate files. Running 'strings' shows fragments of RDF, JavaScript. It's made with some tool called "Notch" and the 'strings' output makes me think it might have a whole web browser in it as well as a whole copy of Notch. It also happens to be a valid zip file, where we can see there's an 80MB WAV file of music, and at least a hundred megabytes of textures and objects.

My first demo had a music wave file. That's because I ran out of time and couldn't replicate the way I'd made it in the DAW program. That was a noob thing to do. But the #1 winner at Revision has the same thing? What is this? I don't like it. Demo-making isn't supposed to be about shipping an mp4 video alongside an mp4 file.

It's a matter of perspective, when I started out it was news when 10mb demos was allowed and it's just continued from that point until they said screw sizelimits. My main point is that 64kb more or less doesn't restrict you in terms of amount of code and you still have a fair bit of latitude for an artist to do whatever they want.

My reference on non-generated is that what is nowadays called chip musicians used to do 10-40kb mod's with regular tracker tools that would easily be compressed with an exe-packer to work out of the box for a 64k, logotypes,etc were commonly just palettized images (wasn't much of an issue for mode13 graphics and even early hi-color things).

But more than anything, the top-2 64k's (And some oldschool prods) at Revision probably were the ones with most experienced man-hours put into them.

As for Notch, it's basically the commercialized version of the Fairlight demotools. Also appending Zip's to .exe files is one of the oldest tricks in the book to package datafiles "nicely", works well since the central directory of a zip file is found from the end of a file and you can compile the exe separately and then just insert data-files by making a zip and appending it.

My point exactly :)

It's also worth noting that the expectations around 64k got higher. We can do so much more in 64k than it was possible in the past. Compression tools have improved. If you take the famous fr-08 intro from Farbrausch and recompress it with modern packers, you would get back many kilobytes. So I think the time investment needed to make a top 64k has increased.

The source code of all my 64k intros has been shared (the last one was added two weeks ago): https://github.com/ctrl-alt-test/demo-archive/ and my plan is to share more code to help people get started. Creating demos using procedural generation is incredibly fun.

Here is a another great place to get started if you want to get into 64k size coding: https://64k-scene.github.io/

I came here to say that :)

In particular, we have a list of making-ofs like this one: https://64k-scene.github.io/resources.html#making-ofs

Coming from the "old ages" when a PC didn't even feature a (3D capable) GPU and all was about getting all the possible juice of the CPU... shaders now allow incredible things in a very small size, as long as you stick to procedural content. shadertoy.com features an insane amount of great samples.

Past HN discussions:

- https://news.ycombinator.com/item?id=14392305 (2017 - 60 comments)

- https://news.ycombinator.com/item?id=16842576 (2018 - 60 comments)

Here I thought it was going to be a 64k Amiga or .COM DOS demo. This seems to be a very expensive and elaborate demo that is out of general reach.

I'm wondering if there are similar demos featuring graphics and sound with strong constraints but written in Python?

Not really, the only way it makes sense for a high level language to be used in a constrained competition is if the rules are specifically set up so that the runtime doesn't count towards the size limit. There are such competitions for Javascript, where you get 64kb of HTML to play with and the size of the browser itself is "free", or for Lua via the PICO-8 runtime, but I don't think anyone has done it for Python.

There's also https://dwitter.net which has a limit of 140 characters of javascript, and the site is fairly active.

Write a Linux demo. You're allowed to use all the tools that come with the distribution (check the rules for the competition you want to enter, to see which distribution). I remember one with a Python shebang that launched SimpleHTTPServer to deliver a brotli-compressed payload to a browser.

Is it 64K period, or 64K plus access to gigabytes of external libraries?

64kB is the size of the binary file. You can rely on an operating system (Windows, Linux...) with its graphics drivers.

Basically, you can use the GPU and you can use most of the 64kB to generate procedural content.

It's an exe <= 65536 bytes. A modern OS does have gigabytes of external libraries, but the functions in them are more "support every graphics and sound chip out there" than "draw cool spaceship" and "play a funky beat". They're not allowed to obtain anything from the internet, nor anything else beside what comes as standard with the OS.

Everyone gets the same base OS and compete against _each other_, so nobody gets a special advantage. If someone finds a clever way to use the base OS for more bytes (such as reusing the standard Windows fonts or MIDI soundfont or CAB compressor), then so does everyone else, and they use the saved space for even more cool and impressive stuff next release.

Here's some tricks used for 4kb demos (some of which are applicable to 64kb demos too):

https://in4k.github.io/wiki/aulds-4kb-introduction

https://in4k.github.io/wiki/aulds-4k-synth

https://in4k.github.io/wiki/exe-packers-tweakers-and-linkers

The executable has to be 64k or less and has to run by itself on a clean install of your OS of choice.

There may be "gigabytes of external libraries" included in the OS, but these are necessary for using current-gen hardware. Libraries and assets that are not just for interfacing with the hardware and basic stuff like opening a window are rarely taken advantage of, except sometimes for text rendering. System decompressors used to be used, but nowadays, productions come with their own specialized unpackers (ex: kkrunchy) that are much more powerful yet smaller than anything that comes with the OS. EDIT: I see MIDI being mentioned, and it is indeed still used. But for 64k, there are softsynths that sound way better than system MIDI and only take a small part of the size budget, sometimes going as far as physically modelling instruments. 64k is crazy.

These 64k intros could be made run on the bare metal with not much difference in size if there was hardware for it. But the way modern hardware is designed, it is impossible. If you really insist on going "bare metal", try an Amiga ;)

You can still make stuff that runs from the boot sector. You just won't have access to fancy graphics drivers.

I found this: https://github.com/nanochess/RayTracer

Reply on news.ycombinator.com