Id Software Programming Principles | Hacker News Reader

Id Software Programming Principles

blog.felipe.rs

340 pointsphilix00176 comments

> Write your code for this game only - not for a future game. You’re going to be writing new code later because you’ll be smarter.

This really stood out for me. I'm always tempted, while writing something specific, to generalize it. I try to resist that, when I recognize it. Writing a ThingThatImWritingFramework risks ThingThatImWriting never seeing the light of day, or never being used.

Yeh totally..simple but so true

> "Write code that is easy to delete, not easy to extend."

Absolutely! https://vimeo.com/108441214

At 3:57 he mentions erlang. Somehow I am not surpised.

It completely removes the fun though, if you only enjoy generalizing, optimizing, refactoring, designing, and take zero pleasure in delivering an actual usable product. If the product is Doom this shouldn't be a problem - but for most of us the product is form validation or calculations involving plywood. Making frameworks and generalization is the only reason I manage to keep doing what I do. (I'm exaggerating somewhat, but I think this is a clear distinction between two kinds of programmers those who love the code; and the craft, and those who like making products. Have too many of one kind and you'll ship a mess. Too many of the other and you'll never ship)

I've never thought of it this way. While I don't necessarily agree, I'm thankful for this insight.

>> Making frameworks and generalization is the only reason I manage to keep doing what I do.

<sarcasm> This could be written on the tombstone of JS. </sarcasm>

I hate writing boilerplate and I'd rather have pineapples shoved up my behind than write code of the kind that should be in the freaking standard library.

But yeah

This is what always bugged me about java. I'm only just above a beginner, but it seems to me that in 2017 you shouldn't have to write so much crap just to read from a file.

BufferedReader, FileReader, blah blah blah

http://www.mkyong.com/java/how-to-read-file-from-java-buffer...

Maybe there's a good reason to make it this difficult, if so I'm not aware of it.

There is - but there is no good reason to not have good naming plus also a simple wrapper to do the most common cases. That the IO framework is very general is good, but that you need tons of boilerplate for a common scenario is bad. C#

  var txt = File.ReadAllText("file.txt");

This is exactly the abstraction you want if you want to read a (whole) text file. The point of abstractions is to pick the right one. For IO it's likely best to have many layers of abstraction so you can choose the simple top level function or use a more complex one when needed.

I'm sure there is something similar in Java these days too. Would be a huge mistake to not have simple IO helpers to the std library.

Like this?

https://docs.oracle.com/javase/7/docs/api/java/nio/file/File...

Something like that, though the method to read a whole file into a single string seems absent for some strange reason.

>It completely removes the fun though, if you only enjoy generalizing, optimizing, refactoring, designing, and take zero pleasure in delivering an actual usable product.

Then maybe that programmer should go into teaching instead?

nope, I've delivered considerable value to my team by essentially letting the people who enjoy shipping end user features do that, and extracting frameworks and libraries so that the overall product stays coherent and easy to work with, or occasionally stepping in and saying "why don't you let me build the general pieces for this feature first, and then you can use them to make your life easier". as long as you have a decent feel for yagni, it's a useful role to play.

> go into teaching instead?

if teaching earned me as much as a commercial programming job, then yes. Unfortunately, most teaching positions are fairly poorly paid (compared with the qualifications required and the opportunity cost). That's a sad fact, but one has to consider it.

You have that backwards. People who don't enjoy these things should go in to management

People who enjoy actually building and shipping something instead of creating abstractions should go into management?

You do realize that this includes all the hackers with the original sense of the term, which are not about finely tuned abstractions and design patterns, but about creating things and hacking/kludging it out to get there faster?

I have seen projects drown to death in a sea of frameworks, supposedly reusable libraries and generalizations developers deemed necessary. But it was fun for some, I am sure.

I've seen projects fail for a variety of reasons. A few fall under scenario you described. But, I've also seen quite a few projects succeed by making use of in-house developed libraries and frameworks. It probably more depends on who is on the team than whether or not the code is written to be generic enough to apply to other problems.

e: fixed typo

No doubt that's often the case. And depending on which type of programmer you have too many of, it might be failing for that reason or the opposite (insufficient generalization).

If you ever think "we are on schedule and this codebase has just the right balance between getting things done and doing it right" then you have a well composed team.

Exactly. e.g. it might be much easier (for some) to come up with a finished product/Game with Unity, with mouse clicking here and there... But it takes away all the fun of game development. On the other hand, I'm pretty sure it is hell a lot of fun working on Unity itself... So all the sweet is for their in-house devs :) (Same goes for Rails, Node, .NET, etc...)

Generalisation is always needed in a code base. However, only generalise pieces that you actually use in your code. YAGNI (You Aren't Gonna Need It) is a great principal. You should follow it -- until you need it. Then, of course, do what you need to do.

People often think that projects get slower as code is added. However, the systems we work with are dramatically more complex and based on more pre-existing code than ever before. Because that pre-existing code has been generalised out, we can write new code easily and quickly.

There are two situations that you need to think about wrt to this issue (IMHO). If someone needs to do something that you have already done in your project, it should be simple and obvious how to do it. The more often something needs to be done, the more simple and obvious it needs to be. However, if someone needs to do something that hasn't been done before (even if it is is very much related to what already exists), then all doors should be open. The code should not imply how new work should be done. It definitely shouldn't try to guess what you want to do and do it before you get there.

Generalise to the simplest expression of the logic of your application. Don't generalise towards the expression of the logic of all future applications.

The article kind of already says this:

  > Keep your code absolutely simple.
  > Keep looking at your functions and figure out
  > how you simplify further.

I thought I was the only one. Thanks for taking off a load of guilt. It's okay to enjoy this stuff. Very sage comment. Never finish because of the fun of building.

"...only enjoy generalizing, optimizing, refactoring, designing, and take zero pleasure in delivering an actual usable product"

Too many projects suffer immensely because of people who don't give a shit about delivering. In that mode, anything is an excuse: we need to reactor, we need to build a better framework, we need to revisit the requirements, etc.

Then real progress gets slowed down because of some ill fitting development philosophy that some of those folks pulled out of their asses.

What you need are wise programmers that care about delivering product, wise enough to balance long term maintenability and code health with actual delivery schedule.

The keyword is "wise".

Of course I take some pleasure in delivering products, and I try very hard to not over engineer or over generalize. I take pleasure in it, that doesn't (necessarily) mean I overdo it.

But if the job didn't have those elements I wouldn't be in it - and our product would be suffering from that too.

Wise devs are hard to come by.

This is basically the Mythical Man Month in a few lines.

You once wrote something specific, and it was great. Then you had to write a second thing, and you remember things from the first, so why not make it more general for the inevitable 3rd, 4th, 5th to come? And, then, you fail to deliver the 2nd.

I don't have my copy handy, but I believe that's actually the second system effect

You're correct, but it is also described in the book, "Mythical Man Month".

Their tools are also a form of generalization. Furthermore, it is definitely possible to create value by generalization.

Yeah, sometimes. And other times, the whole contract gets spent on a framework that only makes sense if there's another contract, and ... well, I guess that never happens anymore.

Didn't someone say premature generalization is the root of all evil?

That's indeed a very good advice.

All abstractions have a cost and no abstraction is better than the wrong abstraction.

I found this talk on the topic very interesting https://youtu.be/4anAwXYqLG8

In the same vein:

prefer duplication over the wrong abstraction ( https://www.sandimetz.com/blog/2016/1/20/the-wrong-abstracti... )

It helps to allow yourself to be philosophical about what you're writing. Sometimes a generalization just moves around the problem. Sometimes the generalization you want competes badly against cut and paste.

Increasingly I try to find a strategy that transitions towards the generalization over time by following a path of "lower friction instead of greater friction." It's easy to increase friction by adding a new configuration step, new concepts and idioms that are discordant with the rest of the environment, and then the point of doing it gets lost - you don't want to pay for that overhead most of the time, you can't ship code that way.

Generalizing is part of how you'll be smarter later though. You cannot generalize something unless you understand its essence.

It's easy to confuse abstraction (choosing to deal with one, possibly newly contrived, concept over another) and generalization (reducing the number of concepts required to explain something to a minimum). The former is like building a Haswell CPU from a description of 70s microcontroller architecture, whereas the latter is like trying to figure out what CPUs were like in the 70s by looking at a Haswell core under a microscope.

> Write your code for this game only - not for a future game.

The quote refers to a future product, one that is very likely unspecified at best, and more likely just something that an individual contributor dreamed up. Unless we have some sort of multi-phase contract, how do we know we'll ever build another thing like this again?

Markets change, hardware changes, experience changes. Hell, building this thing may show you that you should never build another one like it.

Generalize where sensible inside the project, sure. But as the quote says, build what you're building, not what you're not.

If you know you're going to have to experiment a lot on some aspect of the game/product, it seems to make sense to isolate that part, removing the commonalities into some kind of framework (perhaps informal, jury-rigged, so you can iterate on just that aspect.

A bit like the Wright brothers building a wind tunnel, so they could experiment quickly with control systems.

But Id wrote some great games, so maybe I'm missing how they handled this aspect...

There is a saying in game circles, either one writes a game engine or a game, as most studios tend to die spending the publisher's money build the next great engine.

Factories. I've got a codebase to work with using factories everywhere. Static arrays of them. Classes are given hardcoded enums and there's macro magic all over the palce to join classes to the factories that make them, and to generate all the factories in these static arrays of factories.

Sometimes the static array of factories contains a single factory. Creating a single item. No variation, no types, just one kind of one item.

The item is needed once. In one place.

It reads like a "get shit done" manifesto, and good rules of thumb for most programmers tasked with doing just that. Worth remembering that iD had some of the most advanced 3D and networking code for years and really pushed the envelope and the industry forward in many ways (first huge shareware company, first big company to allow mods, first big company to make code open source, etc.)

It is easy to slide into an OCD mindset when programming, to make things tidy and proper. It feels dirty to make stuff just work, to make stuff disposable, but evolution operates a lot like this - many little, reversible mistakes that add up to big improvements quicker than any other method.

> Programming is a creative art form based in logic. Every programmer is different and will code differently. It’s the output that matters.

This one is also nice, especially for mid to large software houses. As long as a common denominator is respected, I guess.

It's enabled by encapsulation.

Some of these things, like focusing on the task at hand and trying to be as simple as possible, not thinking too much about the potential futures is important to me. If all would do that maybe there would not be 1000s of weird half baked npms and such.

He described how literally hundreds of projects were successfully completed in C, targeting multiple platforms, while being first to market with innovative technology, with a small team in a pre-Internet world,

These are achievements beyond belief.

I'd very curious to hear Carmack's take on this. After all Romero was making game levels, not the engines.

Romero made tools like DoomED and also wrote games before id.

Yes, he's a skilled coder as well. He just seems to get overshadowed by Carmack.

He was developing large parts of e.g. Doom: He didn't write the BSP traversal and texture mapper (the graphics engine), but he did write monster AI, level behavior, and level editor.

I think, I heard him state in an interview that he also did a lot of development on Quake in addition to the levels. Obviously the graphics engine was Abrash and Carmack's

I think Carmacks programming principle would be "be a genius"

Then mine would be "don't be an idiot." :)

No, if you learn more about him, its really more work ethic. I think he is smarter than an average person, but not smarter than the average programmer by a whole lot. Just robotic like work ethic.

There's a bit at the end of Chuck Yeager's autobiography, where he takes to task the idea of test pilots being men who have "the right stuff" (specifically addressing the book by that title).

"And in the end, the one big reason why I was better than average as a pilot was because I flew more than anybody else. If there is such a thing as 'the right stuff' in piloting, then it is experience."

Carmack has programmed professionally at least since 1990, he was 19 then, 20 when he founded id Software. Genius or not, he obviously has a lot of programming experience, and familiarity with a subject counts for a lot. He has worked really hard, written millions of LoC. Obviously his "genius" lies not in learning a programming language and syntax, anyone can do that. But he evidently has a knack for developing new techniques (e.g. in graphics). He has a good understanding of especially light physics and hence physical based rendering. Considering his ventures with Armadillo Aerospace it's probably safe to say he's got a good understanding of physics in general. How many game developers went from games to aerospace?

http://www.geek.com/geek-cetera/elon-musk-and-john-carmack-t...

https://techcrunch.com/2015/01/16/elon-musk-tries-to-woo-joh...

Elon Musk practically offered Carmack a job at SpaceX, tweeting "Well, if you love rockets, come work on them with me at SpaceX!"

Carmack is a very enthusiastic person, who is hard working, and genuinely passionate about everything he pursues. I have "followed" him for a good while. It's not difficult to tell he's a lot smarter than your average Joe, but he also seems quite a bit sharper than your average smart programmer. I recommend watching all his talks and interviews.

Some amusing and revealing stuff on his childhood: http://www.giantbomb.com/john-carmack/3040-4576/

"In the gaming industry, there are a lot of people that are specifically in it because they love games and they want to create things. My love for programming is a more abstract thing. I'm taking a great deal of enjoyment writing device drivers for Linux. I could also be having a good time writing a database manager or something because there are always interesting problems. There are some things that are inherently more rewarding than others. Graphics and games are probably the most generally rewarding area of programming."

On software engineering: https://www.youtube.com/watch?v=wt-iVFxgFWk

Another smart and nice guy in the business is Tim Sweeney. About the same age as Carmack, and also founded Epic the same year id Software was founded, at 21. Sweeney got his first taste of programming at 11.

"It was astonishing seeing the IBM PC. It was such a crisp machine. I'd seen a Commodore PET computer before, and it was a crappy device that never really did what you wanted, I couldn't figure it out and nobody was there to show me how to use it. I sat down at this IBM and every key you pressed made this bright, quick sound. [It had] this nice clear screen, this very powerful basic programming language. It just took me a couple of days to figure out how to use that. It was totally love at first sight. From that point on, I tried to dedicate all of the time I had free to learning to do more with computers."

He seems to share a similar passion to that of Carmack's.

This is first class post-hoc bullshit. They weren't making prototypes or reusing code, they sure as hell weren't composing high-sounding principles.

They just got on with it because they were talented experienced and motivated. For that reason, the rest of the talk is very inspiring - so listen to the hour-long video (half talk, half questions), and not the article which only has the post-hoc bits. https://youtu.be/E2MIpi8pIvY

> they were talented experienced and motivated

Agreed and that's the whole secret sauce right there, combined with: no distractions (social media or indeed even "coding forums" with constant flame-wars on this and that language/stack/paradigm) and crucially also no distracting "stack" or APIs to speak of (bare metal coding literally "on top of the BIOS" in these days, no OS/GUI/multithread/GPU APIs).

The initial core team spent their entire youths 24/7 getting insanely good at ASM and C and numerous gfx tricks and then "they just played the piano" to the best of their accumulated abilities. Observe how much longer the Dooms took compared to Wolfenstein and priors, and how much longer the Quakes took them compared to the Dooms.. as they were slowly entering the age of Windows native & Internet multiplayer even they too got slowed down a bit (were shocked how for Quake "just waiting for 'ze engine' took a year"!) compared to the earlier works --- of course still managed to ship high-quality products, but still

Another eg of distraction-free programming: http://www.atariarchives.org/deli/cottage_computer_programmi...

> "No prototypes. Just make the game. Polish as you go. Don’t depend on polish happening later. Always maintain constantly shippable code."

I disagree with this so much, prototypes and proof of concepts teach you so much but usually they are crap you will always write it better a second time. Throw away the prototype and re-write it as a much better implementation.

He addresses this during QA (@28:00).

> If you plan to throw one away, you will end up throwing away two.

For every prototype / POC thrown away there are 9 other prototypes ending up in production or getting sold as business software. If you feel the urge to write a prototype to learn something new please do not show it to your manager/sales rep!

Protoduction!

In games industry nobody ships prototypes. If you manage to put one on Steam it's not going to sell much either. And yes, writing prototypes is normal, read on "Cerny method" if you are curious.

The existence of so many half-baked and abandoned early-access games on Steam would seem to belie this claim. It's not uncommon to see a few hundred reviews for games where the developers have ghosted without finishing.

How so? You think early access half-baked games sell? Or you think few hundred sales is a good number?

People making these games are not in the games industry because they are not professionals (they don't live off these games). Also, it's curious how people who have not even shipped a prototype game down-vote opinions from the insiders. Good reminder when reading comments on something outside of my expertise here.

What if you prototype a game mechanic and it's shit?

The article talks about similarities between common agile practices and John Romero's programming principles. There is a world beyond the gaming industry where you can sign big contracts and ship broken software and no one dares to complain.

They were four people shipping multiple games a year. They couldn't afford to prototype or build proofs of concept. They had to ship.

Doesn't sound like a generally useful tip.

"Make something you can sell" seems like a pretty straightforward strategy.

I think there's a distinction between prototypes of parts of the game and the full game.

I remember Carmack building a prototype of the texturing system for Rage to see if it would all work, so maybe the prototyping was always just an implicit part of JC's programming.

I think the distinction lies in always building production ready code.

You can build things to try out and experiment different modes of play or modes of rendering but the code you write should always be production ready. No taking shortcuts because "it's just a prototype".

>you will always write it better a second time.

No. You won't:

https://en.wikipedia.org/wiki/Second-system_effect

Exactly. I have seen several core business rewrites that didn't make it past the "We must create the one true architecture that will allow us to keep this software from turning to spaghetti like the last one" phase.

I think that rewrites happened, it's just that they were called "the next game." If you were gonna rewrite, you might as well change the costumes and sets some, and sell to everyone all over again.

Disagree or not, but it very obviously worked for them.

Reply on news.ycombinator.com