A Programming Language for Games, Talk #2 [video]

Agree with you, however if you watch blows talks he is very clear that he says a lot of these problems are very specific to games. And its games that really benefit from these low level optimizations.

I really enjoy reading the Bitsquid blog (two guys who made a game engine and sold it to autodesk). They have a much more balanced take on the subject (eg http://bitsquid.blogspot.com/2011/12/pragmatic-approach-to-p...) and lots of practical examples (eg http://bitsquid.blogspot.com/2012/09/a-data-oriented-data-dr... , http://bitsquid.blogspot.com/2014/08/building-data-oriented-...).

There are benefits not just to performance but to clarity and debuggability. It becomes much easier to figure out where your program is spending time or where data is being corrupted because work is done in large, homogeneous chunks instead of being spread all over the stack trace.

Understanding the entire stack is in general a good thing but you're looking at it wrong. These guys are good programmers but they have spent decades doing this stuff so comparing yourself to them is a completely unfair comparison. Plus, like everyone else they are not objective observers and have a bias when it comes to expressing their opinions on this stuff. If all you do day in day out is optimize low level code then of course you are going to attach more value to low level optimizations and machine level understanding.

Now contrast this with the likes of Alan Kay, Bob Harper, Benjamin Pierce, etc. and you notice the same pattern. These people couldn't care less about the low level details and instead espouse the benefit of high level theoretical understanding and how such knowledge advances the field.

Both viewpoints are correct and balancing those two viewpoints is where the engineering aspects of software engineering come into play.

I think it's because there are two cases that require flawless performance: when you are pushing the limits of the hardware, and when you are adding a layer that people will build lots of software upon.

Games are a perfect example of the first case. Have you seen how much optimization goes into the rendering of a modern 3D game? They are not wasting hours to shave a single byte like the olden times, but the whole thing is piled with low level optimizations.

And languages are an example of the second case, where you are adding a layer. This is crucial because layer performance is compounded. If the OS is using 75% more operations than it should be, and the language is doing the same, the final application slowness goes up by 3x.

Sure, your smartphone app should conserve battery, and it would be nice if it finished its operations a tad faster. But that's a case where you don't need to scrub assembly code.

I think there are a number of attitudes engendered by the nature of doing game programming, especially low-level engine programming, for a long time. Game programmers aren't unaware of the rest of the programming universe, but the risks mean that they tend to indulge only one or two new ideas at a time for production code.

One such attitude is that the code is disposable - you cut down the buglist until the game ships, and then boom, new project, greenfield code. (or sometimes, same engine, but you can go in and gut it until it fits the new design) Another is that your primary goal as the engine programmer is to push out data as quickly as possible - more things onscreen, with more detail and more effects; if you aren't doing this, then you're probably spending your technical budget to explore a completely new technology instead. Higher-level abstractions are weeded out if they aren't supplementing these goals.

This means that game programmers see the code as something that you scrub out and redo over and over in order to match the new data model you're optimizing against(which necessarily changes as hardware and design goals changes). You're gonna throw the code out tomorrow, so there's no sense in spending a lot of "thinking and theorizing" time on it to reach artificial ideals. You have a clear destination to reach instead.

The ideal provided as an alternative is to reduce your coding to an efficient, even boring process that consistently gets you towards a working solution that can be optimized later, even whilst you're keeling over from crunch. Game coders don't optimize everything right off the bat because they need real-world data for the test case, so the engine only really gets performance tuned in the later stages of a project as more of the content is completed. The early days are often more important because they're done with the understanding that all the maintenance work needed to ship must be done before ship, so you can only take on technical debt on a feature if it means the feature will never be touched again; otherwise the only reason to do it is to hit one of those stupid publisher milestones that requires a feature to be in place prematurely.

Another issue is that not all abstractions are available. Stuff reliant on GC is generally out, because control over memory allocation is a must. Stuff that bloats memory usage, also generally out. Stuff that leads to unpredictable execution times(e.g. lazy evaluation) is dismissed immediately. Imperative style is still the groundwork for game code, if only because simulated worlds encourage massive amounts of mutability and global state.

So at the same time there's a tunnel vision that comes out of this approach. Not every game needs that level of optimization, and not every optimization is a matter of peering at the hardware and seeing what form the data needs to be in for go-fastness(which is 90% of the Acton slides). But because the programmer also can't predict "how much" performance is enough, they tend to estimate conservatively, defer to C++ and to their hardware once more, and just pull out the old bag of tricks to make it go. And then ship at 30hz when it turns out that the estimate still wasn't good enough for 60.

I find the ideas of the slides good, and as a coder that learn programing from the database point of view I agree a lot. But the specific examples are too "the small stuff". I think that a large-scale code organization could be a better example of how data-oriented programming will yield a better/easier solution that focus in what happened in a loop.

This kind of example reinforce in the newbie the idea of obsess for the performance of the tree and not of the forest. But really, where are a better example?

You needn't be bothered about not sharing the same frustrations as others. Jonathan Blow is a game developer so the low-level memory management is very important. As a high-level programmer, you may never run into the same issues. That's perfectly fine and doesn't imply anything about "good programming". Enjoy doing what you're doing and at some point you may come up with your own list of grievances and ideas about implementing a more perfect language.

Kill your idols. Those people are brilliant and extremely talented at their craft, but like all of us they suffer from the near sightedness that results from decades spent in a few especially detailed fields of expertise. And at the end of the day, they're just humans - susceptible to biases as much as anyone else.

Of course you want to listen to them and incorporate their ideas and viewpoints; but part of growing as a professional is knowing how to balance that with your own experience and knowledge, and having the courage to say (after thinking long and hard) "this guy is full of shit".

DHH's opinion on web frameworks is more valuable than his opinion on game engines. Torvalds' opinion on operating systems is more valuable than his opinion on robotics programming. The same applies to anything you hear from experts. In fact, being in a position of expertise can often lead you into a sense of false certainty about your knowledge: I am a master of X, of course it applies to Y! Well, some insights might, and a few general truths might emerge, but that's the extent of it and it's certainly not systematic.

I don't have any extended experience with non-game programming so I may be completely delusional but it appears to me that the most of non-game programming is not as much about programming as about solving some problem with a computer. E.g. there are programmers working for Coca Cola, Match.com or FedEx. But these companies don't sell any programs and none of their customers really care if their business involved any computers at all. I am pretty sure about this because some businesses that hire programmers today had existed since before computers had been invented.

Game programming, on the other hand, is exactly opposite. The game does not solve any problems at all. Its value is in itself. It's not just gameplay experience or art, it's the binary code and data: there are quite a few cases when a game with the same gameplay and art but running on a different platform does not sell nearly as well. Of course, the gameplay and art are vey important factors in the game's success and the reasons the sales of the ports vary so much are not related to the particular code as much as to the competition and consumer preferences. The point is that the game companies literally sell binary executables that the game programmers create as opposite to the other businesses that sell services or products, which are not binary executables.

So my answer to your question is that no, it is not "good programming" in the mainstream sense of solving a problem with the help of a computer. It is, however, pretty awesome programming in the literal sense of making a computer do what you want it to do.

This is quite common in the gamming industry where performance matters more than anything else.

You won't see them worrying to write portable code like FOSS advocates think they do, because it is everything about extracting to the last drop of performance juice of the machine.

The funny thing about Mike Acton's talk for us old timers, is that if this talk was 20 years ago, he would be bashing C and Pascal dialects for being too bloated, while advocating straight Assembly code.

> This is not what got me into programming, and yet these presentations by coders I respect make me feel like I'm a dummy who's completely "doing it wrong".

You've taken their necessity and assumed it's the same as yours. AAA game developers don't do these kind of optimizations because it makes them feel smart or better, they do them because they have to.

> parallel outlook on how programming should be done.

Nor do they think that this is how programming should be done. It is how programming should be done within the context of AAA game development.

Don't make the mistake of thinking that software development is a singular or uniform field. It isn't. If you took nearly any of these "black magic devs" and asked them to iteratively develop a MVP and they would struggle to do that in the same way you struggle to understand their development practices.

You are better at solving your type of problems than they are.

> Is this sort of ascetic, hardware-savvy code really what "good programming" looks like?

Again, It all depends on where you are coming from. I don't actually work in the gamedev industry, but spent YEARS doing this stuff on the side as a hobby and so I do understand it. I very rarely get to use that knowledge in my actual job, possibly once a year - however that knowledge has turned me into a key man dependency: on the extremely rare occasion that this knowledge is required; I'm indispensable.

> Because I want to get better at my craft, but it just looks like a whole lot of pain.

As you should. Once you get a handle on this stuff it's intoxicating and addictive. It was painful at the start.

I think why you might be having issues is that you are starting at the bottom. Don't start at the CPU, start at the layer that sits directly beneath you. Java coder? Learn about the JVM. C# coder? Learn about the .Net runtime. C coder? Learn about GCC. Once you understand that move down to the OS, then move down to the CPU.

You can't understand what's happening on the CPU if you don't understand how the code you write gets there in the first place.

Just remember, this is personal enrichment and not requirement. There are very few industries that require this type of knowledge.

This is so true! When this type of developers starts talking about design of the architecture they start discussing optimization on second minute.

Here is a good example (it's a good blog and people are talented though):

http://bitsquid.blogspot.co.uk/2014/08/building-data-oriente...