John Carmack: Script Interpreters Considered Harmful
codingthewheel.com
codingthewheel.com
In game, in-world "toy UI" like that is intended to look like the sort of sci-fi UI you see in movies, but also be interactive. The stuff in movies is made by a motion graphics designer with a copy of Adobe After Effects. Replicating that movie like look, with all the animations and visual effects you see on the big screen, but ALSO making it work as a fully user-aware, state-aware computer interface, isn't cheap, either on an artist man-hours standpoint, or from a lines of code standpoint.
I imagine its expense could be reduced if a lot of the stuff up on screen was boiled down to some kind of binary data - a lot of work is put into optimizing performance of engine-native animation files and effects which are played on the characters and environments of a video game world - but usually UI (including fictional in world "toy UI") isn't considered part of the main tool chain or process, so it ends up either being highly special cased by hand, or very highly interpreted (sometimes running interpreted code through middleware which reinterprets the code again) requiring quite a lot of bloat to look and behave in a way that is presentable and on par with the rest of the game. (Usually at this point in the games industry, people build their UIs in flash, and then run them through middleware like Scaleform, which reinterprets their flash based bitmap and vector artwork as polygonal art, and reinterprets their action script into an engine-savvy language... not efficient for the game, but efficient for a former marketing guy who now wants to do game graphic and UI design, I guess.)
(For what its worth, I agree with Carmack's assessment. Making highly authorable and also highly efficient at runtime, "AAA visual quality" game UI is just a hard problem to solve, and it's a problem which few studios deem worth solving because after so many years of the current system, the benefits of a change have almost become an intangible.)
Because he's clearly talking about ScaleForm/Flash specifically, though, I don't think that his comments can be correctly generalized to all "script interpreters". It's kind of the equivalent of buying a Maylong Android Tablet and going around saying that Android sucks or the tablet market is dead. You just happen to have picked the worst possible representative.
Also at some point certain integration between the engine and scaleform is needed, and you would have to lose a programmer there supporting it.
We tried it, and the consensus was it's not of use to us. I wasn't the one evaluating it, so I can't tell for sure. But resource budgeting (memory, cpu, video memory (ps3)) is very debated thing in the console game development, and everyone wants more budget for his need (audio, animation, builders, ui, etc.)
We also gave a try of an early flash middleware for Dreamcast for NHL2K2 (2000) - there were only 16mb of RAM, and flash was taking significant amount (no scaleform relation). We actually got some pretty good menus, and for a video sports game you need very detailed UI for rosters, trades, etc. We also got people to do it back then.
But then the problem was during gameplay. While in the main menu we had the memory to pull it off with flash, during gameplay all memory was for vertex buffers, textures, game data, sound, animations, etc. We were thinking of swapping out game data (and reloading later) when the PAUSE button is hit (and the menu appears), so flash would have enough memory - but it turned out to have some bad latency - few to couple of seconds. Ideally PAUSE ON/OFF should take no time (otherwise it's annoying).
Another thing was that we still had to render the rest of the HUD (scores, replays, etc.) not using flash - so we had to support TWO technologies for UI.
Hence we turned our backs on it. It was nice, but not for our game.
Rolling your own UI has always been time consuming, and a breeding ground for bugs.
Any middleware recommendation?
I find that dynamic languages with first class functions provide much better abstractions and opportunities for code-reuse than languages like C and C++.
I agree with the point, but not its reasoning. The reason why scripting (read: dynamic) languages fall apart at scale is largely because they're too flexible. Abstractions that are convenient in the small can create unmanageable complexity in the large (unless you are extremely disciplined).
Lua was certainly fine for, say, Angry Birds, and has clearly been a huge win for Rovio in rapidly porting the game.
I've seen at least the lua levels described there.
Also on the dissassembled apk (it is compiled lua by the way, so you might need this[1] to see it more clearly).
It's not "interpreters vs C++" as Carmack supposedly said, but crappy slow badly implemented user languages vs efficiently implemented user languages vs a guru writing C++. The middle option is the one to go for.
Especially with LLVM which makes compiling your code very easy, and it's got a BSD-ish license, so no problem linking it with your awesome proprietary game.
Enter compiled code. Now we have the opportunity to run unknown binary code right within the game's address space. Bad News in many ways. To get the Happy Medium (compile code within the game engine, load and run the resulting executable safely) requires implementing operating system concepts. It's a Big Job.
This leads my thoughts on a slight tangent: why isn't there a Super Awesome Open Source Game Engine that's the heart of all 3D games? SAOSGE could even provide the extensibility framework (compiling, loading), ready for you to expose parts of your games to player programmers.
Unreal is the closest thing, but there's still lots of new/strange/innovative stuff going on in rendering, which means that lots of bits are hard to reuse. Rage, for instance, has a pretty nifty virtual-memory for textures scheme, which no one else has really done.
In particular I'm thinking of Criterion's Renderware engine. Originally funded by Canon it was very popular. Then Criterion was sold to EA and everyone dropped it.
Because developing a AAA engine from scratch is a multi-million dollar enterprise at this point.
You actually need to develop your system faster than the leading edge so you can catch up. That's an expensive ask. Most companies arrive, after 2-5 years, with something that was as good as UDK or idTech n was ... years ago.
Paying the $500,000 or x% of revenues is often the fastest, cheapest way to get your game out.
Reality: taking it to production was nice, but debugging and other tools are not available. I spent a lot of time making it possible to debug the PHP and trace that back to C#. This was less than idea.
Then IDE support. While I can happy without an IDE, most people can hardly function.
Trying to educate people on it was (a) marketing fail [web devs hate static typing], and (b) too hard for people to grasp (functional programming + new stuff = fail sauce).
Also sand-boxing, but this is not of concern to the actual scriptures, it's to the rest of the team.
And something to easy deal with tasks/threads, or state machines, and such.
Also a language that allows you to declare easy what needs to be kept persistent (goes in the game save), what can go through the network (game-play related feature affecting all in multiplayer), or what can be local "effect".
Bytecode interpreters must be carefully optimized, and still usually have an unavoidable performance hit from their cache usage patterns (e.g. from bytecode being in d-cache).
Most people don't know about these things because of the poor / non-existent education in computer science and lack of any recognized qualifications, but that's no excuse. One day this branch of engineering will need to grow up.
It's perfectly possible to write a statically-typed language that looks exactly like Python or Perl or Ruby. It's just that nobody's done it yet.
So in the mean time, you have to pick between expressiveness and ease of development versus safety and speed. There's no concept in computer science that prevents us from having all four other than "it's hard and people are lazy".
Haskell and Go are good examples of good progress in this direction, though.
To be a statically type language you would need to throw the error at compile time.
How do you raise (or not raise) error of this code at compile time and still make it feel like dynamic type language?
x = (rand() > 0.5) ? 42 : "42"
y = x + 1;The type of (+ 1) is Int -> Int
The type of y is Int
Because x is of type Int|String and it's passed to a function that can only operate on Int, the program fails to compile.
Really? You should definitely show us how.
I've been thinking about the issues of dynamic/static languages for a long time, and haven't yet thought of a unifying design (maybe I'm just stupid).
The big difference isn't type inference/dynamic type system, it's the difference between compile-time and run-time. In Go and Haskell, classes are compiled. In Python, classes are created at runtime, so if you wanted type-inference in such a language, it would have to be performed at runtime... Also, proper typing of OO languages is very complicated, see Scala (e.g. sometimes you want the function to return the type of this object, even when it is inherited...).
Python 3 added optional type annotations for function parameters and return types. Unfortunately, Python does not use these annotations for compile-time or run-time type checking. The annotations are just for documentation or tools that want to analyze the type attributes.
>". . . but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it!"
Edit: Fixed formatting of quote.
Neither was C.
There are languages developed afterwards with large-scale development in mind, but you have to ask, what did they add? True type safety? Nope. True encapsulation? Nope. Automatic resource management? Nope. C++ is C with several times more ways to write horrifyingly unmaintainable code. All big C++ shops have a very long lists of constructs that must never be used and very rigorous code reviews to ensure that you don't use any of those features by accident.
At this point, we see that C++ has one advantage over C, and that's namespaces. Helpful, but not helpful enough for "large-scale software development". The major innovation that C++ brought was waking people up to the reality that a maintainable codebase must be curated with extensive automatic testing and manual code reviews. Anything else leads to epic failure.
But wait, that's easy to fix! Let's invent a new programming language! This time we'll call it Java. It will be like C++ but with all the ways to write bad code removed. No operator overloading! No multiple inheritance.
And it's true that Java helped in a number of ways. But it didn't solve the real problems. There is still no type safety; null is an instance of every class in Java, but you can't call any methods on it, for example. So instead of a segfault, you get a NullPointerException, but all that means is that the source of the error is easier to determine. But you still have to write a lot of tests to make sure that your code handles nulls properly. (This is compounded by laziness in design like writing "loggedInUser = null" instead of writing a subclass of User that indicates it's not logged in.)
Multiple inheritance in C++ was a mess, but Java's solution isn't much better. What's the conceptual difference between abstract classes and interfaces? Abstract classes are non-composable class-parts that contain API, implementation, and state. Interfaces are composable class-parts that contain API. Why are these two separate concepts? Why not have a generic "traits" feature? (The answer is because Java is mostly a copy of C++, and C++ focuses on irrelevant OO features like 4 different levels of member visibility rather than semantic annotations like "if you compose this method into another class, it should run after the method from the class it's being composed into". Of course, when you have these annotations, multiple-inheritance or multiple-trait-application works perfectly -- see CLOS or Moose. But all C++ had was public/protected/private/friend, and so that's all Java has.)
This is getting ramble-y so I'll get to the point. No modern programming language helps you write huge codebases. If you want to be able to maintain millions of lines of code, you are going to need very rigorous standards and millions of lines of automatic tests. It's the only way we know of. All more modern languages did was push us from "we're leaking memory because we forgot to call free()" to "we're leaking memory because it's pretty convenient to keep all these huge objects around".
They certainly haven't helped us write more maintainable large projects. If anything, a dynamic scripting language embedded in your game means that you'll have fewer lines of code and clearer separation between components. But it's not such a win that you won't have to review code or write tests anymore.
One I think would help would be the inability to interface with concrete APIs. If you want to have a variable or instance attribute or argument to a function or method, its type must be declared in terms of an interface, not a particular implementation. This way, you never get yourself into the situation where you say, "I don't want to inherit from Foo, I want to implement its interface myself" but can't.
Beyond that, there is only so much a language can do to help you. It's nice to have guaranteed privacy, but if you use it like the Java standard library does, it becomes the worst language feature ever. To prevent misuse of a powerful feature, you need to be smart and you need to have other smart people reading your code.
The "way forward" is to realize that software development is not easy.
Have you used Go? This is one of our design goals. Go is a really simple language. Its features are easy to understand and predictable in use. Go code is also very readable in that you don't need a lot of context to understand a piece of code. There aren't any colossal Go codebases yet, but so far things are looking promising.
Not really. Go keeps all the old errors everybody should know are errors (nullable pointers, shared mutable state, raw types, ...) and then proceeds to add new ones, and packages all of that in a less regular syntax just in case there was any chance to get a good language out of the previous clusterfuck.
(also, which language is Go less regular than? Lisp?)
w/r/t regularity: most of the functional programming languages (e.g. ML, Haskell sans GHC extensions, various Lisps) are incredibly regular, especially in the semantic sense of providing a few semantically simple features and milking them for all they're worth. Go has quite a few special cases (e.g. the make versus new distinction, the iota keyword) and some odd omissions (e.g. simulating union types involves what I perceive as interface trickery; const only allows numbers or strings as values.) Coming from C++, Java, &c, Go seems incredibly regular—the lack of OOP goes a long way towards keeping it simple—but it's not a simple language except in the context of "modern, Algol-derived applications languages." Which it is an improvement on, but it's not regular in the strict sense.
And because of the absence of exceptions, Go forces you to deal with errors at the call site (see the number of times you see "ok, err = Foo(); if (err)..." which is not scalable to large scale software.
On the other hand, return-value-error-reporting does not give a way for deep callers (caller of the original API when the error happens 6 frames down the stack) to try and recover (instead of just bail out, or more generally customize the error recovery policy) the way condition systems do in Smalltalk, Common Lisp or Dylan.
In the end though, bad code is mostly due to bad programming and bad process. Go is obscure enough that the bad programmers haven't heard about it yet. Start offering high-paying Go jobs to anyone with a pulse and you'll start seeing why people hate C++ and Java so much. It's not that the language sucks, it's that the programmers using it do.
It always surprised me to hear that Google "got by" on C++ and Java but I lightened up a bit when I was reading some of Android. Normally you open up Java and are immediately stunned by the smell it's emitting, but when I started reading the Android code, this didn't happen. Classes did one thing and delegated to other classes when they needed something done. The methods were small and made sense. Line of code inside methods were in "paragraphs". There were no comments like "// hack around bug in SomeOtherClassIWroteButAmTooLazyToFix". It was clear that it was the work of someone who knew what she was doing.
I guess I knew it was possible, but was never convinced of it by any concrete code. The standard library, for example, is horrifyingly bad.
Ultimately languages can lead you in the right direction or the wrong direction, but which path you take depends on the programmer. Google requires code reviews for nearly every commit, and they get to hire the top 0.01% of programmers. Imagine what the rest of the world is like, without code reviews, testing, or good programmers.
Go isn't going to fix that little problem :)
Yet even with our quality of code we are still struggling with our massive code bases. C++ build times alone are reason to find an alternative. We hope that Go will work at scale while having many of the productivity advantages of scripting languages.
Between your comment about Android and this one, it looks like your problem is much more with the Java API's than Java itself.
Besides, I disagree: the Java collections are fairly solid with the right mix of abstraction and efficiency. Compare with those of Scala, for example, which require a lot more work before they become decent.
In this instance Carmark is talking about a specific instance - scripting for high-performance games, that still need to work with the tradeoff of high levels of reliability.
He even mentions tempted by functional languages such as Haskell and Caml. Then he lists a couple of reasons it's not appropriate - performance is one, the learning curve is the other.
He then goes on to say that he'd think about it differently, but that performance is such a dominating factor and issue for them. That's driving the majority of the logic.
So I'd not draw too long a bow to suggest that he's canning scripting languages in general. I think the author is stretching with that statement.
you'll have fewer lines of code and clearer separation between components
I think that's the most important statement. If you have millions of lines of code, the answer to to _write less_... and that's where dynamic & functional languages are so important.
I've worked on large codebases. It's rare that one language can cover the whole domain without some big potholes appearing (when I say cover I mean vertically in terms of low-high level code and horizontally in terms of function). That's where your DSL and dynamic languages some into play.
1. checkable function purity 2. checkable transitive immutability 3. code can be divided into checkably memory safe code and unsafe code 4. anti-hijacking enforcement 5. modules with closed name spaces 6. memory is thread local by default (shared memory is typed differently) 7. look for non-null pointers in the near future
It was interpreted but it was compiled. QuakeC was not a scripting language as one would say. You had/have to compile it. The engine has a VM to handle it.
Quite embarrassing, since quakeC was one of the first languages I played around with, and I don't remember the compile step. Then again, I was working from tutorials so it's possible there was just a 'compile & run' batch file or something.
The best thing, though, was the obvious love and enthusiasm he has for his team and their software.
The perceived advantage of something like Lua is that it's easy to write, allows for fast iteration, and designers/artists can use it. The issue is that designers/artists really shouldn't be using it. To write fast code it requires a programmer and if you have a programmer why not just do it in C++ where it will be several times faster?
A common game development pattern is to write the bulk of gameplay code in a scripting language and at the 11th hour in a desperate panic start moving as much of it as possible to C++ until you fit in 360/PS3 memory and are fast enough.
To write fast code it requires a programmer and if you have
a programmer why not just do it in C++ where it will be
several times faster?
Anyway, all the programmers I've known would want good debugging facilities as well, like the ones you get in Visual Studio/ProDG/Code Warrior/Xcode/gdb/etc. - oops, only kidding about gdb! This thing really needs a GUI. This all just expands the time-suck further.
That's not to say it can't be made to work, of course! People can do anything, if they set their mind to it hard enough, and if they wait for long enough, maybe it will even become value for money.
However, we also had a scripting interface. Update a script, tell the software to restart, done. Easy.
I already had a real time app dev runtime / framework in Java for J2SE that immediately started porting to also run on Android. A modern SoC and Android w/ my framework on the desktop and on the device for 3rd party app dev is now driving my future audio hardware dev efforts. First though I get to do a general release of the software dev framework as a product which never really would have happened without Android showing up. Things changed a lot recently!
Anyway, good idea with the scripting / Blackfin deal though as that can pay off big.
I would certainly win this bet if it involved all the experienced programmers of my acquaintance.
This is basically an equivalent model to web-browsing. You don't load compiled Javascript into your web browser's process address space and jump into it; you put an interpreter between you and it, both for the sake of the code (JS has its own debugger, which treats the browser as a black box that scripters rarely have to think about, and never see backtraces from) and for the sake of the web browser (which doesn't have to deal with any consequences of bad scripting affecting the validity of the browser's own data structures.)
Take that and shove it up your cross-functional agile team :-D