Hacker News Reader

Hacker News

Comment by Xeamek | Hacker News Reader

Parent Full thread

Xeamek

>Large codebase (10M+ LOC) that is routinely compiled on your machine

Is that really 'an average' for modern AAA game?

Damn. That's an order of magnitude bugger then I'd imagine

View on HN

For reference, from Tokei from Unreal Engine reports lines of code (no empty lines)

Just C++ 11,375,669

Total (of everything) 31,379,114

That’s fairly representative of just the tooling side of things for a AAA engine. That’s not counting the logic of the game itself.

it's partly because of the engine code. there's even bigger stuff, especially especially if it's a company with any legacy codebase that's 10-20 years old or whatever (e.g. EA / Frostbite.) one i worked on took hours to compile the first time on a machine with 128gb of ram and a threadripper. the onboarding doc suggests getting some coffee at that point haha

a big part of working on them as a generalist ends up being the ability to know how to even navigate something like that (especially since they're often haphazardly documented)

(part of it is that most of the games "fork" the engine rather than using it as a standalone thing)

it's probably not everyone on the team building that whole thing each time, but yea. hundreds of solutions and millions of LOC isn't unusual

*i just did a quick check with unreal's source, it's ~20 million LoC (assuming I didn't mess up the filtering somehow)

If you've got a 10M+ LOC project and you're not doing some kind of distributed build you're throwing money and time away.

The main time killer in day-to-day work on such big projects is usually the linker step, which is terribly slow with the MSVC linker and doesn't benefit from incremental or distributed compilation (not sure though how much the MSVC linker has improved in the last 5 years or so).

Is it necessary to build and link windows games with msvc? Pretend I know nothing about the subject.

Large code bases built and linked with open source toolchains have solved this with, for example, thinlto. And by "large" I mean orders of magnitude larger than the mentioned game.

Not necessary, but definitely the path of least resistance for developing Windows games, and AFAIK MSVC is required for developing Xbox games. Commercial closed source middleware is sometimes distributed as compiled C++ static link libraries plus headers without being able to recompile yourself.

Genuine question, what code base is 100 million lines of code (I think that is the meaning an order of magnitude bigger that 11 million LOC). I can not comprehend how that much code is anything but dead weight saddled on ‘generations’ of a business’s employees.

If it helps, even by 10 million LOC (assumedly most of which isn't unused legacy cruft) you're starting to dig deeply into templates and codegen instead of actual manual coding. I'm sure with all this AI stuff that will only get larger and larger.

But yeah, outside of some of the largest of FAANG software, there aren't a lot of codebases that can truly justify 100m lines of code. I'm sure GTA VI is over 100m lines but can be cut down to 10m if that was a priority (it never is)

You can use clang for windows builds. It’s what I use for my games. However, I assume that most popular engines might not compile on clang+windows for random reasons.

It's a lot more common than you'd think. I'm not in gamedev but similarly weird (multiple supported userspaces and OS's for embedded an device line) our "full build" is probably approaching 50M+ lines and only quite recently do people do incrementals from a build server snapshots. No bazel or distributed ccache or anything.

It can be pretty difficult.

Especially for games or OS development, you might have shifting toolchains and SDks. Different teams may move out of sync because different teams want different things at a given time.

obviously many teams do (i've seen incredibuild used pretty often.) that's not a cureall, though.

I just cleaned out my build folders and did a full build. 40M+ lines, 3 mins and 35 seconds. If you're not getting similar speed then maybe you should look into adding more machines. Last time I was on game dev the best you could do was share other programmers machines via incredibuild. No thought about adding more machines just for building or using cloud infra

Isn't that average for any C++ software development?

I would have thought maybe 1M LOC at most, but 10M?

We've been around 1mloc for the Drakensang single player and MMO games, and that was 15..5 years ago, with a relatively small team (up to 20 programmers), and budget-wise far away from what's considered an AAA production.

While I wouldn't want to work on a 10mloc C++ code base either, it sounds totally realistic to me.

realistic, but probably far from "optimal" (not that LoC is a good target for anything other than complexity). There simply isn't much incentive to properly refactor legacy code into something more managable. Rather eat the cost with devs ramping up and working around the cruft.

Why on earth is anyone normally compiling more than one or two files at a time? Why do a full build each time you change a function?

Change a basic thing in a header of a core library, or even add a flag to the compiler, and you need to recompile everything.

most of the builds are incremental, but even incremental builds still take awhile when it's that massive (linking, modifying a header / template code, etc)

True, but linker times still suck and don't parallelize well.

Also, sometimes you need to iterate on some very core .h file and touching any of those brings the whole house of cards down and triggers a full or nearly full rebuild.

Reply on news.ycombinator.com