Software Is Hard (2007)
gamearchitect.net
gamearchitect.net
...and Unreal had been licensing their first person shooter engine since 1996, though it wouldn’t be until two years after this post that they dropped the price on UE3 to the point where hobbyists could afford it, then made UE4 damn near free, to try and grab some of the indy game love that Unity has been getting since its first much-lower-price release in 2005. And don’t forget Id, who’d been licensing their engines since, what, the Doom days?
Big games are still obscene sprawling messes, and so are little games I’m sure, but we are kinda at a point where nobody ever needs to write a FPS engine ever again, unless they specifically think that sounds like a fun project. Or any other kind of basic game format, it’s not like the four people who made Untitled Goose Game had to do much for the initial steps of getting a goose wandering around a 3d modeled world full of objects with a decent caricature of basic physics.
OTOH FPS mechanics like running, jumping, shooting are the core of the game and get re-implemented differently each time. Game engines don't really have much interaction implemented, and the physics they provide still has a lot of bugs, so behaviors end up with custom code most of the time.
Assets are kind of hit or miss as to whether they're reusable, and usually need some tweaking if they are reused.
Overall I'd say the reason games have moved to 3D is because of hardware support and better gameplay mechanics, rather than code reuse. Untitled Goose could have been an isometric tile game made in RPG Maker or whatever but they would have had to spend more effort on graphics and the physics glitches wouldn't be as funny.
Apparently Minecraft is still the top-selling game in the world; the code there was written completely from scratch.
I wonder if it's really software that's hard though, or that life in general is hard and software failures are just more visible.
So, yes, software is complex. And it will always remain complex. It is doing monstrously complex things under the hood, all abstractions are leaky, and we are definitely past the point at which one person can even reasonably understand the entire stack from transistors all the way up through OS, compiler, language, etc. We must accept this fact of complexity and formulate ways to deal with it, to contain it and reduce it when we can, but it will always be with us.
Beautifully put and we definitely are, I think the trick now is knowing which bits of the layers underneath you need to know to interact with things well (in terms of your goal), I came up on computers in the 80's and used C and Pascal, those early experiences have been useful for 30 odd years because often I have a 'feel' for what the computer is doing underneath that younger (and really capable!) devs lack.
Often when performance is really critical I'll go look in the source code for the tool to get a feel for what it is really doing (even though I haven't written C in a long time) which is regarded as voodoo.
On the flip side of course is I can build things in an afternoon or a day or two that simply wouldn't have been possible with months or years of work and a dev team of dozens back then.
My next side project is a tool for motorcyclists, you drop pins on a map where you are going to be at points in time and it then pulls the complete meterological data for those points and does some calculations (are the roads likely to be wet, icy, show the direction of the wind, show the windchill at 30mph, 40mph, 50mph etc) with the ability to set recurring routes and email you the day before something like "Tomorrow morning, there may be ice on the roads, wind chill will be 5C, feels like temperature at 40mph -2c, Sunny, low winter sun so wear your shades/visor"
The data to do that didn't exist 30 years ago and the GIS tooling (I'm using open street maps) to process the entire UK would have cost millions.
Spoken like a man who has never lost an afternoon to Wikipedia.
All the way back to nickb: https://news.ycombinator.com/item?id=62453
In the choice between simple (and cheap and correct) vs complex (and expensive and buggy) customers and stakeholders with very few exceptions go for the latter.
Our software is exactly as buggy as we want it to be - a decision we make with the tradeoff of cost and complexity.
Often simple is not correct. In much of the code I have worked on there is simple code for 99% of cases, but correctly handling the other 1% can be much more complex.
Also, simple is often not fast. An implementation of bubble, insertion, or selection sort is almost always simpler than quicksort, mergesort or especially heapsort.
Web browsers are a really good example that some software needs to be complex to do its job. Web pages can do pretty much anything so web browsers have to support that which requires a bunch of code off the bat.
Not having correct behavior in the 1% of cases is unacceptable because that introduces security problems. And not speeding up webpages as much as possible will make web browsing very unpleasant.
I think if we got a fresh start, we could redefine what a browser needs to do and make it simple and faster, but supporting the web as it exists today requires complexity.
The point is that web pages don't inherently need to do "pretty much anything." The web could have been simple and browsers could be simple. Stakeholders decided that no, we want more and more and more and even more features.
And when you say simple is not correct, it is often because someone wants something complicated instead of acknowledging that simple, in fact, does all they really need.
I did kindof address that in my final sentence. But there is a fair amount of inherent complexity in what I would want from a replacement for the web.
The web interface for Github should still be possible. Doing that would require a graphical layout engine (eg something like css), some way to manage authentication, and some way of submitting user data from a form. None of those require a turing-complete language, so I would perhaps be in favor of not having a JS equivalent, which would dramatically simplify the task of a browser, but there is still a lot of complexity there.
> And when you say simple is not correct, it is often because someone wants something complicated instead of acknowledging that simple, in fact, does all they really need.
The easiest case to say the simple is not enough is with encryption. I am not aware of any simple encryption algorithm. If you want to achieve, communication that others cannot eavesdrop in, simple is not enough.
Numerical stability in all sorts of applications is important and it is hard to achieve. Projecting the real number line into finite bits is hard to do in a consistent way.
You didn't contest my simple vs fast claim, but a good example of that is pathfinding algorithms. Dijkstra's algorithm is simple (relatively), but for many video games, a more advanced algorithm like hierarchical A* search or jump point search is needed.
I disagree very much. Most crypto is very simple to use as well as implement. From DES to Chacha20, you can fit an implementation on a business card or two.
> Numerical stability in all sorts of applications is important and it is hard to achieve.
Counterpoint: I hardly ever need to worry about numerical stability, and most of the time I can just throw more bits at it.
> You didn't contest my simple vs fast claim, but a good example of that is pathfinding algorithms. Dijkstra's algorithm is simple (relatively), but for many video games, a more advanced algorithm like hierarchical A* search or jump point search is needed.
I wish the problem of complex software were just algorithms. Because most algorithms are easy to abstract in a box with loose coupling, and even the more complex algorithms usually boil down to some dozens of lines of code.
No, the millions of lines of code in big bloated applications are not made up of that.
Similarly, I can tell you that dealing with ill formatted feed data, parsing any given value can be trivial, until you find some random example of data that abuses some seperator. Then suddenly you need to do thing like try seperating on X and see if you get data that looks right, else seperate on Y. Oh, and include some data (if it exists) from a prior seperation based on Z. I feel bad for anyone who has to modify my code. Hopefully they won’t ignore the unit tests that include all the gnarly cases...
Yes, if you accept complexity, this is where you end up. The other alternative is to reject complexity. Stop accepting and trying to make sense of broken data.
At this point, we get back to my point because you'll say that someone (a stakeholder) demands that the program works with existing legacy/broken/misguided systems/users. Sometimes that is genuinely the only reasonable option, but all too often I see people introducing more and more features and complexity instead of figuring out whether it's really necessary or whether the intended end result can be achieved with less.
I think you can see evidence against your point and for the GP's point if you look for "Falsehoods programmers believe about X" articles. Whenever software interacts with the real world, corner cases abound.
I'm well aware of those corner cases. And, at work, I'm dealing with.. well, not corner cases, but "real world" stuff right now, related to dates and timezones. None of what I'm doing right now would be necessary if people had the guts to decide that internally and in logs, everything is always going to be in a single format such as Unix time. Unfortunately, people made bad decisions and there are stakeholders so I'm adding complexity to the software.
If it were my software, I would outright reject this complexity. It is not needed for the software to do what its core purpose is.
People like to argue that the real world is absolute and software is strictly inferior if it cannot deal with all the complex cases people in the real world attempt to shove into the software world.
My argument is that a good engineer can look at a (seemingly) gnarly real world issue and finds a way to make it simple. Not all complexity is inherent.
Using OpenBSD after Linux is quite illuminating in this regard. At points it might seem like it's lacking features, but on the other hand you find lots of cases where it achieves exactly what you could achieve on Linux in a simpler manner and with fewer features & less complexity because they took a simpler approach to it, and in doing so, made a bunch of features a Linux user would look for simply unnecessary.
> No decision is ever final. Time and again, the Chandler team hashes out compromises on complex issues, only to hit reset when someone new joins the project with new ideas or when it turns out that someone wasn't really satisfied with the compromise.
God, this whole process happens within my own head on most of my projects, especially games. It's infuriating.
"Painting is easy when you don't know how, but very difficult when you do." (E. Degas)
(or for a more literal translation: "Painting, that's very easy when you don't know how to do. When you do know, it's very difficult.")
(*) (Although Ecce Homo by E. G. Martinez is a good image of what can happen to the software you're working on when management adds someone to help you out ;).
I think about this sometimes. When I was 15, and just learning to program, I thought I could code anything. And if you look at history, I was perhaps right.
Now, as a seasoned professional software developer, all I see is pitfalls everywhere...
And this brings me two sides of the argument: should software capture every aspect of human life with all of its complications, or humans should change their lifestyle to fit a software framework? This is probably a never ending discussion, and I'm keen to listen to both sides.
Think about food. Pretty much every world cuisine is rooted in a bunch of non-optional requirements imposed by local climate, soil, etc.
Even language has shifted to adapt to writing systems over time.
Unicode exists because people all over the world want to represent their written language on computers, and that seems like a reasonable thing to want. Software needs to serve reasonable human needs such as writing systems.
ASCII doesn't even cover all of the characters and symbols that are normally used by English speakers.
Such as?
Why didn’t you use a larger font size to ensure everyone sees your message?
People adapt their behavior to their communication media instinctively and continually, and are as routinely deliberately controlled thereby.
Yes and a huge downside of this is that we're now stuck with an incomplete set of them.
About half the programmers I know (including myself) already write dates as 2019-10-27 even when using a pen, it really is a more convenient format.