Hacker News Reader

Comment by bob1029 | Hacker News Reader

NIH is amazing as long as you are realistic about what you are taking ownership of.

For example, the cost of maintaining a bespoke web frontend "framework" that is specific to your problem domain is probably justifiable in many cases.

The same cannot be said for databases, game engines, web servers, cryptographic primitives, etc. If you have a problem so hard that no existing RDBMS or engine can support, you should seriously question the practicality of solving your problem at all. There's probably some theoretical constraint or view of the problem space you haven't discovered yet. Reorganizing the problem is a lot cheaper than reinventing the entire SQLite test suite from zero.

View on HN

> The same cannot be said for databases, game engines, web servers, cryptographic primitives, etc.

It absolutely can in many cases.

- Many applications don't need much more of a database than files plus perhaps a runtime index. Spawning a database server or even embedding SQLite can be overkill in many cases.

- Most games would be better off with a bespoke game engine IMO, especially those made by small teams. The only real advantage to using an established engine is the familiar asset pipeline for replaceable artists but you pay for that with a huge overhead. Remember that custom engines or at least heavily modified ones maintained by the game developer used to be the norm - "just use Unity^WUnreal" is a relatively recent trend.

- There is no real difference in complexity between a simple web server and FastCGI application.

- Not all uses of cryptographic primitives are a security issues. If you calculate a hash to check against corruption you can implement that yourself (or copy an existing implementation) instead of pulling in a huge library. If your use cases is decrypting existing data offline then again you don't really care about cryptographic security only about getting the correctly decrypted output.

And there are many more cases like these. Learned helplessness when it comes to "hard" subjects helps no one.

Just because some existing solution "solves" your problem doesn't mean its the best or most efficient way to do it.

> files plus perhaps a runtime index.

The latter already adds a lot of complexity, especially if the application has to scale up or out. If scaling up, the index will increase in complexity and re-implement some clever algorithms that have already been solved by the other. If scaling out, you'll need some locking mechanism if multiple processes can write. It quickly becomes logical to switch to an established solution then.

But the choice becomes more logical then; if one picks a scalable database or whatever without knowing they're going to need it, that's a lot of wasted effort.

> - Most games would be better off with a bespoke game engine IMO, especially those made by small teams.

I disagree; writing a game engine, even for simpler "indie" graphics, requires specialized knowledge and a dedicated engineer, whereas with off-the-shelf engines, non-programmer / creatives can build a good game without in-depth knowledge. A small team needs to be especially careful with what they choose to spend their time and money on, and solving a solved problem like building a game engine is a huge investment. Even big studios struggle with this, see also the dramatic release of Cyberpunk 20something by long-time own-engine-building veterans CDPR. They're switching to Unreal for Witcher 4.

Any mature game engine also comes with mature developer tooling. Think profiling, debugging, assets management, map editing etc. That’s a lot of initial hurdles to overcome before you can even start working on your game ideas. For indie / solo developers who tend to be, you know, driven by one’s creative urges, this is not just a huge time sink but also a good way to ruin your motivation by never moving to the part where you get to actually create the game. When your time and energy is constrained you need to choose your battle and for indie developers this often means forgoing the technical part (unless you really need to) and focus more on the game part.

I have seen many people say “there are more games engines written in Rust than games written in Rust” and I wonder if what happened is that the software developers fell for the allure of building their whole stack from scratch and ended up finding out it sucks to actually develop a game for an engine with no tooling and “making an entity inspector that lets you click on an enemy and edit its attributes” isn’t exactly the sexy part of game development.

The trap of "building a game engine" vs "making a game" has existed long before Rust.

When games were really simple, so simple that "making a game" vs "making an engine" had no real difference (say, building a game for 8 bit home computers) then it wasn't a big deal. The concept of "games engine" didn't really exist back then, or was uncommon at least.

But nowadays? A small indie team, or a single dev, are likely to go down the rabbit hole of making their own game engine, forced to solve a lot of problems that are already solved in existing engines, rediscover the wheel, and get frustrated because the step of making the game never actually comes.

I think it makes more sense to use a pre-made engine, unless the game really is so different this isn't an option, or the devs really enjoy making engines (which is valid, and you learn a ton of cool things, but is also a hurdle in the way of actually making a game).

Every games programmer enthusiast I know has a half-made, abandoned, badly performing engine with which they've made a couple of demos or a barely functional unfinished game... (I'm also guilty of this, but given my age, mine was written in C++!).

The author of one of the most popular games of the last decade disagrees with you: https://noelberry.ca/posts/making_games_in_2025/

In which part is Noel disagreeing? He didn't write his own engine either, he just didn't use one (but didn't write everything from scratch either, note he uses lots of game/graphics frameworks, which in a sense is like using a "meta engine").

He's also making a 2D game, which traditionally is closer to game making of old (think 8 bit home computers).

Finally, he's not arguing against engines like Godot or Unreal, he says those are acceptable but he just didn't need them. Much like he doesn't need Windows and finds coding with Linux more comfortable.

PS: out of curiosity, which one is the popular game you mentioned? Looking through his portfolio I see lots of interesting games I'd like to play :)

> He didn't write his own engine either, he just didn't use one

It's hard for me to take in good faith an interpretation of that article that determines Noel "didn't write his own engine," but in case a couple direct quotes would help:

"I don't have much to say about assets, because when you're rolling your own engine you just load up what files you want, when you need them, and move on." ... "my City of None engine"

Noel literally describes his workflow as "rolling your own engine" and has a name for the engine he wrote.

> He's also making a 2D game, which traditionally is closer to game making of old

Please read the context of the thread you're replying to, which is arguing against writing your own engine even for games with "simpler 'indie' graphics," specifically called out.

> Finally, he's not arguing against engines like Godot or Unreal

He's not arguing against anything. He's advocating for writing your own engine.

Noel made the indie hit Celeste: https://en.wikipedia.org/wiki/Celeste_(video_game)

Ok, ok, it's indeed an engine of sorts. And I am arguing in good faith.

I'm not familiar with Celeste and I cannot be sure it's one of the most popular indie games of the last decade (I read that it was a hit, good for Noel because it seems like a cool game), but regardless, it's a very specific kind of game! Mostly of the kind I've argued "writing the game" and "writing the engine" is almost the same thing. Nobody is making the argument you must always use an existing engine; only that it's what makes sense as the default option.

> Please read the context of the thread

If you've read my other comments, you know I'm aware of the context of the thread.

> He's not arguing against anything. He's advocating for writing your own engine.

Oh please. Advocating for a thing and arguing against not doing the thing are two sides of the same coin.

> Most games would be better off with a bespoke game engine IMO

Perhaps, but that would mean most of those games would never get shipped.

> If you calculate a hash ... instead of pulling in a huge library.

This is a bit of a strawman; what hash calcs require "a huge library"?

Most games that use an existing engine get like 60% on metacritic and ship like less than 1000 copies and are a generic flop. “Shipping no matter what” is a lot less valuable in games.

Survivorship bias. Most games are indie games. Indie game devs who deside do use custom engine don't make it to sales.

You think the only game you will lose are generic Unity shovelwares when in reality you will also lose good games made by people with good sense in game making but likely not the resources or technical expertise to make their own engines. Think Cities Skylines (Unity, the studio had only a dozen people in total when it released), Dusk (Unity), Undertale (GameMaker), Spelunky (GameMaker)..

Another comparison is game modding which is essentially using the base game as the engine of your modded game. Do you think Team Fortress or Counter Strike or DOTA would have been made if their creators thought they have to build their own game engine?

Lowering the friction to make a game means more games get made. Yes there will be a flood of bad ones, but we get good one too.

I don't think the engine has much to do with the creativity or originality of a game.

If anything, not having to build an engine frees up more time to get creative with the actual game.

Most games are bad, of course, but Sturgeon's Law applies.

> - Most games would be better off with a bespoke game engine IMO, especially those made by small teams. The only real advantage to using an established engine is the familiar asset pipeline for replaceable artists but you pay for that with a huge overhead. Remember that custom engines or at least heavily modified ones maintained by the game developer used to be the norm - "just use Unity^WUnreal" is a relatively recent trend.

I think we can take Square-Enix as a case study: Final Fantasy XV and Final Fantasy XVI both used their own engines (though XVI used a modification of XIV's engine, from what I hear) and both took a long fucking time to make. Granted, XV had many issues outside of just building a new engine, but it definitely added one more problem onto the pile. My understanding is that Final Fantasy XVI's development cycle was pretty smooth, it still took 7 years from the launch of Final Fantasy XV!

Final Fantasy VII: Remake and Final Fantasy VII: Rebirth both used Unreal Engine 4. While Remake had a long development cycle, Rebirth was done in 3 years and contained a much bigger world!

I do totally get that part of the argument is "reuse the same damn engine and tooling," and I totally agree with that, too. It's just a bit easier overall when you can reach out to a community with your problems instead of learning every problem for yourself.

> - Most games would be better off with a bespoke game engine IMO

You have it backwards.

Most games won't benefit from a bespoke game engine at all. Most games should use an existing engine (not necessarily one of the big names, just one that works) if their goal is to actually deliver a game.

Some games are so unique they won't benefit from an existing engine, and only then the dev team must take the extra step (and eat the delay) of building their own engine.

I mean commercial games of course. Experimental/hobby devs are free to do whatever, since in that case it's the journey and not the destination that matters.

The adventure of writing a custom DBMS, game engine, or other complex and advanced software classes can be justified by 1) knowing that no existing solution is adequate but 2) it is possible to write something suitable and 3) it will be useful enough to justify the cost.

For example, to insist on the example of Tigerbeetle, 1) no current popular or specialized DBMS is particularly optimized for ledger use, 2) but suitable techniques are well understood; I'm not sure how often 3) it will be preferred to a more general purpose DBMS.

Why are there so many different database engines then? Is each instance of a different database out their an instance of NIH?

The answer is of course that every computer system of any complexity has to make trade-offs - do you need constraints, do you need scalability, do you need concurrency, security? Is your data of a specific form that benefits from a particular compressible storage form, is it optimised for write once read many or read/write etc etc..

In terns of the thrust of the main article - which is about the cost of dependencies - I find that I'm typically much happier to take a dependency if that dependency is very keen on minimising it's own dependencies! - ie I don't want to take on a whole forest of stuff.

In my view many automatic dependency management systems have helped create the mess that they claim to try and solve, and using careful manual dependency management can help cut down the burden.

> do you need constraints, do you need scalability, do you need concurrency, security

Don't forget, do you need someone you can call for support (or throw under the bus) when problems come up?

This is the thing.

If the database is essential to your business - as in if it goes down your revenue drops to zero - do you want to be the one debugging it at 3am?

Or would it be more cost effective to have a well-known database or even a managed DB

If you are running into a problem that is bringing your site down and it's truely a problem/bug with a commercial database then you are in trouble and unlikely to see a quick fix as these products are complicated and while the problem might be existential to you, it's not to the bigger vendor.

However, it's much more likely it's a configuration issue - ie you didn't understand the product properly and have configured it incorrectly - and here support can really help.

The point I'm making above is not that you should be writing fully fledged all singing and dancing databases - but that you might be able to get away with something very simple, more reliable and faster in your specific use case.

If on the other hand your use case is complex and a good fit for something pre-existing - then sure you shouldn't be writing it yourself.

i can think of two reasons for using a third-party dependency

1) a dependency on a third-party service provider that publishes the dependency. So long as that service provider is current, the dependency should be maintained 2) short-cut to code i don't want to write

I have no arguments with (1), there's a business reason and the lifecycles should match. However, I should still expect major version breaking changes in order to keep my application running. For (2), the wins are less clear, more dependenent on the perceived complexity of what I can avoid writing.

Taking on any kind of dependency means that someone else can dictate when I need to spend time updating and testing changes that don't add anything to my product. Taking on a third-party dependency is always taking on a responsibility to maintain a codebase or the risk of not doing so.

Most important reasons imo:

1) even though reality has proven us wrong time and time again, we can just not look at the dependency too closely and just act as if it's written and maintained by competent, caring people and is of highest quality. No worries!

2) in case shit hits the fan, let's assume worst case and there is a vuln in the dep and you get hacked... It's somebody else's fault! \o/

> 2) in case shit hits the fan... It's somebody else's fault!

A contrived example but, good luck explaining to the lawyers that openssl had this bug that caused all your customer data to leak and your company is being sued. If your motto for dependencies are “we can just not look at the dependency too closely and just act as if it's written and maintained by competent…” I’m reasonably sure someone is getting fired in that situation if it doesn’t sink the entire company.

Move fast and break things isn’t exactly a viable strategy in all industries.

Now as I said openssl was a contrived example but what if instead it was your ORM that didn’t use templated queries but rather just did string interpolation and there was an SQL injection attack. Considering this is still one if the most popular vulnerabilities on the web someone is messing stuff up somewhere and you are blindly like hoping it isn’t in the 10k lines of ORM library you pulled in instead.

This is where using boring languages like Go and Java come into their own: less frequent breaking changes.

There are some languages where communities aim for libraries to be stable for several months, and there’s others that think in terms of a decade or longer.

I would argue that 2 is the much more important reason. A dependency is worth it if it saves you a lot of time building the same thing. For example, if you wanted to ship a general computing device, you should probably run Linux on it rather than building your own OS from scratch, because you'll save literal years of work. Even if Linux had stopped being maintained, this would still be the right call compared to building your own (of course, it would be even better to choose some other OS still being actively maintained, if any is available).

I wish we could have this discussion without going into exaggerated extremes like "Linux". The OS rarely gets into discussions of rewriting dependencies.

I recently had to rewrite a table component and it took about an hour with extra performance and it became faster-to-use and tailored to my company's needs. One hour, give or take! With measurable performance improvements, and it's simpler to use for our use cases, and it doesn't prevent anyone from using the old one.

Comparing one-hour or even one-day work with rewriting Linux is ridiculous fear-mongering.

But this is exactly my point. Some dependencies don't actually save you any time, they're trivial to write yourself, so there is no point in getting them (perhaps the most absurd and infamous example of this being left-pad). Other dependencies are extremely hard to write, so finding a version someone else built, even if they are not maintaining it anymore, is worth it.

There is another solution which is to just retain the source for your dependencies and bring them in-house if needed. You still may need to abide by licenses, and many things shouldn’t need to rely on external deps, but I’ve seen a lot of wasted time that continues because we came up with our own way of doing something that others do better, and that can be difficult to get rid of because of unwarranted team pride or familiarity in the homegrown code or not wanting to trust the lone dev that says try this instead.

Theres a third one, when it comes to compliance and security tools, you don't want to build it even if you can because.

1. It is a liability

2. There is trust deficit during audit and other events. If audits are internal only sure you can build it but when it is 3rd party audited, auditors often know the product and familiar with the features.

> auditors often know the product and familiar with the features.

or what if you chose a dependency for which this auditor is unfamiliar with, and so it takes even longer (where as if you NIH, you'd have the full source and thus can give the auditors the materials to audit).

I am not sure most auditors work on that level of detail. If it is a library they don't consider audited yet, they might just call it a day and make a statement about your code excluding the dependencies they are not familiar with. Otherwise you would have to pay for all third party dependency audits, which no one else paid for yet or the auditor is not aware of someone else having audited already.

You are right. Thats not the level they often go to.

I have also seen this narrative. If things go south in the org. btw we were using product A for doing our workflows, that replaced if with product B for the same, now everything is going to better.

You fairly quickly run into problems a RDBMS can't solve. They're above all incredibly hamstringed by being built around supporting concurrent mutation of the data set and mutable datasets in general.

If you don't need that, even a fairly naive index implementation will get orders of magnitude more performance out of a computer than you will with a RDBMS if you can assume the data is immutable.

I'd tend to argue the opposite. Unless you have a problem that very explicitly is a poor match for an RDBMS, they are the easiest, most reliable, and most flexible way to manage data, separate your data from your application, and adapt or add functionality as new requirements come along.

That RDBMS example of yours is fascinating.

There are plenty of RDBMS here (wikipedia lists some 100+ of them), there are plenty of problems most of them can not solve, but some of them do solve.

These people considered practicality of their solution and went forward doing the implementation.

Absolutely. A bunch of them also predated the www, or started just when the www was gaining popularity, meaning that information on possible products might be more limited than it is today. Some have narrow use-cases, and some might have started as really narrow in terms of scope, but then ballooned after gaining popularity, sales (and feature requests), and some probably started because the alternative was deemed too expensive compared to just making it in-house.

I think the main point bob1029 was trying to make is that it can be worthwhile doing somehting in-house if the alternatives doesn't match the use-case, are too expensive or whatever else - but that you seriously need to consider if your architecture is the best way to solve the problem before going down that route.

What is NIH? Skimmed the article. Still don't understand, after doing so twice. Google says national institute of health.

Not Invented Here; the desire to always build your own rather than buy / borrow existing tools.

> as long as you are realistic about what you are taking ownership of

Including training and onboarding. There's a reason "popular" libraries, languages, patterns, etc. are popular. One "win" is the network effect when looking for talent - they can hit the ground at least jogging.

> Reorganizing the problem is a lot cheaper than reinventing the entire SQLite test suite from zero.

Sure, but if you aren't happy with existing DBs you are probably wrong in thinking you need a general DB instead of a file layout on a filesystem.

It seems easy, but actually managing files as a database safely is actually very hard. Especially ensuring atomicity.

Portability is a problem as well.

You do not need an SQL database however, there are others available.

Of course these questions are hard and if you are asking them you may need a general db.

The whole industry is wrapped up in these bizarre ideas that for the slightest requirement they are going to add just one more abstraction layer and it will do everything and the kitchen sink. But sometimes the requirement is simple and doesn't deserve solutions to anticipated problems and other times it does and is worth looking for an existing solution. Rarely the problem is complex and deserves a custom solution and then one should probably make a new startup for the one yak shave, etc..

The technology is only part of the problem. If you aren't happy with any existing database, throwing even more technology at it is definitely not going to help - statistically speaking. You're up against millions of engineering hours in this space.

At some point, we need to get up from the computer and go talk to the business and customer about their problem in terms that don't involve anything more sophisticated than excel spreadsheets. There is definitely some aspect you missed if you think you need to build a fully custom technology vertical to get them to pay for a solution.

Of course, all of this advice is pointless if what you are doing is for amusement or as a hobby. I think a lot of arguments about technology would be diffused if we prefaced with our intended applications.

What I want is sqlite but in native Java.

> You're up against millions of engineering hours in this space.

Tangential, but the fact that many orgs choose to not use native DB features like foreign key constraints, saying that “they’ll handle it in application code,” has always seemed like the pinnacle of hubris to me. You’re going to ignore some of the most battle-tested pieces of technology, and recreate it poorly instead? Quite the decision.

While there are valid reasons to not want to use FKCs, IME much of the time when you peel apart the layers of why someone chose not to, you find that they made several erroneous assumptions that led them to that conclusion. Other times, it’s that they simply didn’t know something existed, because they never read the docs. Windowing functions are a good example: RDBMS can do quite a bit of manipulation to data, rather than just sending huge result sets and having you parse them. There are trade-offs, of course, but you should at least know that it’s even possible before choosing to ignore it.

Using files instead of SQLite very much is NIH. Good luck not corrupting or losing your data.

The Gang Learns About Fsync.

If only that’s all it took.

It’s certainly something you can invent yourself in a cross-platform way, sure.

I thought we were talking about NIH, H is for here. If you can't scope your requirements then obviously you need to import something a bit larger than the whole world to solve the whole world's problems.

Reply on news.ycombinator.com