Hacker News Reader

Comment by vosper | Hacker News Reader

The "not invented here" pendulum has swung a long way, and I think that there's now a real, hidden anti-pattern in industry: "never invent here".

At some point, it's important for us as programmers to indulge that "I could do this better" instinct. If you're right you could make a real contribution to your employer or the community. If you're wrong then you'll still be a better engineer (and possibly a domain expert) for the effort.

If everyone embraced "not invented here" as a philosophy then the whole industry would stagnate.

View on HN

Most of the time I can do it better because my problem space is so much smaller.

And sometimes I look at components and I'm so disgusted by the code or the API that I merely use it as an example and re-write the code for my own purposes. Just because somebody put it on Github or built a package doesn't automatically make the code good.

Most of the "not invented here" I come across is from developers who never look for 3rd party solutions. It doesn't even occur to them. And it's usually obvious stuff like XML or JSON parsing! Which is why the message of "Not Invented here" is a good one to repeat over and over.

> Which is why the message of "Not Invented here" is a good one to repeat over and over.

So, there is a problem where many developers would rather work on abstractions rather than concrete value-providing things. I suffer from this myself - if abstractions weren't interesting to us, we wouldn't be working in software.

However, over the years I've discovered that many software projects are gaining negative value from some of their libraries, and I'm not sure which is the larger problem.

http://yosefk.com/blog/redundancy-vs-dependencies-which-is-w... describes the essential problem.

So true. You always have to weight the cost of carrying around a dependency and what that will mean for others who will have to use and build your code in the future and on platforms like Windows where dependency resolution is far from trivial. Seen this way sometimes it's worth spending a few of hours writing a JSON parser to avoid adding a dependency. Probably not woth an XML parser though.

>obvious stuff like XML or JSON parsing!

There is one case when those wheels can be worth (partially) reinventing: if you only need them to parse some files which use XML or JSON in a very restricted way. Say you've got a bunch of XML files which only go one level deep, only use ASCII, don't use namespaces, etc. A full XML parser comes with a whole ton of bloat you don't need for that! Plus, if there's ever a problem with those files, the outsourced XML parser will tend to give the end-user absolutely useless error messages.

This is often a slippery slope. Assumptions tend to erode over time, eg. I've seen code assuming ASCII-only input fall over due to people copy/pasting from Word (which automagically turns hyphens, quotes, etc. into non-ASCII).

Also, whenever there's a lot of data being looped over, it can be very tempting to add new features to the existing loop, rather than adding a new pass (inefficient) or trying to collate the results after they've been spread out (complicated & error-prone). This can turn a simple parser into a core piece of business logic, and of course it's then only a matter of time before custom, XML/JSON-incompatible 'directives' start creeping in to control that business logic.

I guess one cannot pick a non-controversial example. I recently had to deal with someone generating XML by string concat and it worked for years until recently. Something with international characters suddenly started causing problems. The solution was simple; I replaced the code with a proper XML library and it just worked. I didn't even bother to figure out what the underlying issue was.

Worrying about the bloat is premature optimization. If your process is actually too slow and parsing is the bottleneck then by all means rip out the XML parser library and roll your own.

I suspect that the current patent cold war results in a number of programmers, never mind managers, being vary of glancing at third party solutions...

I really like Spolsky's take on it:

> If it's a core business function -- do it yourself, no matter what.

http://www.joelonsoftware.com/articles/fog0000000007.html

I would say that over the past 10 years or so, especially with the rise of frameworks, PaaS and web API's, "never invent here" has become a much bigger and more troublesome anti-pattern.

Especially since the latter seems to have a much deeper and irreversible impact on the engineering culture.

Just grab 20 different containers, slapdash them together with silly string, and prey to your deity of choice that none of them hold some glaring security hole?

> At some point, it's important for us as programmers to indulge that "I could do this better" instinct.

Sure, and that point is pretty easily defined: the point at which the existing solution has produces real, tangible, definable problems with achieving your current goals is the time to invent a replacement that "does it better", with better defined in exactly the terms of the problem that you are addressing.

Its also okay to do it as something exploratory, off the critical path of a real project for understanding/skill-building.

What's not okay -- and is instead a dangerous, expensive (both in the short-term and in maintenance terms), ultimately unproductive diversion of effort -- is making a reinvention of something that works adequately a dependency for some other effort.

And then there's that cold, dark morning when you find the profound, deeply embedded assumption at the core of a library that means you now have irreconcilable differences...

NIH is part lack of communication and understanding. Sometimes it's really hard to get the scope, qualities and failures of an external project.

Reply on news.ycombinator.com