The software industry is due for an industrial revolution | Hacker News Reader

The software industry is due for an industrial revolution

scaletheplanet.com

11 pointschton17 comments

Software already went through the industrial revolution. To take your analogy, the car designers (the ones who work to build the prototype) are all craftsmen. Once the car is designed and we make copies of it, that is the industrial process. In software, the industrial process is trivial.

The car designers who build the first prototype, tweak it, make it actually work would call themselves craftsmen. They might use parts of other car models (same as we use libraries), they might duct tape something (same as we hack things), but the fact remains they are craftsmen.

I would also like to point out to one of the better arguments for this from Dijkstra - https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E... - you actually have to think across many abstractions and as a result, writing software remains a craftsmanship.

I disagree that in software process the manufacturing is trivial. Software is so often designed as more general systems, that need to be configured and installed and integrated, etc. Those steps aren't part of the design process anymore, they're specific to every sale, to every client. They are what I would consider a manufacturing step.

I do agree that car designers could call themselves craftsmen, their jobs are closest to what craftsmen used to be and do. But they don't anymore, they have evolved into new processes and new systems. That is the evolution I think software is due for.

To add to NhanH's earlier point [1]: I think it's an overall interesting thought, but the whole argument hinges on an unfortunate association fallacy [2] itself premised on a leaky and questionable analogy.

Besides, as you well know, there might well be very good complexity-theoretical reasons [3] for drawing a distinction between the easily scriptable, repeatable, limited-state steps required to implement/carry on a solution for a given problem; and the non-trivial steps it took to come up with that solution in the first place. It seems to me the great successes in automation during the industrial revolutions were mostly about parallelizing the former, not the latter; while most day-to-day computer programming predominantly involves the latter, not the former. One way or the other, it's far from evident the analogy is even applicable in this case.

[1] https://news.ycombinator.com/item?id=7724501

[2] http://en.wikipedia.org/wiki/Association_fallacy

[3] http://www.claymath.org/millenium-problems/p-vs-np-problem

The "car manufacturing process" doesn't even happen anymore in the software industry. A few years ago the manufacturing step was to press the installation CDs. The closest analogy today is to deploy the software for distribution.

Instead software development is the design process of a new car, coming up with new ideas that make the new car model better/more efficient/cheaper then the past model, testing and tweaking all these improvements, making sure all the components and little pieces work well together.

The industrial revolution didn't really improve this step, it only made the manufacturing step at the end of the whole process more efficient.

I consider industrial designers and software developers still as craftsmen in the best sense of the word.

It's a big industry out there. There are those who make bespoke products, and those that take a number of pre-built more general purpose libraries and just stitch them together, and everything in between.

Stitching together libraries is easy and generally quick to produce an initial product, but can become difficult to maintain/extend and is wasteful on both run-time and development resources. This method is pretty forgiving to those that don't know exactly what they are doing, and you don't have to have any experts on your team to produce a "working" product. You will accrue lots of technical debt this way (but in a lot of projects is doesn't matter much).

Bespoke products are difficult to architect and build (properly), take a long time to develop, but are performant, malleable and predictable when done right. If you do it wrong (or your architect is hit by a bus), you have a huge mess on your hands.

Most of my experience has been about two thirds bespoke, one third stitching. In my experience, finding the right ratio depends on the amount of scaling the product needs to do, how much you're willing to spend on talent, how unusual your requirements are, and how screwed you're comfortable being if things go wrong.

I have never seen an instance where a well built bespoke product did not have a competitive advantage over a stitched one. Generally a large competitive advantage. I have also seen a number of poorly built bespoke products crash and burn; so take that as you will.

I find myself quite convinced of this today. I believe that, like the industrial revolution, there needs to be a reduction of time-consuming effort to make it happen.

My thoughts lead to thinking that machine-aided development will be a major step forward. Part of this I believe is discoverability in what we make. Perhaps another major step would be to reduce the time to express the intent in code.

Reduction of time-consuming effort is exactly one of the points I'm going for. It's necessary for a proper software future.

I considered machine-aided development, and it's definitely a very good potential future. In the context of the post, I see machine-aided or machine-driven development more akin to the 3D printing revolution going on now, where you don't need intricate knowledge of the component parts at all. I'll mention this in part 2 (starting to sound like I should add a part 3 too :p)

> Like craftsmen in the past, every line is bespoke, every function something toiled over. [...] None of them worried about how to build the screws that hold it together.

Many years ago I wrote a video game in x86 assembly. (It was a shitty Snake game, but still) That code was burping out pixels direct to the VGA framebuffer. That code was pretty manual, and every assembly instruction that ran was designed and typed in by me.

Contrast that with modern web development. You're getting TONS of functionality for free. Did you toil over every line of the node.js code? Did you worry about how the interpreter stored your data in a pretty optimal data structure, gave you the illusion of nearly-infinite memory by garbage collection, or automatically cached a bunch of stuff that you never had to think about? And let's not talk about the encryption library, or the networking code, or the OS, or etc, etc.

In other words, I don't think modern web developers worry about the screws that hold it together, but man, they are still still there.

But you probably didn't write the assembler in machine opcodes before you wrote the snake game. Now I'll grant you that writing in assembly is not a fly-by-night skill, but at some level everything relies on abstractions.

Any common programming language comes with some set of tools, in the form of libraries and built-in keywords. The only difference between modern web libraries and language libraries is that modern libraries evolve and change at a much faster rate.

While a web developer may not be looking as closely to what is running on the metal, they are every bit as focused on being a craftsman at their level of abstraction. And they are able to take advantage of the fact that the tools they use are constantly evolving and providing ways to be more expressive.

Proposals to industrialize software development have a long history, for example at the NATO conferences in the 1960s which coined the term "software engineering":

http://homepages.cs.ncl.ac.uk/brian.randell/NATO/

and in the "software factories" of the 1970s and 80s:

http://en.wikipedia.org/wiki/Software_factory

Part 2 is now live! You can read it at http://www.scaletheplanet.com/post/the-industrial-revolution... and comment on HN at https://news.ycombinator.com/item?id=7758174

Hi all,

For what it's worth: this is my very first blog post, and it's a bit of an unusual one. I'd love to hear any feedback, good or bad.

I disagree with most of the points you made (mainly because I think the analogy is very weak). But I'm waiting for your next part before commenting more. For a few short notes: - Software, nowadays, are both services and products. And we are already providing the services to a much larger numbers of customers than developers.

- It could be said that "build" would be a good description too. But "build" as in "we build a company", rather than "we build a wall". There are big multinational corps as well as mom & pop shop, and there are big distributed programs as well as simple CRUD apps. We'd have the industrial revolution in software when we have one in starting a business (which some may argue is happening too, actually).

- And finally, "engineer" would be an apt description too, we're merely engineering the program, with all the declarative programming and what not :-). The compilers & co are our factories. Luckily in our case, no sweatshop is involved in manufacturing.

Thanks for the input! For the purpose of this post, I didn't make the distinction between services and products, since at their most basic level, they are the same. The difference is in business model, not implementation or process.

'build' a in company is a very interesting avenue of thought. It might imply different ways to work on software. Food for thought :)

Compilers as factories are something I considered, but for software they are a part of the creation process, not the manufacturing process.

FYI, Part 2 just went up on http://www.scaletheplanet.com/post/the-industrial-revolution...

What's the software analogy of a factory? Making copies is simple in software so you must have something else in mind?

Software needs to be configured and integrated. Those steps resemble the manufacturing process a lot more. I'll go more into this in the second part, I hope I'll be able to make it clear enough.

Reply on news.ycombinator.com