Why does programming suck?

And things in programming are getting emphatically better! For all the incidental complexity and pointless thrashing about [1], almost everything about writing software is better than it was just 10 years ago, which is not a very long time. The people who say things like "nothing ever changes" or "we're just reinventing old technology" have some seriously rose-tinted glasses. It's so much easier to do just about everything. The tools are better, the hardware is better, the quantity and quality of information is better. And we have better, bigger software doing more important things, which is why the industry is doing so well. The sky is doing the opposite of falling.

That isn't to say there isn't room for improvement, or that complaining about broken things is wrong. That lets us keeping fixing things. It's more that "everything is broken and stupid" isn't a very useful lens, and leads to fool's errands like this:

> It’s time to kill the aimless, bottom-up, ad-hoc incrementalism in programming in favor of purposeful, top-bottom and planned innovation.

I'll stick to my massive pile of working software that solves real problems, warts and all. And I'll continue to help improve it wherever I can so that it works better.

[1] I joked recently that Javascript library developers would be more efficient if they had their CI servers deprecate their APIs automatically before releasing them

Sure, some engineers (especially junior ones) enjoy reinventing the wheel more than using something that already exists to solve users' problems. We all like to trick ourselves into thinking we're unique snowflakes (this problem extends into our personal lives as well). But more often than not, when multiple popular standards emerge, it's because there are legitimate engineering tradeoffs that are being made.

JSON vs. XML? XML is far more capable but complex. JSON is far simpler but less capable. Sometimes when you're delivering value to customers, you need that extra complexity, so you use XML. Other times you don't need it so you use JSON. It has nothing to do with ego or politics. Multiple different JSON libraries? One may prioritize ease of use to get up and running, the other may prioritize strong typing for serialization speed. One may prioritize strict compliance with the spec, the other might prioritize speed above all else. JSON vs. a binary format? JSON is more readily compatible and easier to debug. Binary is faster but more complex to setup. It goes on and on.

Again, sometimes competing standards or libraries or languages emerge because of political or capitalistic concerns. But usually when you're talking about competing open standards, there are multiple because there are legitimate engineering tradeoffs being made because engineering is not a one-size-fits-all science. (Few sciences are, otherwise they wouldn't still have people working on them.)

When an open source project rejects a contribution, or five projects exist to solve the same problem, they're often making legitimate engineering tradeoffs that are in no way arbitrary, and any one project would suffer to try to be every thing to every person. The article in question doesn't even point out a single example of a set of standards or libraries that are entirely arbitrary in their differences or could be collapsed into one solution, which further highlights how this is a theoretical argument, not a practical one.

>What would programming be to you in an ideal world?

More visual, especially when observing execution.

When you're debugging anything non-synchronous—be it events, messaging, distributed systems—it usually sucks, even in the rare case that you do have good tooling.

I'd like to create, view and debug my code in an almost entirely visual fashion. I want to see events and messages perfuse through the structure of what I'm building, with complete temporal control of the view.

[0] http://www.chris-granger.com/2014/03/27/toward-a-better-prog...

However, programming may be lambda calculus at a theoretical level, but execution is a turing machine. All these little real world things that can be abstracted away in abstractions(!!), have to be unabstracted to meet a bunch of very real NAND gates somewhere, limited memory, resources etc.

Thats one reason it "sucks", but thats why its engineering and not math. Maybe one day computers will just be so powerful that we can afford to treat memory as infinite, runtime as infinitesimal, and build all the abstractions we want for free - then programming will be done :)

In this fun world, when i want to build something ill just draw a bunch of squiggly boxes and describe an alarmingly underspecified thingy i want done. Then the program will automatically figure out all possible inputs, quickly perform a "32 questions" kind of thing and resolve any ambiguity, then produce code. Managers will once again be confused why "adding this textbox is so hard", but this time they will be right - i will just tell lambda to go make a textbox.

Look at everything we built out of arbitrary conventions of boolean logic that has real impact and is shapping the modern world!

>>The computer shifted the complexity of our solutions from the world of atoms to the world of ideas.

And that's fucking awesome! Thanks everyone that was part in making that a reality!

Complexity can pile up in this new world, sure, but we are only starting to get our feet wet. Outside the boundaries of processing and storage, we are our only limitation. We can create things as complex as we can logically describe them.

It's also incredibly difficult to create something that will acomodate everyone else's solutions.

Software development is in its infancy. The fact that we can create absurd structures that still work is great because we can use them even if our skills are not mature enough to make truly elegant structures.

Saying programming sucks and wishfull thinking does not help anyone. Not happy? Start hacking on the next thing!

Imagine you're a painter and then comes some moron (who is probably not a painter) yelling at you:

- Why are you holding the brush that way? Also, I don't know that brand. - You should make thinner strokes, and always start coloring the surroundings so you don't go over by mistake. - Do not bother with the first brushes until you have a clear pencil sketch. Actually, here, use this one that I paid someone else to make. Limit yourself to add these colors. - The painters at X draw ultra realistic portraits, that doesn't even look like a cow!

Programming sucks because we need to eat.

There are too many wrong "conclusions" and premises in the text, just an example, here's the author writing about Babbage's and his Analytical Machine:

"he was focused on doing math fast and without errors. He didn’t care how complex the machine was as long as it accomplished that."

Of course, not true, Babbage had to care. He was designing a purely mechanical machine for years at the time when nobody had electrical logical circuits. He designed the simplest "universal enough" machine in the limits of the technology of his time. And the design was never built. But to claim that his design was "too complex" the author would have to design a significantly simpler one under the same limitations and goals.

The whole article is, unfortunately, like this. Uses something that actually happened or somebody did say, "concludes" something else. And the claims that aren't that kind of "conclusions" also sound like from somebody who only very partially understands what he writes about.

This doesn't mean that the author shouldn't attempt to write more articles, he can learn something from it (although I'd suggest him to base his writing on his own, specific, experiences) just that this article as it is is a quite bad potential basis for a serious discussion, IMHO.

He asks "Is there another hardware, general-purpose or not, that would fit our needs more precisely while being cheaper to program?"

The answer is a simple "no." The author just doesn't understand how much of everything was invested up to now for him to pay so little for the CPU's he uses:

https://en.wikipedia.org/wiki/List_of_semiconductor_fabricat...

Note just the "plant stat-up costs in billions" column. And that's just a piece in the whole puzzle. There are big gains for those who'd improve a current state, so the humanity as the whole is much more motivated for the real improvements, where they are possible, than the author assumes.

When we remove wrong conclusions after quoting others, the rant can be summed up to just a few claims, one of major being "I'm confronted with different frameworks and libraries, "everybody" solves similar problems, it "sucks" to work this way, maybe the hardware is the main problem." No.

Programming isn't something that can be "solved" any more than math or writing is. There is inherent complexity in building apps and there are an infinite number that could be built. The fact that it hasn't been "solved" is no indication that we've "never asked ourself this question!"

> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem (there’s not much we can do about this)

Agreed. That's where "lean programming" comes into play. If you have the capacity to make the code change as requirements change then I think the issue is pretty much solved. However, not all languages/solutions are equal in that matter. Obviously TDD, Agile techniques help.

> If we assume that typing code is the way to program, we will never look for better alternatives.

Typing code is a medium. The fact is, for an engineers it usually the most convenient way to create a program.

> But without an end goal, of course you have millions of developers releasing Javascript frameworks every day

How many of these are still maintained 1,2,5 years after?

https://www.nczonline.net/blog/2015/12/why-im-not-using-your...

So I don't think that is really a problem, there is an handfull of well maintained frameworks and libraries in the JS world. Most are not. Most are flashy with a nice "flat" page then you never hear about them again.

It's easy to produce shitloads of code, it's harder to maintain it and build a real community around it.

> Building software is 90% a human problem

Agreed, but like everything humans do.

> What would programming be to you in an ideal world?

There is no general answer to that question. That's why we have all these languages, frameworks and libraries yet we keep on creating languages,frameworks and libraries.

What drives me as a "programmer" isn't the medium, but the goal, the product.

Creating something useful. Languages, frameworks and development methods are the necessary constraints.

http://www.infoq.com/presentations/Simple-Made-Easy

> Of course we have never questioned any of this. We don’t have a direction, a path or an overall goal for programming!

> Let me ask you this: What would programming be to you in an ideal world?

> We don’t know! We’ve never asked ourselves this question!

When at the start of the "essay":

> Turns out, we are horrible at predicting the future and we end up discarding that extra complexity.

Too bad outsiders think (and will for a long time) it's a done thing. Society thinks software is the best thing since sliced bread so it's all around us. It's both a curse and a blessing.

Compared to other fields, CS or engineering is relatively new. It's exciting, it will keep breaking and evolving. And that should be it. It should keep going places without restrictions, precisely because we don't know.

Similarly to scientists ripping open humans on anathomical theatres in the 17th, we are having fun.

[No ability to execute, no debugging, no compilation step, no unit tests....]

We (who do we mean when we say "we", anyway?) don't have a direction, path or overall goal for life, society, the human race and so on. I don't even have a plan for what I'm going to have for lunch next Tuesday.

Programming is contingent and incremental because rationality is bounded and the future is uncertain. We don't know what will be successful in the next few years or how the world will want to use it or what it will be prepared to pay for.

We can build jewel-like systems that are perfectly adapted to their use case using technology to the utmost. This nearly always results in a stranded evolutionary dead-end. The BBC Domesday Project (live video database on an 8-bit micro using PAL laserdisc!) is my favourite example, but there are many.

These days it's trivial to create a CRUD web app that is even save against CSRF when you use a framework like Django.

But of course, nobody wants a simple CRUD web app anymore. It has to do business logic, it must have responsive design, usable on mobile, incrementally load pages, and be an enterprise integration framwork that checks with the credit card processor, the warehouse software etc.

It's just that we don't talk anymore about the things that have become trivial, because we don't spend much time on them.

So of course, in our perception, programming and tooling suck. And they always will. And that's OK, because if everything was breeze, we'd be bored.

Very interesting thoughts about this. I wouldn't think programming sucks, until you think about how it could be in an ideal world.

I'd love to see a follow-up article about what the OP thinks about that question.

* There is always a deadline and pressure to deliver that exceeds quality controls. * Software development shifted from engineering to crafting a long time ago, mostly because of the prior pressure to deliver. * Poorer quality code that works and building new features on top are easier to sell, than rewrites and paying off technical dept.

I think it comes down to building in pieces that you don't mind throwing away when a better piece comes along.

> Hiring developers is an extremely difficult task. Learning to program is ridiculously difficult as well.

If programming was a solved thing, that would mean problem solving itself was a solved thing. It has by nature to be difficult. The question is more whether we manage to avoid additional difficulty on top of that. He's talking about the cost of bugs, but I posit the cost of burdensome frameworks and artificial bureaucracies in software development processes is by far bigger and harder to come to terms with, both of which exist in part to manage this inherent difficulty (often achieving the opposite).

> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem

This is not unique to software development by any means. Every organization that deals with end users has to negotiate this interface.

> anywhere from 50 to 90% of the cost of building software goes towards maintaining it after the first release

Whether that figure is real depends largely on the type of software you're writing. If this fits what you're doing, the assumption that at some point the program is supposed to be "done" and every bit of time spent evolving it further must be a sign of inefficiency is simply unproductive. That kind of program is more usefully viewed as an evolving organism, and the sooner management recognizes the nature of this the better it will be handled by the organization.

> Our tools don’t make a good separation of essential and accidental complexity[2]––we are forced to crystalize the essential parts of our programs in a programming language and make them coexist with irrelevant, usually platform-specific code (accidental complexity).

This too depends on the nature of your chosen ecosystem. On the whole, interop has steadily improved throughout the history of computing. We have meaningful universal data formats now, and many options for moving code between platforms. Where interop troubles exist, they mostly exist on purpose (like the iOS example you made). You can absolutely solve most of this problem by throwing computing power at it and choosing an interpreted or cross-compiled environment if you want to avoid writing glue code.

We may be programming in our different teams, companies, organisations and countries, but the stuff we make makes an impact and affects the world around us.

This impact is an input into future similar projects and, given time, the technologies that results from wherever will be better for it. Please note that this idea does not mean that any group now is going to make these technologies, but what we do now will influence the future.

The same pattern has been observed in the industrial revolution - many tinkerers, manufacturers and inventors were all sort of learning from each other, all improving. This happened over decades. The main difference with programming is the time dimension. We've only really been doing it at a cultural level for the last twenty years - we're only just getting started in discussing and sharing what we are doing.

I'd argue Math is still slow and error-prone. Ask any math PHD candidate. Computation is fast, but Math itself has more layers, and more branches than ever. Sound familiar?

That said, some smarties are trying to remove unnecessary layers in programming. I feel like rump/Uni-kernels are looking to simplify the stack. [1] The clear linux project is removing some pieces from the stack. [2]

TL;DR Math is about as noisy as programming :/ I don't buy the premise. But, I enjoyed the read!

[1] http://rumpkernel.org/ [2] http://clearlinux.org/

Things are getting so complex for us that we find it difficult to service the requests.

Solution: We need someone to help us to solve the problems. Or more practical - we need intelligent tools which attempt to guess what we are really trying to do.

Far more intelligent autocomplete: "This looks like you are writting a postcode splitting regex, here is one that has been used by many programmers.". "Looks like you are adding a new business layer class - do you want to create an entity as well. Oh also it looks like this is going to part of the new view model you have been working on - do you want me to add a projection in for bl Class, mapping to the un populated properties on the vm?" etc.

We

The solution is totally obvious. The code we write has a form defined by it's input->output mapping over all possible inputs.

Write a function and you immediately define this input output mapping. All you have to do at this point is search through all previously written functions to find the one that most resembles the one you are trying to write.

Trying to write a nested for loop to pull out pixel values from a jpeg? Start writing it, shit you get it wrong - but the input output mapping is close enough to the cluster of previously written solutions that the correct code pops onto the screen.

Structure aware code search across all open source code would make a lot of the pain just go away.

But programming? It's an expression of the mind, and as a nice side benefit, the ability to communicate explicit commands to tireless machines to act on our behalf. There have been a couple of long, recorded speeches that I need to transcribe (for searchability and research purposes) and so I've been checking out the various speech-to-text APIs (Watson and Oxford) as a first step (e.g. before sending it to Mechanical Turk) in automating the process. The APIs are so easy to tinker with that I thought it'd be fun to pass in clips of Malcolm Tucker to see how well the APIs did. Then I remembered the IDEA project (a collection of audio of people all around the world with various accents attempting to read English) and thought it'd be interesting to gauge which accents are the hardest for machines to figure out...and all it takes is some scripting to automate the feeding of these recorded clips into the APIs (the analysis is tricky because of the high variance in audio quality, never mind the accents themselves). All of this is useful in figuring out a solution to my original problem, except it's a lot more fun and I get to think a lot more about how language works.

If I weren't a programmer? I'd set aside a day, brew some coffee, and manually do the tedious work of transcription by re-listening to those speeches several times over just to get them into a text format. How boring.

Yeah, programming can be difficult, but it is completely the right level of difficulty for the power it gives us. Even the things that make me nearly throw my laptop into the wall -- such as string encoding -- are a result of something kind of wonderful...the attempt at having an automated, standardized way to translate every squiggle among the hundreds of human languages, and cultural quirks, too, such as a pile of poo.

I don't think I've said anything the OP would disagree with. But I kind of checked out after he got into ranting about JavaScript, as if the world needed more bundles of words that insinuate that programming == web development. Things could be smoother, and they are getting that way because of how programming coordinates the flow and storage of information and assertions...but a lot of the OP's essay is not much more useful than someone ranting about how literacy sucks because we all can't agree what makes for great literature these days.

No, you have to present your reasoning before I agree with you.

Worse, it's often accompanied with 'but profession X is so much better and they are all certified and do things properly unlike us awful cowboys!'. In the worst case you get the whole 'in the future everyone will program and programmers will become totally redundant'.

I have a feeling the developers writing these posts don't have a huge amount of experience, and have a natural tendency to want to turn the table up side down and start afresh to get rid of all this cruft. I understand it, but it's frustrating

But what they perceive to be the cruft isn't just cruft. In programming the possibility of starting afresh is there, and has been done several times - haskell, erlang, go, rust, clojure, etc. etc. etc. on the programming side. In the web world the front end is filled with the latest great solution that just pushes aside all this terrible accidental complexity, and this happens in other realms too, yet we still see these articles.

I think there are a number of things at play here:

1. Programming is hard and often in annoying, frustrating, and painful ways. Nothing you do can change that, and blaming it on tooling or legacy or whatever doesn't change the _intrinsic_ complexity at hand. People like to imagine it's hard in challenging and enjoyable ways, and sometimes it is, but more often it's just frustrating.

2. Abstractions won't save you. They help, but always involve trade offs of one kind or another. Something that is declarative enough hides imperative details which eventually leak in horrible ways unless you come to understand how they work, and something too imperative invites duplication, complexity and bugs. Having seen giant T/SQL stored procedures, Microsoft Access queries that take hours to run and excel sheets that are so complicated nobody can vet them, I have seen my fair share of collapsed abstractions which started wonderful.

3. Programmers are needed. A great example is excel - some people point to it as the philosopher's stone of the 'everyone is a programmer' world, but as soon as you start doing things it's not good at, or exceed a certain complexity, it explodes into horrible pieces and fails in all kinds of terrible ways. Then the programmers need to step in. It's not something special or conceited, personally I think only a certain % of the population are capable of thinking in a sufficiently logical way to understand what a computer needs, whether employed as 'programmers' or not, and these are the only people who can help beyond the simplest case. Nothing to do with intelligence, just a kind of thinking. The power of software + the rarity of this way of thinking combined with people actually using it is why programmers are expensive, it's not an anomaly imo.

4. Real world work is largely boring and/or painful. I think particularly here people miss this point, in fact PG wrote a great piece relating to this (http://paulgraham.com/schlep.html), but it's so important - even the most interesting problem involves a great deal of schlep to get things _nicely_. No matter how wonderful programming could possibly be made, there is no avoiding this. It's true not only of programming but any job. I think there's too much of a delusion sold by many that work can be nothing but a joy if only you pick the right one - no. It can _net_ be wonderful, but in that net are wins and losses. If you love programming, the kick of solving problems and seeing an application do something for somebody easily outweighs the pain of schlepping to that point.

5. Other professions aren't magically better. I did a civ eng degree at arguably the top civ eng school in the world, and worked in a practising office for a couple of years and believe me the insight I got from both the theory and practice of engineering made it incredibly clear that the kind of abominations and messes you see in programming are nothing compared to that. You think you lack freedom in programming (you don't btw), ha! At least in programming things can be fixed only conceptually and not risk lives :)

6. Programming is actually wonderful. Despite the schlep and pain, you can build things that are actually useful using only 'thoughtstuff'. Everything that is broken can be fixed, and a great idea and a decent community around it can be enough to change anything that really is broken out there. That is totally counter to most industries out there. Problem solving in such a pure fashion, even in the compromised and often boring ways commercial programming causes you to do it is still an amazing privilege.

7. The possibility for shifting your area of programming are huge. In how many other industries could you change your specialisation by doing hobby work on a project open to anyone? You'd usually either have to 'start again' or simply not have the opportunity to change what you work on.

I feel like I have a blog post in me about this that refines and expands on the points above, but these are my thoughts on this perennial and understandable pattern of articles.

As a final note, a sad aspect is that this kind of article (not necessarily in this case) seems to downplay programming skills as just knowledge of the mess we have, or suggest it is not a nice job to have, or whatever else, which can have real world consequences. I think we have a lot to improve in programming, but to suggest it's some terrible sucky mess is a mistake, and a fatal one if you think that in the cases where in fact there is something wrong, they can't be fixed.

I remember entering a long and painful depression after reading an article by a well known author (who will go unnamed) who talked about how programming utterly sucks but if you do a real job and use programming then it's amazingly useful, just avoid the former. I think that was not a well thought out point to make and reflected more on the author than reality, including the grass is greener imagining of how somebody in another job could use programming and how great that could really be.

These kind of discussions have real world impact, let's not leap to 'X sucks' before we really know that it does.