E.W.Dijkstra Archive: Interview (1985) | Hacker News Reader

E.W.Dijkstra Archive: Interview (1985)

cs.utexas.edu

48 pointsmelvinmt24 comments

Full relevant snippet:

(before the submission was renamed, its title was "Coding is not allowed to start before seventy percent of the scheduled time has elapsed")

---

van Vlissingen: In practice it often appears that pressures of production reward clever programming over good programming: how are we progressing in making the case that good programming is also cost effective?

Dijkstra: Well, it has been said over and over again that the tremendous cost of programming is caused by the fact that it is done by cheap labor, which makes it very expensive, and secondly that people rush into coding. One of the things people learn in colleges nowadays is to think first; that makes the development more cost effective. I know of at least one software house in France, and there may be more because this story is already a number of years old, where it is a firm rule of the house, that for whatever software they are committed to deliver, coding is not allowed to start before seventy percent of the scheduled time has elapsed. So if after nine months a project team reports to their boss that they want to start coding, he will ask: "Are you sure there is nothing else to do?" If they say yes, they will be told that the product will ship in three months. That company is highly successful.

This is my favorite quite from the interview:

---

...I feel that the effort to use machines to try to mimic human reasoning is both foolish and dangerous. It is foolish because if you look at human reasoning as is, it is pretty lousy; even the most trained mathematicians are amateur thinkers. Instead of trying to imitate what we are good at, I think it is much more fascinating to investigate what we are poor at. It is foolish to use machines to imitate human beings, while machines are very good at being machines, and that is precisely something that human beings are very poor at. Any successful AI project by its very nature would castrate the machine.

UPDATE: Can't help myself. This Lisp quote is too good=).

---

Lisp was great at the time of its inception if only for a radically new way of using machines. It has suffered the fate of having been elevated to a status of de facto standard with all its defects. Despite its conceptual novelty a lot of the design of Lisp is terribly amateurish even by the standards of the time. When I read the first manual, the Lisp 1.5 manual, published in 1961, I could not believe my eyes. It was an extremely poor language. Now it has become the defacto standard of the AI community, which now suffers from Lisp, the way the rest of the world suffered from Fortran.

Here's my favourite quote:

"My overall impression is that it does not matter what mistakes the USA makes, because they will be faithfully copied in Europe only 20 years later. Europe can learn an immense amount from the United States. By studying what is happening and then not copying it. In some aspects we might even try to copy it."

Unfortunately, still true, of so much beyond software 'design'.

Are you trying to steer the discussion to the quote "Coding is not allowed to start before 70% of the scheduled time has elapsed"?

I think it's important to take this in context. EWD was a proponent of Computer Science as mathematical proof, and as such pushed hard for structured programming methods. This was the mid 80s.

In hindsight, we have learned that there are too many unknowables in true software engineering, so we need to be more Agile. Try something, fix it, and try again.

One of his big points in that paragraph is something that we should take to heart. Programming by cheap labor is very dangerous. We do need to think as well as test and do.

Coding using mathematical proofs only works if your problem is defined at a similar level of rigor. That sounds obvious when stated so simply but I think it is the major downfall of formal coding methods. Depending on the field, business software seems to be the worse, defining the problem seems to be more than half the battle.

It sounds pejorative when stated this way, but there is algorithmic/scientific programming and then there is plumbing. Different techniques are effective for the two. Plumbing is non trivial, you realize it only when you have to do it yourself.

What I see among many plumbers is the notion that the world of algorithmic programming does not exist ("I dont give a $#@! about big Oh, algorithms are entirely unnecessary, I dont even know what quicksort is and I dont care LOL"). Similarly the other camp trivializes the act of plumbing. Neither do our craft any favors.

You are invoking a false dilemma here that implies using structured programming methods is not compatible with an agile approach to designing of software.

Perhaps I can go back to the original quote that spawned this all... I don't think waiting for 70% of the time to expire to start coding is consistent with the iterative cycling of agile methods.

I'm sorry, I can't connect "Computer Science as mathematical proof" and "structured programming" at all.

I mean, structured programming was much better than unstructured, goto-riddled, ball-of-mud programming, but... mathematical proof? Not with that foundation.

You can actually do formal mathematical proofs of functional programming (at least in the small). You can do it for structured programming, too, but in the much smaller.

Systems were waaaaay smaller, in that time, and the idea that one might even talk about invariants was foreign to many.

Also, we don't have hardware that is functional all the way down (no pun intended, but it is somewhat unavoidable), so even a purely functional world must be built on something else. That better not be a ball-of-mud.

I love it so much my heart is crying.

Taken of course to mean, that you will do tons of coding in the first 70%, just knowing it's ALL THROW AWAY CODE.

This changes nothing, it's what already happens, you just are doing it wrong if you keep the code you write in the first 70%.

" the word user is perfect French. Then I discovered that it is also perfect Russian and the two of us also know more Japanese than you think. Well the mere fact that that little word is not translated, but it is taken over as a foreign body, in Dutch, French, Russian and Japanese discussions, means that it has lost its original meaning. Now, if you start to analyze the many character traits of that literary figure, you discover that he is most uninspiring. He is stupid, education resistant if not education proof, and he hates any form of intellectual demand made on him, he cannot be delighted by something beautiful, because he lacks the education to appreciate beauty. Large sections of computer science are paralyzed by accepting this moron as their typical customer. Rare are the computer companies that are prepared to make a Mercedes, the analog the high quality product for the discerning customer. As it turns out, particularly in the USA mathematics is the pinnacle of user unfriendliness, if you read the catalogs of text book publishers, then it is quite clear that the major recommendation that they give a book is that it is a-mathematical, that it does not require mathematical knowledge, etc. So, user friendliness is, among other things the cause of a frantic effort to hide the fact that eo ipso computers are mathematical machines."

Couldn't agree more.

At the 71% schedule point, one day after coding on the perfectly designed spec has commenced:

Programmer: "@%*#! We didn't think of that, the design will have to change."

This is more of an history lesson than anything else. It just used to be that way, back when machines were slower and there was an actual cost involved in wasting cycles.

Not really, you should spend a reasonable time architecting before the coding begins.

the hopes of functional programming is that...it will be easier to exploit a lot of concurrency

Is this not exactly what is happening today?

Because computing is now supposed to cure all the ills of the world and more... The current boom in computing is an immediate reflection of absolutely unrealistic hopes.

Are we not repeating history exactly? E.g. medicine-quackery, chemistry-alchemy, astronomy-astrology

Regarding AI: Can you research something that is not science? I feel that the effort to use machines to try to mimic human reasoning is both foolish and dangerous.

Amen

Ahh, this.

> That is a very sad comment, but the moral of the story is that if industrial practice suffers it is largely due to the conservatism, technical incompetence, of the managers of those projects. If however the internal organization and hierarchy of industrial programming organizations can be changed, there is room for great progress. And I think it will change, because the pressure from below becomes stronger and stronger.

It seems that everyone found a snipped that resonated with them. Here's mine: "1975 was about the time that the Swedish logician Per Martin-Löf convinced himself of the fact that a well documented program was an object logically isomorphic with a constructive mathematical proof. ".

So much for code being its own documentation (which I suspect is an ideology promoted by arrogance).

The seventy percent statement is an example of a classic waterfall approach. A modern approach is to take agile philosophies and apply them to software development.

Waterfall works well when requirements never change and priorities don't shift.

Agile works well when you have full control of the requirements and full control of releasing.

Your mileage may vary. My 2 cents.

how do they verify that the third party libraries they are going to use actually do what they promise to do in all cases?

"Trust, but verify"? Test the libraries to the degree of confidence that you need. (There was a link here a few days ago that advocated testing an algorithm with all possible floats, because there's only 4 billion of them. If you really need the library function to work the way you expect, that level of testing is not out of line.)

If you need your unit tests 'show' that 4 billion floats are giving the desired answer, creating some formal proof instead doesn't seem far fetched either. I guess it all depends what you are writing things for; personally I would like to see the radiation equipment in hospitals to have formally verified software on board. The fact that 'this is not feasible' which is management speak for 'this is too expensive' remains to be kind of a depressing truth for me when we're talking lives.

For non life threatening stuff the crap people deliver now is ok-ish as no-one seems to want to pay (or ask) for anything better. I'm just wondering how you validate something is non life threatening; a bug in some financial handling code could be life threatening if someone loses all because of it and kills himself?

As I reached the ripe old age of almost 40, I get requests to review code quite a lot and it makes me want to take up gardening instead. It pays a lot less but at least I can kick my alprazolam habit.

In Dijkstra's world, there is no 'real software' but what has been mathematically proven correct. Horribly unworkable in practice, but man, having core libraries & frameworks be formally verified sounds like utopia.

Mathematically proving correct is possible if you have a formal mathematical definition to prove against. So what you're doing is translating the problem from one language to another. This is not the same at all as programming to solve user's problems where the only formal definition is the program itself. The most we can do in this respect is proving the correctness of small parts of ours programs with tests, i guess.

Reply on news.ycombinator.com