Who Builds a House Without Drawing Blueprints?
cacm.acm.org
cacm.acm.org
http://paulspontifications.blogspot.co.uk/2013/01/software-h...
TL;DR: Source code is the blueprint for software.
However that doesn't mean I completely disagree with Lamport's arguments. Specifications are necessary in large-scale software because otherwise you can't know if you have actually implemented everything the program is supposed to do. When you release it the end-users get angry and insist that they told you in that email / meeting / watercooler conversation last year how important it was that the program frobnicate the warbles.
Lamport makes much of formal specifications. I agree that in some cases they do add value; trying to prove the correctness of a distributed protocol in any other way is dumb. But in my experience once you get up into application-level requirements a formal specification is basically source code you don't know how to compile; there is no formal correctness criteria against which you can verify it, and the stakeholders can't understand it in order to verify it.
As an example, imagine you're paid $1m to assess my super awesome new database. I give you the source code and tell you "the source is the blueprint." After reading through the code you discover there's no DELETE function. Have I built a clever immutable data store, or am I just an idiot who forgot a critical function? You can't tell that from the source.
Technically the source is a blueprint. But this comes down to scale.
Among other things I've worked on banking and corporate management applications. You cannot reason about the codebase at that scale. In these instances it's not possible to have a good outcome without a blueprint.
Using the building metaphor, you can get away with building a bird feeder without a blueprint. You can't get away with building a skyscraper without one. Houses, maybe. Either way, high level blueprints help everyone.
Taking the building metaphor, you can't clone a building locally and make changes to it, then ask another builder to review them, suggest some changes then push them out to a test environment before finally putting them into the real world. So for me, the building metaphor only goes so far but there are definitely some useful points in the article.
Definitely. But anyway, unless you have built several bird feeders or similar constructions before, your first version will probably be ugly and you'll have to iterate on it (i.e. find the right angles etc.) to make it better. What is of course fun, but costs you time.
But all modern systems have both structure and behavior. For every database there is a MVC model somewhere, for instance. And looking at those controllers, you should absolutely be able to tell whether the database is well-constructed or not.
Now there's another consideration: does the database structure map into some problem domain such that the introduction of new controllers will be supported? That's usually what people mean when they talk about architecture: the mapping of a problem domain over to a solution domain such that changes in the requirements have minimal impact on the solution.
In System-of-Systems work I'd argue yes, such work has some value -- as long as it is continually driving some kind of delivery (and not just "smart guys drawing boxes"). But in regular green-field system development? Not so much. That's because the tendency is to begin focusing on the wrong thing, the model instead of the user. Timeboxes help with this, so much so that I wouldn't advise any stand-alone architectural work in such an environment without team integration and time-boxing.
If I hand you a blueprint for a skyscraper and I realize there is no lock specified for the front doors. Does that mean it's a lockless skyscraper or I'm speaking to a jackass who forgot a critical function related to security? You can't tell from the blueprint.
If a specification is required to be 100% complete for all things that could possibly relate in any way, then even blueprints don't qualify.
Its an absolute nightmare when something doesn't work as expected, because rather than checking a specification that will take a couple of seconds to read, you basically have to debug software, which takes a lot longer.
In any case it's a good discussion to have in this thread, which is why I gave you the up.
So yes, it's exactly like code.
But I just wish people would stop comparing software development to construction. They're not alike at all, not even vaguely.
Programming is not like bricklaying, because bricklaying is deterministic and uncreative: there is one correct method of laying the bricks, and any two competent bricklayers will lay the bricks in exactly the same way. But there is never one single correct method of writing a program, and no two programmers will implement a feature in the same way.
Any attempt to create a specification so exhausting that it allows no deviation by individual programmers should stop pretending it's a spec and just write the damn code. In fact, the author of the article should probably stop writing specs and just iterate prototypes to achieve the same end: there's no difference between "understanding our programming task" in formal English or understanding it in code, except that the code is more useful and a lot easier to test.
I am no architect, so all I think about it can be true or false. But what I remember from an architect friend is that an architect's work is not really that detailed. It's like the photoshop mockup of a website, not like its source code. It doesn't say exactly which materials should be chosen or how many screws should be used to connect two ends. An architects design says "There is a wall x cm thick and y cm long". The source code says "There z bricks and the last one we had to cut to n cm" (sometimes a comment will state why it had to be cut). For the source code it is to some degree by accident the amount of bricks creates something that can be called a wall.
But all the information about the correct numbers of bricks, the slump of the mortar, the number and type of screws - these are all just library functions. Why would you re-write all that perfectly serviceable code for each project?
But I digress. Here is how bricklayers and programmers are similar: they commonly work in the vacuum of their own workspace, without awareness of the environment that impacts their projects. So once the projects are launched, they're impacted by things that weren't accounted for.
If the foundation is too weak, the brick wall will sag. If nearby trees grow too close to the wall, the wall will bow inward or outward. If someone builds on top of the brick wall and doesn't reinforce it over 8 feet, the inspector will condemn the work. The bricklayers are paid to just lay bricks, so they won't care about the nearby trees, they won't care about the foundation, they won't care about what you put on top of the wall.
Programmers will build the project to spec, and come launch time, not account for a particular load signature because that was never their concern. Seen that time and time again. Substitute foundation for baseline server power, trees for amount of interrupting processes and meddling managers altering the original program at random, and you have a pretty reasonable comparison to bricklaying.
The problem here is not that you have insufficiently detailed specs, but that you're treating your programmers like bricklayers.
I strongly suggest you try bringing your programmers into the room during the specification process, even get them to write some prototype code to explore the tricky areas (something bricklayers don't get to do either). You'll probably find you get better code as a result because the people writing the code are aware of the environment the code will be deployed to.
* Would you bring bricklayers to the design meeting? No. Just the foreman. * Would you bring all programmers to the design meeting? No. Just the lead developer.
Yes, ideally you'd have all the programmers in the room, but more of them you introduce, you increase the communication overhead. And some projects involve dozens to hundreds to thousands of programmers and are complicated enough to not be able to bring everyone to the discussion table.
So what you have to do is use bricklayer programmers- trust them to do a single task right (warehouse uploaded photos, talk to APIs) but recognize their limits. I'm not faulting neither bricklayers nor programmers, but defining their borders more concisely.
I've never met a business problem in my 30+ years doing commercial IT that couldn't be split up into smaller problems with smaller solutions, but there seems to be a bias towards large teams and complex solutions in commercial IT.
I don't know what causes this (I suspect management ego or consultancy fees, the cause of most commercial IT problems), but it's a real problem. The vast majority of these huge projects fail, go massively over-budget, precisely because they're being managed wrong.
Programming is fundamentally not like bricklaying, because it's creative not deterministic. Better, more detailed and more complete specs won't make more successful projects.
If you have too many programmers to bring them into the room when you're specifying the problem, then your project team is too big. The answer is not to exclude programmers and create more detailed specs, because programmers working to a spec like that won't write good code.
I know, I've been there. One project in particular stands out. We had detailed specs for a range (a few hundred) of middle-layer business routines. I received a function spec, wrote the function, wrote a few tests that the function did what it should do, ticked the box. Rinse and repeat for months. There were people working in a similar fashion on the front end and database layer. The system went over-budget, never worked, and was finally scrapped. It's not hard to see why, when the entire system was being written in this fashion. Programming is not like bricklaying.
If you think about it for 5 minutes you'll see that "bringing more programmers into the room increases communication overhead" implies that there is some information that is required to be passed from the customer to the programmer. The solution to this problem is obviously not to exclude the programmer and reduce the information passed to them, because that will lead to inferior code because the programmer is missing information. Instead reduce the amount of information the team is dealing with by segmenting the problem better.
But blueprints are done at various scales and they differ in complexity and format ie. sketches, models, schedules. They also often come with a book of specifications, and these are typically only referred to when ordering parts or if something goes wrong - and they are always open to interpretation..
...and the 2x4s are the blueprint of a house and the steel beams are the blueprint of the skyscraper....
I'll take the TL;DR at face value since it was written by the author. The statement completely obliterates the entire concept of a blueprint or plan. By that reasoning, it is true everything is its own blueprint, and the concept has no meaning (of course this is a standard tautology).
And, simply nodding to CYA documents (for when the customers/end users come back saying that your product doesn't do X), or to docs for when formal proofs are needed, doesn't cover the concept.
The point of planning (or blueprinting) is to think about what you are going to build before you get out the tools, abstracted away from the specific tools that you'll use, and to then record and communicate those thoughts to all the stakeholders. It doesn't have to be a complete formal, engraved-in-stone blueprint, but a modest amount of actual thinking and planning goes a loooong way.
I've often found that with a little thought, we'll use a different tool than originally expected. If you expect to just dive in and sort it out in your source code, then you've already a priori restricted your choice of tool, and probably also your library, algorithm, etc. (of course if you only have or know one tool then, well, 'everything looks like a nail and who needs a blueprint?').
Without planning, you'll get what grows out of the first thing that comes to mind, not the best thing you could build. The mere fact that it manages to run or stand is not a defense of the bad process.
Planning is where the real thinking takes place. Sure, if all you want to do is exercise your programming skills and avoid thinking, then don't plan. But don't pretend that no one else should think or plan, or complain when someone else presents a plan to which you are expected to build (and don't be surprised 10 years on when you don't advance as far as you expected).
I think it's a straw man argument. An unfair comparison. Buildings & bridges etc have been designed and improved for literally thousands of years. Software is a half a century old.
Aza Raskin's blog piece about human powered flight which bubbled up on HN has always stuck with me[0]. "The problem is we don't understand the problem".
The confounding case is the Linux kernel, which is both large and systems-focused but seems to eschew both kinds of methodology in favor of undirected chatter on mailing lists. Then again, maybe that's just part of a two-level process. More than a few patches to the kernel have real specs behind them, though developers don't call them that. They're just "kind of long, kind of detailed" emails. This tendency to embrace a thing while rejecting the term for it is one of the weirdest tendencies many developers seem to have.
You can apply the same metaphors to software ;)
Also from a practical standpoint you probably won't be able to get a building permit without blueprints or drawings of some sort (though this varies).
In software you most of the time have the 2 things running parallel because of time constraints and other business needs, which almost always leads to failure.
This probably one of the reasons some agile practitioners scoff at documentation - if one is using the same framework to implement the similar website time and again effectively the model remains the same and understanding of the model is easy to retain and retrain.
> But few programmers write even a rough sketch of what their programs will do before they start coding.
deserves a reference to a study. (or, and this is the first time I use this word: a #strawman flag.)
- or am I the only one who sketches up data models and data flows before going to work?
Also: Prototyping comes early in many other designs as well, often way before detailed blueprints are signed off I think (not a mechanical engineer, just outside observations).
In software it just so happens that prototypes are often made by the same people and from the same material as the final product, only reinforced to withstand production load and abuse.
Anecdata: Yes, or rather you're one of a vast minority. I work in an office with no notebooks in the supply closet, and one 5x5 whiteboard which is usually full of UX layouts.
Proof systems have their uses and then there are places where they are just too much.The best is in the mid!
>> "am the creator of the TLA+ specification language." I think TLA+ is/was being used at Amazon [1]
Safety with Preservation and Progress guarantees of critical systems demand design audits at a larger scale however if the scale of the program is actually quite small , things like type-systems themselves are sufficient proof of safety.
Ofcourse there are cases where even typing is too much!
One of the middlepaths is actually using inspection tools on code/programs as another aspect altogether.
[1]:http://research.microsoft.com/en-us/um/people/lamport/tla/fo...
If it is a start-up with 5 people, the software probably is the specification. If it is a large organization of devs (25, 50 or 100 people), then you will see more documentation, some of that specifications.
There is likely a threshold in the number of people where the number of communication paths between them is too large to rely on informal ways of communication. Maybe its 15-25 developers. The larger an organization is the more bureaucracy gets introduced and you now have people whose role is to write specs and requirements docs, etc.
I would expect a house to last 100 years. A family member once bought a house that is over 400 years old (and still standing).
Then again, construction and houses are way more expensive in Europe/Germany. They're planning to tear down and rebuild my old high school from scratch, current estimate: 85 million euros total.
http://www.monomoy.edu/page.cfm?p=931
(A regional high school school for about 700 students, 168,000 square feet)
http://delivery.acm.org/10.1145/2740000/2736348/p38-lamport....
An error occurred while processing your request.
Reference #50.cf633f17.1427707361.40a63048
This.
The process of producing code is the process of specifying what the computer should do -- the deliverable is the specification. On the other hand, few civil engineers will start designing plumbing without at least having a sketch of the floor plan and an idea of the building uses.
I'm sure coding is "specifying what the computer should do", but that's not what people are talking about when they talk about specifications. A specification defines what the application should do. Producing code is implementing the specification. Without having any specification first you wouldn't have a clue what the code needs to do, so you wouldn't know what to write.
The problem with the analogy is that most people don't understand what a blueprint actually is. A blueprint is an architectural planning document. It defines the building in very broad terms - where the walls go, where there's a staircase, and so on. It doesn't go in to the detail of how the building is constructed. That comes from all the other documentation. As an analogy with software it's a pretty good one - a building's blueprint is equivalent to the high-level planning documentation that any big project needs.
An experienced carpenter/contractor can build a house, without excruciating detailed notes. In fact they can build it faster, adapting to conditions as they change and solving problems efficiently.
I would take ABNF as used in RFC:s and a few clarifying diagrams over verbose UML vomit any day :)
For anything of scale we really need blueprints.
Thanks but no thanks
I don't love it, but it works (better than the alternative)
And we don't have any more weird conversations about trying to get the entire software, all the tasks, in one sitting.
Oh, and the famous feature requests disguised as bug reports for the features that didn't make it out of the backlog.
I have done both ways and still haven't found a way to getting it working across the board with all parties.
> just weird conversations about when feature X is fully ready in which sprint,
What stories compose that feature? How are you going to plan the following sprints? (are there other stories or only those?). Prioritize.
> features that were done getting new "improvements"
Yes, you can split stories into 2, or create a new story for something that you want changed/improved. Create a story, put it through the system and do it
> and getting sprints replanned in the middle of execution.
It happens. But it shouldn't be frequent.
> Oh, and the famous feature requests disguised as bug reports for the features that didn't make it out of the backlog.
Well, that's not really an Agile problem, that's scope creep (and shouldn't be allowed)
It seems your problems are due to people implementing "half agile" or just using the buzzword without the important elements. Especially in small teams.
(been there, done that)
My point is that I have never seen any company doing agile as their founders created it.
Trying to push for agile way since 2006.
But there are some places that managed to "get it".