Hacker News Reader

Comment by Chris_Newton | Hacker News Reader

I like the saying “slow is smooth and smooth is fast” as applied to software development. For all the talk of Agile processes and breaking creative work down into bite-sized chunks and prioritising collaboration and team productivity over individual contributions, ultimately our work is about thinking and fundamentally it requires that each individual doing the work understands what needs to be done. Not investing enough time in gaining that understanding early on means wasting far more time later.

IME, a huge amount of rework in modern software development results from diving into the implementation before the requirements were properly understood, a reasonable design was created and both were recorded in some useful form and reviewed by the relevant people to make sure everyone was on the same page.

We seem to have gone from one extreme to another with our processes. We realised that waterfalls that start with months of requirements capture followed by months of BDUF aren’t a great idea. But now we often seem to have one-paragraph tickets, maybe spend a few minutes thinking of some class names that look about right for each one, and then dive into coding. And then a week later we find that two developers working on three tickets have implemented the same concept four different ways and each of them was a different 75% of what the customer actually needed.

View on HN

> slow is smooth and smooth is fast

Another phrase with similar meaning (and interesting history) is "festina lente" ("make haste slowly," in Latin). I named my company after it (:

It's certainly an ancient concept. I feel like software is ultimately about humans organizing complex activities, and humans haven't changed that much in the past few millennia, even if the activities have gotten more complex and abstract.

I feel like the ones-size-fits-all approach of trying to say approach X or Y is superior (eg: scrum vs. kanban vs. waterfall, etc) is totally off the mark. Different sorts of work require different sorts of effort. Creative and full-of-unknowns work is different from assembly-line sort of work is different from maintenance work, etc.

On a meta level, I think part of the "slow is smooth..." philosophy is to take a deep breath at the beginning and think about what sorts of approaches are going to be best suited for the tasks at hand. As opposed to some mindless "scrum is in, so we'll do that for everything". Or "everyone is using NoSQL now, so we should too" or any number of other cargo-cult-y behaviors.

Festina Lente!

>I feel like software is ultimately about humans organizing complex activities

Organizations are much like software: embodiment of domain knowledge. It needs to be codified, and if you're not actively cultivating it locally, you're outsourcing the one thing that matters: learning.

It's a 100% in the interest of incumbent players to divulge ready-made pretenses of universal solutions. It precludes competition.

Another saying "work smart not hard", i.e. you can save a lot of time by thinking things through before jumping into executing.

Fast is fine, but accuracy is everything. In a gun fight... You need to take your time in a hurry. -- Wyatt Earp

Another one is Blue Origin's motto: "Gradatim Ferociter", which translates to "Step by step, ferociously".

The famous UCLA basketball coach John Wooden used to say "be quick but never hurry".

My mom has a saying in Arabic that roughly translates to "he who rushes is delayed"

The German press agency (DPA) has (had?) a saying/motto:

> Be first, but first be right.

Another one for the collection: vísteme despacio que tengo prisa.

I like the Spanish saying "Vísteme despacio, que tengo prisa": "dress me up slowly, since I'm in a rush".

The directly equivalent proverb in English is: "Less haste; more speed."

Isn't changing requirements one of the key principles of the Agile manifesto?

If so, why do we want to dive so much into requirements that are likely to change anyways?

Not saying we should jump into design without understanding the requirements, but sometimes requirements are incomplete or not clearly defined and we cannot sit and wait until a new revision is out or someone becomes available to answer our questions.

I guess this really depends on the area we operate.

In my business domain we call requirements "wish lists" as the solutions we build are for complex hardware that generally doesn't exist yet and there's a good chance the requirements spec you got today will be rewritten in six months, yet we need to start working as soon as the project is launched.

So we have learned to design things that are reasonably easy to adapt, replace and test, we even developed an entire framework for this very purpose that has become some sort of spinoff business as some customers wanted to use it for their in house development.

Isn't changing requirements one of the key principles of the Agile manifesto? If so, why do we want to dive so much into requirements that are likely to change anyways?

Who says all the requirements are likely to change anyway? That’s a whopping great hypothetical on which the entire premise rests, but YAGNI applies to development processes as much as to code!

Yes, requirements often evolve and we often start with some gaps or ambiguities in the spec. However, that’s a very different situation to having no detailed requirements at all. In one case, you know where you’re trying to go right now and can set a course that will take you there. Even if the destination is changed a bit during the voyage, you can adjust your course to compensate and still be heading in roughly the right direction. In the other case, you literally don’t know which way to start.

Moreover, jokes about skyscrapers turning into spaceships aside, the central requirements for a software project usually evolve relatively slowly. Business priorities or some of the details might change faster, but in most fields the underlying domain concepts and actors that the software models are going to be relatively stable.

Big pivots where you’re effectively trying to build a completely different product afterwards do happen. However, a team that pivots so often that it never really understands where it’s trying to get to before changing direction again won’t last long because it can never get far enough in any direction to deliver real value.

The tricky bit, in my experience, is that when you straight up ask the customer for requirements you get a list full of details around specific functionality -- basically all the stuff that's going to change in a month.

It takes skill and effort to abstract from these detailed wishlists into generalised primitives which remain somewhat constant. This work is similar to the fuzzy logic social scientists work with, and that's exactly the sort of thing most techies are not very good at.

It takes skill and effort to abstract from these detailed wishlists into generalised primitives which remain somewhat constant.

100% agreed. It’s well worth doing it if you have the chance, though. Understanding how your customer sees their world and what problems they’re really trying to solve is valuable both for building a useful product and for keeping that customer happy!

In 20+ years I don't think I ever worked on a "well-defined problem".

It's always a fuzzy nebula and the only way to get insight is to build some prototype, see how it behaves and improve upon it. With the inevitable "suboptimal" early design somehow enduring and adapting through successive iterations, never have time or be able to afford a full rewrite from a business perspective: stuff works for the client, regardless how terrible the engineers view the build process. So people love to lament how awful the early choices were and how far in outer space we'd be, should some shiny late-stage idea have been applied. But it's a lot more like the x86 CPU architecture, in spite of all the detractors, it endures.

Ahh, and I've also worked for a company that did rewrites. A lot of them. Like every time the product reached some 80-90% of functionality, they'd throw everything away and redo from start. Never shipped anything in my entire stay there. Again in my life I never seen such a thing, it's like the managers were not right in the head and developers were lacking any common sense. So it shouldn't come as a surprise that now (after some 5+ years of this) they closed and fired everyone.

On the flip side the code bases I’ve worked with that were built with exploratory coding, instead of deeply understanding the problem and designing a solution, tend to be buggy, slow, and stressful to work on. I think it’s a balance like anything else.

The changing requirements part is mystifying part of Agile. How do you build the a proper foundation without knowing the base requirements it needs to support day one and plan for day 30/60/90/365 days needs. Its almost like its written by the PO/PM without really understanding anything else but their slice of work and how they can make it easier for them.

Money and time (for us humans) are finite, big things at lease have to be targeted precise efforts, a good plan and architecture that maps back and supports the requirements.

I couldn't disagree more. The worst codebases I've seen in my career have been codebases that were religiously planned upfront, and then religiously adhered to The Plan. Irregardless of the revelations that popped up in the midst of development.

This usually takes the form of pattern driven developers: the developers that must fit the entire problem into a predetermined pattern (which is ridiculous because every problem is unique and there are very few patterns that are truly applicable everywhere). This will also take the form of Amazon's famous 6 page design upfront. It's funny, they wanted me to prescribe what the expected latency of a system that would be running over an unreliable network was and figure out an SLA based on that, before I wrote an ounce of code! How in the world are you even supposed to begin getting accurate measurements, without measuring!

So, planning is good yes. But the reality is every problem is unique. Unless you have somebody who's solved the exact problem you're trying to solve, planning is going to do no good and you're better off exploring first. Once you became a domain expert in your tiny niche domain you just explored, then you get to make a plan.

But enough with this nonsense that we can magically figure out what the exact requirements are for a problem before we've even begun to measure the appropriate metrics associated with the problem. And one of the key ways we can measure stuff is by exploring the problem space with code.

I've recently become a big believer in Python notebooks to plan out these sorts of designs. It allows you to quickly mock up and iterate on a few prototypes to build your expertise, and then actually document a proposed design alongside working code that demonstrates the functionality.

The bonus is, being a Python notebook, you usually avoid the common pitfall of shipping crappy prototype code to production; you are forced to rewrite it because putting a Python notebook into prod would make any software engineer shudder.

> slow is smooth and smooth is fast

I've been in enjoying the converse of this i.e. "fast is smooth". It was provoked by using an ultra-fine fountain pen nib - too slow and it would be scratchy and dig into the paper with poor ink flow but using it very fast and lightly and it would be really smooth and delightful.

I am prone to perfectionism, overthinking, procrastination, majoring in the minors and getting derailed by little hiccups. I get motivation by seeing progress. If I pause I can struggle to get started again so momentum is really important or I fall into a negative spiral.

For my personal productivity, it helps to force a pace and do things faster than I naturally want. I need to keep up momentum and quickly do the known things and discover the problems early that need time to percolate in the back of my mind to reach a creative solution.

I guess we all need productivity epithets that counter our dysfunctional traits - too slow then speed up! too fast then slow down!

Sometimes software engineers forget coding should be the final step, and not the main step. Defining as much as you can,80/20 before starting will save you time. Certainly there are cases where you have an idea and you dive right in and it will work out but usually that won’t work out well from my experience

For me, as a rookie developer, I find I often have little to no clue what needs to be done until I've started coding and figured out through trial and error how to implement it and what the requirements even should have been up front.

Just because I feel more people should hear this early in their career, that’s perfectly normal. I fairly routinely answer questions about how long a piece of work is going to take by saying I’m going to need n days to work out what needs to be done, after which I’ll be able to give an estimate.

That sort of exploratory work is really good because you’ll end up reading relevant documentation, then actually trying to use whatever it is you’re learning about. Often I’ll throw together a hacky version of what we’re aiming for, but ignore anything but the happy path. No error handling, no UI beyond the bare minimum, just the very core of what’s being done.

And on that note the ”trick” to being more experienced is being able to anticipate what’s coming up next and already have a mental model for how to solve that.

Easier said than done though

Heh, I don't think that will ever really change. What changes in my experience is your knowledge and confidence when it comes to translating requirements into maintainable technology. The real world stuff seems to be always messy. My attitude is to try to locate areas of potential unknown risk, plan how to deal with the known risk and then hack away, knowing that no matter how much time spent, the plan will be flawed. During this whole process, communication is really important (i.e. 30% feedback, where I just write comments of what I would change in the codebase and then talk about it with a collegue)

On the flip side, a lot of exploratory coding is often necessary to learn exactly what needs to be done. "Oh this API sometimes returns garbled garbage results, guess we need to write a sanitizer for its output" isn't something that is going to be known until at least some code is written.

Definitely, exploratory coding can be very useful. It’s essentially writing code to help identify the requirements or design instead of to help implement them. As long as management understands that you might be writing different code for each case and you can’t necessarily turn one into the other…

For me it’s the main step. Decision making up front would require literally 40+ people, none of whom claim they have authority but all of whom say they MUST be involved in planning.

Or, I could deploy something to QA and ask for feedback and forgiveness. Then a 3 week discussion becomes a 30 minute demo + a few small notes.

On this theme I can heartily recommend SovietWomble’s essay on the game the Forest (Soviet was a QA specialist in a former capacity)

https://www.youtube.com/watch?v=PUWg905fGTA

Both funny and insightful for anyone starting a project.

> ultimately our work is about thinking and fundamentally it requires that each individual doing the work understands what needs to be done.

Usually the individuals doing the work do understand that, but it is really hard to properly divide work and describe requirements. Why people try do divide it in even smaller tasks which increases this kind of complicated creative-writing work is beyond me.

> We realised that waterfalls that start with months of requirements capture followed by months of BDUF aren’t a great idea.

What we should have realized by now is that we never did waterfall right because it's inventor Winston Royce required it to be done TWICE for each project. Two iterations.

What this world needs is Atlassian hiring a number of work psychologists and theorists and have them streamline JIRA based on their learnings. Not in a UX "I want to create a story"-story sort of way. But in a "How do I deal with 50 tickets that are vastly different in quality?" sort of way.

They should answer questions like: "How can we make it so that the user always immediately knows whether to put this text in JIRA or confluence"?

Then Microsoft needs to hire the same people and improve text creation in outlook.

>What we should have realized by now is that we never did waterfall right because it's inventor Winston Royce required it to be done TWICE for each project. Two iterations.

Reading the original waterfall paper was eye opening for me. I saw that we're falling into patterns that are decades old. Someone insightful tries to explain that learning and building are two sides of the same coin, and companies read it as "ok, I need a learning team and a doing team" and run with it in the wrong direction. Taking the conceptual structure of some paper of manifest and applying it cookie-cutter style is missing the point entirely.

What Conway and Royce were trying to convey is that the social process that underlies software matters. What people do with it on the field is using the examples employed in the explanation and mistaking them for the solution to their lack of respect for the intricacies of social processes.

A somewhat related tangent is the idea that "writing is rewriting". Hardly, if ever, is a first draft suitable for publication, not just because it's rough-hewn or unrefined but also because requirements for some chapters change once others are written. The feedback loops between dependent parts of a written piece are dynamic, not static.

(Yet another reason why LLMs are so limited: they take their earlier writing as sacred and unchanging as a direct consequence of their autoregressive structure.)

You might like the book "The heretic's guide to best practises".

Looks very interesting, thanks for the tip!

People keep saying Royce invented Waterfall, but he didn't. At most, he used it as a straw man. What Royce actually advocated was an iterative model. You're right that his approach recommended at least two iterations, but not that his approach was the Waterfall model.

Waterfall also uses an iterative model. Inescapable when building software. The myth needs to die so we can progress. Waterfall, as a typical corporate tech fad, WILL emerge eventually.

I love that you quoted “slow is smooth and smooth is fast” -- I learned it as a volunteer firefighter! It's a great saying, and can be applied to just about anything.

Sex?

Another wacker in the wild! I'm taking Fire 1 (final burn today actually) and the instructors are always yelling "don't run, just move with purpose. MORE PURPOSE THAN THAT NUMBER 14"

>breaking creative work down into bite-sized chunks

In my decades of experience in multiple companies doing 'agile', that's something that we've always ostensibly strived for, but in practice it almost never happens in reality. At least not as intended. Occasionally it happens, but only by pure chance.

It seems the 'agile way' is to attempt to manage a project by taking a sizable but coherent and cohesive concept/feature/requirement/design/plan, shatter it into a thousand incohesive random fragments of varying sizes (all estimated to be bite-sized, but true size unknown until they're actually done).

Then everybody grabs a few fragments and works on them, placing them somewhere more-or-less where they're hopefully supposed to be, hoping that in the end everything will come together somewhat similar to the picture on the box. Then everybody has to try to duct-tape and glue them all back together again on a herky-jerky schedule, like doing a puzzle with constant interruptions and distractions for the agile rituals.

The results aren't necessarily worse than waterfall/BDUF, but not necessarily better either. There is in truth a lot more potential to course-correct along the way. But since everyone's spastically going different distances in different directions at different speeds on as-yet-unconnected fragments of varying sizes, and no one can see the big picture, that theoretical course correction en-route is not nearly as powerful as it sounds.

I've seen some debates about shorter vs. longer sprints. The die-hard agile people tend to push for the shortest sprints possible or even shorter, leaving no time whatsoever to clarify requirements, work out integrations, or do any QA/testing. No time for thought at all, just spit out some code, any code, as fast as possible and then it's review/demo/retro time. The veterans push for longer sprints, with time to understand and coordinate, experiment and test. But that gets a lot of pushback for not being agile enough, so we usually end up somewhere in the middle. And a lot of things don't fit neatly into that timeframe.

Which in my opinion is not the worst thing, but also nowhere near as efficient or as effective as it could be if things were not broken into bite-sized chunks with arbitrary deadlines, but instead developed as parts of a cohesive whole. Whether from the foundation up (risky) or by building vertical segments and linking them together.

"Move fast and break things" indeed. That's the state of the art - breaking things as fast as possible. We could do better.

It all stems from the desire by stakeholders (who are paying the money) wanting to know when something is gonna ship, before committing the money.

I think it should just be accepted that it is not possible to know. Unlike building a building, you cannot know. It is possible to do a feasibility study, or a prototype, and time box it, but that is as good as it gets.

The stakeholders don't want to hear this unfortuantely. They refuse to believe it isn't possible to know ahead of time how much work is needed. In order to appease, the engineers and managers have developed a set of "practises" such as agile, but in the end, it is a lie.

Yes. Nobody asks a mathematician to "story point" the conjectures they are working on. Nobody asks them for daily status updates. This is pure and harmful micromanagement because software engineers are not saying stop it.

The optimal sprint length is none. It's an awful concept that is misused to micromanage highly skilled professionals that often didn't learn to say no. You want to work on three to six month projects, not two week "sprints".

MVP works when everyone on the team is seasoned. All of the members of the Agile Manifesto were seasoned. As were many of the people they tended to work with.

For the rest of us “slow is smooth” means you should work on something that is analogous to the most important functionality first, to build your ability to do that work. Then as you gain skill, tackle harder and harder problems.

Sometimes the rule of three lets you go back and fix so,e of that early poor ordering, but that doesn’t always trigger at all and sometimes triggers when you still don’t know what you’re doing. And then everything you do later is hampered by getting things wrong up front.

Might I suggest that instead of working on "the most important functionality first" you consider identifying the riskiest aspects of the project and working on those first?

Some times those are the same, but often the part of a project that sinks it isn't the core functionality but some prerequisite that turns out to be much harder than anticipated.

No that’s the problem. The riskiest bits stay risky forever if you tackle them before you understand how. Junior people or new problem domains require exploratory development before committing to a strategy.

In an 18 month or three year project, it doesn’t affect your deadline much if you do the hardest bits in month one or month four. Except in how much rework you end up doing to those parts that have already started to cement bad habits.

When we have a Rule of Three situation in the product backlog, I usually recommend one of two strategies. Do the hardest one second, or do it third and allocate very little time for the first two - you’re going to rework the whole thing when you get to #3 anyway.

On one project, no matter what order we did them in, the third one took the longest time. Even when we tried to take our time on the second. Took me a year to train them to stop doing that. I’ve never seen a team so sure of their abilities yet with such a low opinion of their product. Big ol’ echo chamber.

Fast and stressed is where The Mistakes™ happen, and depending on the thing you are working on those may not be recoverable in time.

When I am under time pressure I slow down a notch. Over the years my slow speed has gotten faster as well.

At my company we have a nightmarish combination of a lot of docs and meetings but also the pressure to produce a lot of artifacts, so we have “planning” but it could be a lot better, uh, planned.

One of my faves, for sure. I find the "slow is smooth" transition to "smooth is fast" especially holds true for physical tasks. Learning how to do something for the first time - carefully, slowly, awkwardly - you lay the initial record grooves of muscle memory, and if these are "right" the first time around, then you don't have bad habits to unlearn once you become proficient enough for them to matter.

The typical waterfall assumptions are very overblown. We are doing ourselves a great disservice by throwing it away. Hardly anyone developing today has ever used waterfall much less agile.

The waterfall:bad, agile:good trope is really tiring. Like listening to a teenager that knows jack shit explain how the world works.

Down voted by know-it-alls trapped in an agile doctrine.

One paragraph tickets? At my previous job that would be considered a miracle!

Agile processes is an oxymoron.

"Agile" coaches scammed you

Not at all. There are repetitive elements of agile development too. Or do you invent a new way to e.g. deploy your code every time you put it into production? Come up with a new format for retros? Vary the lengths of sprints and the check-in points in them?

One problem of delaying the coding is that it takes 5 minutes to dream up requirements that could take a millennia to complete. Coding is often the hard part actually, especially in some of the large slow moving corporations where coding is often delayed by lots of requirements brainstorming.

Right; but the problem is that you usually don’t really understand what you’re building until you’re halfway through building it. Specs inform code, and code also needs to inform the spec. Both because we learn more about how it will be used, and because there’s usually a clean, simple, straightforward way you can build something that won’t exactly match up to feature list in front of you.

Worse, I never seem to get those abstractions quite right until I’ve gotten them wrong a few times. If you don’t take the time to iterate and build some knowledge of the solution space, anything you make will be a bit of a buggy, fragile mess.

And I think it’s much easier (and more efficient) to do that iteration during development than spend years building something messy and then try and remake it from scratch, cleaner, later. But that means it takes more time to get to market, and that’s a really hard pill to swallow (for good reason).

> and fundamentally it requires that each individual doing the work understands what needs to be done

Which is precisely what agile is about, specifically the estimation process. No story enters the sprint unless there is unanimous consensus among the engineers as to its point value, and it's the process of coming to that unanimity that makes sure each individual understands what needs to be done.

A lot of people like to complain that agile slows down their coding, and this is by design -- to achieve exactly what you're asking for.

> But now we often seem to have one-paragraph tickets, maybe spend a few minutes thinking of some class names that look about right for each one, and then dive into coding.

That isn't agile. If you're estimating a new story the team hasn't discussed before, it's usually going to take between 5 and 30 minutes to come to consensus on its size and what it is conceptually.

So the main problem you're describing is what Agile solves.

Then you seem to have a secondary problem which is about different team members working in incompatible ways in terms of coding architecture. That require a team lead who is in charge of architecture decisions, that team members consult with whenever they start a story that there's no precedent on how to build. And code review ensures code is written as planned.

>No story enters the sprint unless there is absolute consensus among the engineers as to its point value, and it's the process of coming to that point value that makes sure each individual understands what needs to be done.

I call bs.

Are you telling me that the less experienced / less confident devs on the team don't cave to pressure from the team lead or whoever happens to be the most forceful / loudest to agree that, ok, this is only a 5 pointer and we all understand what needs to be done, because DAVE says it's easy, 30 minutes work tops, and DAVE is tired and has better places to be.

You write as if the team is all top engineers with equality and not one leader and his minions.

That’s not an agile problem so where’s the bs? If your team is not following agile you can’t just blame it.

You can certainly criticize a methodology for being difficult or impossible to follow.

But it's easy to follow if a scrum master, facilitator, PM or team lead actually just does it.

There's nothing difficult or impossible about it at all.

But it's true that if the leader doesn't want to follow it that it's not going to work. It requires the person in the position of authority in the room to buy into it. No process will ever work if it's not actually being implemented by anyone.

And also, do people actually plan it out or do they just approve every idea that doesn’t sound terrible?

Software is complicated. Half the time I don’t know what the hell anyone is talking about due to labyrinthine requirements, but I’m not going to stop the meetings to ask questions unless I’m the main guy working on it.

But nobody knows who will work on it and you'll stop the meeting during the blind estimation process because you have to estimate points and you don't know what anybody else is estimating.

All of the points you're raising are solved inherently by the way points estimation works with the points playing cards.

You're not stopping the meeting. But it will be your turn to explain why you think it's the number of points you chose, so you'll do that.

Well, yes, I'm telling you exactly that. That's how planning poker works, you have to speak up. You don't even have a choice because the process requires you to.

It sounds like you have a very dysfunctional team if you're characterizing it as a team leader and his "minions".

I'm sorry that's your situation.

Some larger corporations have agile facilitators precisely to nip this kind of problem in the bud.

Playing "poker" at work and having agile helpers sounds like extreme dysfunction.

Well sure, anything can "sound like" extreme dysfunction if you don't actually know what it is.

Do you know what planning poker is, and are you aware it's not a game for fun?

To be fair having demeaning names for the engineers' work activities is not the worst part of corporate agile. Unfortunately.

I have heard some third tier companies and lower play "planning poker" at work. It doesn't sound like any fun or efficient but a cheap paycheck for an "agile helper". I am not sure how many of them have remained employed during these layoffs.

It's not supposed to be fun. It's work, not a game. It's productive because it results in more accurate estimation of tasks, and a better understanding of the work entailed.

Are you against those things? "First tier" companies use planning poker as well, you know.

You seem to be oddly hostile to the concept without seeming to understand it at all.

I don't think there's much evidence if any at all that it provides more accuracy or understanding. First tier companies tend not to use "sprints" at all.

There's nothing to understand, the agile manifesto or worse the "scrum" pamphlet can be read on a lunch break. Somehow engineers who put ten-thousand hours into their craft need a layman agile helper to understand it?

Every sentence of your comment is factually incorrect or irrelevant. I'm not going to continue a conversation with someone who doesn't have an interest in facts.

What you might actually mean is that you are going to try to sell agile snake oil to someone who knows less. Try a middle manager without technical background? They tend to be more receptive.

Reply on news.ycombinator.com