Hacker News Reader

No, me too.

At one point, years into the Agile movement, we were wrapping up a very difficult meeting and I said words to the effect, "You know, we knew going into this that most of the problems in a project come from the requirements process. We have tons of studies that show that. And instead of tackling that problem, we just spent 10 years trying to get better at our part of the development process. Why did we do that? Did we think we would shame the business and management people into doing their jobs better?"

We are the drunk looking for his keys under the street light because the light is better here, instead of in the alley where we dropped them. I want another book called "the inmates are running the asylum" but about software development instead of design concerns.

Scrum, in particular, is a fascinating beast. Looking at how it was in the early days, you can see that a huge motivation was to try to get the non-development bits of the process under control. For example, one of the big ideas behind the whole sprinting thing was to limit the time during which the requirements could be changed to one day in, say, every 10. The whole idea behind "as a/i want/so that" was to try and make it so that a clear motivation and context were being communicated to the development team.

And then, over time, it became clear that, for whatever reason, management just wasn't going for it. So all these different bits and bobs pivoted and morphed and re-scoped into a process by which the stone tries to squeeze more blood out of itself.

IMO, the most valuable bit of Scrum isn't user stories, or story points, or sprinting, or having a Scrum Master, or burndown charts, or anything like that. It's the idea of having a single, designated person whose job is to tell people, "no": The Product Owner.

The dirty secret is, velocity is a crap metric. It unskeptically measures total things implemented, even though we all know that only a portion - I'm guessing often less than half - of those actually needed to be implemented. Meaning that the best way to increase a product team's actual productivity isn't to increase velocity, it's to maximize the average usefulness of features being implemented. Preferably by identifying the useless and less-useful ones ones, and deciding not to implement them. Or, equivalently, identifying the ones that seem most likely to be useful given the currently available information, and making sure those are always the ones at the top of the to-do list.

My suspicion is that, the better your product manager is at that particular job, the less need there is for any of the fancy ceremonies. Because most of those ceremonies aren't there for the developers' benefit; they're really only there to make it easier for the PO to scope and prioritize features.

Scrum has been twisted into a way to gaslight developers by shaming them for their estimating skills when that was project management's job the entire fucking time. It's the biggest dodge going right now. A massive case of deflection and, dare I say, projection.

If the quality of your estimations has ever come up on your annual review, that's them bargaining you down by making you feel bad about yourself.

Someone in a video I watched recently pointed out that story points per week is a graph plotting time on both axes.

One of the earlier agile methodologies (FDD) had one thing figured out: the law of large numbers works just fine for long-term estimation, as long as you can identify the stories, and the range of story 'sizes' is within an order of magnitude (eg, a day vs 2 weeks). You don't have to give a shit if a story is 4 points or 7. That's a waste of everyone's time and especially energy. It's horizontal aggression condensed into a management model. We need to start refusing, as a collective, to engage. The only discussion you need to have is whether this story is less than two weeks, more than two weeks, or way more than two weeks. Those happen at a much lower frequency.

There is one thing I really like about story points: Disagreement about them drives a whole lot of useful conversation, and can reveal communication problems and misunderstandings that are difficult to root out otherwise. That, in turn, should give the PO useful feedback to help with refining the requirements. Which means that story pointing meetings should, in theory, have a huge multiplier effect on productivity, where every hour spent on activities like story pointing saves many hours effort wasted on building unnecessary or mis-scoped or miscommunicated features.

But that requires a very engaged PO who really gets what grooming is really about. Also, I don't know what it is with MBA types, but it really seems like anything that can be turned into a KPI, will be turned into a KPI, without ever pausing to think about whether it makes any sense to do so. And that makes story points radioactive: In the absence of intense and intelligent regulatory oversight, their potential value is more-or-less negated by their potential for abuse and misuse.

Incidentally, this is what fascinates me about the Forth approach: The unflinching dedication to stripping the system down to only the things that you actually need. The problem I see is, the Forth way of doing it seems to assume you're working with an army of one. How do you scale that up to a modern product team that may comprise 10, 50, 100, even 1000 people?

> most of the problems in a project come from the requirements process

Yep I agree with that.

Although getting the development process right is critical too with all the complexity in today's web stacks etc. 20 years ago dev processes were horrible and error prone compared to today. For example if you had a word doc with the manual install steps that got printed and filled in with specific details for that customer deployment's to hand over to the team that does custom installs, you were lucky. We now have hooks from Github to deploy. That's one example.

On top of the requirements engineering that's left implicit here, there's still the matter of understanding the event trace that you're supposed to be modifying. If every patch is a transformer from traces to traces, then it stands to reason that you must understand the event trace at least as well as the program you would otherwise be modifying. And program traces are an inherently dynamic thing -- how do you gain an understanding about the potentially very complicated, interwoven traces you're developing with if you can only see one trace per execution?

A great deal of the problem of concurrent programming is just coming to grips with these traces, and decomposing them in a comprehensible way within the structure of your codebase.

>And program traces are an inherently dynamic thing -- how do you gain an understanding about the potentially very complicated, interwoven traces you're developing with if you can only see one trace per execution?

The example with the ATM makes this clearer: by designing the system so that a trace has one clear path for a given input.

Which might not be workable for larger systems of course, but holds for certain categories of systems (like ATMs and industrial control systems).

While true, this still restricts you to learning one trace per input. With symbolic representations of a program, you can perform symbolic execution (whether in your head or using a formal tool) to understand whole classes of traces in one go.

Determinism is important, but it’s also not enough. You want some analogue of continuity / smoothness to be able to generalize observations based on one input to others that you haven’t tried.

This "append-only" style also seems like it could lead to new code being added just to "undo" what the existing code is doing, which is obviously waste.

Reminds me of a process I saw at work years ago. An employee was given an Excel macro to run on a weekly spreadsheet file. The instructions included "Unhide cells D through F". It turned out that the macro had code in it to hide cells D through F, but the employee didn't know how to just remove that code from the macro, so they had to manually undo something the code did, every single time they ran it.

It is more than just a waste - how do you know what might depend on what you are undoing? Other, that is, than by having a deep understanding of what the code does and how it works, which is the problem that this approach is supposed to solve.

>But, at some point, someone has to understand if those requirements clash with the old ones that are currently implemented.

Which you could do by checking the "event source" for the old commands.

I think the idea is that the system is a black box (you don't care for the implementation) and everything happens through those events, e.g. like through commands in the GoF Command pattern. By studying the existing commands issued, and by being able to interject new ones and block old ones you can change the system without caring about the implementation of those commands.

The way the overall system is described (both for the water example and the ATM) is that the system is also kind of immutable and composed of atomic facts, where e.g. a counter action -X will revert a previous +X action (e.g. if you have 3 "add hot water" commands, you can issue 3 "remove hot water" commands to cancel them out).

>Finally, the third b-thread is added that interacts with the two previous ones to cause the hot and cold water adds to be interleaved. What requirement does that meet?

Obviously to not have the water jump to hot or code by 3 units before it gets to lukewarm, but to incrementally make it lukewarm.

> By studying the existing commands issued, and by being able to interject new ones and block old ones you can change the system without caring about the implementation of those commands.

There's a sort of semantic shell game happening here, though. It's like:

1. Understanding the code is hard.

2. So instead we'll use these event things and you only need to understand the events.

3. And events are so great that we'll design a language around them and implement the whole program using them.

Step 3 is equivalent to step 1 now, except that instead of "statements" and "expressions" you have "events" and "yields". It's still "the code" for any meaningful definition. It's an authored system with behavior that you need to understand before you can profitably modify it. All you've gained now is that instead of textual code you can read and at least somewhat reason about statically, you have a dynamic stream of events coming from who-knows-where and you have to somehow hold the entire possible state space of them in your head in order to add new code that patches them on the fly.

> Obviously to not have the water jump to hot or code by 3 units before it gets to lukewarm, but to incrementally make it lukewarm.

Obvious to whom? What is to prevent some future maintainer from breaking that requirement when they bolt on their change?

There was clearly a previous requirement to have the water jump to cold by three units. Is that requirement null and void? How do we know?

>Step 3 is equivalent to step 1 now, except that instead of "statements" and "expressions" you have "events" and "yields". It's still "the code" for any meaningful definition.

Sure. The author doesn't communicate their idea their well, but it's an interesting suggestion (even if not practical at large). It's not far to what some protocols/systems do (though they do it at a higher abstraction that the code level).

>Obvious to whom? What is to prevent some future maintainer from breaking that requirement when they bolt on their change?

Nothing. What is to prevent some future maintainer from breaking that requirement when they bolt on their change on any other codebase? As long as one has access to source, they can break any and all kinds of contracts and requirements trivially.

>There was clearly a previous requirement to have the water jump to cold by three units. Is that requirement null and void?

That requirement (or function) never went anywhere. As the post explains, the new functionality is only triggered when the water level is low (and the old is untouched).

"The b-thread from before remains untouched. We added a new one to the right. For now there’s nothing surprising. The behavior of this program (compromised of these 2 b-threads) is that whenever “waterLevelLow” happens, we add hot 3 times and then add cold 3 times."

> What is to prevent some future maintainer from breaking that requirement when they bolt on their change on any other codebase?

A bunch of stuff:

* A cultural expectation that you understand what the existing code does before you change it.

* Seeing the tests and documentation for the code when you're in there.

* Having to physically delete some existing code in order to remove functionality, which forces you to reckon with that choice.

I mean, the general thesis of the article is "here's a methodology that allows you to add stuff to a program without understanding what it does". And my claim is that that's bad goal in the first place because it means you can discard behavior users rely on without realizing it.

I admit I didn't read the entire article, but didn't we already implement this engineering methodology in WordPress with actions and filters? You essentially register functions to either do something (action) or process something (filter), and add a priority to it (i.e when is it meant to run with regarding the sibling hooks.

And no, this does not save one from understanding the rest of the code. :)