Still arguing about agile software development

1. The manifesto for agile development has turn into a sort of religion and through all my experience and from what I've seen Jim Coplien, David Thomas, Robert Martin, etc. are saying, it seems pretty clear to me that us people from the IT industry have no clue what we are doing, and are constantly reinterpreting and clarifying the basis of our professional activity, in a time where we should have everything set and well-oiled.... I mean it is 2021 and we are still discussing the fundamentals... How on Earth is a house supposed to stand if the foundation is brittle ? We need to take a cue from other, more established industries, we need to use the right set of practices for the right set of people and in the right context. Not every project is the same, in terms of complexity AND budget. Budget is something that is highly overlooked in this talk about agile software development. Managers, especially CFO's are very attentive to the money problem, because if you keep taking money from a bag, without thinking about the consequences after that bag is empty then you don't have determinism in project, it's just putting money in a black box and waiting to see what comes out of it, maybe it is something useful, maybe it is something half-baked, and usually it is the latter. If you make a simple website, and you have the experience, then it is easy to estimate that a project will be done in 1-2 weeks and that it takes ..idk $1000, but if it is a distributed system that does some heavy number crunching on one side and some fancy UX/UI on another side, then you need to consider prototyping, feasibility, system design, onboarding experts and onboarding juniors or less-experienced people that can learn from those experts, you need to allow for learning, experimenting. Agile development and scrum has been turned into a mean of milking the developers dry, of their energy and ability. Agile and scrum has become a tool for managers to force rapid development for BUDGET reasons. It all ties down to budget, every company wants to get a BANG for their bucks, and it usually involves agile development with the software developer at the end of the food chain .... It is especially interesting when the project leader decides a specific deadline without consulting the technical team on the feasibility of that deadline.

2. Taking the idea of the budget from point #1, and eliminating it, and combining the lack of budget with open source software, you can see that the quality of the software can be pretty high, and the libraries and interfaces that these libraries expose, plus the user guide offered alongside these are of high quality and high completeness, especially because the software developers that put time and thinking into these libraries and tools have the time and are not constrained by the budget and the deadline. So I agree with Jim here on the weird economics of software development.

3. Making your own tools can be interesting, especially when developing highly customised products that have features that are very different from what framework A or B has to offer, and it would be absolute murder to try to beat with a hammer a TV screen hoping it will turn into a guitar... and there's plenty of examples where something like this happened, like turning a Wordpress setup into a customised e-commerce website with very specific wizard like panel flows that are not covered by the actual e-commerce plugin, having to hack that thing out to smithereens, fiddling with the core code of the wordpress framework and the e-commerce plugin, which is absolutely painful and counter-intuitive. If you want something specific, very different from what a plugin or a framework has to offer then you would better start thinking of how to write it from scratch. If however you don't have the budget of doing that and you are fine with using the features of framework A or B, or plugin A or B, then that's the only thing you can do and ...so be it.

4. Related to #3, I do not however agree with an idea that every time you build a product you need to make your own tools, it would be a complete waste of time and budget in some cases, where you could easily use some library that solves your problem in a direct and practical way. You don't want to rewrite a HTTP request/response protocol library or a TCP handshake handler every time you want to create a backend for a website... instead you want to use existing tools and libraries that do that for you. It all boils down to pragmatism and pragmatism is learned by doing, by exercising your skills, by finding out what works and what doesn't, and this is tied to the fact that in many teams apprentices don't learn from more experienced programmers, and that is also due to a lack of role definition in the industry, I mean a conventional one.

5. Related to #4 on the topic of role definition, what should a junior programmer do? what should a senior programmer do? What are the responsibilities of a dev, of a database administrator, of a UX person. Do we even need a database administrator? What is the flow of communication between these people, when should the UX guy do this thing, when should the dev start implementing the UX guy's concepts? Why is the product owner doing UX design and database design when he does not have any background on the psychological concepts of human-computer interfaces or RDBMS theory? We need to understand what everyone is supposed to do, and we need to have the right people practice the right things for their knowledge. This is something that I have mixed feelings about in relation to Jim, because on one hand he repeatedly talks about having the hands-on knowledge in a particular domain like banking and telecom, but on the other hand he absolutely despises specialisation (because "specialisation is for insects"), but what does that mean? Does that mean that the UX guys should write code sometimes? Does that mean that the backend dev should know if a customer trying to checkout should see product A or B? I hope not.... the UX guy didn't learn in school why merge sort is better than bubble sort, so how can he decide if code A is better than code B, hell, how does he even know how to design code in the first place? Just by learning in the weekends some python fizz-buzz functions doesn't get you to write professional-grade code. Even in medicine you have specialisation. You don't ask the liver surgeon to perform a heart transplant because he does not know the subtleties of the procedure! It is all about skill set, about what have you put a lot of effort into and have a good grasp over to be able to practice flawlessly. It's about how and when you acquire those skills and how long does it take to apply those skills. There is nothing in agile or scrum or any of these sets of principles and procedures that even gets close to these notions.

6. We need people involved into research on programming languages and program writing. We are still fighting on which paradigm to use and how to ensure the correctness of our algorithms. People are now creating cults between functional programmers and non-functional programmers. They are two different ways of writing code and it affects the way we think about code. There are cults (and Jim and Trygve are partly to blame here - see the DCI Architecture) on what is true object oriented programming! When is this going to end? We need to agree on the fundamental technology itself, we need to agree on what style of programming to use when, we can't go on for ever bashing person X or Y for not programming with insert cool way of programming here. People that understand programming languages and that do active thinking about it need to draw these conclusions ...

In short, my opinion is that:

- we need to consider the skill set of people and you need to allow the right people to work on the right things, and also to teach in a given amount of time the ones that don't know

- we need to account for budget

- we need to account for feasibility and we need to know our limits as a team, as a company, what we can produce and in how much time

- we need to not use "agile" or "scrum" as a tool to torture software developers and stress them out into a corner where they will likely refuse to give estimates or do any design thinking because of culpability reasons. This also means that we need to allow room for ERROR.

- we need design, especially in software that is related to complex systems, which is not a "sprint deliverable" in the "agile"/"scrum" cult thinking, and we need to allow the people that know how to design to do the actual design

- we need to be looking at building code by learning from people that have thought hard about how to build code in a good way, like Barbara Liskov (the person not the principle), David Parnas, Tony Hoare, Alan Kay, Trygve Reenskaug, Jim Coplien, Robert Martin, Grady Booch, Tom DeMarco, Martin Fowler, etc. etc.

- we need to be looking at other industries and at how they do it

- we need RESEARCH in the matter of programming languages and program writing, and we need our industries to draw their practices from this research and NOT based on the cargo-cult-which-is-the-coolest-10-frameworks-you-need-to-learn-today.