Hacker News Reader

Comment by Ozzie_osman | Hacker News Reader

Lots of hate in this thread.. People asking for proof: what sort of evidence would convince you? I take these blog posts, apply it to my experience, and take what I think makes sense. Sometimes, an idea will solve an obvious pain I've had. Sometimes, an idea will show me a pain I didn't know I have. Sometimes I disagree with the idea because it won't work for me. That's fine. I'm still much better off thanks to this literature, and I would be worse off if it all had to be evidence-based.

If you think you can write better based on your experience, please go for it. I'd love to have more literature to read, analyze, and learn from. If you can't (pressed for time, etc), and only have time to post in HN comment threads, then at least discuss concretely which parts you disagree with so we can learn from that too (I'm serious, not being facetious). If you can't even do THAT, then sorry, all your valuable experience is way less useful than someone like Martin Fowler, because I'm not learning anything from you.

Having a good understanding of how to architect code is hard. Being able to write about that in a way that's useful for others is even harder, and is a skill of its own.

So personally, I thank Martin & Co for all their writing.

View on HN

Thank you for writing this. As an engineer reading these style of articles, I've witnessed two common reactions: Either the content resonates, and our eyes widen at the site of a simplified and intuitive explanation for phenomenon that we've somehow already observed, or... it doesn't. And maybe that's the source of much of the negativity we're seeing. These engineers haven't worked in domains, projects, or organizations where they could encounter how these lessons play out, and so they're dubious about the whole thing. One reaction to being in that position is to downplay the author and complain about lack of 'evidence'. The other, is to put down our egos, and consider that maybe we're just not there yet to appreciate what the author is trying to tell us.

That's not at all the source of my negativity. Instead, I am confronted in my work with developers of different experience levels which justify their design and coding decisions based on what e.g. Robert Martin or Martin Fowler wrote in a book or blog.

Some of these ideas, like "code should read like prose" or the obsession with unit testing are causing problems, such as making the code harder to follow or neglecting other kinds of testing that are more effective.

I'm not going to claim that everything these authors say is false, but it's high time we say: PoC or GTFO - show us the systems where these rules are applied and let us judge if they're worth anything, don't tell us nice stories.

> unit testing [...] or neglecting other kinds of testing that are more effective

Could you elaborate? (happens I come from a background where there is no kind of systematic testing, which is a bit of a downer)

And 'PoC' = proof of concept?

> which justify their design and coding decisions based on what <authority> wrote in a book or blog.

Happened to me. We had an SQL-based product that ran like a dog. Much faster by the time I'd finished it (I was the local guru), but on the next iteration I found the lead programmer had ripped out the big complex SQL statements it generated, and replaced each with many simpler queries. I asked WTF he'd done that without telling anybody and he justified it by pointing to a blog article recommending that for speed.

The article ended up 'trust me, it will be faster'. Oh yes, it ran faster because the query optimiser (very cpu expensive) now had very little work to do - so in ran like a rocket when the lead programmer (ie. one single user) tested it in his laptop. But it would scale like a dog because the optimiser had been sidestepped!

In addition, lots of little queries picking data from here and there would allow inconsistent views of the data because he didn't know about transaction isolation levels.

Thanks, idiot blog author!

i dont think having a blog outlining random advice is the source of the issue - it's the programmer who chose to take advice without first measuring and/or critically evaluating it.

Even if martin fowler and co didn't write this, soembody else would've, and the same sort of mistakes would continue to happen. The root cause, as always, is incompetence.

If people don't know much then they are not in a position to discriminate. It's the unknown unknowns. In the case I gave I'd reasonably agree that the lead programmer should have sat down with a stack of MSSQL books and done his homework.

But even more so, the prat of a blog author, a well-known blogger in MSSQL at the time, should have done his homework and not published dangerous crap. Proclaim yourself an authority, you'd better be one! I've other examples of this too.

But sometimes the end-users just can't know. I'll skip the details but due to overload at work I couldn't help our support staff properly, so they did a web search and grabbed a snippet which did what they needed. Well done for initiative, lads.

It also silently disabled full trans logging on our clients production DBs. Not so good. Support staff couldn't know, they did their best, blame lies with idiot blogger 'expert'. Again.

I was referring to system and integration tests. If a system has any kind of user interaction, user testing is essential.

The articles on Fowler’s blog are written by well-established senior developers/architects with years if not decades of experience.

There’s no evidence that Fowler, et al, are saying there is One True Way and he even eludes to the fact that any such proposal makes him uncomfortable.

I also have decades of experience and would add my observations:

Leadership matters. If you’re a part of team without authority or with weak a vision, you suffer at the whims of your leadership. I find this to be the first point of failure. Without a strong vision, enterprise software will develop code debt instantly and it will grow exponentially. Enabling a development team should be a priority.

Vision and Strategy A significant effort should be made to reason through objectives and put guardrails on those objectives. Are you re-platforming? Are you only rebuilding one boundary? Are you migrating to the public cloud and leveraging IaaS and PaaS? How brittle are your existing systems? Can your existing systems support the expected growth of the business?

Someone needs to consider these things before engaging with system change, develop a plan, socialize the plan with everyone, then develop execution strategies to enable the plan to succeed. This has less to do with code, but drives coding standards.

If you have complex systems, understanding Domain-Driven Design is a critical aspect to improvement.

If you have a “big ball of mud”, pulling it apart and rebuilding those parts according to the business is critical and can’t be done by “winging it”.

Opinionated vs Unopinionated Deciding how high your development guard rails need to be and managing them is an art, but it still needs to be explicitly managed. In an enterprise, opinionated is probably better. In a startup, the opposite may be true. Someone or some team needs to think about this and talk about it.

Communication Everyone, from Product Owners to Testers to QA needs to be able to model, whiteboard, and discuss problems, scenarios, and agree on solutions. Throwing work over walls will destroy the expected results.

I think there is some tribalism at play. Fowler and friends are writing about enterprise architecture. HN is probably more oriented towards startups which haven't yet reached that kind of challenges and constraints (and in most cases never will), and who feel superior to the dinosaurs. And then there is is the "computer scientists" who think software development should be based on proofs rather than experience and results.

And then there are people like me. I've worked in big enterprise software departments (a huge homeowners insurance tech company, big banking, government contractors), "companies" of three people including myself, and some in between. Currently I'm at a 500-ish person company that is not quite enterprise but not really a start-up.

My experience is that "Enterprise Architecture" creates as many problems for the business as it solves for technology practitioners attempting to solve problems in a huge battle royale of competing departments. I think it often helps cement and provide feedback loops for the causes of the problems seen in large enterprise systems.

This is where I get confused. Are you criticizing a particular architecture, or architecture in general? Is it even possible to have a system without an architecture?

The Winchester Mystery House does exist in software somewhere.

Even the Winchester Mystery House has architecture. Pretty good architecture. It's still standing, in spite of earthquakes. She bought it in 1884.

Now it has some fairly odd "features" based on her very eccentric "requirements." Doors that open into walls. Stairs that "go nowhere." Very expensive windows that look out at brick walls.

But given her odd sense of "requirements," it's the house she wanted!

The word enterprise is not included in the article, which makes it a generic article about architecture.

Experience and results are better then nothing. Where are the author's results?

Based on which project experience was the article authored? Without knowing such things, we can't distinguish what kind of outcome will the proposed ideas result in. After all, good results can be good on many dimensions.

Yes. And Computer Science is not a science, and Software Engineering is not an engineering.

Guides to it are more like guides to writing. Strunk and White, where is your empirical data? What's your control? Where are your PDE's?

Some also have aspects of business productivity books, with well-known common-sense ideas, like In Search of Excellence (which made up case studies, and a co-author said what's the problem?)

Even Fred Brooks' Mythical Man-Month - informed by his leadership of the massive bet-the-company "360" project at IBM - lacks data. It's still great though (in, uh... my subjective opinion, I guess).

Engineering is the application of science. A civil engineer uses laws regarding compression, stress, etc... to build its buildings. If you apply scientifically derived principles when building software then You are a software engineer.

That depends on what jurisdiction you are in. The title of engineer is protected in lots of places and the description of software engineer is very specific and effectively regulated. That is, a software engineer is licensed.

I'd love to see what those scientifically derived principles are. If I'm building a bridge, there are equations based on physics I can use. Like I mentioned in a separate thread, software is far more removed from those physics.

GP may be implying from the conditional that there aren't any software engineers.

While not exact physics we have plenty of real world constraints that we skillfully (or not) work our way around or use for our advantage, here are two :

- Amdalhls law

- Moores law

Software engineering is engineering in the same sense the mechanical engineering is engineering.

The only difference is that with software your materials are descriptions. I.e. you describe something, give to the computer and the computer turn itself into this something .

In the end, there is a physical entity in the world (the hardware), which turned itself into a unique machine based on your description (for example an ATM).

While in mechanical engineering, you materials are physical, which might make it easier to explain.

That is a valid argument if your concept of a mechanical engineer includes anyone who designs a machine; for example improvising levers, pulleys, gears to achieve some end.

I guess it is all "engineering" in a casual, tinkering sense. But a professional engineer, with a Mech Eng degree who uses mathematical models of stress and strain etc, who did lots of tinkering as a kid, might not agree.

Again with the over emphasis on “state sanctioned recognition”.

So will you call anyone that tells people what chemicals to ingest when "a doctor"? Will you call anyone on the Internet commenting on legal issues "a lawyer"? Or would you prefer people emphasize the meaning of "state sanctioned recognition" in these cases?

Engineering is like law and medicine, in that it requires high skill and can hurt a lot of people if done badly. Not every country has engineering licensing, but I find myself increasingly agreeing with those that do. I feel calling software development engineering is mostly false advertising right now, only trying to capitalize on (and devalue) the status that engineering has in society.

I disagree. Engineering disciplines vary in how far removed they are from the underlying physics and how malleable they are. Building a bridge (or civil engineering) is very connected to the underlying physics. A set of equations give you the answers. It's also not malleable. You design it once, then build it, putting in some margins of safety to account for any variances in underlying material or potential usage, then it's pretty much not malleable.

Mechanical engineering is a little more removed. You have moving parts, interconnected pieces, wear and tear, etc. Now you have to lean heavily on preventative maintenance, ability to replace worn parts, etc.

Hardware is more removed. Software is even more removed. The laws of physics are there, but there are so many man-made layers of abstractions on top of it that "good architecture" is a lot less like what you'd expect in a typical engineering discipline.

Not saying any is easier than the other, just that they are very different disciplines.

Your layering does not really have practical consequences that would matter to exclude a field for engineering. Plus you are focusing on the technologies, and while this is important this is really not what distinguish an engineer from a non-engineer technician (even an extremely skilled technician), and that part have way more things in common regardless of the field.

There is not such thing is "partial" engineering.

Engineering is the creation and ongoing management of any real world object (building, cars, bridge, computer) which has affect on its environment. I.e. they are placed in the real world to achieve some sort of a function.

The role of the engineer is:

1) Find out what function a machine should provide

2) Create this function by building the artifact on time, and in cost effective manner.

An artist on the other hand:

1) Does not care about the function. I.e. if the thing is useful or not.

2) Does not care about the time/cost etc.

The fact that software is just a description (bunch of text files), does not mean that it does not affect the real world.

As I said, the machine in the real world is the general propose computer. The computer understand this text files, and turn itself into the machine.

I didn't say anything about partial engineering. Just that engineering disciplines are different. If you need to design a column to hold up some load, a set of equations will tell you the dimensions of that column, how much cement, sand, etc to put in the concrete mix, how many steel rebars of what diameter and specification. Designing that column is very different than anything I do as a software engineer.

But, if you need to design a data processing system with a certain throughput, a set of equations can tell you how many servers you will need, how much storage, constraints on latency or network bandwidth, etc. So it's not so different after all.

I strongly disagree.

Engineering is the study and practice of problem analysis and design synthesis. Whether that means using abstractions of the physical world or how data is structured and processed is unimportant.

Any approach to the understanding of problems and how to solve them is engineering. Sometimes it's very structured, sometimes it deals with the physical world - and other times it's incredibly abstract and divorced from the real world. It's all engineering.

> Even Fred Brooks' Mythical Man-Month - informed by his leadership of the massive bet-the-company "360" project at IBM - lacks data. It's still great though (in, uh... my subjective opinion, I guess).

I have just read that book.

It is the computer book with most tables pr page of any I have seen.

How you come to this conclusion is beyond me.

The "data" in this context is empirical data in support of the hypothesis that the man-month is mythical (aka adding people to a late project makes it later). You also need a control.

Similar for other hypotheses, such as a "programming product" and "programming systems" take three times longer than a "program" (written by e.g. a garage duo).

BTW Your "beyond me" was unnecessary. https://news.ycombinator.com/newsguidelines.html

Thought I'd better check this. I went through the book's eponymous chapter 2 The Mythical Man-Month (p.14, 1975 ed), and it has not a single table; not a single datum.

It has many graphs, without data points, and the text and their smoothness indicate they are graphs of functions, not data.

I have a disturbing feeling that the downvoters in this thread simply don't know what it means to validate a mathematical model with data.

BTW It's not a criticism of the field that it's not yet an engineering. It's just where it is at the moment.

A philosophical book of essays about software from 1975 is not relevant for a set of recommendations on architecture 44 years later.

You are correct that chapter two lacks supporting evidence. And that's a problem for the book, it doesn't mean it's ok to omit data everywhere now.

I seems to me that you and hyperpallium agree.

I still hold that

1.) That book is still one of the most relevant books I've seen lately (which might be scaring, I haven't made up my mind yet)

2.) While he might not have (or at least not include) data, he argues hos case well, and while I wouldn't bet money on experiments to follow his smooth curve I would be willing to bet (a small amount of) money that for non-trivial software his ideas would still result in better estimates than one that accepts "man-months" as a unit of progress.

And for those that haven't read the book yet it might be worth noting that while some people might find the book quotable more or less from cover to cover there is also a number of observations to be made that takes a full reading to realize. Here is one I'd like to point out:

Some quotes from the book might leave one thinking that he says programming cannot be scaled. I'd say the opposite is true, he just argues that it must be done in a thoughtful way instead of by mindlessly adding people to the project.

I don't have an opinion on the book, I read it too long ago and just skimmed chapter 2 now. I don't think it's comparable with or relevant for this blog post.

hyperpallium's original message, the one that started this thread is a feeble attempt at legitimizing the blog post by comparing it to a bunch of unrelated and generally appreciated things.

I think it's quite clear from context that all graphs in Mythical Man Month are based on formulas and theory, not on double-blind experiments or empirical measurements.

Still, I often find it quite useful advice even for today's projects, where people commonly assume that they can put a late project back on track by adding people to it, and that by doubling the staff, the product will be produced twice as fast.

"Engineer" Origin: Middle English (denoting a designer and constructor of fortifications and weapons; formerly also as ingineer ): in early use from Old French engigneor.

Also shares same root as Engine I assume. Engineers are experts in system building or some kind, the earliest being weapons, then transportation, etc.

Software Engineer is 100% an engineering role IF you are making what could be called a kind of engine. Prime example is working on a game engine's source code, and design (i.e. Unreal, Unity...), or working on a custom operating system. I've seen many people try to downplay how much software has these "engine-like" properties, where you have to pay attention at the "moving parts" (runtime), and those people simply don't see the invisible machine that's doing the work.

Or in a non-medieval context, an engineer is member of a professional association of engineers whose rights and responsibilities are legislated. And imo, anyone who calls themselves an engineer but isn't a member of a professional association of engineers is misrepresenting themselves.

That's like saying no one can be an artist because they're not a member of the official artist club. I'm sure the term engineer outdates any professional association.

Actually it's just being respectful. I'm not an engineer, but I work with a lot of engineers who actually have the qualifications to call themselves engineers, so if I call myself one, I'm essentially lying.

I am an Engineer, in that my degree is in engineering, and my job is in engineering.

I am not a Professional Engineer, in that I have not taken the Fundamentals of Engineering exam, passed, worked for at least 4 years under a Professional Engineer, and then taken the Principles and Practice of Engineering exam. So I can't put EIT or PE after my name, stamp projects which require it, or operate as a PE in any state.

I misspoke then. I would call you an engineer without your being a member of a professional association. I guess in my mind the degree is enough. I'll never have PEng on my business card, but you at least have the potential.

Highly depends of your region. In France being an "engineer" means having an official Engineering Degree, or simply occupying an engineering position (the degree is not strictly necessary, you can have another one or even none at all).

The common ground of engineers throughout the world is more simply that they practice engineering... (and yes, this has a meaning -- managing requirements and developing in accordance to business values and economics comes to mind -- and yes, this can be applied to SE, and this is even quite "simple" to apply, at least this is not more difficult than for any other field)

I don't really see why some people pretend all the time that SE is different or young and it is not proper engineering. I never read a convincing argument about why it should be so. Of course they are differences but they also are differences between other fields.

Much programming is more art or craft than engineering.

For example, what separates a mechanic from an engineer? They are often working on the same project, and appear to be doing the same thing. In my view, a mechanic follows rules of thumb, eyeballs a design, etc. An engineer applies math and derives design, or checks the design using math. A mechanic will say "this beam looks thick enough to hold the load", and engineer can do a calculation to select the correct thickness for the load.

For software, a programmer can tell you that quicksort is faster. An engineer can tell you why it is faster, and be able to calculate how much faster it is.

That's a pretty bad example because any good programmer will have at least a rough idea how to profile code in their environment.

Can you name any software that's "engineered"? I can't. All software is built by craftsman of varying degrees. Nobody sits down and engineers anything up front.

The original point "agile" was that code is an iterative craft, so everybody should get together and work toward incremental improvements and respond to change, rather than try to engineer the whole project upfront. In other words, the way software is "supposed" to be built in most places these days is explicitly the opposite of engineering.

* Individuals and interactions over processes and tools

* Working software over comprehensive documentation

* Customer collaboration over contract negotiation

* Responding to change over following a plan

Doesn't sound like "engineering" to me, and that's the process with the most adoption (even though it's bastardized now, that's still the idea that's being sold).

> Can you name any software that's "engineered"?

Most compilers. GPS and mapping systems. Basically, software whose core operation founded on well understood data structures and algorithms.

No doubt there is heuristic fiddling around the user interface and that's not engineering, but the core of plenty of software ought to be well engineered.

I'm interested. Could you elaborate on "most compilers", with a specific compiler, and how it is "engineered"?

Although much care and thought goes into compilers, and they are related to discrete mathematical models (though, more descriptive than prescriptive for hand-written recursive descent parsers; unlike the central PDEs in some engineering), there is much tweaking for efficiency and especially for helpful error messages. I would describe that as "crafted" (though "engineering" has some crafting too).

Even something like diff, based on longest common subsequence, is full of ad-hoc tweaks to give excellent performance on typical real-world input, much better than theoretical complexity. (I mean the specialization that is in addition to the general optimization in the implemementation of the algorithm.)

I suppose your concept of "core operation founded on well understood data structures and algorithms" is the best we can hope for.

CompCert is the most extreme example of course given it's fully verified and the strictness required entails a high level of attention to details. Almost every compiler will be based on graph or tree transforms; Haskell and Rust are both engineered around reduction to intermediate, lower-level languages.

I don't think the tweaks you mention necessarily negate the core engineering, unless the system is completely unusable without them or they define more of the behaviour than the core itself.

Edit: for an analogy, consider whether the ergonomics of where buttons are placed on the dashboard of a car suddenly negate the fact that the car is engineered. That seems silly right? Any number of performance or other tweaks can be made to a car and not negate its engineeried quality, until you start compromising the actual operation of the car itself, like swapping out wheels for a tank track, or something like that.

Thanks, I was thinking of more used languages, such as c, python, java, javascript (they'd have greater need of good engineering, wouldn't they?).

I don't know Haskell or Rust well enough to appreciate their compiler engineering (and haven't heard of CompCert). But I might have a look at their source - I have a long-term project about tree transforms ATM and hadn't thought of looking at what compilers do (even though I was aware of Knuth's attribute grammars).

Yeah, nah, I think your "core operation" is a reasonable definition, both as primary function, and as the basic design (despite crafted tweaks atop).

I agree with your argument, but can you predict performance of an algorithm from big O complexity? (I assume you mean)

Perhaps a few runs of data points could enable curve fitting as it converges.

I feel like the "physics" of software is immature, closer to math than to practice, than for engineering - or something like that.

So I'm also agreeing with the gist of the other replier, that software isn't yet mature enough for precise estimates. We still flounder on what's needed, and stumble around what kind of a thing to build.

Precise specification is the exception.

> Software Engineering is not an engineering

For those who refuse to practice engineering, casting that self-fulfilling prophecy over and over instead, yes, what they call "Software Engineering" is not engineering.

They are tons of people who practice Software Engineering as proper engineering for decades, as most people (everybody?) should do.

At least take an other word if you want to designate your practice related to software development that is not engineering, and define it in detail.

I concur.

I'd like draw a parallel to philosophy: There is not one Grand Unified Philosophy Of Everything, but there are many different philosophers you can read, and take some of their views as their own, and still be aware of the rest.

I'm sure Fowler has been in contact with way more projects in enterprise software than me, so I appreciate his perspective. That won't stop me from reading folks from the more functionally inspired crowd, from the DDD corner of software design, etc.

Thanks for this. The initial wave of hate on this post had me worrying that we are collectively forgetting the lessons of the past. This industry continually re-invents the same concepts over and over again and just re-brands them. Once you recognize that, the writings of Fowler & Co become highly applicable to many scenarios.

Is this because what they've written truly works or is it because they've simply managed the same thing as the loudest engineer in the office, at larger scale?

Well HN knows a thing or two about the latter ;)

It’s because I write better software in a team setting when I apply the techniques and philosophies of Fowler & Co. The biggest complexity in software is people. Coordinating them, navigating egos, and leveraging their experience levels. It’s a hard problem and the place where I see the most consistent system problems.

IMO Fowler is just providing accessible common sense strategies for dealing with that complexity.

Interestingly enough this social factor is also the hardest to deal with in other scenarios (expeditions, film crews, multi-party-projects, ...)

> If you can't even do THAT, then sorry, all your valuable experience is way less useful than someone like Martin Fowler, because I'm not learning anything from you.

There is a small gap in this reasoning: that something is written down and sounds like a good idea doesn't make it a good idea. If you select on "people who have successfully written books about software architecture," you're in no way guaranteed to select for actual good ideas. It's quite possible none of the advice works.

That said, your broader point is taken, and I'll think about how to write things up about actual design lessons I've learned in practice, so still thank you for calling this out.

There is a small gap in this reasoning: that something is written down and sounds like a good idea doesn't make it a good idea.

That's his point, doesn't matter if is a good or bad idea, but that someone took the time to sit down to think through, write and publish some ideas that will give developers a new perspective on how to do things that they would never thought about, whether those ideas are good/bad, apply or not to the software we are currently creating is up to us to analyze and decided.

Suppose we weren't software engineers, but instead bridge engineers or physicians. Would you then accept "someone took the time to sit down to think through, write and publish some ideas" that we could either take or leave depending on how persuasive a writer the author happens to be?

I think it is entirely reasonable to ask "what reason to we have to think this is good advice"?

Perception, Comprehension, Analysis, Publication repeat.

Somewhere in there, it's doesn't matter if it's "advice" or whatever you want to call it, as long as it's shareable elucidation you can apply your own reasoning...but if you don't bother to follow the pattern, it doesn't matter what you think. Asking someone else to have done these steps for you, is a proxy for a lack of interest, imo. Everyone wants to be lazily told things they either already know or can agree with easily, but being critical in a constructive way is work.

I think the point of both 'geofft and 'bradleyjg is that these writings are mostly opinion pieces, and it's justifiable to not trust them based solely on popularity and author's status - none of which involve anyone actually verifying the soundness of all this advice. It's also justifiable to ask for more concrete evidence, since an ethical responsibility of an author is to not waste people's time.

If you were building a bridge, you wouldn't accept that you have to use particular type of beams just because a certain well-known bridge architect Fartin Mowler recommends it, and tells stories of how good bridges are when they're using these beams. You probably wouldn't discard the advice either, but you'd ask for more detailed, first-principles explanation, you'd ask to see the plans and the studies of the bridges with these beams, etc.

I think the article is really on to something, but fails to make it clear.

Her is my current belief system;

A) Architecture is strategy. Ted Neward was my first inspiration for this. It makes sense particularly when considering systems with long live cycles. Less obvious for smaller designs that don’t se much change over time.

B) Strategy is most clearly defined by Good Strategy Bad Strategy: The Difference and Why It Matters Book by Richard P. Rumelt Describes the kernel of strategy as

1) diagnose

2) guiding policy

3) coherent action

C) diagnosing is difficult, but both wardley maps and cynefin can help direct us to methods of diagnosing. Unfortunately the immaturity of computer science puts a lot our subsystems in the complex adaptive domain. Partly because we haven’t discovered the best patterns and partly because our tools them selves are a mess of abstractions.

I remember struggling with my first non-trivial CRUD web app using no web framework, just libs, it had to have a CLI frontend too, his PresentationDomainDataLayering[1] really helped me in refactoring the spaghetti code I had. Probably obvious to many experienced devs but it was enlightening to me.

[1] https://martinfowler.com/blik/PresentationDomainDataLayering...

Software development is as much of an art as a science.

Personally I've seen people double down on bad design decisions early in a project that becomes major pain to deal with later on. It is easy to see that if they spent a little extra time on quality in the beginning it would pay off in a few weeks or even sooner.

I also see others spend too much time trying to be perfect in the beginning and nothing ever gets delivered or the over engineered solution creates its own form of tech debt.

Every situation is different so you need to use judgement whether to apply someone's advice.

Software development doesn't need to be an art and it doesn't need to be a science either.

By science I mean using data and statistical evidence to drive development, by Art I mean using your gut and your intuition to drive development.

A computer is a deterministic system. It is 100% predictable and follows an exact set of rules. Software development is amenable to proof, logic and theory. You don't need to make decisions driven by data or your gut if the problem space can be described by a logical and axiomatic theory.

The problem is that the current "theory" is incomplete and not all parts of systems are amenable to statistical measurement. Thus a proper system architect needs to unify Theory, Science and Art to form a cohesive framework of system architecture.

Theory is priority because it says definitive things about the system, Science is secondary and operates as a data driven backup for places theory is currently incomplete, and Art is basically the catch all admission that there is no data or theory to back up the problem space and you're just going to have to rely on your intuition.

The problem with Martin Fowler is that all his stuff is 100% Art. It's just made up stuff from intuition and buzz words. Art is largely ad-hoc and does involve a lot of guess work that is often wrong. People identify with Martin Fowlers stuff because they identify with it on the "gut" level. It "feels" right. Your feelings are not a good way to do engineering, you rely only on intuition when you have no choice.

Fowler is also a big Object Oriented Architect. The thing with Object Orientation is that it has no basis in theory or science and is almost 100% art. The pattern is starting to fall out of favor because people are realizing that they tend to build over-complicated systems when following the pattern because they are largely just relying on their gut intuition to build these things using primitives that are validated by the same flawed intuition.

> The thing with Object Orientation is that it has no basis in theory or science and is almost 100% art.

Absolutely incorrect.

The data part of OO is based on set theory as formalised in the database design research and type systems.

The dynamic part of OO is based on finite state machines.

The data abstraction and information hiding part is based on the work of barbara liskov (for which she won the Turing award) - https://en.wikipedia.org/wiki/Barbara_Liskov

>Absolutely incorrect.

I said OO didn't have basis in theory or science. By basis I mean it wasn't derived from theory. Rather it was created as more of an "artistic" endeavor and later it was fit into a framework of an existing theory. You should know almost ANY system can be broken down to fit into an existing theory from a projectile object, to a flying plane, to the weather, to an object oriented programming language. Thus following that line of thought ANYTHING can be a part of a theory but I use the term "basis" to emphasize that something can be derived out of theory.

>The data part of OO is based on set theory as formalised in the database design research and type systems.

I highly disagree. Type systems are not exclusive to OO, additionally the OO type system allows for mutation which is basically not at all part of set theory at all. Also type systems and database systems didn't formalize set theory. Set theory comes from mathematics.

>The dynamic part of OO is based on finite state machines.

This I can agree. A state machine modifies a change of state which is closer to the definition of an object in OOP.

>The data abstraction and information hiding part is based on the work of barbara liskov (for which she won the Turing award) - https://en.wikipedia.org/wiki/Barbara_Liskov

This has more to do with the type system then it does with OOP. Also note if you read the paper. Her paper has no data, statistics, formal theory, rigor, theorems, proofs or axioms. So although she won the turing award, although this is a research paper... it is largely "artistic" in the sense I defined it. See the paper here:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.136...

Note that her paper is important, I am not discounting it. But it is largely more in the realm of design then it is in the realm of theory and science.

She developed the axiomatic theory in a later book:

https://mitpress.mit.edu/books/abstraction-and-specification...

Were she propose an axiomatic theory for OO interface design.

Definitely not artistic.

The only thing artistic in OO (as someone who practice it in the last 30 years), is finding the initial set of objects. But you will have this issue with any technique.

>The only thing artistic in OO (as someone who practice it in the last 30 years), is finding the initial set of objects. But you will have this issue with any technique.

OO is highly artistic. It's not just the initial set of objects you decide on, but you have to decide how each and every single object composes and which objects compose. There's multitudes of techniques for how to compose objects and when to compose: Inheritance, dependency injection, Object composition, etc, etc. Within each of these techniques you still have to write custom code, what is the parent, what is the child? What property overrides what? How does the parent use the injected object? etc etc... You are literally designing the system rather than operating within a theoretical framework. It is largely art.

If you operated within the framework of an existing theory you would be able to much better predict the outcome of a system at scale, but because in OO you are creating the system from scratch for almost every application it is largely unpredictable.

In mathematics all theorems flow from your choice of axioms. There is little room for deviation.

Interface is not exclusive to OO. It's a seperate concept. I don't have access to that book so I can't say what she talks about in the book specifically but an interface is not exclusive to OO. The concept exists in mathematics as well which deals more with functions and types rather than objects.

Additionally OO to my knowledge came later after her paper on abstraction. Alan Kay developed OO. Then OO as we use it today came when JAVA and C++ grafted parts of the concept into the framework of procedural languages. This is not deriving a language from theory, it's more of an organic evolution of concepts. This isn't bad, but it's not axiomatic, scientific or theoretical. It is art through and through.

As I said, any system artificially created or not can be fit into the framework of some theory. There is a huge difference though between systems that are designed artistically vs systems that are derived from theoretical knowledge.

> A computer is a deterministic system. It is 100% predictable and follows an exact set of rules. Software development is amenable to proof, logic and theory. You don't need to make decisions driven by data or your gut if the problem space can be described by a logical and axiomatic theory.

That's true, only if you can assume perfect knowledge about the problems/requirements, the data, the environment, and how (if at all) those will change.

In practice, none of these tend to be true, and that is where a lot of gut feeling will come in.

An experienced practitioner will just know that certain modules / feature sets will grow and grow and grow over time, and will take special care to build a particular robust approach for that area. Or that certain kinds of data are often of low quality.

Yes this is where the "Art" that I described also comes into play. We have no theory that can predict the output of a human brain therefore we can't theoretically derive what requirements a customer will want or change over a span of time.

However...

There is something sort of new from the academic side that could potentially be a theory that can deal with problem you describe.

At a high level the theory looks almost like a formal language for "design" and "abstraction." It's called category theory. It's new-ish so it doesn't yet solve the problem you describe but I see the outlines of it within the framework of the theory.

What's the relation between category theory and changing requirements? You lost me there...

An example:

Since category theory looks almost like a formal theory of "design" and "abstraction" is looks like if one day someone could come up with an algorithm or function to formally derive a solution given a set of requirements. To anticipate changing requirements you simply need to adjust the initial set of requirements to be more broader and then this hypothetical function would output a system that can handle potential changes that you anticipated.

This is all hand-wavy and not concrete. Many people who study CT see it as currently the closest thing to a formal theory of "design" and "abstraction." If this is true then techniques or algorithms for deriving architectures of systems can become turn key.

Of course let me emphasize, I'm just speculating here. Not talking about anything that actually exists. Just saying that if you study CT you can kind of see this kind of thing existing sometime in the future.

Proving diagrams commute with abstract nonsense is not going to help the average programmer. At most it's a formalism that will help people who design programmer's tools reason about their system, much like relational algebra for databases or Hoare logic for imperative programming, or UML.

Relational algebra is more or less isomorphic to SQL if you're doing SQL you're doing relational algebra.

Hoar logic can prove your programs 100% correct. This is useful for anyone who wants to write a program that's bug free.

UML is more of a visualization tool then a theoretical tool.

>Proving diagrams commute with abstract nonsense is not going to help the average programmer.

What in the world is a proving diagram, and what is abstract nonsense? Please be more clear with your explanations. I am not talking about diagrams and trying to say they commute with anything.

Category theory is nearly a century old; what are you talking about? People write code today to solve these problems, and it's category-theoretic code: https://www.categoricaldata.net/fql.html

Yes yes, you are right. I mean new-ish in it's application to things outside of math. And also very new to programming.

In terms of just pure math though, category theory is very new when compared to most of the disciplines around today.

Also you are completely wrong about it being a century old. It was invented in the 1940s.

I'm assuming you just googled it to find counter points. That being said you missed something even OLDER then CQL and the most popular and definitive use of CT in programming today:

Haskell.

Anyway my main point is that Category Theory can answer some questions about 'deriving' an architecture for a problem rather than having a person 'design' one. The way category theory is used and haskell and CQL is as a framework. In haskell and CQL it's more about designing the system within the constraints of category theory then it is about deriving a system. The derivation of a system from a formal theory was ultimately what I was trying to get at.

You can sort of see an outline for the possibility of this happening when you study CT. I'm being really speculative here in case I failed to convey that. Such a system doesn't actually exist (yet). That's all.

Haskell is not really category-theoretic. The category Hask doesn't quite line up with real-world Haskell. The disconnection is well-understood and well-documented [0][1].

Category theory can directly give us systems from theories. See the papers behind FQL/CQL for more details, especially [2] and [3].

[0] https://wiki.haskell.org/Hask

[1] http://www.cse.chalmers.se/~nad/publications/danielsson-et-a...

[2] https://arxiv.org/abs/1102.1889v2

[3] https://arxiv.org/abs/1706.00526

Well you're not technically wrong about the Haskell thing. I'll look into FQL and CQL. If such a thing exists, what in your opinion is stopping an architect from using it?

Is it just that it's not popular or are there flaws?

If you use it now, and it ends up being popular like Java or Python or Haskell are popular today, then you can expect a two-decade delay before you're vindicated.

>need to use judgement

I'd put this as the only real Golden Rule of software development. There is so much blind cargo culting around best practices / code / architecture in our industry that, with some engineers, human judgement seems to have been replaced with someone else's paint-by-numbers rules. Sometimes, man, it just makes sense to take a step back, look at what you're actually building, and ask what makes sense _for this case_. Sometimes the stupid thing is the best option. Sometimes it's not. Judgement should trump all else.

> People asking for proof: what sort of evidence would convince you? I take these blog posts, apply it to my experience, and take what I think makes sense.

I love opinions. But if you provide a plot like the one for software quality, you should provide backing data. OR you should clearly state that such plot is totally made up. Nothing else.

I have a blog where I write about my opinions. But they're CLEARLY opinions, UNLESS I have some backing data.

If you want to see how a properly referenced text looks like, pick any of the following:

* Code Complete - McConnell

* The Economics of Software Quality - Capers Jones

* Facts and Fallacies of Software Engineering - Robert Glass

* Making Software - Oram & Wilson

... and there are many others. Not all references have survived the test of time, L. Bossavit describes in his book " Leprechauns of Software Engineering" how several software development "facts" are in fact myths.

The approach you suggest suffers from a chicken & egg problem because you first have to be experienced enough to be able to evaluate such reference-free content, as the blog written by Martin Fowler. It's precisely the inexperienced that are likely to internalize such advice and perpetuate personality cults.

Your objections are invalid; one does not need to be better at a task than an individual in order to criticize them. Asking for references and proof is perfectly legitimate, at least if we want to approach our discipline more as engineering and less as "Strunk & White" as someone comically suggested in another comment.,

>Your objections are invalid; one does not need to be better at a task than an individual in order to criticize them. Asking for references and proof is perfectly legitimate, at least if we want to approach our discipline more as engineering and less as "Strunk & White" as someone comically suggested in another comment

Software development, as often as comparisons are made to engineering or science is neither. It's a craft. The tools differ from person to person, each piece of software is different, a lot of practises are learned and tacit and transmitted from one pracitioner to the next.

Writing code is more like being a blacksmith than being a scientist, as much as people yearn for formalistic answers and peer reviewed studies so that they can have the appearance of objectivity, a lot of software development is fundamentally subjective.

Software development does have a certain Darwinian character - whatever survives and thrives can't be wrong.

Embracing this seems misguided, because as software becomes more complex, it's no longer enough to have software blacksmiths banging away with their software hammers and unit testing the end result into a releasable shape.

Teams working on high-availability, high-reliability software know this and take a rigorous approach. Too many take the BDD approach (blog-driven development) and when the thing inevitably falls apart, the customers pick up the broken pieces.

> Software development does have a certain Darwinian character - whatever survives and thrives can't be wrong.

I am not sure of that. A lot of what we see today is just adding an abstraction on top of other abstractions. People use five frameworks for a static webpage that could have been easily made in half a day using editor, css and html without ever writing a single line of javascript.

In a evolutionary sense it works for them, but the reason isn't that html and css wouldn't also work, the reason is that they believe they have to use it.

So I'd be careful with the darwinian comparison here, this is more a mimetic or meme-like evolution of ideas and once the layer below is abstracted away it is quite easy to just forget about it alltogether.

A lot of what we see today is just adding an abstraction on top of other abstractions. People use five frameworks for a static webpage that could have been easily made in half a day using editor, css and html without ever writing a single line of javascript.

Not to speak for them, but I think it's more that despite decades of ridicule, there's still a lot of COBOL out there humming along. Some of it might even be deciding whether or not your health insurance is going to cover that thing that you went to the doctor for the other day. See also: Visual Basic, ColdFusion and so on.

> Teams working on high-availability, high-reliability software know this and take a rigorous approach

Approaches that are, nevertheless, still more akin to philosophy than engineering.

Please detail your understanding of what philosophy is and how it resembles what such teams are doing. My gut feeling is that they're quite different, but I could be missing something...

I don’t reward sealioning.

> Sealioning is a type of trolling or harassment which consists of pursuing people with persistent requests for evidence or repeated questions

Asking what you mean by what you wrote in a public discussion forum isn't seasoning. You aren't being asked for evidence for you opinion, but what that opinion is, and the context of a discussion forum implies you mean to discuss the topic. Also, I can't see the "persistence" or bad-faith that would make it trolling.

I looked at the correspondent’s other posts in the same thread. They qualified.

All right, but if we take the standard definition of philosophy then your initial claim makes no sense.

The study of the fundamental nature of knowledge, reality and existence is unrelated to writing software.

When I build applications the questions of “What is truth?” “What is the essence of a thing?” “What is useful?” are top-of-mind throughout, because there’s no rigour without context and purpose.

Software development is applied philosophy. Wittgenstein would’ve been good at it.

Not sure I agree. One example. Theres a lot of mainframe code still in production, it survived and it’s still doing its job to this day. The reason it survived is because replacing it is very very costly.

The rampant semantic games about art vs science vs engineering vs craft littering this thread are nauseating. It’s all of those things in different contexts and situations, and regardless, seeking evidence backed advice is the right thing to do no matter what context you’re in.

My electrician father doesn’t base his craft on thought leading blogs. He has actual textbooks on electrical equipment, and guides with examples on using tools and solving certain common problems, full of reference material to other books.

I happen to think software is something like 70% craft, 25% science and 5% art, and to me this suggests not putting much stock in armchair blog opinions (and yes that includes Fowler’s) and instead seek scientific or data-backed analyses, even for craftsmanship such as any of the references in the comment above yours.

Software development is indeed not "engineering or science";

But that doesn't mean it has to be "craft" or other highly subjective fields, nor that we can't aim for similar standards as other engineering disciplines.

> Writing code is more like being a blacksmith than being a scientist ... a lot of software development is fundamentally subjective.

But writing code is more like engineering than science, so why not compare those? There is subjectivity to engineering design too, but plenty objectivity in the constraints - what make something "fundamentally" subjective versus some other kind of subjective?

Is it possible we don't need to compare programming to any other profession? We can instead identify there is subjectivity in software development, and that some (but maybe not all) it it could be more objective for a benefit.

"Your objections are invalid; one does not need to be better at a task than an individual in order to criticize them"

He did not say that. He said, that he wants concrete criticism, what ideas are wrong and if possible, why ...

Referencing and evidence has never been a benchmark requirement for works of philosophy.

There are many arguments that sound logical on their face but completely miss the mark.

Laffer curve is one that comes to mind immediately. The saying goes: "It can be explained to congress in 6 minutes and used for 6 months."

It can be argued (with data) that it failed.

Martin Fowler is no doubt a smart and greatly respected person in this industry.

Saying something qualitatively doesn't mean there isn't quantifiable data behind it...but with his vantage point I believe we should press for a higher source of truth.

I assume he can understand how important that higher source of truth is in having these conversations regarding what moves the architectural needle.

Else we stay in this echo chamber.

ok, then. I will ask for no proof and believe in anything He writes.

Please don't post unsubstantive comments here.

Reply on news.ycombinator.com