Does Software Understand Complexity?
michaelfeathers.silvrback.com
michaelfeathers.silvrback.com
This was all over the place behind the core theme of my CS master's degree, for which I happened to also have background in both mechanical and electrical engineering. There are a lot of things that felt quite natural to me but that were proving elusive and challenging to people I worked with who lacked an equally diverse background by being exclusively focused on CS.
Two points I raised were designing with, leveraging, and modelling feedback loops in component-based software using control theory (application was on auto-adaptative AV streaming across a network of varying quality† but the overarching theme was largely generalisable); and modelling execution flow using laws from electrical engineering (using high level concepts such as power potential and current intensity as well as qualifying code paths as being resistive, capacitive, and inductive) as an alternative approach to complexity and optimisation.
The resistance to cross-domain mixing was surprisingly high, entrenched thought processes are a remarkably deceitful thing, even when faced with provable results.
† Something I would find out to be implemented a couple of years later with zero-delay Xbox Live movie streaming.
It's only a master of advanced studies (Diplôme d'Etudes Approfondies [1]) so it's way shorter than a doctoral thesis. Also, the thing is probably half-assed in some way, as there was some, dare I say, "lack of support" on challenging ideas which was highly non-motivational on getting a proper paper out (The moment I talked about leveraging AI/NN in component supervision I was outcast as a sci-fi author). It doesn't cover half of my work (had to cut it down to only the core part, enough to get the gist of things, but that's only the tip of the iceberg). And it's in French.
If you (or anyone else) can ever get the full text please forward it back to me so I can laugh at my arrogant young self!
[0]: https://hal.inria.fr/inria-00000787
[1]: https://en.wikipedia.org/wiki/Master_of_Advanced_Studies#Fra...
According to Taleb: complexity is good for system reliability in the long term. (As well as redundancy in every aspect like head count, components count, technology stacks used in the same system) to mimic biological complexity.
But I strive to make everything as simple as possible, remove unnecessary components, etc.
I still cant understand contexts where complex is better than simple
But this cost association might be sort of special for systems built with code. Or systems that are built for handling discrete use cases rather than for continuous or approximate use cases.
This is quite speculative and a bit out of depth but: For some systems it might make more sense to duplicate a lot of functionality and use redundancy to achieve stability. Thinking of evolutionary systems for example.
But complexity can also be emergent from the aggregation of many small, simple components. E.g. the internet, which grew out of some relatively simple protocols connecting a handful of computers into the globe-spanning organism that it is today. So you may not design a complex system as a software developer, but your simple system may become a component in a larger, complex system. And that system will be more robust than the individual, simple components are. Such as the replacement of HTTP with HTTPS at the protocol level or fiber replacing copper in the physical system of the internet. Those simple systems have (or rather will soon) perish, while the more complex system that emerged from them survives.
Funny you should say that w.r.t. to reliability.
I just got back from vacation today and have just had 2 SSL-related issues I needed to solve, not to mention one related with dependencies on smaller components, which when updated broke my chain of software.
All these would have worked more reliably 1. without HTTPS and 2. if everything were packaged together :)
In short, the simpler solution would clearly be more reliable here.
I've wondered whether that is why we find ourselves surrounded by complex software. Legacy code is code that provides more value than the cost of its replacement. Bad code endures when that cost goes up.
If there are some, I'm not really aware of them. Systems we create should be human-manageable. The thing I find interesting is that, even with decades of attempts at better practice, we still end up with complexity that makes our work difficult.
I'm applying to the systems science PhD program at PSU this year and was amazed at the both the depth and breadth of the systems research and discussion to be had, but also noted the lack of software developers or computer/information systems discussion.
I'm looking forward to being able to bring some of my expertise to this field, while also mining the knowledge and concepts for use in better software systems design.
I'm going to definitely look into this conference for next year, though hopefully they won't be running the same week as the ISSS conference again.
It's Dr. Marty Zwick's favorite picture since he's the one who teaches DMM (Discrete Multivariate Modeling) which deals with RA. I haven't taken it yet, but he was at ISSS and gave a plenary on RA, as well as chaired a full day of RA related presentations.
Here is some more information, which also has the diagram I think you're referencing:
https://www.pdx.edu/sysc/research-discrete-multivariate-mode...
Maybe someday we can have an AI watch our log streams and tell us the ideal architecture for our systems. Then the trick will be to generate fake log data for what we want to happen in general. Perhaps we can just keep it coarse grained and say the AI can fill in the rest. The future of systems architecture!
Even more interesting is Conway's law. What if an AI could tell you how best to organize your company?
Companies organize around office politics, in my experience efficiency is quite low on their priority list, especially in big enterprises.
It's more rational from a business standpoint for it to introduce often-extreme efficiency penalties in order to achieve the CFO's will. This is an actual scenario that happened to me by the way. I couldn't analyze it with quite this amount of granularity until recently.
Everybody in the company that could have formed an opinion agreed with me about the efficiency gains of doing it my way. But what I was wanting was politically impossible. So me and the company parted ways.
Since then Magento's new version provides an API layer that makes it way way easier for companies with competent technical leadership to have their programmers use disparate skillsets and still get along. I can learn Magento well enough to work on it on the backend, and maintain the Rails frontend that I would have wanted to work on. If I had saw this coming at the time, I'd have stuck around and helped manage the replatforming. But all I was looking at was the extra $X0,000 I was going to make elsewhere.
(Stories about the mystery conference)
2nd sentence of 2nd paragraph: “I was at ICCS2018, a complex systems conference.“
Why is this normal now? It’s ruining the internet. Everything I read lately is wound up into a desperate suspense-arc.
I understand not everybody has liberal arts degrees (and definitely not this person) but that might be all the more reason to not make such an attempt.
Introductions come first. It’s a lot like meeting a person for the first time. Creativity is not a bad thing, but if you’re sacrificing respect in the process, you’re blowing it.
> suspense arc
Kids these days don't have any attention span. /s