Less is Moore
samgentle.com
samgentle.com
If you have some real-time large-ish-data need, you'll need to evaluate your options based on how they manage concurrency and locking. You may even have to implement a custom cache or archive layer. (I had to.)
I'll admit, I haven't dealt with SVG rendering, and it wasn't important to any company I've worked for. In cases like that, you can say "whatever, fast enough on today's computers, this one doesn't work so I'll replace it with a png." But if you really need stuff to work right, if you want it done right, you don't have to do it all yourself, but you do have to understand it all yourself, otherwise you can't correctly choose and marshall the pieces that do it.
EDIT: I feel like adding an addendum: yes, the specific views described in OP are historic relics. Still... the number of levels that exist today between do-it-yourself c programming with an alternative/minimal libc, and a ruby/rails/activerecord/auth-gems/assets-gems project, is really amazing.
Turtles all the way down. I'm working right now own an OS-less Forth system to run on a SoC, and keep thinking "man, how much easier would this be if I had a kernel and a library!"
Meanwhile, somebody else is probably working on a pure logic processor and thinking "wow, I wish I had a full SoC to work with..."
There aren't that many giants; there are a lot of people with a few years of experience that think/claim they're giants.
You're writing SQL queries and claim you don't have the time to research how the database you're using works internally without destroying your productivity? You can write SQL without knowing the database's indexing strategy or how it optimizes queries, but it's likely to be of a lesser quality than someone who has taken the time to do so.
IIUC is that how the Go team sets out to do stuff? I have no references but I seem to recall I read something along the lines of "writing a Min(uint, uint) function is easy enough, so there's no real need for it in the stdlib (whereas Min(float, float) is there because it relies on optimized hardware), and you can always implement it and have it published" and "container types are best implemented on a case by case basis because their traversal is best suited to each specific case, so generics aren't needed that badly".
I do undestand such arguments, yet encountering the fifteenth slice Mirror function {which is unconcerned about the type) being either non-optimal, or just wrong due to an off-by-one, or giving up on static typing, gets old fast. And I'm not even talking about tries, b*-trees or whatever other interesting data structure there is.
To expand on this, and why Moore's views are a niche and will remain that way for a long time: in _Mote_, the aliens in question have been trapped on the same planet for millions of years, typically overpopulated, and have evolved to an extremely high degree to cope with living at (and sometimes beyond) the Malthusian limits. The Engineers are an example of these adaptations at work - because there are so many Moties, Motie time and labor is dirt-cheap (specifically, they are at the Malthusian limit where their wages equal how much it costs to live the most minimal life) while resources remain at their usual finite amounts; and so, it pays to have Engineers finetune and customize each product, for the same reason evolution leads to ultra-optimized (but often unclean and inelegant) solutions.
In contrast, as ugly incidents like the wagefixing scandal at Google/Apple/etc show, we face the opposite situation. Real resources are extremely abundant, and computations have never been cheaper, while programmer time remains expensive. So it makes little sense to take a Moore/Engineer approach, and instead generally the tradeoff of performance for more generality is made.
This will remain true for as long as the cost of programmers remains the main part of running software compared to the resources like CPU or RAM or joules consumed to run the software. (The software could be run at Internet-scale, in which case the possible efficiencies from specialization are worth the programmer time; or programmers themselves could become much more abundant, such as in a Robin Hanson upload/emulation SF-like scenario.)
It has always been a cost/benefit analysis. The hard part is figuring out the costs and benefits when it comes time to make that decision.
Case in point. My current work is in C# with some Go sprinkled around. Go happened because the c# Console an IO api's are a hopeless mess. They are bloated with everything under the sun and special cased in the API. And to top it all of they are just about the farthest thing from composable. The creators of C#'s standard library created a monstrosity in the name of abstraction that ultimately was harder to use correctly. What is the cost of that decision? Longer development times than is necessary for projects that do IO in C#. Late refactors when you realize you should have been using a TextReader instead of a StreamReader and you have these seams running all through your app that have to change now.
(Now, the Forth way isn't the one I would choose to be easy to understand: http://yosefk.com/blog/my-history-with-forth-stack-machines..... But that's a separate story..)
Part of your point still stands: the `for' loop is harder to read and write specifically because it is more general. `for' loops can do all kinds of wacky things.
The first kind is exhaustiveness. Being general by accounting for all special cases. That one is complicated, and rarely worth doing beforehand.
The second kind is genericity. Being general by ignoring all the special cases. That one is often simpler and more solid than any special case. Parametric polymorphism (or generics, or templates) are like that: ignoring the specifics of the parameters limits what you can do with them, making the generic thing simpler.
The layer at which you choose to apply exhaustiveness or genericity is irrelevant in my opinion. C++ often leans towards the exhaustiveness end of the spectrum despite being a language, for instance.
I think that maps to your distinction, except that I don't believe in 'exhaustiveness'. The state space of a program isn't some fixed thing for most programs. It evolves and grows in response to what we want it to do, which dimensions we choose to generalize along and where we stay stable. I think it's equally reasonable to view the evolution of special cases as equally exhaustive at every point, it's the boundaries of the state space (requirements) that is growing in strange ways.
Not that I'm trying to be all dogmatic about referential transparency- more like, without it, is there a large benefit to functional style vs. procedural style?
From all I've heard people in impure languages have begun to embrace the notion of restricting side effects voluntarily.
In fact, the more design you do, and the more you're exposed to, the better you can get at design. And the better the design, the easier it is to understand.
An "ugly generalized abstraction" is pretty much by definition not a good abstraction. Moreover, a good abstraction is simple. It's easy to understand and it's easy to manipulate. [1]
But you need to practice to get better, so it may take a few ugly monstrosities before you can start creating masterpieces.
[1] Granted 99% of the code I've encountered in Java qualifies as ugly abstractions; Java's nature seems to encourage poor abstractions. Or attract poor developers. Or both. But even in Java it's possible to create beautiful and easy to understand abstractions.
I was deliberately avoiding the d-word because it is vague enough that I'm not sure there's anything useful around there. The good designers are all good at designing within some domain. Jonathan Ivy, Christopher Alexander, Ralph Lauren, they all had their specialty. You wouldn't ask Ralph Lauren to design a rocket ship. Software is even wider a set of domains than meatspace. So I'm not sure you get better at design by 'practicing design'. That's just a set of vague words like 'simple', 'good', 'beautiful', 'abstraction', 'decoupled', 'coherent', 'cohesive', etc., etc. I think you get better at design by better understanding a domain. You get better at design by avoiding premature design.
"When you take on a framework, you're like a consumer buying a product: if it does a hundred things you don't need, or doesn't do things the way you want, well, tough. That's what we've got for sale. But as a programmer you're not a consumer. You're a producer. You aren't forced to accept an abstraction that doesn't work for you, or solves a problem you don't have. The option to build the Engineer-style specialised solution that conforms exactly and only to your needs is always there."
If you're solving problem X and realise "hang on, that's actually a special case of problem Y", it's actually the single most dangerous point in the development of your solution; you're only one step away from the logical conclusion of "I should solve Y instead". Now you're solving the wrong problem.
I think this resonates with most of us but we will still pursue the general case. I think its because problems are simpler when expressed in their general case than with all the hoary detail of a special case. Its close to a rule of nature: whenever we see complexity we can discover simple elegant rules that govern it, and as programmers who's main challenge is taming complexity, we are prepared to risk anything in our pursuit of simplicity, including more complexity...!