Paul Graham on Spaghetti Code
kludgecode.com
kludgecode.com
Isn't the whole point of well structured programs to be able to work on them while holding as little of the program in your head as possible, so you can work on part of it and be able to "make all these types of modifications without even looking at the rest of the code"? If you have a program that can "evolve" successfully without the need for anyone to "hold it in its head", then a large team can work on it and you can usually replace a 10x programmer with a swarm of 1x ones, basically a pointy haired boss's wet dream...
Why is that the PHB's wet dream? I mean, by observation I agree with you. That is the case, so I am not challengingyou on that. But what could be the underlying reason, seeing as 10x programmers are only 1.5-3.0x as expensive?
It seems like PHBs shoot themselves in the foot, repeatedly, without mercy.
I think it's easier to understand why middle managers are the way they are once one understands their incentive structure. They participate in an employee's downside but not upside (she did it herself) so of course they're going to be the way they are.
I may be radical but I think open allocation is the only answer for software development. All the nonsensical process that grows up around closed allocation seems archaic and stifling. I think typical middle management is useless. What you really need are HR-independent mentors. Aside from mentoring, the other thing managers can do (firing) should be so rare in a well-run company that you only need one per few hundred people.
[1]http://www.ribbonfarm.com/2009/10/07/the-gervais-principle-o...
I didn't read any of it yet, but just the thought of having more to read about it makes my life already better :)
EDIT: I just realised who I was "recommending" this too. Well, at least it was worth it for me :)
Take any project (or platform) that uses quirky and individualistic style, preferably written in an obscure language. Any programmer that has been working with such a project for a year or two, would be a 5x programmer. Any programmer that has actually started that project would be a 10x programmer. Why? Because an amount of effort that any developer, unfamiliar with an obscure approach has to spend is 50x.
But. If that code is approachable by any regular developer, when the original programmer would not be 10x. And management and peers wouldn't think about the original developer as 10x.
But if, the code is quirky and individualistic... a funny thing happens. Suddenly whoever works (or can rise up and work with that code for whatever reason) is 10x. And peers and management would see them as 10x.
I've seen that quite a number of times. Many times as 10x, as 1x and even a few as 0.1x.
Getting to that point is difficult, and may require a 10x programmer. But the end result is simplicity.
quicksort=: (($:@(<#[), (=#[), $:@(>#[)) ({~ ?@#)) ^: (1<#)
Nope, still got it backwards.
The characteristic of a 10X programmer is how FAST he can write GOOD code that solves a particular problem. Neither individualistic, nor quirky. He doesn't even have to read Ayn Rand.
If it's not code that his 1x coworkers can understand, then he's not 10X, just show-offish and hackish.
To give some examples: Dennis Ritchie and Brian Kernighan. Linus Torvalds. Poul-Henning Kamp. Zed Shaw. Notch. John Carmack.
All of those are/were very much 10X programmers. But no obscure languages and nothing quirky and particularly individualistic about their code.
I think you have mixed the notion of the 10X programmer with the notion of the "cowboy coder" (with traces of crazy genius, e.g the autonomous, quirky, Lisp/Haskell/J/whatever yielding programmer who completes some huge project in his own peculiar way in a short amount of time).
And peers almost always lack the information to make the objective call. If they'd had that information, they wouldn't have called that developer a 10x. And that's one of the reasons, why you never see normal people calling themselves 10x, by the way.
(For my part, I'm only a 6-8x. I'm fairly new to software and I also spend too much fucking time on activism.)
Assuming the PHB can hire 10 junior devs or 1 senior dev at the same cost:
If you have 10 junior devs and 1 decides to quit, the PHB can swap him out fairly easily - he just has to wait for graduation day.
If you have 1 senior dev and he decides to quit, the PHB's hands are severely tied, especially if the company is located in a place where talent is lacking.
This is also the reason why mainstream languages will likely remain popular forever in the enterprise world - no one wants to take a chance on a language that only 5% of devs know.
Unless they have a legacy system. Or what percentage of developers know Cobol these days?
Problem is, the industry standard used to be COBOL. Sooner or later, if/when Java falls out of vogue and stops evolving, they will end up with two legacy systems instead of just one.
The true plague or our industry and everyone else's is short term thinking, and I don't see how it could be fixed at this point.
I'd venture that it's because something like less than x% of all programmers are elite; where x is some number between 5 and 10.
In this case, if you are PHB and have a budget of y for staff (where y <= average) and time variable of z (where z <= average) for a release date - it would appear that you are an incompetent PHB if you believe you can hire an elite programmer to solve your problem(s).
Basically the 10x guy is uncomfortable for all guys above him (not only in software but in any other field that has a 10x skill gradient) that lack self-control, self-knowledge and people skills (what's called "being wise"), and most PHBs are "very not wise".
Because it makes the business more failure tolerant. Finding even one 10x guy is tough - many teams will have only 1 or 2 of them, and a swarm of 1x guys.
If the 10x guy is on vacation, quits, or has something more important to do, you can throw one of the 1x guys at it.
You don't need nefarious "I hate the 10x cause I'm a psychopathic PHB" motives to explain this one.
That was exactly my thought as well, I had to reread the part twice to make sure. This is the definition of maintainable (or maybe even well-designed) code when all its parts are carefully isolated.
Adding that together, well designed code is the foundation of really bad code. That could explain a lot.
I won't guess at Graham's definition today. He can speak for himself, anyway. But it resonated with me in the context of Rich Hickey's ideas - particularly "Are We There Yet?" http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hic...
As I understand it, an issue with [some] object systems is that when objects are made by aggregation, rather than inheritance, they are of a different type than each of their components. And this requires the programmer to write some code [however minimal] for the new type in order to reimplement any desirable feature of the objects from which it is composed.
In other words, the behavior of an aggregate of objects is undefined. Hickey's comparison is to data. Which he claims aggregates into more data. When, I ask myself "what is data?" lisp-a-cadabra - it's code. Which is the [structuralist] way in which I read Graham's note.
I've only worked on one or two code bases that were truly spaghetti code and I replaced them both completely in short order.
"The object-oriented model makes it easy to build up programs by accretion."
I've seen people with literally dozens of objects who still had no idea what they were building or why and couldn’t answer basic questions about the problem domain.
The SQL database (a brilliant, wonderful thing despite the hatred it gets) is an example. When you write a query, you don't tell it which indexes to hit and how to micromanage details. You send a declarative query to this "black box" that, most of the time, does the right thing.
I think Objects are a good thing, but they should be rare. Very rare. As with macros, only use them when the simpler tools (referentially transparent functions, immutable data) won't do. Files are a case where the object metaphor makes sense-- there's a bunch of tightly-wound connected state, and I/O is inherently stateful.
The problem with "OOP" is that it's "programming as micromanaging, inept businessmen understand it". Instead of math, it's Factories and Visitors and other nonsensical "big picture" bullshit that doesn't map in any coherent, reliable way to the underlying substrate (mathematics, logic) of computation. I'm sorry, but programming is a game where if one doesn't understand the details, one doesn't understand the fucking big picture (which is made up of extremely important details).
And user interface widget libraries, which are all about I/O. Even Racket's widget toolkit leverages the superb racket/class encapsulation mechanism.
This makes UI code more declarative and allows you to create things like neat combinators. As an example, it's very easy to write a function like `when' with FRP, which could be used like this (code roughly from a game of life app I wrote):
when (not <$> paused) (step <$ time)
So you get signals and event streams like `paused' and `time' from UI widgets (or the system), and then combine them into new streams that you can plug directly into UI widgets. This then lets the widgets themselves be much simpler structures with not internal state. In essence, this replaces arbitrary mutable state with an explicit dependence on time which leaves less room for making mistakes while also making it easier to write high-level functions like `when'.
There's a ton of other cool things you can do--this is just an example meant to illustrate the sort of abstractions you can use; it is by no means exhaustive!
I've found that FRP leads to much nicer UI code than callback-based OOP.
Superb? The racket class system strikes me as awful/embarrassing. The language at least looks functional in nature, but to do OO they implemented this ridiculous single dispatch mechanism (manually calling "send" no less!). Why on earth didn't Racket end up with a CLOS like system? They could have made anything they wanted, and yet they ended up with a system that will require people to write the Visitor pattern. Shocking.
CLOS-like systems can be built on structs with the racket/generic module and Typed Racket makes multimethods trivial.
That sounds very much to me like 'A SQL database is an abstraction', and very little like 'A SQL database is somehow related to OO'. In facts it's really unnatural to map objects to relations, which is why you need yet another ORM abstraction on top of it, just to satisfy the OO silver bullet.
"Alan Kay's idea of OOP-- encapsulate complexity behind a simple interface, and separate the two while preferring stability in the latter-- was utterly sound and right. He wasn't saying, "go off and write opaque, complicated objects". He was talking about what to do when complexity becomes inevitable."
Late nineties, good times, OO was the silver bullet back then :)
After about 10 minutes of no progress, I suddenly realized I would be working with and for idiots so I excused myself from the rest of the process.
The tyranny of the dominant decomposition is no joke, especially when single inheritance and single dispatch are so popular.
If by "functional language" you mean JavaScript, then that's a completely different matter... :-)
In the production code, stay with the mainstream. And limit your extreme approaches to research grade code, with limited lifetime.
This may mean that we can't just go hire your average programmer, but we have also had a bizarrely low rate of bugs in production despite a high level of inherent complexity in what we do.
So let's wait a few years, and see, shall we?
(It's not that I don't like, or don't use shiny 'new' toys. It's just that in the long run it really really helps to use these toys only in the 'research grade' code. Depending on anything even a tiny bit out of industry mainstream in your key production code is a huge liability few years down the line.)
I do not fully agree with his opinion there, but he certainly has some valid points.
I've had to refactor code that my co-workers have written several times because they end up throwing code in a class where it works but doesn't "belong".
For example, the REST API I'm building has thin controllers, thick domains, and thin data access layer with various supporting models. I had a co-worker write a Domain object that acted like a model (a single object that should be instantiated, properties set, and a calculation function run).
I had to refactor all his code out into a reusable model and then build helper functions in the domain for the controller to consume.
Whenever I see that I need to do the same type of operation in multiple places, it tells me that code should probably be consolidated into a structured class to make sure that any bugs in the code can be fixed in one place instead of a dozen.
So in a way the code was structured in a class but was still a mess. OOP isn't just about classes and inheritance but about structure, architecture, and how classes interact with each other.
Inheritance (typically assumed to be part of OO, but not a requirement) is a big part of it. It gets nasty pretty quickly. Inheritance is the 21st-century goto, although it shares features with the even wonkier "comefrom" of INTERCAL.
Spaghetti code isn't just "bad code" but a specific kind of bad code where understanding a piece in isolation requires pinging about an ungodly number of places. There are a lot of causes of that, but usually it's poor software management and bad development practices.
Inheritance is a great way to maintain encapsulation in the face of a messy world. Unchecked composition quickly gets you to a world where everything has to be public anyway.
http://www.pcworld.com/article/249951/if_it_aint_broke_dont_...