Steve Yegge: Notes from the Mystery Machine Bus

Perhaps another way of framing this is to ask the question: are you optimizing for the best case or the worst case? This ultimately is a form of risk management. And I'm not talking in the algorithmic sense, meaning complexity expressed as the asymptotically worst case. I'm talking about people, software and ecosystems.

Let me illustrate this idea with Java.

- C++ has operator overloads. Java does not? Why? Because people might abuse them. That's optimizing for the worst case (ie bad or inexperienced programmers). Properly used, operator overloading can lead to extremely readable code;

- Java has checked exceptions and uses them liberally (pun intended). C#, as one example, only has unchecked exceptions. Why? Philosophically the Java language designers (and many of its users) feel that this forces callers to deal with exceptions. Pragmatically (IMHO) it does not and leads to more cases of exceptions being simply swallowed. But again this is optimizing for the worst case ie programmers who should deal with a particular error condition but won't;

- Java has no multiple inheritance. Same story: it can be abused ("it is known"). But also mixins can be a powerful metaphor.

- Rinse and repeat for duck typing, extension methods, etc.

Putting Python two steps from Ruby strikes me as an interesting choice. I'd say the difference is at most one.

I'll also agree that Google as a company (based on my own much more limited experience than Yegge's) is firmly conservative. The style of writing Javascript that he refers to is about writing Google Closure code with all sorts of directives to aid the Closure Compiler (I describe Closure as putting the Java back into Javascript).

I also see a lot of Python code that isn't really Python. It's Java expressed in Python syntax rather than idiomatic Python and that is kind of sad.

Which isn't to say that any of this is necessarily bad (or good). It's just a (software) political viewpoint you need to be comfortable with (or at least can tolerate) or (to quote the South Park meme) "You're gonna have a bad time".

One of the comments linked Worse is Better [1], which is worth a read too.

[1]: http://www.stanford.edu/class/cs240/readings/worse-is-better...

Steve claims that Clojure folks come from the Haskell/ML world when the survey lists the "former primary language" of survey takers to be 1% Haskell, 0% SML, 0% Ocaml. Whereas they actually come from Java, Python and Ruby mostly.

The Clojure "replacement" in the survey is all over the map with Common Lisp, Erlang, Haskell, Java, Python, Ruby, Scala and Scheme all performing well. The liberal/conservative thing is a false dichotomy and Clojure community is probably living proof of that.

I'm trying to think of an apt analogy for this post that doesn't involve vomit or defecation, but it's hard. From the introduction proclaiming how readers will be stunned by how clearly and resoundingly true the revelation revealed within will be, to the literary diarrhea it's followed by... it's like a little kid proudly telling his parents he finally used the toilet properly only for them to find he completely missed the bowl. Yeah, I failed.

If the political spectrum is deeply flawed, as he said, then why even try to hack it onto something completely unrelated, made of individual technical points where each programmer may have a different approach?

I don't know, I like a lot of his past posts, but I don't dig this one. I don't think it provides any useful insight whatsoever.

In particular, I like typing as machine readable documentation, that makes IDEs simpler and more accurate in code navigation and refactoring. Dart in particular shows this false dichotomy really well, but having a type system for human and machine readable docs, making the IDE experience far more pleasant, but which can be turned on or off. Unsound, untyped, programs can still run. Yes, dynamic languages can have nice IDEs too (Smalltalk), but they are harder to engineer.

In terms of optimization, typing is a must for many types of hard real time programming. You can bet that the Mars Curiosity rover isn't using a garbage collected dynamic language. Nor are the inner rendering loops of most mobile games or console games. (Lua is another story when it comes to actual game logic)

Lots of bold claims have been made for Javascript JITs for example over the years, include a lot of hype about tracing JITs, but the reality is, Javascript still doesn't even come close to native performance, and it's hideously slow on mobile platforms in comparisons, with basic, idiomatic programming (e.g. dictionaries with fields) having terrible numeric performance. All this has actually served to threaten the Web with a native-takeover because we don't have a fast web language CPU and memory efficient on these devices.

I don't think that Tim Sweeney or John Carmack are prematurely optimizing when they decide to write a game engine(rendering) in C++, because experience has taught them that it is highly unlikely they'll be able to optimize a dynamic language implementation later to a satisfactory level.

I think many people use a mix of languages depending on the context. I certainly wouldn't write a web service in C++, nor would I write a 3D game in BASIC. I wouldn't use anything but Perl to masage text files, and I'd use R for data analysis. 

In general, when I fall on the liberal side of an argument, I think it is for the reasons he gives (no fear, resist ossification). But when I fall on the conservative side (which happens just as often) it is not because of fear, but because I think that mathematically rigorous abstractions (pure functions, persistent data structures, etc) offer a more powerful way to approach the problem.

I think my attitude toward bugs (which he suggests is the defining issue) is more nuanced than this spectrum allows. Bugs are inevitable, and it's not the end of the world when you have a bug. We need debuggers. But I think that over time, if we are doing anything right, we should be growing a stable core of our codebase that is increasingly flexible and bug-free. I guess maybe this could be considered a centrist view. It is certainly neither liberal nor conservative.

Someone in the thread mentioned pragmatism vs. idealism. I think this is a much more useful distinction. I would definitely consider myself an idealist. But the pragmatic-ideal axis doesn't map to the conservative-liberal axis at all.

Unfortunately, your business is not Aunt Marge. You need to be able to make the tough calls and say that, no, banning the color yellow is not a viable policy. Software engineering and programming languages are both seriously-studied disciplines, and all too often, the evidence comes down conclusively in favor of one position.

To pick an easy target, in many languages like Java and C++, null can be passed in place of (almost) any type. But

1) Empirical studies show that values intended to be non-nullable are more common...

2) ...which means that many method definitions are cluttered with is-null checks (to cut down the exponentially-increased state space)...

3) ...and it's just as easy to provide a feature to turn it on when it's wanted (option types/Maybe monad)...

4) ...which many companies hack into C++/Java anyway (various annotations and preprocessors)

This is a pretty solid case. Liberals win -- it's less code. Conservatives win -- there are fewer bugs. Sometimes things really are that one-sided.

[1] http://blip.tv/clojure/christophe-grand-not-dsl-macros-45407...

From a German point of view both liberals and conservatives are right wing. And his article is totally missing any left wing politics. He is missing the green, the socialists, the communists, and the anarchists point of view here.

And making it one dimensional also does not fit: What about Christian socialists, what about conservative greens, what about anarcho-capitalists, or national socialists?

e.g. I would classify myself (when it comes to programming) as anarcho-capitalist: I'm using the language that best fits the problem, without bias. And I'm earning most of my money with free software.

I also can not agree with how he positioned some languages, e.g. Perl might look liberal at first, but the CPAN community is more conservative when it comes to constraining regression test or documentation than the Ruby or Python people. Also ASM might look liberal at first, but if you ever worked in a closed shop, you know that ASM/370 coders have a really conservative approach to create software.

"A measurement of length of a piece of writing, particularly when indicating a length excessive for the genre. A Yegge is approximately 4000 words or 25 kilobytes.

Named for well known programmer and technical blogger Steve Yegge, whose blog up to about 2009 was notorious for entries of approximately 1 or 2 Yegges in length, vastly exceeding the typical length of blog entries in the genre."

Political talk and thinking is poisoned by the ideas of "left" and "right" even though those phrases haven't had a connection to reality since the French First Republic.

It'll probably be fine. Politics is in many respects a zero-sum game, and polarises participants into two camps. Software isn't like that.

For example of what's pretty close to a Yegge rant of the 1980's, see Edsger Dijkstra, "On the cruelty of really teaching computing science" [1]. It seems to be bemoaning similar debates in the software field, though from a very "conservative" perspective, since Dijkstra prefers that a formal proof should be required with every program, and one should completely avoid anthropomorphism in discussing software design.

Another thought that's crossed my mind: one of the reasons for the evolution of these approaches to politics and risk over the years has to do with the scale of impact. Liberally messing around with a social and economic systems can lead to widespread human disaster: China's Great Leap Forward comes to mind. How the software is used and how reliable it needs to be is an engineering tradeoff with regards to cost & time. This is often why you tend to see much more liberal approaches to software in smaller companies - the scale of impact is much smaller (in terms of customer base, and investment) when you cock up.

Now, it's clear that larger companies, particularly many IT shops, could learn a thing or two about being "progressive conservatives", as they've "conserved" for far too long and are caught in the trap of special interests (i.e. traditional vendors dependent on their cash). Fear of dynamic languages, open source, or cloud services, or non-waterfall development is mostly a reactionary ideology grounded with some kernels of truth - static typing DOES help codebases scale (but you shouldn't be proud of a large codebase), you can't just pick ANY open source library as some are DOA, or ANY cloud as some are unreliable, and tinkering with your delivery methodology can reduce your performance greatly due to confusion, plus there's plenty of cargo-cult agile consultants waiting to rope you in. So, you need to think these things through. But that's not an excuse for avoidance. Perhaps that means I'm a software moderate.

[1] http://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD1036...

(edit: typo)

With Clojure you keep one feet on the ground (the Java ecosystem) and the other feet is in the unknown dark water.

One may find many aspects of a contract annoying yet still prefer a world where contracts are widespread to a world without contracts.

In a sufficiently complex ecosystem contracts make some things easier and some things more difficult. But contrary to Yegge's assertion, it may be that some organizations have a contract that if something breaks, then one of the parties involved does a rewrite/redeploy. This is not a replacement for a contract, as Yegge implies, simply a different contract.

One contract might be: "All code must be unit tested". Another might be "If you don't write unit tests and your code works, that's great, but if it fails then prepare to pull an all-nighter if necessary."

My guess is that most developers, if asked which kind of methodology was appropriate, would generally pick a methodology that was appropriate for the level of risk involved. If the code is going to manipulate a robot arm holding a knife as it jabs quickly toward the programmer's body, few programmers are going to think that the bugfix/redeploy approach makes sense. But when it's a social site then everyone starts to feel more like a cowboy.

I think it is an insult to the professionalism of programmers everywhere to assume that risk decisions are a function of internal constitution rather than a rational risk assessment.

The type of tools they use and like are circumstantial. It's more of a empiricists vs. authority thing.

If it's a library that someone else will use, I try to play as conservatively as possible, and I fully expect that despite my best efforts those developers will uncover lots of bugs regardless.

If it's a mission critical application, I'll also be conservative. The more pain there is in fixing things later, the more careful I'll be up front.

When I'm writing a game, I'll play it more fast-and-loose. So long as any glitches don't crash it or open an exploit or ruin the user experience, it's usually an acceptable trade-off.

When I'm writing stuff for myself, I go crazy, trying out all new fads and methodologies just for the hell of it. This way, I learn new things and get enough experience with them to mark them as conservative-safe and liberal-safe.

My only sticking point is this: If someone competent cannot follow your code, you haven't documented things properly.

A bit annoying that the post text only takes up about 33% of my available screen real estate, even on the "view single post" URL: http://cl.ly/image/1p1B2o3D262g

So this is the project that Yegge mentioned would turn "all code [...] into Wikipedia." Man, my ongoing project is more similar to it than I thought.

I find it curious that he would even bother to mention (IV) though. (IV) falls right out if you start from the correct data representation, which I would have assumed from (A). I wonder if he's still listing dependencies explicitly.

And because of that you can find yourself in an endless argument with someone.

Why he dressed it up in political satire (allegory?) I don't know, seemed to me to make his point less forcefully.

Is there a reason nobody speaks publicly about Apple's engineering culture? We hear a lot about the culture at Google, Facebook, Microsoft, and Amazon. But I've never met an Apple engineer. And I've never read any detailed accounts by an insider.

Does anybody have any good resources? Or is there a very restrictive confidentiality policy?

In my estimation, if the answer here is unclear, then this metaphor breaks hard.

If that happens, I predict a fucking hellhole and I can only imagine myself leaving the industry at once.

This means that promising projects like Clojure CLR have no chance of taking off the ground, unlike say Ruby vs Jruby vs Ruby EE vs Rubinius.

Conservative: the existing system must not break!

Progressive: we must add new features!

Political terminology in the United States is MASSIVELY warped by entities who spend literally millions of dollars in an effort to connotatively redefine words. When you (re)define words like "conservative" or "liberal" you risk alienating people who have been conditioned to have an emotional response to either definition. If you define a term in a way that seems pejorative to your reader, the reader will weight every subsequent statement against the rigor they think you brought (or did not bring) to those definitions. Your essay will entirely fail to persuade if it rests on (perceived) faulty definitions.

When writing to a largely US audience, no attempt at honest communication will be aided by the application of political labels.

Here's a test: can the essay be rewritten without that political terminology? Reader "nirvana" used the terms "cowboy" and "architect" to describe his two axes. ( http://news.ycombinator.com/item?id=4367328 ) I very much prefer those terms. Yes, they're also connotative and subjective, but as analogies they map much better onto the subject being discussed.

Scheme, Erlang and company are more like communism: extremely liberal applications of extremely liberal beliefs. So much so that people tend to view them as "good in idea, but flawed in practice".

Python and Ruby are more like traditional liberalism. They bring in new ideas, but not in excess.

Scala and Clojure seem remarkably centrist. They both bring in good ideas from both the liberal and conservative camps.

Lastly, C is "old guard" conservatism, C++ is a Bush-style "compassionate conservatism" that tries to please everyone while being labeled "conservative", Java is a neo-con, and C# might be compared to the Tea Party.

I think there are way too many variables for a linear scale to provide meaningful comparison.

Put another way, it's the political liberals who came up with OSHA (I hope--I'm somewhat ignorant of the actual history :P). A liberal or conservative outlook is not characterized by some particular processes or tools--it's characterized in an entirely relative way. The conservative approach is in choosing the familiar over the novel and in avoiding change. The familiar could be safer--Java vs Python--or it could be less safe--Java vs Haskell. I've talked to some ardent Java adherents, and they have lucid cases for not going over to Python or Ruby or Clojure or what have you. But--critically--these cases are virtually identical to their cases against going over to Haskell or Scala. There are differences in details, of course, but it's a difference in degree rather than kind.

Another even more extreme example is TDD. In particular, the arguments people have against adopting TDD are essentially exactly the same as I've seen from TDD supporters against using formal methods. Once again, some details differ, but the core idea seems to remain: some people are inherently wary of change.

It's also interesting to note how Yegge categorizes certain concepts in multiple "buckets". Either he's just being inconsistent (which is plausible) or he's making a deeper point: it's not about the particular concept, it's about the philosophy behind it. If he wasn't making that point, I've made it for him :).

That is, anything called "something calculus" is conservative, but lambdas (e.g. lambda calculus) aren't. Type-based functions overloading (like type classes, I guess) is conservative, but Scala implicits are liberal.

In my view, the languages that are the most conservative (at least in my part of the world) are Java and Python. Why? Simple: they are the default language for almost everyone I know. You're at an enterprisey company? You're probably using Java. You're at a startup? You're probably using Python. You're using C or Scheme or Haskell or Erlang? You're crazy. (I should note that I don't know very many people in systems or embedded programming, so my view is obviously rather biased.)

All this rambling (I certainly see why Yegge always writes long posts) has left me with a fairly concise conclusion. Namely, mapping programmer attitudes to a spectrum vaguely inspired by politics is a reasonable idea. Sure, the reality is that there is no total ordering so a one-dimensional representation is fundamentally lacking. However, it's good enough to give some insight.

But I would not map technologies there based on the technologies' innate traits. Rather, I would map them there based on the thinking behind the people who use them. This is similar to how--if you don't know the background--it's hard to guess which political party supports which regulation. Gun control is the opposite of liberal, but it's exclusively heralded by liberals; deregulation seems liberal but, of course, isn't. Yet, on other issues, people on either end of the spectrum behave as expected!

This is why I think languages like Java and Python are fairly conservative. Not because they try to offer some sort of safety but because they are safe choices. This is also why I would probably place C# as significantly more "liberal" than Java--it may be the most "conservative" .NET language, but it is far less afraid of embracing new ideas than Java. So that end is simple: at least for enterprisey companies and startups, it's populated with Python and Java. But what about the other end? I think this is where the languages that most people consider too crazy to use go. Haskell, Scheme, Erlang and so on. Only very brave--very liberal--companies are going to use Haskell or Scheme in actual production. Too many weird features. Even the sentence "Scheme in production" just sounds weird.

So it's not a matter of wanting handrails (Haskell) or not even wearing a helmet (Scheme); rather, it's a matter of being willing to choose something more advanced over something more understood.

Assembly language: Batshit liberal.

Perl, Ruby, PHP, shell-script: Extremist liberal.

JavaScript, Visual Basic, Lua: Hardcore liberal.

Python, Common Lisp, Smalltalk/Squeak: Liberal.

C, Objective-C, Scheme: Moderate-liberal.

C++, Java, C#, D, Go: Moderate-conservative.

Clojure, Erlang, Pascal: Conservative.

Scala, Ada, OCaml, Eiffel: Hardcore conservative.

Haskell, SML: Extremist conservative

Woud be nice to overlap James Iry's chart[1] with it...

[1] from http://james-iry.blogspot.com/2010/05/types-la-chart.html

(Dynamic Languages are Static Languages by Robert Harper): http://existentialtype.wordpress.com/2011/03/19/dynamic-lang...

With HN discussion here: http://news.ycombinator.com/item?id=2346590

Also some classification of technology as conservative or liberal depends upon the competing technology it is running against. I was a bit surprised to see Python classified as liberal - but it makes sense when comparing it to Java. If it is being compared with Ruby it is very conservative ("do things one way").

There are some non technological risks that influence technology decisions. Visual Basic may be "Hardcore Liberal" from a language perspective, but it is pretty conservative politically (backed by Microsoft, lots of available experienced programmers).

Steve as usual has interesting insights. I am not sure that this is a paradigm that completely fits - but it does provide a perspective for understanding fundamental beliefs that can lead to disagreements in software projects.

His conservative definition is seems to be "everything that is wrong with the world" or something.

---------

Also, this older rant reads to me as Yegge saying that this "conservative guy" was responsible for Google's success:

"I've been debating whether to say this, since it'll smack vaguely of obsequiousness, but I've realized that one of the Google seed engineers (exactly one) is almost singlehandedly responsible for the amazing quality of Google's engineering culture. And I mean both in the sense of having established it, and also in the sense of keeping the wheel spinning. I won't name the person, and for the record he almost certainly loathes me, for reasons that are my own damn fault. But I'd hire him in a heartbeat: more evidence, I think, that the Done, and Gets Things Smart folks aren't necessarily your friends. They're just people you're lucky enough to have worked with.

At first it's entirely non-obvious who's responsible for Google's culture of engineering discipline: the design docs, audited code reviews, early design reviews, readability reviews, resisting introduction of new languages, unit testing and code coverage, profiling and performance testing, etc. You know. The whole gamut of processes and tools that quality engineering organizations use to ensure that code is open, readable, documented, and generally non-shoddy work.

But if you keep an eye on the emails that go out to Google's engineering staff, over time a pattern emerges: there's one superheroic dude who's keeping us all in line."

(http://steve-yegge.blogspot.com.br/2008/06/done-and-gets-thi...)

urban - accepts a certain amount of chaos for the benefits of reducing redundancy, wants to centralize code, reuse via shared centralized code

rural - wants to be isolated, wants to "see" everything, wants to have control over everything, reuse via cut-and-paste

2001: http://developers.slashdot.org/story/01/11/13/0420226/kent-m...

1994: http://www.nhplace.com/kent/PS/Lambda.html

Nevertheless, I don't know which was worse, the claim about what political conservatism/liberal is or the software one. I guess the software doesn't matter because he's just making it up anyway, but still. Before anyone thinks too highly of this piece, I recommend people study some actual philosophy and humanities.

At it's very core this is just an elaborate (not in a good way) dressing up of the static vs dynamic typing discussion; hardly a revolutionary insight. I also don't see how this is "risk" based debate. But in any event, this is just hard to take.

[1] safety meaning certainty of correctness, and speed meaning speed of development (as opposed to execution)

I think the issue he captures has been well known for a while. It is structured versus unstructured. Neither is better in an absolute sense, but one method can be better than another in a given situation. This is beyond just software, but business approaches in general. Startups are generally liberal/unstructured, Dow Jones firms are generally conservative/structured.

I did find it interesting that he considers himself a liberal (both in software and politics), then then goes on to create a "type system" categorization for programmers (and programming languages) to be placed into.

Then again my actually political views tend to span the liberal/conservative spectrum so I suppose this is not surprising.

Didn't we know that already?

I would like to believe that the political bit was just a well timed hook to get people to read his blog. He understands that the right incendiary rant will bring readers. If it is sincere, though, it shows a disturbing level of crankiness.

http://lesswrong.com/lw/no/how_an_algorithm_feels_from_insid...

https://dl.dropbox.com/u/704864/software-politics.html

priestc, your account was hellbanned 26 days ago: http://news.ycombinator.com/item?id=4245644

If you like dynamic typing and can write good, legible code in a language like Python or Lisp, do it. If you like static typing, knock yourself out. If you want to use an IDE, go for it. If you want to use emacs, do it. Hell, if you like object-oriented programming, try it out. I think 95% of "object-oriented" programming (as currently practiced) is junk, but the other 5% is legitimately valuable. If you have the taste to pick from that 5%, go ahead.

What you shouldn't have the right to do is impose complexity on other people. Use whatever environment you like, but if your code depends on your environment, that's bad. If people can't get work done because they're cleaning up your messes, that's bad. Be as liberal and as kinky as you want in your own sandbox, but don't impose your wacky, untested DSL on everyone else.

That said, I like statically typed languages. ML is the only language I've encountered where reading average-case code is enjoyable. (Scala's a great language, but average-case code is ugly due to the Java influence. There's a fair amount of spaghetti code written in it due to the cultural legacy of the SpaghettiFactoryFactory Java culture. I can't speak for Haskell because I haven't seen enough.) I think that's neat and very rare in the programming world. How much code is enjoyable to read? 1 percent, maybe? In Ocaml, that number is a lot higher. Probably around 50%. 50 percent of Java code isn't even legible. Being able to actually read other peoples' code is nice, and it's one thing I miss about working in Ocaml.

I'm probably more in line with the hard-line conservative camp in terms of my view of complexity: avoid it unless you need it. The Unix philosophy works. Religious X-oriented programming doesn't. Big Code leads to fail. Small-program methodology's little programs (Unix philosophy) are written to solve problems: do one thing and do it well. Ambitious projects should be structured and respected as systems, not all-or-nothing, massive single-program megaliths with no clear communication policy among modules. Small-program development works. Big Software is written to get promotions. That produces the next generation's legacy horrors. Also, structuring your company around 17-day "iterations" is stupid. Et cetera.

I also tend to think that a lot of the features that scare typical software conservatives are genuinely worthwhile. Macros in Lisp are important and can be very beneficial-- if used conservatively. Cleverness for its own sake is bad, but there are times when macros are very useful. Document what you're doing, and make sure it's tasteful and makes sense before you let anyone else depend on the work, but go ahead and do it. I wouldn't have learned what not to do with macros had I not made a few mistakes when I first encountered them.

So, with a mix of opinions from the "conservative" and "liberal" camps, I can't say where I fall. I like macros (when used by disciplined people) but I also like static typing. Both turn out to be very useful tools. Consequently, I find that I like a lot of different languages and insist not on a specific one, but on small-program methodology so that people can use the right tool for the job.

I'm conservative because I dislike complexity (I think "software liberals" are OK with complexity as long as it's under the hood-- most metaprogramming involves extremely complex solutions that, when they work and the abstractions don't leak, although this is rare, allow clean interfaces-- whereas I'm not comfortable making that distinction) but I (a) understand that liberalism is essential to driving innovation, and (b) can't classify myself as a conservative because management is inherently conservative and is, in software, almost never the solution to the problem. Usually, it is the problem. Most companies fall to shit not because they have some difficult code-- every codebase has some crap in it-- but because management mandates that they use the bad code (often for political reasons, like the original architect being a crony of the manager) as-is instead of taking the time to understand or replace it. I'd like to see how Valve evolves over the next 5 years, because I think management in software is usually a source of undesirable complexity, rather than the safeguard against complexity that it thinks it is being. If Valve can give us a determination either way on whether software can work without managers in the first place, that'd be incredibly useful information.

Not surprisingly, software politics also has a lot of individual inconsistencies and hypocrisy. Corporatism (in politics, not software) is neither socialism nor capitalism but a system designed to give the best of both worlds to a well-connected elite and the worst of both to everyone else. (Consider air travel as a microcosm: Soviet experience and service quality and comfort, mean-spirited and capricious-- but very capitalistic-- pricing.) I think the same exists in software politics and the typical managerial conservatism. People and organizations can identify in name with liberalism or conservatism, but tend to pick and choose as suits them. (For an example of this inconsistency: Google, a bastion of software conservatism, allowed GCL to exist.) What makes 90 percent of software jobs so fucking miserable isn't those rock star, "undisciplined" Rails liberals or stodgy gray-haired conservatives. Rather, it's a corporatist "crony capitalism" state where people who win managerial blessing get liberalism (i.e. the autonomy to do whatever they want and freely impose complexity downstream) while the rest of the drones get stonewalled with a supposedly stiff-lipped conservatism (e.g. "you can't [fix that codebase | use that language | pursue that side project] because we can't afford the risk") that is presented as principled, although the drones see managerial favorites getting through that wall on a daily basis, so they aren't fooled.

What's interesting is that there has been a shift over the years to where "programming" has come ot have a strong ideological bias to only one type.

The types I noticed then I call Cowboys and Architects. These are just terms I'm using for convenience, not meant as pejoratives.

Cowboys are now more common: Some programmers write a bunch of sloppy code without bothering to ever design anything. Their methedology for making a product seems to be akin to bashing to the code into the shape it needs to be eventually. This group of people tend to advocate policies that assume everyone else is writing crappy code as well- unit tests, agile, etc. For instance, agile rejects design and assumes you can't know what the right shape for the code is going to be more than a week in advance and that all code is maleable without repercussions.

Architects are now rare: Other programmers will sit for a week and think without writing any code. When they do, they sit down and over the course of an hour (or however long it takes to type it) will write out the code for the complete system or module. It will be bug-free with the exception of typos. Once the typos are fixed (Generally by getting the compiler to accept the code) the programmer can move on to implementing the next bit of functionality. Sometimes there are errors in the code working with other code, generally integration errors, but not errors in the thinking of the programmer. These programmers only accept that Agile makes sense because the business side of things can't make up its mind, so they architect core systems that are flexible to support multiple business needs, but don't need to be rewritten or bashed around, because they did it right the first time.

I'm of the latter type, and I have tested this objectively, by producing a 10,000 line iOS App like this, which has been in use for several years by tens of thousands of people with no crashes or other defects (a few minor conceptual bugs- mismatch between the features and the expectations of the business, which were fixed)... and not a single unit test. It compiled, it worked, and almost all the development time was spent on the UI. It has had major releases (eg: going from being iPad only to a universal app, etc.)

It has built in reporting for exceptions, and all of the reported exceptions are the result of things other than my code (eg: there are several situations where iOS will crash an app if it needs to or due to problems with pre-release versions of iOS, and those are what generate the exceptions.) No customer reported bugs either. (though they do have requests for things working slightly differently and new features, no programming errors reported by customers.)

The industry is so dominated by a culture of "all code is crap" that I think many people think that all programmers are cowboys and even Steve here is delineating types of Cowboys, and nobody believes architects exist.

Can you imagine someone saying "unit tests are a waste of time, they just double the amount of work with no benefit.". It produces a litany of excuses for why this isn't true. ("You need them if other people work on your code!", "maybe for a team of one", "you're assuming you'll never forget a design decision", etc.) I know this message will get responses along those lines-- its because Cowboyism has become an ideology. Yegge is right - programmers are ideological.

I'm not a savant and I'm not rare. Architects like me were about %50 of the programmers out there when I started out. I think the mainstreaming of "hacking" has produced a lot of people who are taught to be cowboys and a culture that encourages cowboyism.