The demise of the low level programmer
altdevblogaday.com
altdevblogaday.com
I don't want to praise ignorance - probably I will read some of the stuff he linked under the article - I just think it's ok that people can do something useful with computer not knowing all the details about it. It's worth knowing though - but just as you can have no idea about physics of sound and how piano works and write nice songs, you can build nice stuff with high level tools not knowing low level.
Did, and are doing, and, optimistically, will continue to do. The job is not done, in fact, it will never be over.
I think that what is happening is that the low-level programmers are becoming outnumbered by you high-level programmers. That's OK, and, in fact, that's how it should be.
This is not inherent, as abstractions could have fully specified the performance characteristics of their primitives. However, most don't, so to get great performance out of your tools, you typically have to see directly through the abstraction layers.
When you write a quicksort algorithm, it is pretty important that you know about cache lines and the relative costs, or your quicksort will not be nearly as good as the one that does take that into consideration.
http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...
I agree with you on most counts, but when you traverse your 2d array one way versus the other and are left scratching your head as to why one of them is slower, it's usually because something is going on in there that you don't know about, that the abstraction didn't make clear.
We need those low level programmers. We also need high level programmers to know a little about this stuff. However, we don't need everyone to be a low level programmer, and that's why the current state is an improvement over the past.
That's a huge step from knowing the number of instructions that fit in a trace cache of a CPU or when loop unrolling becomes optimal. Cache theory and understanding how IEE 754 are basics and are fairly universal. It's completely different from programming a fast subroutine in assembly optimized for a specific processor generation.
But I am not sure where a lot of things he notes were ever "taught" in universities or formally to those of us who were in the trenches. Much of the hard-core low-level stuff was learned either on your own, or when you got into an environment where it was necessary to know.
There are some additional skills that are falling by the wayside as well. One is how to optimize instructions on a type 650 rotating drum computer. Or more realistically, how to program co-routines in an interrupt-driven environment. Or how to write a floating-point division routine if you are working on a 286 target that does not have a 287. Or what order to write items to a disk and where to place them to minimize arm movement in a real-time data-driven system.
I admit that some of these are actually less necessary now due to technology being replaced, or machines being ridiculously faster.
But I was around when those technologies were necessary and useful (well, not the 650, but almost). We built a real-time system in assembler that had to be seriously well-optomized, and we had to modify the operating system to remove hazardous a latency that was fogging our 2ms data sampling rate.
But you know, simultaneously with that were COBOL programmers who had to be schooled in the idea that writing backups to tape was a good idea.
So there always has been a gap--this article suggests that this is something new.
Having said all that, this is an excellent article, as it is a challenge to programmers who would be better, and has a forest of valuable links to learn more. (This site now bookmarked.)
* Lots of college courses, especially of the Java school variety simply do not cover anything remotely near low level code.
* For people looking to learn themselves, there are far fewer books and websites with resources than for other subjects. If I walk into my local book shop, there are books on Java, C++, C#, PHP (lots of PHP), Flash and Python. Then a few language agnostic books like Code Complete and Design Patterns (Stuff like this is lacking in quantity, but at least it's there). None on anything low level. There isn't even a x86 assembly for dummies type book there. The internet has free books for other languages too (Dive Into Python, Why's Poignant Guide to Ruby, etc..) and plenty of free resources on good functional programming or good OOP. For assembly, all I keep running into is a wikibook, which I'm rather skeptical about as they tend to pretty poor in my experience.
And then, once you've gone through the initial learning phase, how are you going to get any experience? For 95%+ of projects, it's a case of YAGNI. There aren't any interesting open source projects coded in assembly today. Maybe a few Linux drivers, but that's something that would need a lot of domain knowledge on top of low level knowledge to understand.
I don't think there's a better - or easier - way.
Also, doing what you suggest on a number of combinations of hardware would be useful, so you can compare various processor architectures.
That's probably true for CS courses, but a lot of courses in my computer engineering undergrad years touched low-level stuff. The most relevant are the real-time systems, microprocessor architecture, and configurable processors courses. We had to unroll loops, reorder instructions according to data dependencies, and otherwise optimize assembly routines on paper during exams (based on a description of the architecture and the number of cycles for each instruction). Having learned that it's rather easy to pick up a book on whatever processor interests you and start writing.
I thought, or maybe you don't grasp that one need not understand it first, or in many cases, at all.
If I had to write my entire operating system from scratch everytime I wanted to get something done I'd find a different profession.
However, I don't think that's the point being reached. Somewhere out there exists a magical corpus of knowledge that every programmer should be able to memorize from heart. And then there's the world we live in now where you have a corpus of knowledge you do know and can build upon. Somewhere in between is a practical subset of knowledge that should be common but some feel isn't being represented well. The debate rages about what that suitable subset is.
What I think most people who bring this argument up fail to realize is that the full corpus of knowledge that embodies all of programming is far too large for a single programmer to understand. Good abstractions can be trusted to hide the unnecessary details. Great abstractions shouldn't mean piss-poor performance (and should infact provide just the opposite). They also get out of your way (or you avoid using them) when working in the problem domain where your "low-level" knowledge is more useful for the optimizations you can provide to your code.
I'm just not as OCD as some programmers. Yet I can still get good work done. I don't think I could do it without useful abstractions.
I think many low-level optimization tricks each generation of programmers need to find for themselves, the technology changes so much that old tricks are not as useful anymore.
Memory wasn't as slow (compared to processors) as they are today etc. Many people still count FLOPS, while memory accesses is probably the most relevant metric today, especially on GPUs.
I see people lamenting that their favorite low-level programming techniques aren't being taught these days.
That's because the field has expanded so much! Entry-level programmers learn entry-level things. If they're good programmers, they're going to eventually teach these things to themselves... If they need them. Chances are they won't need them, though.
There are still plenty of programmers doing low-level work, and there's more all the time. There's just so many more high-level programmers that you don't notice them.
I mean, yes, web & desktop coders don't really need to know about the details, but there are people who write operating systems, compilers, microcontrollers... Those are very low-level areas.
There are phone app programmers, and there are phone programmers. If the phone programmers wasted the power in that processor on the core APIs, the energy and expense would be wasted.
http://www.amazon.com/Hackers-Delight-Henry-S-Warren/dp/0201...
http://www.amazon.com/Programming-Pearls-2nd-Jon-Bentley/dp/...
And I also feel that that guy in the article should have grey hair. It would give more weight to his argument.
My take on it is that "low-level stuff" isn't going away any time soon, and that there will always be a equilibrium reached between those who understand it and are fluent with it (and, sometimes, are going to be able to write code that is 5-100x faster) and those who don't.
It's orthogonal to an understanding of algorithms. No amount of bit-bashing can fix really poor choices of algorithms (N^2 vs NlogN, say, on a big input). That being said, there are a lot of tasks where everyone is going to land on the same linear or logN basic algorithm and the main difference is going to be the algorithm that misses cache frequently and is stuffed with branch mispredicts and pipeline stalls vs. some algorithm that avoids all these problems and runs 10x faster. Sometimes you've just got to go do something to every bit of data and there's no classic algorithmic trick.
For example, I helped a former academic colleague write the 'worlds fastest floating point minimum' routine using SSE and lots of unrolling/software pipelining and I think he got about 8-10x (he had a library for most of the common operations like vector add, mul, etc but it didn't do 'min'). No amount of rampant algorithmic cleverness would avoid the need to look at each data element once when trying to calculate the min of a vector and if you're doing WORSE than linear, you've got real problems.
The major point that I've discovered from doing this stuff for years (on considerably more complex cases than the example above) is that programming efficiently for a modern architecture is qualitatively different than designing algorithms for the abstract '1 operation counts as 1 operation' machine in your average algorithms textbook. Oddly, quite a bit of improvement in my scalar, non-parallel programming came about after having used CUDA fairly intensively - if you're writing code for a Core 2 Duo or beyond, you're already parallel programming even if you're designing code that's single-threaded. Understanding how to rethink your algorithm to have data-parallelism (not to mention using SSE) is just plain conceptually different and more akin to parallel programming than scalar programming.
Knowing when to do this is important; I like bashing out a quick Python script as much as the next guy, or perhaps some pretty random C++ STL code that's probably an order of magnitude from where it should be (not because the STL is bad but because, say, I've been bone-lazy with design). So all those 'I have 10 layers of abstraction above this level" nincompoops shouldn't get too smug; we (low-level guys) can go there too - just because we know how to optimize C/asm loops to within an inch of their lives doesn't mean that we're going to compulsively do it with every last one.
Also worthy of note - the right thing to do changes frequently; some of the resources (especially on branch prediction) listed are already out of date. Don't bring P4 knowledge to a Sandy Bridge fight. Some bit twiddling hacks are great, others (especially ones that assume multiply is ultra-expensive) are obsolete on recent x86. The concepts still keep their validity a lot more than, say, all that newfangled crud that you youngsters fill your heads with (LAMP stacks, etc.) :-)
If you're looking for game development work, I can see how being stuck in St. Louis must be tough. Good luck!
Out of curiosity, what's different about RAD's hiring?
Its also worth noting that sometimes an algorithm with higher O() complexity could actually perform better. Eg, a brute-force linear search may be faster than a binary search if the elements can be efficiently cache prefetched, or if the entire dataset fits into cache.
I think this is a great point. Low-level programming has always required an in-depth understanding of the underlying hardware and that hardware is changing as fast as ever today. It may be true that a large percentage of development can live happily in a land of high-level abstractions, but there will always be a need for low-level optimization and that space is still rapidly changing in very interesting ways.
We have bit-twiddling hacks because we have machines that do bit-twiddling. If we had had different kind of programming environment, these low-level hackers would've been creating different tricks of trade. Nobody probably ever taught them anywhere formally, except for programmer peers to each other.
It's not the low-level stuff per se, it's the people who go for the low level no matter what stuff there is because they have to do it to get the job done.
An exception might be if the game's core gameplay mechanic is fundamentally based around doing some trick on a console or handheld device that can't be done without careful optimization.
When I got my first job I had to deal with BASIC in ROM and 6502 assembly. Aztec C on the Apple II was very slow and it wasn't possible to tap the faster routines in ROM because it messed up page zero completely.
Still, I can't complain. I loved the 6502 and the routines built into the Apple II+ ROM were incredibly efficient.
1980: Being too slow to run or not being able to an amount of memory we'd consider tiny is what kills projects. Large programs are relatively uncommon because the default languages (C and assembler) are so low-level that writing them is pretty much impossible. Programs exist to solve defined problems, not to be million-line ecosystems. In this world, being a decent programmer means you have to know about things like the performance trade-offs of pre-increment vs. post-increment.
2011: It's rare that a program, unless it's using O(n^2) sorting algorithms on large sets, is actually too slow to be useful. What kills software projects is illegible code. Paying a performance penalty for readability is generally a good idea, and highly-optimized but illegible code has fallen out of favor. This also means that fewer people are learning how to write such code, because people encounter less of it to read.
Do you have any supporting references? Legible code is a minimum requirement.
It is like saying "code which dies every full moon is killing software projects." Sure, such code would kill most software projects -- but it is common today?
(My educated guess would be that slow development speed is the real mass murderer of projects.)
Technical debt is something that sinks projects, but it's by no means the only thing. This keeps engineers up at night, not project managers. A project manager worth his salt is worried about selling the wrong thing. Plenty of systems are messy, impossible to update, and wildly successful. I've worked on some of them. But few - if any - systems get away with solving a problem nobody has.
You hint that the goalposts have shifted for engineers, and that's a good point. Moore's Law gave us the room for a much stronger culture that worries about abstractions and meta-things like maintainability and cleanliness. But more importantly, technical restrictions have been lifted that previously prevented users from getting they software they need.
Our abstractions are still leaky, but they're getting better. Soon enough they'll be good enough that you mostly don't have to think about what's happening in the physical box at all— just like you shouldn't have to think about different kinds of wood when you're laying out a housing development.
So since my metaphor was obviously unclear, the point I'm getting at is why would I want to be "punching tape" (writing assembly or C instructions) for a "Turing machine" (a physical computer with unlimited resources that executes one — okay, two — instructions at a time) when I could instead tell a robot (a high-level programming language) what I want it to do, and let the robot figure out the individual instructions to make that happen?
And just for future reference, "You know that #{thing I assume you don't know}, right?" is a really douchey way to pretend to try to educate somebody.