Why it's impossible to become a programming expert | Hacker News Reader

Why it's impossible to become a programming expert

blogs.techrepublic.com.com

32 pointsbaha_man36 comments

This definitely resonates with me. I got my first paying Java gig in 1996 (really). Back then, one person could be reasonably expected to know all of Java, and work on any project that was feasible to do in Java, whether that was applets, servers, fancy graphics, database access (no JDBC back then!), anything.

Fast forward a few years, I knew JDBC and Swing pretty well, a little of the EJB stuff, there were whole vast swathes of the Java universe that were simply out of scope. I got out of Java a few years ago, but I imagine it must be even more so now. With my current main languages, Python and OCaml, I still feel as if I've barely scratched the surface (and in the company I count as an "expert").

But this isn't limited to languages, back then a DBA could know all of Oracle 7.3.4, now Oracle 11g is too big for any one person to know it all. There's no such thing as a generalist DBA anymore. You know enough to operate anything your organization uses day to day or fix it if it breaks in the middle of the night, but a tricky problem outside your specialist area will need to wait 'til whoever in the company specializes in that can take a look the next day.

A programming expert is not a person who has memorized the contents of the programming equivalent of the Oxford Dictionary, but one that can write code the way Hemingway could write books.

The usual term for somebody who knows endless language details is 'language lawyer'. It's mentioned in _The Mythical Man-Month_ as a member of the "surgical team". Nowadays, the language lawyer role might include exhaustive knowledge of the dusty nooks and crannies of complex APIs.

I think I will end up reading this book someday.

In the words of Fred Brooks himself: The book is called "The Bible of Software Engineering" because everybody reads it, but nobody does anything about it!"

It's actually got some good ideas but I think you'll find it less of the guiding light that you might expect. It was written in the world of the waterfall method, batch processing computers, and systems development. It has some really interesting stories about OS/360, but I think you'll find more guidance in a modern book on Agile methods.

You've got half the picture, the art, the intuitive grasp of flow and balance.

You've ignored the other half, the mechanics. The mechanics are not about memorizing volumes of texts, Aspects of an expert include having been there and done that, learned from own or other mistakes. Knows the roadmap, overall rules of the domain. Consider a programmer as a medical practitioner (Doctor). What are the properties of an expert Doctor? Much of what I've briefly stated above: Lots of experience, knows what is most likely (not guaranteed) to succeed or fail given known conditions.

Now imagine you are an expert doctor and the human physiology radically changes every 2-3 years? The field of given known conditions also radically changes. How can you keep up on these changes and all their nuanced inter-relationships?

Most things in programming haven't changed and won't change. Yes, we programmers do float on a river of coding, passing through an ever-varying scenery that is the programming environment.

But the river itself and its flow remains the same.

There's a lot of fuzz all the time but, for example, in the last twenty years what has really changed? The playground has invented itself a few times and we're juggling primitives of a much higher level of abstraction as before. But there are always the set of primitives and, out of them, our job is to create the rest of it, the new things, for we are makers.

Doctors aren't makers of new but practitioners of what's known.

In the last twenty years we've gone from where microcomputers were generally isolated to our now highly connected world with social networking and botnets, all the benefits and security risks that entails. Programming languages in the mainstream have seen great changes in the past twenty years.

River flow does not stay the same. The rate of flow changes with the seasons and geographically, rivers can move over time, depending on local geological conditions.

Doctors are makers of new, otherwise where do our medical advances come from? They have to explore the unknown to make it known.

Yes, "primitives", our foundations in digital technology and programming fundamentals are the same. We still use collections and algorithms, and our computing technology is still rooted in the core theory of the Turing machine. This knowledge is very important, but it really is still just the foundation which we need to be effective, develop expertise. The tools we use today, the applications and libraries and languages do change rapidly. And that is where the interactions are complex and change all the time.

I would call that a hacker. An expert in the real world just knows a lot of stuff.

An expert is someone that can code without using google to teach him how to make every other function.

from http://www.zedshaw.com/essays/master_and_expert.html

> The main thing I noticed about the experts I’ve encountered is they are into impressing you with their abilities. They are usually incredibly good, but their need for recognition gets in the way of mastery. Everything they do is an attempt to prove themselves

An expert programmer is a knowledgeable, yet ostentatious programmer. :-)

You right an expert hacker! :)

"I find it quite frustrating at the personal level. I miss learning things in depth. I miss the sense of satisfaction from attaining a level of expertise. I miss getting to explore obtuse and obscure areas of knowledge."

How about getting to know your problem domain in depth instead? Gaining a sense of satisfaction from being an expert in the business and automating the work so knowledge of obtuse and obscure bits of data is no longer needed?

I'd rather be known for having built an amazing product that anyone can use than for mastering some obscure bit of the technology.

How about getting to know your problem domain in depth instead?

If I had wanted to be a biologist, or a businessman, or a banker, I would have become one of those things. My domain is computers, and computers are what I am interested in. I will take playing with pointers over $different_kind_of_work any day. I think it's just a personality thing, but that's the way it is for me. I would guess that a sizeable minority of programmers are like that.

The difference is that the thinking process has become more 'externalised'.

Ideas or structures are produced not so much by mental action, but by the mechanisms of external tools and representations. Instead of items being put together by knowing them well, they are connected by web linkage. We are being ever more subsumed into the community brain.

Interesting how he says he relies heavily on IDEs. If those IDEs weren't there as crutches how do you think those frameworks might be written differently?

I think the Rails super-over-loaded Do-What-I-Mean method would become more popular. Last night I tried to futz around with some of the form helpers, and almost every parameter is optional. It makes for some pretty challenging parameter munging at the beginning of the methods... but usually when coding there is a particular method you call, and you just kind of pile things on. (The plethora of form helpers notwithstanding)

I focus less on knowledge and more on skill esp. API design. This makes me a good programmer, I hope, although often an uninformed one. In an "open book" world I can't help feeling like I made the right choice. The _right_ skills are resistant to change.

Taking the time to learn things in-depth is what grad school is for.

Taking the time to learn things in-depth is what L I F E school is for.

That's a nice feel-good statement, but grad school gives you an environment where you can spend all day researching what you're interested in. Most people can't do that at their jobs or they'll get fired.

Yes, but you can also say that most people become experts at whatever their specific job is.

that sort of grad school is for people who need their hands held to learn things.

In some grad schools you're assigned projects, but at others you need to continually figure out even what problems need to be solved and which avenues of attack would likely yield the best results. This is much like what startups do, and in fact figuring out the important problems/attack strategies is much harder in my experience than performing the experiments themselves. I don't see how this is "hand holding", but perhaps the grad schools with assigned projects feel more like a normal job.

You'll have to elaborate on that. When I'm not on an internship, I'm supported by NSF grants to do my research. The support isn't much, but it's enough to pay rent, groceries and a few other things.

The general area of research I do is of interest to many companies in industry, but what I do specifically is usually not. Put another way, most companies would not pay me to do what I'm doing. Being a grad student gives me the freedom to spend 50-60 hours a week on research that I find interesting, but may not directly benefit anyone financially. This circumstance is unique to people in academia, some industry research labs, and people wealthy enough that they no longer need to work for a living.

Is it possible to create an equally powerful modern programming environment with reduced complexity (as measured in pages of documentation or whatever)?

If Apple, or someone else, were able to it would be a competitive advantage.

I'm not suggesting that Apple hasn't already done so, but I don't have the first hand experience to make the comparison myself.

OP is an expert, he just doesn't realize it.

If you define "expert" as someone who has memorized all possible syntax, then the only experts would be autistic and would need help getting dressed.

OTOH, if an expert is someone who knows how to find the right tool to get the right job done, then OP is clearly already there. As are many of us too, I suppose.

That's the difference between theory and practice.

When I studied C in College, I could tell you all functions on the ANSI library. More often than not, it meant that I knew if a library had something already done so we wouldn't need to do it again. Even simply things like using freopen to redirect standard output to a file, or using sscanf to parse an user input where outside of the class curriculum that I only got to know because I studied the language somewhat in depth. Now that I work with C#, nearly everytime I need a class that is somewhat uncommon, I am forced to do a research, and it usually DO exist in the framework (exception in mind, double indexed lists) somewhere. But I do not know it. Even the expert C# guys do not know it. We need to research. Because the framework is too big, it is not possible to know it. And because it is not possible to know it, every time I am coding, I feel like blind because of the possibility that of what I am doing is actually already there and I don't know it because I didn't think of the right words to research it for.

I wonder if we're any more productive - whether we spend more time doing research to find a library routine than it would take to just code it.

This is one problem in the OCaml community. Uptake of the new "standard" libraries is lukewarm because everyone has already half-written their own standard library.

We are more productive. It is not simply a matter of have/not have the routine. It is also a matter of usability. If you did the routine yourself, you would make it specific. When you needed it again, you would need to either write it again from scratch or modify it. The framework has its methods coded in ways to be the least specific possible so to avoid rewrite (in an ideal world, everything would be like it).

Does anyone think this would be an interesting problem for AI? I mean while you are coding have some software watch what you are doing and suggest standard libs that you might want to have a look at? It seems like standard information management techniques are already being applied to the documentation. While I'm coding in java I can get java doc for the class I'm working with to come up as mouse over. Maybe it could work like a recommendation engine for libraries.

A good first approximation already exists: Searching by function type in languages (such as Haskell[1]) that have sufficiently expressive type systems. There might be integration with e.g. the Emacs mode for Haskell* , and if not, it would certainly be feasible.

  [1]: Hoogle: http://www.haskell.org/hoogle/

* I haven't used Haskell for a while, though, so I don't remember if the Emacs mode currently supports searching by type.

And do you think that _just_ knowing every function that already exists and how to use it would you make an expert ???

A little more productive maybe, but this is indeed neither a necessary nor a sufficient condition to be called an expert.

Higher levels considerations are far more important.

To me, a lot of this is what the "design" process is for. You have a certain set of requirements that you need to fulfill, so before you set about coding it, you enumerate how it is possible to implement it. Now, I won't go into details like "Use a System.Web.UI.WebControls.Label here", but if it's something that I know my peers don't have much experience with (like apparently the entire System.Drawing namespace, dammit), then I'll specifically indicate in the design that that is an option. It makes for a much more streamlined process for development. No, it doesn't answer all of the questions that come up in coding, but it certainly slays the big dragons.

Reply on news.ycombinator.com