Coding Horror: Separating Programming Sheep from Non-Programming Goats
codinghorror.com
codinghorror.com
“The act of programming seems literally unteachable to a sizable subset of incoming computer science students”
There is, of course, an alternate explanation:
“We in Universities seem literally incapable of teaching programming to a sizable subset of incoming computer science students”
Experiments going back to the 1980s with teaching children how to program using Lego blocks and robots with Logo seem to indicate that nearly everyone can learn how to program, but possibly not when they’re 18 or 19 and possibly not with the kind of academic environment represented by the test.
Its more likely that people are being taught the wrong way, rather then the idea that lacking a natural aptitude for programming you don't have a chance.
http://www.bricklin.com/wontprogram.htm
It's not that Johnny can't program, it's just that he won't.
We teach children the basics skills needed for most studies well before that age. Reading, writing, math etc. You can't expect someone to successfully study literature if they don't know how to read.
Programming is a skill, not just a matter of knowledge, and the foundations for that skill must be laid at an earlier age. Most skills we learn after that are just extensions of those basic skills.
I'm not sure about the current generation, but in my age group (40+), everyone who programs started in their early teens.
What basic skills are the foundations for programming? Abstract thought? Deductive reasoning? Maybe someone is more inclined towards programming versus some other field, but I believe everyone has— at any age— the ability to learn how to program, much like everyone has the ability to learn a new spoken language at any age.
I only learned how to program at around age 20 or 21, but at that point I had already been a CS student for 2 years, and of course had failed most of the actual programming-related classes (even though I had earned a A+ in calculus, with congratulations from the professor). I am now a professional programmer, I'm in my early 30s, and as such I'll quickly write down my thoughts on this, maybe it helps someone make some sense of it.
Now, about the particular example in the article, I can still remember the first programming class during my first semester at Uni. I can still remember about how baffled I was when seeing almost the exact same code as in the above article, I was wondering to myself: "assignment works left to right, is the only natural way, what do you mean now "a has the value of b"? You first write down a, the 'equal' sign, and then b, it's only normal that b, coming at the end, takes the value of a". And so on. Someone also mentioned the "competitive" nature of a CS-class, which is 100% spot-on. I had never owned a computer during high-school, while some of the my then-colleagues had been programming for 10 years already, and because of that I was afraid to ask questions or even to ask for help from my colleagues.
And in case you're wondering how did I finally became a programmer, well, it was thanks to Python. At first I started writing some ugly scripts in PHP, but after 6 months I stumbled upon Python. I don't know exactly why, but it's something about this language that makes it very easy for non-programmers to grasp programming language concepts. After one year of doing Python I finally had the balls to try and take my most difficult programming class again. I got an A (the class was taught using the C programming language), with the professor surprised about how had I managed to do that.
I spent a year learning about website design shortly after turning 14 and during that time eventually came across PHP. For about another year until 10th grade started, I kept doing simple PHP sites. I had looked at Python once and felt kind of offended by it, and I didn't understand OOP. "Why do I have to do mystr.split(' ') instead of explode(' ', $mystr)?" "Why don't variables have sigils?" "What's this import crap?" "Why do I have to have indentation?" (I commonly didn't indent my PHP code during this time.) I also used Windows. I still hate cmd.exe.
I took the AP CS course in 10th grade, and Java forced me to at least partially understand OOP and why it may be useful or cool. (We also did everything on a Mandrake Linux computer.) I looked at Python again about midway through the year and started loving it (I was in the habit of indenting with one space at that point since the teacher required at least some indentation so Python's forced indentation didn't bug me anymore). After several months in Python I began to understand that real OOP went far beyond what Java supported. Now that I'm 21 Python is still my favorite language, but I do really like Clojure.
So I still think Python is a perfect beginner's language, but I'm not entirely convinced that Python as Python is enough to spark an interest to go do your own thing. The first thing I reached for when I learned Python well enough to do anything was video game programming. Making interactive applications that aren't limited to the terminal held my interest a lot more back then.
(My CS teacher also resolved many difficulties with the = sign by always reading code like "a = b" as "a gets b". Some people have trouble enough with homonyms in language, seeing the '=' mean something completely different to what they've been used to for 10 years can be startling.)
This is one of the great things about Pascal. The := sign is used for assignment, and the equals sign alone is used for equality. It's very natural to call := 'gets' and keep a clear distinction between assertions and assignments.
If we know that the problem is a lack of a consistent mental model, the solution is to learn how to teach adults to have a consistent mental model, not to throw up our hands as these researchers seem to do. To me this study suggests we aren't teaching the important skills necessary to program. We may need the equivalent of Drawing With The Right Side of the Brain for code.
After all, a 22% failure rate is still pretty pathetic.
...replications of the test have had issues; from http://crpit.com/confpapers/CRPITV78Bornat.pdf:
> "We now report that after six experiments, involving more than 500 students at six institutions in three countries, the predictive effect of our test has failed to live up to that early promise.”
And http://www.eis.mdx.ac.uk/research/PhDArea/saeed/SD_PPIG_2009...
> “A test was designed that apparently examined a student’s knowledge of assignment and sequence before a first course in programming but in fact was designed to capture their reasoning strategies. An experiment found two distinct populations of students: one could build and consistently apply a mental model of program execution; the other appeared either unable to build a model or to apply one consistently. The first group performed very much better in their end-of-course examination than the second in terms of success or failure. The test does not very accurately predict levels of performance, but by combining the result of six replications of the experiment, five in UK and one in Australia. We show that consistency does have a strong effect on success in early learning to program but background programming experience, on the other hand, has little or no effect.”
I see this at work. I've known programmers who, when they get a new hire, will basically let them flounder. I guess the basic idea is that if you're "good" you'll "figure it out" (with minimal time investment to boot). And that will work with some people. My own philosophy is that there are a significant number of people who need direction. Don't overwhelm them with "everything". Give them small, structured tasks to learn processes, small parts of the system, build confidence and give some measure of progress.
That all being said, I honestly don't know how I learnt to program at all. I can teach people who know how to program (the basics) more. But I can't teach someone who doesn't know how to program how to program.
I've come around to thinking that Zed Shaw (as just one example) is right about this. With his "Learn X the Hard Way" books he starts out by basically saying "just type this in" (and don't copy and paste). Don't worry what it does. You'll figure that out later.
I see people (in many different areas) who are under the mistaken impression they need to know "everything" before they can do anything (or even in games, make a decision of any kind). They feel the need to weigh up all the options and consider all the consequences. That's not actually how people learn (IMHO).
Children don't know any better so can just learn this way. Adults often let self-doubt, fear of embarrassment or fear of "what might happen" get in the way such that they won't allow themselves to learn.
So perhaps what universities should do is separate students who already know how to program from those that don't. Those that don't don't take ANY theory at all until they do an introductory course that starts with "just type this in". If you lack the ability (and willingness) to turn a thought into a program, no matter how simple, you're doomed to failure. This, I believe, is something that can be taught.
The ability to just try stuff and see what happens in a key element in continual learning (IMHO) and something that needs to be ingrained in programmers (to be) from day one.
I'd really love to see the outcome of this approach versus more traditional approaches.
I've found that education, at least K-12, is slowly shifting attitudes from 'Why are students so dumb?' to 'How can we teach this better?' Computer Science is notoriously bad at believing the former over the latter, and papers like this are demonstrations of this.
When "teach a bunch of freshmen to program" is your mission statement then it's perfectly understandable to prefer the students with solid fundamentals.
Already know how to program and have several jobs under your belt to prove it? Step right in, of course we'll accept you over the schmuck who wasted his high school drawing or playing music.
It angers me that what is primarily an educational institution insists on having their students pre-educated. You'd ideally want your students to have solid fundamentals, yes. But the reality is that most of them don't, and casting away those kids is the easy way out. I listened to a talk by a dean of one of the best technical schools in Turkey, and he said, "we spend your first year erasing all the junk put into your brain in highschool, and re-teach you the fundamentals". I think this is the better way.
I think it may not be so much a failure of education as so much the economics of education. The use of a single curriculum reduces complexity and administrative overhead (teacher time) which at the professor level is expensive. If the computer science courses were designed to use multiple methods you could indeed better educate a variety of students but at an increased cost.
In my mind the question then is whether or not it is worth the money to educate those that can't succeed with the current teaching method. The entire CS department would need to charge students more per class in order to have more successful students and in order to justify this cost there would need to be a corresponding beneficial pressure for the student to pay those costs (increased demand or higher pay for programmers).
I guess my point is that just because this group learns how to assign variables doesn't mean they'll be able to program recursively, understand how a compiler works, or be able to debug the stack and they will need just as much help in those classes as they did at the onset.
But the students who flunk out of CS or other STEM fields usually don't go away, they end up in the humanities instead. There, professors aren't getting a lot of money based on their research, and they usually have a surplus of grad students that they need to put to use somehow. And the college gets the same tuition either way.
Good thing professional educators are almost as observant as you in regard to teaching.
http://lmgtfy.com/?q=differentiated+instruction
I really love how HN conceit lets people post about entire industries as if those people just have no clue what they are doing. And not only do people here hold those pompous and generally ignorant positions, but other people support them!
Demonstrably true here in India as well.
True that I can't speak for all of India, or you can't speak for all of US; but I believe I have enough data points for the extrapolation to be valid.
I think that's enough to infer we have a problem, and it's largely ignored.
> cletus believes he is a more thoughtful educator than people who spend their careers on it.
I don't think it matters what cletus thinks. Conceited or not, I see his observations mostly hold in the real world.
I would say that it is demonstrably not true in many EU countries. Maybe "non personalisation" is the tendency of lazy teachers, but I remember pretty well how my math teachers repeated the same subject thrice: the first time describing its core in an abstract way, the second time with concrete examples (apples and pears) and the third time in very little incremental steps. The first explanation was for naturally able math learner, the second for those how needed practical examples and the third for less able pupils. Some of us complained about the fact that it was a boring way to teach (and you know how horrible bored kids can be in the class) but I'd say it was worth it to have all my classmates learning more or less at the same pace with nobody left behind.
Actually, I complain more about the fact that we had to change math teachers so many times because of educational reforms and budget cuts/reorganizations.
(This is not to say that I think everyone can do math equally well, but I've seen some otherwise intelligent people making such comments that made me want to rip my eyes out.)
Look at the percentage of programmers that are basically not any good at programming all and yet are able to continue being called programmers; if we apply standards of expectation uniformly, then you would expect to see even people in technical fields be able to be useless at math and have everyone basically just shrug and accept it.
That's how I read it too. And that is a finding of international studies of primary mathematics education. The newly industrialized countries of east Asia do much better in primary mathematics education, using textbooks (and teacher professional development to ensure that the textbooks are well applied in the classroom) such that key concepts are presented from multiple points of view, to reach as many learners as possible. The TIMSS test results
http://pirls.bc.edu/timss2007/PDF/T07_M_IR_Chapter1.pdf
suggest this is working out well for those countries. A book-length description of how mathematics primary education differs between much of the United States and much of China
http://www.amazon.com/Knowing-Teaching-Elementary-Mathematic...
is an eye-opening read.
Further, why be an ass and post a lmgtfy link when the op is clearly not aware of the term to google. You see the contradiction there I'm sure. Next time just post a link to the search results.
The fact the original poster fabricated his post out of naked overconfident ignorance is more than enough to warrant a snide reply. His post was not a critique on current educational theory or instruction but based on his experiences as a grade schooler that might be decades out of date, and seen through the eyes of a child.
Chances are your school implemented a similar style of teaching, except your teacher didn't come into a room of 3rd, 5th or 10th graders and say :
"Ok students, I spent 2 weeks this summer on my "vacation" training 8 hours a day and now I will be differentiating my instruction! I sure hope you don't all grow up to be overconfident ignorant asses who can't see beyond the direct and personalized instruction I give each one of you. I hope you realize that I might be explaining these concepts to different students in a way that suits them! Also, I hope you don't make fun of the kids who are in the low classes or need the help of an in class aide.
Now here are three ways to look at multiplication."
The best math course I ever took was geometry. The teacher never lectured - instead, he had written a worksheet of problems for us to do during every class period. Each worksheet would start out asking deceptively simple questions, which built upon other questions or challenges to prove something true. By the end of the worksheet, you had taught yourself a new concept in geometry (or proved a series of fundamental theorems) without really realizing what was happening. If you got stuck, you were encouraged to ask the students next to you, or the teacher himself, who would wander the classroom providing hints and alternate ways of approaching the problem.
There was a textbook, but we only touched it for homework problems (and occasionally as a reference).
The original article is saying based on a simple test you can throw half of the students directly into a 'learn to program' class and the other group into something else (I wish they had explored that a bit more). At no point is the student offered a choice. From their perspective the school only has one track.
Now, your school may indeed have been terrible... but I think the point that parfe was trying to make (ineloquently) is: don't presume that your one experience makes you an expert in the system. You probably weren't even aware of things going on behind your back in your own school, much less all of the other schools in the 'education system'.
This is one of those things like graphic design.. Everyone has an opinion; some people have training. Those who work in the field might come resent those who have opinions without knowing about the training.
1. If you spend three weeks on only the topic of assignment, any idiot (who has been admitted to university) can master that one concept to the level required to answer this stupid simple question correctly.
2. The fact that the students failed to exhibit mastery of even this one simple point at end the of three weeks means the class moved past this very fundamental concept too quickly.
3. It follows that this study doesn't really address the question of how students would do at programming if this fundamental mental model were taught.
Point 1 is the weakest, as it is justified only by common sense, but I'd put money on it.
>I've come around to thinking that Zed Shaw (as just one example) is right about this. With his "Learn X the Hard Way" books he starts out by basically saying "just type this in" (and don't copy and paste). Don't worry what it does. You'll figure that out later.
That's basically the 10,000hr approach - the earlier you start programming, even at Zed's most basic level, the earler you start rewiring your neurons to do it more efficiently and effectively. For the natural born programmers in the article's study, this isn't necessary, but for that whole other group aren't natural born, I do believe something like this is very necessary if any of them hope to become effective programmers. It's not impossible for them, as the study implies, but does require taking into account how the mind works (which we know now thanks to modern neuroscience), how it is malleable, and then designing an instructional course around that understanding.
Additionally, trying to educate the 'goats', the second group, at the same pace as the 'sheep' is going to fail. It simply takes more effort and more time for their minds to adjust, for things to 'click', for their neural pathways to rewire. Especially when that the thing that is getting rewired is their fundamental need to grok meaning amongst the relatively 'meaningless' rules that programming consists of. They're not a lost cause, but trying to educate them with the same curriculum and pace is.
Finally, the programming instruction should, as you imply, be bottom-up exploratory, not top-down memorize-apply-repeat. Part of the problem may also be that people with weaker working memories have a harder time with the academic approach to learning, even project-based ones. One of my favorite pieces of anecdotal evidence for this is from a story Paul Graham linked from 'Beating the Averages' [0].
So, given that it takes significantly more time and effort for the second group, it is all the more imperative that they start earlier, as early in life as possible. As soon as they can read and type, basically.
And in order to increase the amount of time per day that such students can spend practicing programming, it may also be worth delaying math education, or integrating it with programming education. Something's gotta give. No more pen and paper math, or even calculator math, if possible, rather learn how construct and solve math problems in code.
That may sound extreme, but given where the world is going [1], I think nothing should be off the table when it comes to teaching people, even (especially) the 'goats', to code. Even if they go on to non-programmer jobs, the ability to think both abstractly and concretely simultaneously, and the ability to handle meaninglessness, are both supremely valuable, no matter what you endeavor.
0. http://www.trollope.org/scheme.html
1. http://online.wsj.com/article/SB1000142405311190348090457651...
Similarly, when learning to be a Jazz soloist, students are encouraged to practice the great solos from Jazz legends of history over and over, by rote - just keep doing it. Eventually an aptitude develops and the student just gets to reach in and feel for why a particular set of notes might be chosen, or a certain ornament here, a pause there. Somehow, this can then develop into their own particular style and personality too, and evolve beyond mere emulation.
As musicians progress, they begin to develop higher and higher level abstractions for music; being able to use subtle nuance to relate complicated lines of melody, harmony and counterpoint is one example. Another practical example is how musicians over time develop better models for music itself. There's a natural progression; a novice will struggle to play even a familiar melody on their one instrument, an experienced musician will be able to play it with ease on more than one instrument - even if they have never played it before on that instrument, a master musician will be able to transpose the melody into a different key, rework it, add nuance ... and so on. At the beginning the novice can only model the ordered movement of fingers, but the master is modeling relative pitch changes, fluid timings, potential note substitutions and so on.
I find programming to be similar. With a range that goes from being able to type in a hello world program in a procedural language, to higher-orders of thinking required to model concurrent functional systems and have a see that a tiny defmacro might be the best way to re-work a whole piece. There is a progression, and it's intrinsically mysterious how the mental models work, but it's clear that they come with practice.
Through this lens, "Talent" might best be thought of as "An innate willfulness for practicing" rather than some genetically encoded aptitude for the task at hand. There are many communities where musical education is essentially mandatory, and near enough to anybody can become a good musician if they are required to practice. Some militaries arbitrarily assign soldiers to pipe regiments, treat it as a learnable task, and make them get on with it - and it works.
It may be well worth looking at how music is thought, and seeing if there are any dividends for the educating of programming.
I firmly believe that either you are born with the ability to be a programmer or you're not. That's not saying that it's good or bad, but I believe a lot of it is just how your brain is configured. It's as innate as the ability to shoot baskets, or throw a football, or as innate as gaining energy when interacting with people (extrovert) vs losing energy when interacting with people (introvert).
Case in point: there are some people who are extremely smart, but don't have the ability to visualize things like a map in their head. I know someone who loves cars, has been into cars since he was a teenager, and loves driving. However, for the life of him, he can't read a map. He's not dumb at all, he's very intelligent and excellent with computers. But he has no ability to visualize things in his head and spin them around. So when he's given instructions on where to go, he needs landmarks, ie. "Turn left at the McDonald's and turn right when you get to the 4th traffic light." He can't process "Go north on Main Street, and then head East along Central Ave."
I believe programming is the same way. There's a type of inner mental gymnastics that you need to do in order to program well, because a lot of the things are very cerebral. Some people just can't do it. I'm not saying that they can't be taught how to program to an adequate level, but the fact that it so goes against the mesh of how their thought process works, it just won't be enjoyable to them. They will never be better than adequate.
Of course, most of us here probably can't picture that, so many of you might think I'm wrong. But put it in another perspective. We can all be taught to throw a football perfectly. But does that mean we'll be any good at it? Probably not. If we don't have the physical ability to throw a football more than 10-15 yards (like me), will it be fun for us? Probably not. But if you were a football player, hanging out with a bunch of other football players, you wouldn't understand why everyone can't throw a football more than 40 yards.
The same goes for Sales. I've come to learn from my friends who are salesmen that the sales cycle is largely algorithmic. So it can be taught. But getting out in front of customers every day, chit-chatting, engaging them, etc, is something that many of us simply aren't built to do. It takes a certain personality type to actually enjoy this. And there are plenty of them that do enjoy this and make vast sums of money. But if I were put in that position, I would hate every second, and would likely get fired.
So yes, programming can be taught, but I truly believe that there is an innate mental configuration that determines whether or not you can digest and learn the information and whether you'll be good at it, and more importantly whether or not you'll actually enjoy it.
It is not a matter of being born with it. It isn't a matter of being smart.
It is all fucking work. It is never being satisfied. Its trying so hard and getting so pissed off that you throw the ball in the dumpster, that you smash your fist into the keyboard and walk away, that you tear up the papers and throw them in trash. But you always come back, because you won't let it beat you. You have to know how it works. You have to know how they do that thing you saw them do. You want to do it like they do. You want to do it better.
So it isn't pixie dust, it isn't innate ability, it isn't being smart. It is all fucking work.
No, it isn't. I knew a girl in high school who worked far harder than I ever did, and had far worse grades (particularly in math and science). She literally spent 2-3 hours every day doing homework and studying, while I spent 15-30 minutes. If you talked to her, she seemed perfectly normal, she just had tremendous difficulty with academic work. While hard work is important, natural gifts make a huge difference.
That is you didn't have a natural gift, she had a problem learning the way everyone else was?
You obviously didn't read what I wrote. The point is that people can always learn to be adequate programmers. But if the experience of programming is so different from the way they process information, that the act of programming is mentally uncomfortable for them, then they will never be great. The discomfort has NOTHING to do with hard work. It's like sales. You could spend your whole life being a decent salesperson but if you hate interacting with people you'll never be a great salesperson that closes multi-million dollar deals. It's as simple as that.
If you can't visualize a map, if your spatial abilities aren't strong, then having to interact daily where you need to do that would be frustrating as hell and you would hate it.
I took two computer-related courses throughout college because they were prerequisites, one being a CS101 using Pascal and another in my junior year doing assembly language on the 68000, which I did poorly in. I actually thought computers were dumb and was planning on going into electronics, but my senior project ended up being about assembly language programming on a DSP chip, and despite not knowing what I was doing, I actually enjoyed it. So that, coupled with the fact that programming jobs were easier to find than hardware, I spend 2-3 months after college teaching myself how to program, and I've been making my career out of it ever since.
For sure, someone may not discover programming until later in life and my anecdotal evidence can't possibly be all encompassing. But I do find if they haven't discovered programming on their own before taking their first CS course, that is very often a sign that they fall into the "cannot program" category that the article puts forward.
EDIT: even the fourth comment on Jeff's blog basically says what I said: "When I was in college, I remember the students in my classes who had trouble were ones who were entirely new to programming. The rest of us had either been in programming classes previously or had been coding for fun since we were kids."
Unlike every other discipline, programming is a completely different way of thinking from what we're exposed to in public schooling. It takes time for your brain to change the way it thinks.
It's not impossible to learn; I learned. It just took me more than one semester of being exposed to something for the very first time. The reason for the two humps are people who have been exposed to programming and those who have not; it's why it appears in intro classes but not in the later ones.
So you are basically saying there is a gene (or sequence of genes) that predisposes you to programming. Could you elaborate on how that might work?
Twin studies have shown a lot about personality being more genetic that was thought before. But I don't know of any brain studies that have linked knowledge/intellectual capacity to genetics... Do you have a reference?
Or is it literally as you say, just a belief? like "I firmly believe in the Flying Spaghetti Monster"
Whether that is genetic is unknown to me, but it strikes me as something that coders have to be good at.
Crystallized Intelligence
Fluid Intelligence
Quantitative Reasoning
Reading & Writing Ability
Short-Term Memory
Long-Term Storage and Retrieval
Visual Processing
Auditory Processing
Processing Speed
I know plenty of people who are "smart" who have a very hard time with that. In addition, I know plenty of people who aren't academically smart, but who are great visualizers. There are many different components to cognitive ability. I have a friend and both he and his father are exceedingly intelligent. My friend's son also is also a gifted child. However, when the son took some cognitive tests, his spacial/visualization scores were average, whereas the verbal and math were off the charts. I don't think that affects whether or not the kid is smart, it's just interesting how cognitive ability is broken up into different measurable areas.
Is it so hard to imagine that computer programming takes a certain subset of cognitive ability that some people, who are considered smart, just don't have? I know a lot of very intelligent people who wouldn't be able to program at all. It's not because they're "dumb", but as I've said, but the way they process information is completely different. You most definitely could develop the skills, however, if their brains are wired a certain way that causes them to be predisposed AGAINST programming, then they won't enjoy it very much at all.
- Some brain visualisation skills are innate
- Programming (probably?) requires some of those traits.
- JUMP
- "either you are born with the ability to be a programmer or you're not."
What you haven't demonstrated (beyond some weak anecdotes), is that those innate skills can't be taught.
Even if you proved that, you'd then have to prove that developing software, will always involve those skills. E.G. programming in the 60s required a different set of skills to programming now (with some overlap).
This sentence made me think. The image of the brilliant programmer who got his start at the age of 12 has always been, in my mind, proof that some people are born with programming talent and some aren't (not to say they can't develop it).
But what if, the only difference is that someone happened to start programming at a young age because he was interested it for some irrelevant reason, and because he was young, he learned it much more easily, in much the same way kids learn languages more easily than adults.
An interesting idea. In the real world, people's ability is probably a factor of both these things.
The example of "mental model of assignment" is ridiculous. a=b can mean whatever. It means something totally different to a mathematician than a programmer. Say:
"The piece of paper on the left says dog and the piece of paper on the right says cat. Overwrite the contents of the piece of paper on the left with the contents of the piece of paper on the right. What is written on the pieces of paper?"
I bet pretty much everyone would have a correct mental model of assignment if the problem was stated like that.
Try this experiment with five people at a public place, say exactly what you said, and give them exactly one try at the correct answer. I'll give $10 to charity for each one that getes it right. If all five of them get it wrong, you owe me an upvote.
(Fair warning, the terms of this bet are ridiculously unfair to you.)
My theory is that those who got it right (at least most of them) didn't get it because they "formed a consistent model". But simply because they made the same assumptions people before them did when creating the language. But of course, would need more data to be more sure of anything.
That's why the first group (the predicted to be successful group) is defined as forming a consistent mental model and not necessarily the correct one:
> 44% of students formed a consistent mental model of how assignment works (even if incorrect!)
I don't know what the notation means and there is no way for me to know what it means. Therefore I'm going to use a different interpretation in each question to minimize the variance in my grade.
> "The piece of paper on the left says dog and the piece of paper on the right says cat.
We are talking about university students. I think they all have had a bit of calculus. That they had to "find the x" or "give the value of y = 1/(3/x) to the third decimal when 1) x = 3, 2) x = 5".
If your 18-year old students need to be told of "dog" and "cat" on pieces of paper in order to understand the most basic abstraction that math has, well, you, as a teacher, will find big difficulties ahead.
Also I think it's impossible to hold a consistent mental model (whatever it is) if people can't form a logic behind the operation.
Let's get one thing straight: the problem is bad teachers. My mentor once said it kind of sucks that UTeach was only available to UT students and by extension those who statistically succeeded in traditional schooling because they are usually the least acquainted with the system's failures.
That "test" they show inside is bullshit, first of all. If they showed that to me as a completely green 11 year old without explaining the syntax or semantics of what was going on, I'd probably have to ask some clarifying questions. And, I think it goes without saying, if the professor is the type of teacher to go into a class with the expectation that half the class will fail, I'd probably have shaky confidence and would hesitate to ask questions since everyone is quickly cast into a mold of either "have" or "have not." I've taken math courses before by these people.
Many so-called "gifted" math students (and we're all students) are great with applied math but are just as stumped by theory as everyone else (anecdotal). Testosterone and typical youthful arrogance turns classrooms into settings hostile to people aren't extremely competitive and who are somewhat unsure of themselves. Combine this with a teacher who was once one of the cocky ones and a few snide remarks about how some students "just don't get it" and what you have is a self-fulfilling prophecy.
If your students fail to learn, it means you did not teach them adequately. Learning is the product of the work of learners: you have to give them the proper resources to work with. Humans are exceptionally capable. Children are extremely inquisitive and interested in so many mundane things; they can be taught math. This is beaten out of them as they get older and they're told there is a cap to what they can accomplish.
Bah. Humbug.
A good set of resources for anyone looking to teach programming and/or computer science is CS Unplugged (http://csunplugged.org/). No computers. Much of it relies on human interaction. It's fun, it's versatile, and most importantly it distills the important parts of computer science while shedding the stuff that gets in the way: confusing, advanced user interfaces; arcane terminology and confusing history; and the stigma that a computer brings to those who've never worked with them before.
For young students, simple activities like these (and other communication / language exercises like sentence diagramming) can have significant benefits later in their education. I don't think introductory CS education should be completely replaced by non-computer exercises, but it's no surprise that many HS students and college freshmen don't "get it" when the first thing they're exposed to is a pile of code.
People have been trying and failing for 40 years to improve CS teaching. Granted, most professors don't try and just teach the way it's always been taught, but some, like you, do.
The big question then becomes: is this an impossible problem, or simply a very hard one? When you start teaching, you will find that a certain percentage of your students fail to "get it". Will this be your fault? Probably not. At the very least, to "get it" requires effort, and some of your students won't put in that effort, and you can't force them to. There are many ways you can encourage the effort, but you can't force it.
What we can do, though, is shift the goalposts. Turn a 70% failure rate into a 30% one into a 10% one into a 2% one, et cetera. It looks like csunplugged is one of those tools that will shift those goalposts.
The fact that the goalposts can be moved is awesome and a testament to what can happen if we assume that it's a communication or motivation problem, not student ability.
There are lots of fringe students who just think oddly. Living in cooperative housing in Austin has really driven this home. They require odd teachers.
In the vast majority of schools in the U.S., however, it's almost impossible to fire teachers after two to three years of employment; in many schools, it's virtually impossible to fire them even before.
There was some mental disconnect that didn't allow them to grasp certain key features required for programming. No matter how you explain it to them, no matter how you tried to change the model to help them. I spent a lot of time with the professor I was TA'ing for trying to understand why, trying to understand what we were missing that could help these students understand. Stuff like scope, pointers, references were some of the difficult topics, which to me come almost naturally they felt it hard to wrap their brains around.
There were students that didn't get it but they could fake the knowledge required and could get the end result but not understand why that was the end result. They did okay on the programming questions, but when it came to knowledge of what was going on they couldn't explain it.
I felt terrible having to fail students, I felt even more terrible when I could see on their face that they had tried their absolute best. I don't think I could go into teaching unless I taught uper-level classes so that I didn't have to deal with the great disappointment and filtering of students. The look on a students face as he was sitting in my office during TA hours and he still wasn't able to grasp the concept of pointers made me feel really sad. I don't think I can deal with that again.
That said, I should have reasoned about pointers by simply handing them a map to a building and saying "this tells you how to get to the building. It's a pointer. The building actually has what you want. This cannot because it is paper."
Perhaps framing that "ceci n'est pas une pipe" picture on the class wall would help, too.
int a = 10;
int b = 20;
a = b;
int a = 10
a = b
b = a
x = x + 1
What we're seeing here is essentially a pun: the use of '=' for at least three different things (binding, mutation, and mathematical equality). To my knowledge, only Lisp gets this right (although there must be others). For example, in Scheme we would have
(let ((a 10)
(b 20)))
(set! a b)
(= a b)
I'm certainly prepared to believe the hypothesis that some people will just never 'get' programming, but we should consider the possibility that bad notation is part of the problem. Given how entrenched it is, changing notation might be unwise, but at the least we can acknowledge and explain ambiguous notation rather than treating it as self-explanatory.
Bill Wadge invented the dataflow language Lucid in the early 1970s by pondering this very paradox. He wrote a nice series of blog posts about it that begins at http://billwadge.wordpress.com/2011/03/23/lucid-the-origins.
His insight was that x = x + 1 isn't contradictory once you recognize that the two x's have implicit temporal subscripts. That is, "the value of x at time t+1 = the value of x at time t, plus 1". Make that temporal dimension explicit in the language and you can go back to having a nice equational semantics.
It's interesting, though, and perhaps germane to your point, that this insight turned out to lead far away from mainstream languages.
I've come to the conclusion that math and programming are very different things because math isn't executable. Programs have to run on a physical machine, math doesn't. The "physical" part changes everything. As someone pointed out in a blog post recently, even programs written in the purest most side-effect-free languages have side-effects when they run. I suspect that functional programming ultimately runs aground on this discrepancy - i.e. on the intrinsic imperativeness of the machine. The "x = x+1" paradox can be seen as a distillation of this imperativeness.
Perhaps we should teach programming not as math but as a kind of mechanics.
The concept that the "non programmers" could never understand was the abstraction of functions. Like:
foo(a) { // stuff }
foo(b);
would confuse them because they'd get hung up on foo's parameter declared as 'a', but called with 'b'.
def count_orange_slices(number_of_oranges):
return number_of_oranges * 8
size_of_my_orange_stash = 10
count_orange_slices(number_of_oranges=size_of_my_orange_stash)
>> 80I understand the impetus to filter out students that are less naturally inclined to it, and if your goal is to establish something of a programming conservatory, then administering a test like this would be an effective barrier. However, I think perhaps more advantageous reading of these results would be to recognize that the ability for students to construct these mental models is the single most important foundation in their education.
Anecdotally, I have had some experience teaching non-programmers basic programming skills at the university level, and it's absolutely the case that some kids, even bright ones, struggle greatly with these constructs at first. They look at the symbols and freeze, intimidated by the arcane expressions, and control flow is often non-intuitive to them. To me at least, that's a pedagogical challenge, not a lost cause. You should have seen the exclamations of wonder and excitement when some of them finally got it, once a particular way of illustrating it stuck for them.
http://norvig.com/experiment-design.html
and attempts by other researchers to look at the problem from other points of view.)
There are quite a few undergraduate disciplines in which it is assumed that most first-year undergraduates will not continue studying the discipline. In some disciplines that is considered a bug, but in some that is considered a feature. It's too bad that many undergraduates registering for a first-year course don't know that their professors don't expect most learners in the class to succeed. What would college enrollments in different subjects look like if there were more disclosure of that pessimistic attitude?
(Basis of knowledge: I have one son who is an adult now, the child of two nonprogrammer parents who is now a computer science major, who first learned programming through a CTD course in programming taught by T. J. Leone
(who now offers a different course, based on different materials) and the EPGY distance learning course in C programming
http://epgy.stanford.edu/courses/cs/
and some more or less haphazard local classes in programming arranged by our homeschooling support group, which includes parents who are programmers by occupation. He was advanced in mathematics before he began his programming studies.)
As the final exam approached the instructors asked his peers to go over the exam he had written up to which they replied something like, "you can't give this to your class, it's too hard and they will all fail and that will make you look bad." The instructor was worried, decided his students could hack it and gave them the exam despite of peer unease. Upon grading the exam, a higher than "normal" proportion (as measured from years previous) of the class left with higher grades and greater grasp of material.
Now hopefully someone knows what I'm talking about so I can re-read that account.
The fact almost everyone has same mental model now is down to very early age education. When your parents are literate you will be too.
It's possible this test could be done today in countries with terribly low rates of literacy
The question is whether this can be taught. When I read the study I see another possible conclusion; that we haven't sufficiently prepared everyone for the basic reality of computing -- that this is about arbitrary machine rules, and how to think about them.
Some of us may come into the world having that 'mechanistic' worldview, or perhaps some early experiences teach it to some earlier than others.
But we're evolved to deal with organic environments. Where precise definition and full knowledge aren't possible, so instead we have fuzzier categories and flexible responses. Where everything intelligent in the environment can be negotiated with, using empathy. In programming 101 you have to abandon that mindset.
Many others really, really floundered. They had no clue what they were doing. I finished the task in about a 1/3 of the time, others were working right up to the deadline and ended up getting 50% or so.
I'm not an excellent programmer. My skills might be above the average in mechanical engineering, but that really isn't saying much. I'd bet that double hump is less to do with some innate ability and more to do with exposure to programming at a younger age.
I don't think that the goats will ever go away, but I do think that if taught properly, that the ratio of people who understand to those who don't can be on the order of someone who understands an art history class to someone who doesn't-- or any other area of specialization for that matter.
I think the misconception some teachers have is that if a student is struggling, he or she is either incapable of learning the material or just not trying. The reality is that it could just be a reflection of how effective the teacher's methods are for that particular student.
I've been fairly skeptical of this paper ever since reading Alan Kay's rebuttal: http://www.secretgeek.net/camel_kay.asp
I trust Kay much more than these authors on matters of computer science, teaching, or ideas in general.
To expand a bit, I've since mentored over a dozen students and I've found that in most cases of students who couldn't program well, the issue was "further upstream." That is, there was some misunderstanding at a more fundamental level that was preventing them from continuing forward. Once that issue was fixed, they rapidly catch up to where they should be.
Also, assignment is rather strange concept relative to math education. I don't remember any test where we had to create an algorithm. Variables where things you solved for, not tools.
How are they to know that "=" is assignment and not testing equality? I guess you could glean that from the context where they immediately ask you the values of the variables after the operation but I could see someone misinterpreting the semantics of that operator.
Rather, the idea was to see whether or not, over the course of 20 or so questions, their answers indicated that, whatever they had decided '=' did, right or wrong, they applied their conclusion in a consistent manner.
Thus it was irrelevant whether or not they got the semantics of the operator right; what was important was determining whether or not they had operated as though operators in general have semantics, or whether they operated as though things behave essentially randomly.
It's more or less an attempt to determine whether the test-taker behaves as though the universe is a rational and understandable place or not.
a = 20 b = 20
a = 20 b = 10
a = 20 b = 0
a <- 10;
b <- 20;
a <- b;
The aggregate breaks down like this, loosely:
20% succeed, always. Hackers/nerds, basically.
80% might succeed. They don't have the talent. They succeed or fail on teacher's competence and their own hard work and background.
20% fail, always. Cheaters, people who are constitutionally incapable of getting it, misplaced students, people with health issues, etc.
The 80% is what a teacher can affect. If they don't have the thinking (math/logic) background or the curiosity, they probably won't do well.
Teaching is hard. It's probably the hardest task I've ever done.
Sorry, couldn't resist. :)
10/80/10 would be how I would break it down.
I guess it is reasonable that people have a hard time with this. They've grown up their whole lives dealing with people and animals and themselves. These act according to wants and gloss over minor errors, a fuzzy world full of fuzzy actors. Heck, even our species has lived and evolved mostly dealing with such. It doesn't make much sense to them that one typo could cause a super powerful computer or expensive device to keel over and be useless and do nothing.
int a = 10;
int b = 20;
a = b;
The new values of a and b are: False!
This is obviously a teaching failure.
I don't accept that there are certain things people "just don't get." Watch the documentary "Waiting for Superman," about some heroic teachers who didn't accept that either and did something about it.
Way back in the 1980s, when I was an undergrad, just switched majors from Journalism to Computer Science, I had a professor that tossed this very exercise out during class, my 1st class (which we quickly learned Fortran, then Pascal at a slower pace, titled "Intro to Programming and Algorithms I"). Except he named the variables BARF and BEER. And led everyone to believe that if BARF = 3 and BEER = 47 and then BARF = BEER, that their storage contents would be reversed. Nodding his head and asking if this was correct -- to which the entire class raised its hand except for the very few (this was 1983 mind you) who had some experience who sat smugly by, knowing the punchline already.
I instinctively also raised my hand, but had an inkling that something was askew, and in a snap, I got it, before he "played CPU" and scribbled an line-by-line execute process on the chalkboard. It might sound trite, but it was a thrilling note, to realize something logical that your brain short-circuits. Today, in languages, you can write something like "a, b = b, a", giving you that sugar sparing temporary variable and confirmed my choice to pursue a computer science degree.
That "aha" moment was repeated when learning about pointers, recursion, regular expressions, etc.…
But I wonder, the way my 1st teacher presented it, if the way he stoked and kindled the curiosity fueled and whetted an aptitude for programming. Not that it made me super-rock-star-programmer but it certainly led me on a quest that resulted in semi-successful (I mean I have made a living, but not built things that sold for millions and billions :)) career. Contrast that to advanced math classes I had where the professors were arrogant a-holes, seemingly solely interested in showcasing their intellectual chops.
This test is intended only for people with no previous programming experience. From the 2006 paper: "none had any previous contact with programming, and that all had enough school mathematics to make the equality sign familiar".
In retrospect, I don't know why I put up with all of that. I think it was 18 year old Greg's way of meeting girls...:)
I wonder if attitude towards programming is responsible for the distribution? Seems to me that people either love programming or loathe it - there isn't a whole lot of middle ground.
>There was an attempt to administer the test to another 30 or so students on a foundation (pre- degree) programming course at Middlesex University. That administration failed, because the students – rightly, in our opinion – were incensed at the conduct of their teaching and the ar- rangements for their study, and simply refused to do anything that wasn’t directly beneficial to themselves.
Uh... what? What is the "conduct of their teaching and arrangements for their study"?
That random bit of information is followed later by this:
>3.3 Third administration
Because of what we found on the first and second administrations, there was no third administration.
That doesn't follow at all. The first quote states the students refused to take the test, while the second states a final test would have been valueless. I very much doubt that it wouldn't have had value, as it could strengthen or weaken their claims, so I'm forced to conclude the students refused to take an exam? Something doesn't seem right in this classroom.
Ultimately though, without any information about how things were taught, I'm not sure this study tells us much. With a lot of studies across a variety of teaching styles we might be able to extrapolate something, but a single one just throws doubt on the methods of teaching which is a massive un-controlled and un-known variable to the whole system.
After years and years of algebra teaching me that variables were inherently immutable, I automatically assumed that was a universal law that was never broken.
Then again that proves that I at least thought there was a system.
Students with no experience programming may be better off starting with a language like Haskell.
http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper3.pdf
More:
1. Students for whom it came naturally 2. Students who could be competent if they worked really hard 3. Students who, no matter how hard they tried, weren't going to learn to speak Japanese. Wasn't gonna happen.
Bad teaching methods might result in some #2 types being mischaracterized as hopeless #3's, but I think no amount of pedagogical innovation can overcome the fact that some people just can't do certain things. (Of course, it could be a function of age and brain wiring, since all citizens of Japan, even the stupid ones, have no trouble learning Japanese.)
This is just retarded. Tell me the rules first so I can infer on right premises.
This test merely indicates that teachers, by tossing out untrained minds and using a subset of already trained minds, can skip the challenging part of their job and take credit where none is really due.
Grrr.
I think that many more people can learn how to program, and become good programmers at that. It's true now that the majority (at least 80%) of programmers are incompetent, but I think that has more to do with environment, tooling, economics, and the short half-life (5-6 years) of a software developer, than intrinsic limitations on ability. I wrote about this here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... . The tools and freedom that a developer needs to become good (1.5+) are simply not available in average software jobs.
I think the big problem is that we teach programming using languages like Java, which are actually very complicated. There's a lot of information in "public static void main(String[] args)" that just seems like line noise to a novice, and that people shouldn't even be exposed to until they develop the taste to know when to use OOP and when not (and that takes years, IMO). In teaching, we should start with simple languages like Scheme so we aren't requiring people to lex and parse and generate .class files in their heads before they even know what those terms mean.
Scheme and Python are better starting languages, in my opinion, and we should be starting programming education around age 6. We can (and should) expose growing programmers to C, Scala, Clojure, and Haskell later.
Also, starting with complex languages and tools (Java-style) means we have to teach in an inverted way. In your first Java exercise, you don't write a program, you write a class. Why? No good reason. There's no intrinsic reason a program has to be a class. This sort of botched, improperly-coupled teaching produces a generation of programmers who think every program must be a class, and who produce shitty AbstractVisitorHandlerFactory programs as a result.
This is one thing that annoys me especially in the teaching of Scala-- one of the most exciting languages to come out of the past 10 years, but one that is poorly understood and generally taught quite badly. What angers me especially (I'm writing a Scala tutorial to fix some of these mistakes, probably making others in the process) is when case classes are taught after classes. For example, the O'Reilly book (which is generally quite good) covers traits (a very powerful and highly nuanced advanced OOP feature) in Chapter 4 and case classes in Chapter 6. Case classes, for the uninitiated, are immutable records (data objects) and provide a lot of niceties (automatic .equals and .hashCode, class name as constructor) for free. Since case classes are simpler than full-featured classes, they should be taught first.
The right way to teach programming is to teach the simple stuff (immutable records, referentially-transparent functions, mutable state in simple data structures) first, and then provide some insight into when and why you might want to use the more complicated stuff (OOP, type classes in Haskell, functors in OCaml).
This is the reason why people who start in functional programming usually become top-5% programmers within ~3 years and people who drink the OOP Kool-Aid almost never do. Mutable state isn't evil, and good "functional" programmers use it all the time, but if you don't start with the right (simple) default abstractions-- immutable records and referentially-transparent functions-- you are not learning how to write clean, simple, maintainable code. Not only is this an important engineering skill on its own, but it's also important for the purpose of growth, because reading good code (one's own and others') is a great way to improve as a programmer.
IF I HAD NEVER HAD ANY EXPOSURE TO PROGRAMMING.
var a = 10, b = 20; // Declare variable values
a = b; // Assign b to a
console.log(a); // Output a to the console
console.log(b); // Output b to the console
// Log: 20
// Log: 20