Stanford study shows success of different programming styles in CS class
ed.stanford.edu
ed.stanford.edu
I have the logic down quite well but my biggest issue is the syntax - it would appear that Stanford thinks I'm doomed to fail.
Anyway, the fact remains that you need much fewer syntax lessons to get started with Scheme compared to anything else.
I'm curious about that too; is this course optional? Would people who can already program take this course?
A large majority of my CS friends in that class had all been programming for either a little bit or quite some time, and the class was mostly a joke for them. Lectures were optional, and a bunch of friends (including myself) only attended three classes the entire semester: the first class (where we found out we didn't need to go to class except for tests) and the mid term and final.
Of course on the other side of things, I had friends who had never programmed before and found the course grueling and struggled to pass.
There are three "intro" programming classes at Stanford. CS106A is "intro to java", CS106B is "intro to C++", CS106X is CS106A + CS106B + a bit more in one quarter. People who already know how to program are encouraged to take one of the latter two, although there is always a contingent of people with /some/ background in 106A.
Or is it just the case that these people just wanna take one course from the "real" CS for a non-coding reason?
Given that the class has been successful at teaching programming methodology to tens of thousands of beginner students over the past decade there's no pressing need to restructure the entire course, even though everyone does acknowledge that Python or some other language could be better for new students. It's basically like any other large software project, yeah it could be rewritten from scratch and modernized to make small gains, but if it ain't broke, why fix it, especially when the professors and TAs have other projects that have far more interest in, like this paper.
Also one thing that many people don't know is that CS106A actually changes the language environment as well as providing a modified version of Eclipse to remove some of the less useful overhead in Java (for example removing main, and the beginner confusion surrounding keyword static, arrays and arguments). Also there's a much simpler introductory class called CS105/101 that uses Python.
Source: I taught sections and managed the TAs for the introductory classes over the course of four years while I was a student at Stanford.
Not as "juicy" as the website - but then again, it's the actual results :)
Many people are making great points - there's a wealth of uncontrolled variables (ie some students already know how to program), but it was a fascinating dataset and I could even see the paradigms play out over the quarter during discussions with my students, both experienced programmers and students getting their first exposure to programming.
I would offer as a larger takeaway that work like this has the potential to vastly increase our understanding of learning and development with the rise of MOOCs, ipads-in-every-classroom, and of course IDEs that save every incremental compilation :-)
Oh and as another mention - working with Marcelo and Paulo was awesome, both are super passionate, intelligent dudes.
“Educational data mining should not be used to reinforce the existing ineffective forms of assessment, but to reimagine it completely,” Blikstein said. “Pre- and post-assessment is a black-box. We do not know much about what is happening in between.
“Ultimately, this work may help us better understand some forms of human cognition because you can see what people are doing and how they are thinking in real time.”
Not being flippant: The conclusion here seems to be evidence of an ability to learn something new, regardless of what, is strongly indicative of performance.
> "The discovery of these “sink states,” and how students
> got into them, offers opportunities beyond predicting
> grades: They open the door for developing systems to
> encourage students to go down more fruitful paths before
> they become lost in the programming weeds."
Very interesting, and intuitively right to me. The difference is likely in the ability to create and maintain a consistent mental model of the program's structure and behavior during the implementation.
The other interesting approach I read about a while back was using tracing quiz data to identify "blind spots". A typical example was identifying students that had problems understanding loop constructs (they would always assume one iteration for example).
I talked to one of the authors of this paper and he had some interesting ideas fro CS education in general: http://dl.acm.org/citation.cfm?id=2526978
My gut instinct and thinking back to my university days make me think that a blind spot for recursion may very well be a thing. I'd be pretty interested in identifying students that struggle with loops and/or recursion and investigating that further. For example...is there a correlation? If not what happens if you give a loop-struggler group only recursive tasks and a recursion-struggler group only loop tasks? Etc.
My knee-jerk reaction would be "No! Everyone learns and thinks differently"-- but being able to put programming styles into buckets and correlate final grades from it suggests otherwise, doesn't it?
All that leaving aside that it's a social science we're talking so... It's not exactly science as in scientific method. It's just models that (under X circumstances) might or might not apply... Go figure ;-)
I learned in a vacuum, having nobody around with any software experience. I was given a PC and then saved up for an Amiga 500 (and later a $500 Lattice C compiler). My only link to software was through the bookstore and magazines.
Being a nerd, my main motivation at the time (of course) was to make a D&D game. I remember my first code was in Basic (no subroutines!) so I eventually learned to create subroutines based on line numbers (100-900 was main, 1000 was map, 1100 was character A, etc)
I managed to make about 50% of a single game before I ran out of memory. So I then set out to shrink (optimize) what I had made. After reading an article on dungeon map creation, I figured out that large arrays could replace the hand-coded drawing I had written. Check.
Ok, so now I start doing everything in arrays and pretty soon my code is smaller but "fixing stuff" (maintaining code) was stupidly hard. Oh okay, so I learned then some that data and code should be together rather than global (encapsulation).
I then needed to create random maps. After struggling with that, I read a paper on fractals, and implemented my first algorithm.
You can probably guess by now where I'm going with all this... I had crudely recreated many things by knocking up against hardware and brain limitations and figuring out how to work around them.
I progressed into Pascal, C, C++, assembly and eventually created my own assembler and graphical operating system at 16 years old. It was not a thing of beauty, but it did work. (Time was free, and compilers weren't cheap.)
I think that most people learn programming by failing, hitting traps, finding limits, crashing, burning and trying again. The only real identifier of outcomes, in the long term, is the willingness to continue trying and learning from mistakes.
So from my point of view, the outcomes of the paper are arbitrary in the long term. I would definitely had been a "Gamma" programmer. I hit every problem, and got terribly stuck. It took years for me to progress, learn good practices and become a proficient developer.
Don't Get Stuck!
Now how does one avoid getting stuck while programming? I still get stuck a lot, but these really helped:
- Test what you write as quickly as you can.
- Hold it in your head! Or you won't have a clue what all your code, together, is doing.
- To get started, ask yourself, what is the simplest thing that could possibly work?
I'm not sure I would go for what appear to be the simplest now ; I try to find the implementation which will have the simplest design to use/modify/delete in the future.
Which may just be correlated with high performance, not necessarily the cause.
If you make "don't get stuck" a motto, you can easily imagine people skipping over difficult problems and not thinking about a good solution.
1. Write down the problem. 2. Think very hard. 3. Write down the answer.
Rich Hickey's Hammock Driven Development, https://www.youtube.com/watch?v=f84n5oFoZBc, is IMO very good for actually not getting stuck in a bad place problem solving.
I guess this is the fourth tip?
- Step away from the computer
It is also important to define, what "getting stuck" means. For the research the alpha performers where the ones who fulfilled the goal quickest. If you switch the point of view from the teachers to the learning student, instead of fulfilling the requested goal quickest, you might follow an interesting problem that you discovered on the way. Is that getting stuck? No! It is following your own path of learning which is a very effective method. It does not look best to the teacher, but it can be very good for the learning student.
Asking your own questions and answering them is an important part of a learning process. It also makes you independent, so you don't simply do most efficiently what someone told you to do.
* Test early, test often test automatically [0]
* Remember the big picture [0]
* The minimum viable product [1]
[0] - https://pragprog.com/the-pragmatic-programmer/extracts/tips
[1] - http://practicetrumpstheory.com/minimum-viable-product/- think back. Students who change style do better. This implies that those students are reflecting on their own style and trying out techniques (performing experiments) to find out what works best for them.
- think ahead. Use lookahead to avoid backtracking. Some amount of top-down planning will make your life easier and avoid traps, even if your natural style is bottom-up.
Until you run into that one problem where you can't test between the small steps, because you need the whole thing to be up (to some degree) and working for any test to work.
Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails.
Another example would be refactoring a complex library into something more manageable. If you keep working in small, testable steps you can move only along the gradient, bordered by "can execute" and "doesn't execute". So you'll be able to reach only a local extremum. Now if you're in the fog and don't know where to go, that's fine. And for most development this is exactly how it happens. But sometimes you can see that summit on the other side of that rift and you know you have to take a leap to get over there.
I spent the past 3 weeks doing exactly that, refactoring a code base. I knew exactly where I wanted to go, but eventually it meant working for about a week on code without being able to compile, not even think of testing it, because everything was moving around and getting reorganized. However now I'm enjoying the fruits of that week; a much cleaner codebase, easier to work with and I even managed to eliminate some Voodoo code nobody knew why it was there, except that it made things work and things broke if you touched it.
> - Hold it in your head! Or you won't have a clue what all your code, together, is doing.
Or, sometimes it's important to get it out of your head, take a week or two off and look at it again with a fresh mind and from a different angle. Often problems seem only hard because you're approaching them from that one angle and you're so stuck with wanting to get it done, that you don't see the better way.
Instead you should write code in a way that it's easy to get back into it.
> - To get started, ask yourself, what is the simplest thing that could possibly work?
And then wonder: What would it take to make this simple thing break. Make things as simple as necessary but not simpler.
How do you test kernel? Run a virtualised kernel? I think at least DragonflyBSD supports that.
Today you can also exploit the busmaster DMA capabilities of IEEE1394 to peek into system memory.
However the still most widely used method is pushing debug messaged directly into a UART output register, bypassing any driver infrastructure, and hooking up a serial terminal on the other end.
So start with something simpler. Start by making a kernel that can run /bin/true, that never reclaims memory, that only boots on whichever VM you're using for testing. You absolutely can start with a kernel that's simple enough to write in a week, maybe even a day or hour, and work up from there. See http://www.hokstad.com/compiler for a good example of doing something in small pieces that you might think had to be written all at once before you could test any of it.
> I spent the past 3 weeks doing exactly that, refactoring a code base. I knew exactly where I wanted to go, but eventually it meant working for about a week on code without being able to compile, not even think of testing it, because everything was moving around and getting reorganized. However now I'm enjoying the fruits of that week; a much cleaner codebase, easier to work with and I even managed to eliminate some Voodoo code nobody knew why it was there, except that it made things work and things broke if you touched it.
Which is great until you put it back together and it doesn't work. Then what do you do? I've watched literally this happen at a previous job, and been called in to help fix it. It was a painful and terrifying experience that I never want to go through again.
In my experience with a little more thought you can do these things while keeping it working at the intermediate stage. It might mean writing a bit more code, writing shims and adapters and scaffolds that you know you're going to delete in a couple of weeks. But it's absolutely worth it.
And, because I love the solution, here is a link to my sudoku solver.[2] I will confess some more tests along the way would have been wise, though I was blessed by a problem I could just try out quickly. (That is, running the entire program is already fast. Not sure of the value on running the tiny parts faster.)
[1] http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s...
The odd thing to me, is you say this still of problem doesn't happen in a large code base. But, to me, this style of problem just happens many times in a large codebase. That is, large problems are just made up of smaller problems. Have I ever used the DLX problem? No. Do I appreciate that it is a good way to look at a problem you are working? Definitely. I wish I had more time to consider the implications there.
More subtly in your post, to me, is the idea that with the right people the problems don't happen. This leads me to this lovely piece.[1]
[1] http://www.ckwop.me.uk/Meditation-driven-development.html
Absolutely agree with "Instead you should write code in a way that it's easy to get back into it" though!
That grabs in the concept of switching between planning and experimenting modes with the end result of not getting stuck.
I may be biased in having learned this stuff a long time ago so when I get stuck its from getting hopelessly backed into a corner at which point the best solution is pretty much start over differently, and first semester programmers might get trapped a little more easily, like they don't even know common ancient anti-patterns so they don't even know where to start looking when debugging.
Reading between the lines it almost sounded like the A level programmers wrote their tests first (which is usually easy) then wrote code to pass the tests (usually not too awful) then they were done, whereas the lower grade level programmers wrote the code first, then tried to debug (and debugging is always harder than test writing or coding)
> Before you can write your own subroutine, you have to know how. This means, to be practical, that you have written it before; which makes it difficult to get started. But give it a try. After writing the same subroutine a dozen times on as many computers and languages, you'll be pretty good at it.
Or, as Fred Brooks put it, “plan one to throw away”.
Do you mean to say: if you can't hold it all in your head, you're overcomplicating things? Because if so, I agree.
I just finished giving an introduction to programming course, using Processing. I tried teaching my students to write structured code from the start, splitting functionality into smaller methods as much as sensible. That way they never have to juggle too much functionality in their head at once.
On the first day I had them call built-in methods. Then I introduced the concept of variables and types, then I introduced defining your own methods, and only after that did I move on to booleans, if/else, for-loops and arrays.
It seems to have worked well for getting them to structure their code from the start, using methods more or less the same way you would use chapters, headers, sub-headers and paragraphs to structure a paper.
I know this isn't the best way to program, obviously, but this analogy is (relatively) easy for them to understand and loads better than the ball of muddy spaghetti you often see with people who just start out. It's also probably fairly easy to make the jump to other more advanced forms of code organisation.
I also completely agree with your other two points by the way, I kept repeating similar statements throughout the course:
- By splitting functionality into small methods, it's easy to test if something does what it is supposed to do (and I could easily correct them: "Hey, this method drawBall() is also doing hit detection. That's not what the method is supposed to do, restructure the code!").
- "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?"
Remembering back to my first year, it seems that some people "get this" and some do not. Which is nothing against them -- everyone is different.
I feel like after you really know a system you almost start thinking only in terms of its components. "Hmm I want to do X...somehow I just automatically thought to modify function Y and call it in module Z."