Computers 'do not improve' pupil results, says OECD | Hacker News Reader

Computers 'do not improve' pupil results, says OECD

bbc.com

56 pointswj53 comments

I'm honestly not all that surprised, seeing as nearly every "technology in education" initiative really just amounts to using iPads for virtual textbooks. If schools used the technology to enrich education (specifically where it would be useful rather than just every single class) then at the very least it would be helpful in teaching new skills. However, treating computers as a magical gateway to better educations will never accomplish anything and I fail to comprehend how people thought it would be in the first place.

> how people thought it would be in the first place.

Naive educators desperate to show they are doing everything possible to improve outcomes + mountains of marketing and lobbying dollars from the companies that make the "solutions".

Source: I used to work in academic technology.

I said in another comment, it's basically enterprise software, bought by one small group of people and foisted on another much larger group (both students and teachers are in this latter group) and suffers many of the same market dysfunctions as a result.

On a similar note, people seem to think that internet access will magically allow people to lift themselves out of poverty in developing countries. I personally find this idea to be pretty stupid; just because you have the sum total of the world's knowledge available to you (a paraphrased quote from the article) does not mean you are going to do anything with it or you can do anything with it.

Probably not for the majority, but for the people who want to learn new stuff the internet is invaluable. I learned programming as a kid in the 80's, and the amount of time I spent only with finding and obtaining information is mind-boggling. Even in the early 90's if you wanted to get the official OS documentation for e.g. the Amiga you had to order them as several kilograms of dead tree from the States.

So, 15 years ago I traveled to Senegal to help evaluate how the Internet could be useful to them, economically speaking. Before my trip, I was imagining things much like you're describing here. What I saw on my trip was fairly eye-opening.

First of all, in much of the country basic services like electricity and running water cannot be taken for granted. Phone service is available -- for a price paid by the minute to some equivalent of a local entrepreneur who has invested money into getting a mobile phone plan and has some means of charging / powering the equipment (solar was the typical method, although I saw more primitive techniques as well). Internet access was generally many miles away in a regional capital, which is probably an all day trip for someone walking or on a horse (cars are for the wealthy). When you get there, access is very expensive.

I'm assuming things have gotten better in the last 15 years. For instance mobile phone penetration was already dramatic when I visited and can only have improved since then. However literacy was a serious problem -- you have to be able to read in order to make sense of almost anything on a computer, and in Senegal at least just knowing French is an elite signifier since most speak Wolof. Very few people were fluent English speakers -- I effectively had to learn pidgin French just to have conversations with the local telecom employees, although thankfully most Internet-specific technical terms were just directly taken from English.

In Dakar (the capital city) things were very different. It was pretty much what I imagined a typical city in Africa to be like -- and Internet access there was still expensive but in the realm of affordable. University students got some sort of access that was reasonable, although the pipe out of the country was pitifully small at the time (we had more bandwidth for our office than the whole country had in 2001).

The really interesting thing was that no one could figure out how to make money off Internet access other than Internet cafes and connecting people to the very large number of Senegalese who live outside the country. I imagine the remittance market is huge now. However things have changed, it almost certainly hinges on the extreme motivation people there have to better their lives. They will move heaven and earth to make a buck -- SV entrepreneurs could learn a lot from the drive on display everywhere I went. If you can imagine a way to make money from a situation, they were way ahead of you.

One last thing -- Senegal is considered one of the African success stories, generally speaking. It is relatively democratic, has a growing economy (without all the problems that come with having oil deposits) and has been the recipient of decades of development funding. Many countries in Africa are worse off than Senegal, and I would imagine the snapshot I got in 2001 looks like utopia to them now.

Thanks for the write up!

Oh, just getting current market prices from the neighbouring towns can be invaluable for the farmers to decide where and when to sell their goods.

Mobile phones without internet already work well for that.

It's not magical for sure, people still need to put in a lot of effort, but having access to vast knowledge can definitely help.

We can learn anything online, and I can't count how many times the Internet has helped me with normal everyday life: proper ceiling painting, good electrical wiring practices, fixing up an engine, properly cutting glass, mixing cement for various applications, more effective gardening, food recipes, building a solar water heater and integrating it with the existing hot water system (awesome summer savings), and more...

But yeah, people need to apply these things. Another obstacle is that all this information is only available in select languages, mostly English, so the first thing everyone needs to learn is English AND to share the information in their own language (which they often simply refuse to do because why would they share these advantages with other people? Now that is a stupid mindset...)

Really great used to be the education software that was available in the multimedia CD-ROM era (Win3.1 and Win95).

Nowadays all the PR and marketing budget dwarfs the development costs and newer education software is a lot less creative and useful and often full of advertisement. Today certain websites fill the niche for good education resources including Wikipedia.

These devices could be doing so much more. Showing which kids are struggling in different areas, perhaps even creating groups of mixed skills to try and share the knowledge around. Letting teachers see where they need to spend more time, maybe even issuing tailored homework assignments. All we've got so far is books that can be read in the dark.

not even that - all you need is a flash-light and a conventional book

Students at my sons school (Australia) use the school computers to research topics, and create PowerPoint presentations. It seems like a reasonable way to introduce early school kids to computer skills, as well as some of the better uses for them.

I taught college pupils to math and basics of cs. I think the "multimedia" materials make things worse in many cases, because people are not enforced to imagine things or just think hard on problems. The mistakes, the way of discovery toward e.g. a formal proof also help them to understand more tougher problems and advanced learning materials and instant solutions on the net take this opportunity away.

I'm honestly not all that surprised because we're still doing education the same way as we did over 1000 years ago. Of course news tools aren't going to help with archaic platforms.

You know how expensive books were 1000 years ago?

With cheap books it's much easier to pick up a random subject on your own.

> With cheap books it's much easier to pick up a random subject on your own.

This is moving toward the correct approach; however, the topic of the article alludes toward elementary/high school. I strongly doubt the majority of youngsters would take this initiative if their parents just dropped off a bunch of books in their rooms, this is the one thing that the current education system does teach a good amount of students to do.

Oh, my comment was purely about your claim that there were no changes in teaching in the last 1000 years, and especially no technologically induced changes.

Kentaro Toyama talks about education and the failed promise of the OLPC program and other technological endeavors in education in Geek Heresy. His thesis is that technology "amplifies" the characteristics in society and self. If you have an underperforming school with ill-equipped or overwhelmed instructors, technology won't fix these issues. But in education research, where studies very carefully select a field site, you don't see experiments involving problem schools; you find experimental deployments in idyllic settings, with instructors who are extremely competent, and the researchers might even take a relatively hands-on role to ensure the technology gets used as desired by the protocol of the study.

In one chapter he points out that studies such as the OECD's tend to find middling results (as this one seems to, although I see more explicitly negative conclusions, like that students do worse with technology in some circumstances) mainly because you're studying students from across all schools; look at the successful schools and see how technology affects them, and he seems to argue that you'll find those students make even greater leaps over the underprivileged students.

His bigger point is that technology's benefit is contextually determined by human factors, and that we need to understand the cultures in which we hope to use technology to benefit the members of that culture. Throwing tablets, laptops, or smartphones at everyone won't magically make the world a better place. It's a good read (so far, at least).

Also, I'm getting more and more annoyed seeing news outlets publishing summaries of third party studies without linking to anything. The BBC don't even link to the OECD's homepage, let alone the study they ostensibly published.

I've got two kids in school (grades 8 and 10, relatively affluent district in the US), and I'm not shocked by this result. Anecdotally, whatever the possible upsides of computer use are, they are mitigated by some severe downsides:

* While promising the potential to be a learning tool, the computer is also an addiction. I've observed that it's almost impossible for some kids to manage their time, and to maintain their focus, while doing lessons on the computer. This has been a huge setback for one of my kids. Fortunately, math is still done on paper, so he gets something out of that.

* I've looked at the online lessons. My impression is that the effort of programming the interactive environment for online lessons tends to limit the breadth and depth of those lessons. Math instruction has abandoned proofs. A huge amount of the computerized lessons are busywork.

* There's no limit to the amount of homework that can be pushed on kids.

This is nothing new. Roughly 30 years ago I had an internship at an educational computing center that had almost every kind of computer and educational software title in a demo lab, for teachers to try out. The vast amount of apps amounted to glorified flash cards.

How great it could actually be is lost on the teachers. Instead of working through canned lessons, or surfing for stuff to paste into a report, kids could explore "real" software such as (just listing some of my favorites), Scratch, iPython Notebook, Arduino, etc.

Humanity is suffering from a collective ADD epidemic under the spell of computers and internet and children are getting the worst of it. Computers are the last thing that kids under 12 need in a classroom. Teach them something that involves quiet focus, like 19th century cursive writing.

Humanity is suffering from a collective ADD epidemic

No, it's not. The US and a few other developed countries are suffering from an epidemic of ADD diagnoses. Unrestricted computer use is certainly not a good education policy, but there's no evidence that it causes ADD.

Oh, it causes ADD, alright. Without a proper way to use it, the Internet just leads people to jump between pages, bits of information and topics, which often devolves in distraction on the many entertainment bullshit resources out there...

You can't read a book properly when you know you can just jump to the middle or the end to see the most interesting bits.

Books just leads to people to jump between books, bits of information and topics, which often devolves in distraction on many of the entertainment books out there. Oral tradition was so much better because you had to be focused on the expert who was giving the information and you had to learn it to a great enough detail you could recount the information yourself. With a book you can just jump to the interesting bits, but with oral tradition you have to stick around for the whole story.

And lest you think I am just being facetious, this was a legitimate criticism of writing back in the day.

>Writing, Phaedrus, has this strange quality, and is very like painting; for the creatures of painting stand like living beings, but if one asks them a question, they preserve a solemn silence. And so it is with written words; you might think they spoke as if they had intelligence, but if you question them, wishing to know about their sayings, they always say only one and the same thing. And every word, when once it is written, is bandied about, alike among those who understand and those who have no interest in it, and it knows not to whom to speak or not to speak; when ill-treated or unjustly reviled it always needs its father to help it; for it has no power to protect or help itself.

http://www.english.illinois.edu/-people-/faculty/debaron/482...

"People reading on the Internet are easily distracted" is not the same as "having ADD".

Well, if it looks like ADD and acts like ADD, it is ADD.

Asperger's, for example, has been pushed under "autism spectrum disorder". And Pluto is not a planet.

I could see Internet overload/distraction be diagnosed as some ADD-like disorder, or the main cause of it...

Yes, exactly.

The largest use of computers in schools (let's stick to the educational software, even though classes may not always)

By far the biggest use is in doing what computers are good at : automating manual, repetitive tasks.

So instead of a teacher teaching spelling, the kids use a spelling program where an annoying animated characters zips around saying 'great job'.

This is just automation of the teaching that needs to be done, but without the added ability of the teacher to see who is engaged and who is not. I have first hand experience of children who don't know / don't feel comfortable on the computer and as such get zero out of the automated lessons.

Any use of computers which relies on the person being a passive recipient of the output, instead of being an active determinant of what the computer does (ie programming instead of consuming ) is just automated teachers. Fine, but let's call it what it is.

Alan Kay's "Dynabook" report is such a refreshing look at what computers could be, in the hands of kids. There may be better links, but here's one:

http://www.mprove.de/diplom/gui/kay72.html

To think it was written in 1972.

> kids could explore "real" software such as (just listing some of my favorites), Scratch, iPython Notebook, Arduino, etc.

Go to any UK school and you'll see this happening as part of the CS Curriculum. It's probably less relevant to an English Lit lesson on To Kill A Mockingbird.

Some software that provides large banks of questions for students to work through (particularly in Maths) is actually extremely useful. Not for instruction, but for assessment. It is an automation, but it saves the teacher a huge amount of time in marking and analysing results to identify areas of strength and weakness within a class and further inform teaching.

From a quick scan of the actual OECD report, it is quite clear that the authors are far less certain of the general conclusions than than the reporter from the BBC.

"With this data, patterns of correlation can be identified, but these must be interpreted carefully, because several alternative explanations give rise to similar patterns"

and

"Nothing guarantees that students who are more exposed to computers can be compared to with students who are less exposed, and that the observed performance differences can be attributed to such differences in exposure."

and

"Non-random selection and reverse causality thus plague within-country analyses, even after accounting for observable differences across students and schools."

http://www.keepeek.com/Digital-Asset-Management/oecd/educati...

There are several scenarios where computer use would not just be great but accelerate learning process:

-Teaching about planets, universe, Big Bang through interactive simulations

-Virtual interactive dissection for biology class

-Interactive geometry for understanding proofs

-Playing with molecular structure of material

-Interactive virtual model of steam engine and internal combustion engines

-Interactive excercise for basic arithmetic and trignometry

-Learning about WWII with photos, animations on map, graphs, data, videos.

I would have truely loved all these to be there in my school years. The key is that you need a great software that is targeted towards specific parts of topics. Remember interactivity is the key and only computers can offer that so cheaply and effectively even to the most disadvantaged students in remotest part of the world.

Instead of all these, when I hear "Tech for Education" it usually means, replacing books by ebooks, submitting homework electronically, ask questions in class chat rooms, recording activities in e-log and so on. Those things are triviality with negligible benefits towards actually understanding the subject - more likely negative benefit as it just adds on distraction. No one wants to do real tech in Ed like above examples because it's hard, requires lot of expensive talent and risk taking. Building chat rooms for class shouldn't even be called "tech in Ed", IMo.

Unfortunately schools are blowing 100s of million on just that and governments and philanthropists are happy to shove their cash in to creating ever more advanced chat rooms and classroom management systems rather than create actual interactive content that helps understanding of the subject.

It would have been nice if study pointed this out instead of denouncing use of computers in education straight up.

PS: No one should compare student performance with Shanghai or Mumbai. Those places have extra-super-heavy emphasis on passing exams and memorizations. You will find tons of students there who can acurately list down every single important date for WWII and without pretty much any understanding of dynamics that caused Holocast or even Holocast itself.

"There are several scenarios where computer use would not just be great but accelerate learning process:"

And dumb teachers across the world have missed all that, have they? Before spending taxpayers' money, how about some evidence that it's cost effective?

I agree with some of your examples, but geometry is best learned old school; with the use of a protractor, compass, etc. for diagrams and reasoning about proofs in your head assisted by paper. I have no scientific evidence to offer so the statement is purely based on experience.

For what you are looking for, take a look into the PLATO computer system, used from the 1960s until at least the 1990s.

It also included chat and classroom management tools.

"Computers in education", at least in my country (Portugal), was never about improving pupil results or education per se.

"Computers in education" is all about clientelism; filling the pockets of a selected few which produce mediocre technology, a quid pro quo between politic executive power and the private sector.

I agree.

My daughter has just hit secondary school in the UK. She was disappointed to find that they rarely touch a computer.

What they did do is on day one of science drop her a proper lab notebook and start talking about the scientific method and proper experimental recording, in mathematics they started talking about propositional logic and in RE they talked about logical reason. They're also not allowed to read eBooks; only paper ones.

I'm shitting myself with joy if I'm honest that they didn't stick them in front of a Flash game like they did in primary school and assume that was the end game for technology in education. They learned close to nothing and we had to do all the educating.

The real promise of education technology isn't to plop computers in front of kids and hope they do better it's to get persistent, dynamic, individualized curriculum for every student.

Right now a kid that scrapes by with a D in 3rd grade English starts up at the same place as the kid that sat around bored because they'd read all the required reading the first week.

TL;DR We need a Young Ladies Illustrated Primer - https://en.wikipedia.org/wiki/The_Diamond_Age

Interesting. I'm most fearful that this will be interpreted as "technology is bad! Keep it out of schools!" - rather than "ok, now we have learned that randomly situating computers throughout a school may not be the right approach to introducing new technology into the school in an attempt to improve outcomes", and possibly, "should we revisit the outcomes that we are seeking in this new world?"

What about the benefits to teachers? Taking a test on a computer won't make you perform any better, but it frees up a lot of the teacher's time grading, which is often spend "after hours".

"The role that the computer can play most strongly has little to do with information. It is to give children a greater sense of empowerment, of being able to do more than they could do before. But too often, I see the computer being used to lead the child step by step through the learning process."

http://www.papert.org/articles/ACritiqueofTechnocentrism.htm...

For most applications computers do not make activities higher quality - they make them faster. And they need software to do this. History has shown that really good novel software is extremely hard to come by.

Before being productive on a computer a person has to learn to think and how to be creative. And after that one needs good software.

Sure, 1 in a 1000 kids is a born natural programmer and given a computer will make programming his/hers lifelong passion. For the rest of the crafts - unless exceptional applications are available like Photoshop and (Cintiq/Surface) for art, Scratch for learning programming, etc - traditional analogue learning materials may be superior.

This has been known for years. The only thing computers really help are learning disabled kids.

It's like exercise. You can attach a motor to the exercise machine to do the work for you, the work gets done, but you don't get any stronger.

The same goes for learning. Learning requires effort. No effort = no improvement. Jobs called computers a "bicycle for the mind". Biking will get you further than jogging, but not stronger.

Clifford Stoll was right. Read Silicon Snake Oil and marvel at his prescience.

I assume this article is a summary of his argument, seems more wrong than right with the benefit of hindsight:

http://www.newsweek.com/clifford-stoll-why-web-wont-be-nirva...

Quote: "Then there's cyberbusiness. We're promised instant catalog shopping—just point and click for great deals. We'll order airline tickets over the network, make restaurant reservations and negotiate sales contracts. Stores will become obselete. So how come my local mall does more business in an afternoon than the entire Internet handles in a month? Even if there were a trustworthy way to send money over the Internet—which there isn't—the network is missing a most essential ingredient of capitalism: salespeople."

(I work in education, and the software is mostly pretty bad, since its basically "enterprise software", but I don't blame technology as such)

Stoll makes a convincing case that between the ages of 6 and 16 computers are harmful to education.

It’s not the tool that yields results, but how you use it.

Agreed! Giving them hammers won't improve the outcome either.

When I was little, I remember this racing game and this block building game. You did multiplications (like 6x7) and if you got it correctly you would get another block in your wall. It was really addicting to be faster than you little friends.

Stradivarius violins in schools won't help if teachers are not musicians. The whole notion of professional "teachers" instead of professionals is broken. My best teachers were practicing scientists, programmers and crafstmen

There's an element of truth to that, but I'd argue an effective teacher in a classroom environment also needs to know how to communicate to a group of kids. I've had teachers who clearly had an interest in a subject but weren't able to get it across (for reasons that were a little hard to judge). Arguably that's the sort of thing that teacher training should give teachers a chance to refine, though I don't know if that's the sort of skill that teacher training focuses on at the moment.

I would have liked to see some data on the accompanying human resources and expenditure, if technology investment is taking from human resources investment then it's not a surprise that the results are poor.

I think they have found this result because technology is not used correctly in schools. These "edtech" softwares are very weak most of the time.

What the "International Tests" are optimized for?

I wonder if, a century ago, the increase use of automobiles led to a measurable decline in buggy handling skills.

Reply on news.ycombinator.com