In a study of senior CS majors, U.S. students are tops in skills
spectrum.ieee.org
spectrum.ieee.org
There is a huge variation among countries in how subjects are taught and tested. I would like to add that many of my peers have resorted to streamlining for whiteboard interviews, and unsurprisingly, they end up with well-paying jobs.
This annoys me to no end, since ABET is used to gate certain federal jobs and the Patent Bar, even in Computer Science.
However, every school you listed is ABET accredited in other engineering disciplines (mechanical, chemical, civil, electrical). So ABET-accreditation is important to some fields.
I don't want to be too elitist, but where would you like to recruit reviewers for your computer-related patents from: DeVry Tech or Stanford? Do you really want to actively exclude Princeton and Yale grads from working on software that runs the government?
I get the appeal of having a standardized 'seal of approval', but for Computer Science, ABET is clearly not that. It looks like several other disciplines are also moving away from ABET (e.g., https://cen.acs.org/articles/95/i48/time-leave-behind-chemic...)
Acceptable Computer Science degrees must be accredited by the Computer Science Accreditation Commission (CSAC) of the Computing Sciences Accreditation Board (CSAB), or by the Computing Accreditation Commission (CAC) of the Accreditation Board for Engineering and Technology (ABET), on or before the date the degree was awarded. Computer science degrees that are accredited may be found on the Internet (http://www.abet.org).
[page 6 of this pdf: https://www.uspto.gov/sites/default/files/documents/OED_GRB....]
A lot of fed jobs have engineering-centric requirements. This one, for example, wants an accredited degree or experience under a PE....to do network management. https://www.usajobs.gov/GetJob/ViewDetails/525193800
>exam that was developed by the U.S. Educational Testing Service
Who could have guessed that?
And how about this lovely question:
> A personal identification number that opens a certain lock consists of a sequence of 3 DIFFERENT digits from 0 though 9, inclusive. How many possible PINs are there?
Putting aside the value of this question in the first place, the correct answer is 720 but it also includes the answer 1000 in case you didn't notice the emphasis on “different”.
Doesn't seem like a great test to me unless it's about your reading comprehension and ability to decipher code with single-letter variable names.
Different can be interpreted in multiple ways.
Reminds me of a physics test where the question is something like:
Susie is almost at her house and comes to a stop in 4 seconds. What was her acceleration.
Don't use that same password for anything else!
For a fair comparison, the translator would have to fully appreciate the relevant connotation of every significant word like "different" in every question, and successfully convey the same in Russian. You couldn't just go to a regular translation service and get a good result.
Although I have no faith in a multiple choice test as a predictor of CS knowledge anyway. That is just a horrible test format anyway.
Does anyone have an assessment of reliability/validity/fairness of multiple choice comparing to other test types? (afair objectivity is very good in multiple choice tests)
Being able to go from some constructed story to a math problem is a skill that can be tested elsewhere (and really, should be assumed if someone clearly has proficient enough numeracy to do combinatorics) Questions should be testing one thing as isolated as possible, not the same things repeatedly.
I can see this in my students in my introductory class. You can ask them to calculate a concentration of a solution and they will reply just fine. But when you ask them to go into the lab and actually prepare a solution of a certain concentration you draw blanks in 9 cases out of 10. Students are trained in AP Chemistry to spot problems according to keywords and apply the appropiate formula, but they have never thought about the underlying meaning of the things they do, and they cannot connect different subjects to save their lives.
The mechanical kinesthetics of combining things is not the same skillset as the mathematical process of computing it. Lab work is its own craft.
Based on what you're saying, I'm assuming you already put your questions in the "Story time" format and are getting that 1/10 success rate because the story problems are yet a Third skill - this time one about reading comprehension and language mastery.
Reading comprehension isn't kinesthetics and that's not computation. One may partially inform the other, but each skill is fairly independent.
Anyway - IMO this is a terrible question.
The sample questions cover amortization, memory allocation, finite state machines, formal logic... If you just studied enough to pass a whiteboard interview, I'd hazard that you probably can't answer half of these questions without looking up what the symbols mean.
Sure, most people here may be able to figure out the PIN question, but it's not about whether any single question is easy or hard, but whether the test can sort the students according to ability. A good student may misread one or two, but a bad student is going to miss a lot more.
What this really measures is corricilum. And pressumbly it's more stuff like this in the US then Russia?
If a CS student taking this test does not get “tricked” by this question, they should definitely be able to answer it.
The first time you pick, you have 10 different numbers to choose from, the second time, 9, the third time 8.
1000 = all possible combinations from 000 to 999.
less 10 combinations with all numbers identical is 990.
now there are three patterns for numbers to be identical left that need to be taken out: 00x 0x0 x00. The x can be replaced with all digits != 0 in this example, so thats 3 (patterns) x 9 ('x' digits) = 27. That times 10 digits that can have double patterns is 270.
1000 - 10 - 270 = 720.
for j = 1-10
if i == j break
for k = 1-10
if i == j or i == k break
log(i,j,k)
(I don't understand HN formatting still)
#!/usr/bin/python
r = [(i,j,k) for i in range(10) for j in range(10) for k in range(10) if i!=j and i!=k and j!=k]
print(len(r)) import itertools
print(len([x for x in itertools.product(range(10), repeat=3)])) #1000
print(len([x for x in itertools.permutations(range(10), 3)])) #720
The reason it's less is because each number has to be different, so you can't have 000 or 111, etc.
The formula is:
n! / (n − r)!
Since there is 10 (n=10) total numbers, 0-9. And it can only be 3 (r=3) digits long. Then it would be,
10! / (10-3)!* which would result in,
10 * 9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 / (7 * 6 * 5 * 4 * 3 * 2 * 1) = 10 * 9 * 8 = 720.
It can be easy for people to forget that many people (even coders) have never done this type of math before.
Really? I mean, to first approximation, all software engineering work is debugging. And to first approximation, all debugging is about carefully reading results and the code that generated them. That sounds pretty spot-on to my eyes.
And the variable naming quip... are you serious? One of the worst mistakes you can make when trying to read code for understanding is to rely on documentation layers like comments and variable names. Those things lie. Read the code.
I mean, obviously people do rely on those things for understanding, so when writing code it's important to have good semantically and contextually clear names for stuff (which certainly can include single letter names for local symbols repeated often!). But for debugging, you really have to train yourself to read the code, not the names.
I don't really agree that these questions test debugging skill. In real life if I read a bit of confusing code and get tricked, I will usually know immediately thanks to my IDE, compiler, automated tests, debugger, or behavior of the program. On this test if I get tricked, I just lose points.
In my experience most debugging presents as trial-and-error (a terrible test-taking strategy) or seeing a bug and reasoning at a high level which component of the software is responsible and gradually narrowing it down. Carefully reading through lines of code is usually the last step, unless I've got an especially elusive bug.
> And the variable naming quip... are you serious? One of the worst mistakes you can make when trying to read code for understanding is to rely on documentation layers like comments and variable names. Those things lie. Read the code.
I'm not sure what sorts of code you're reading but this opinion seems extreme. Do you really just wholly ignore comments and variable names? I have encountered inaccurate examples of both but I have never thought to distrust them by default. As far as I recall this has not caused me serious issues while debugging.
I don't think I agree. You're talking about heuristics for debugging code and I'd argue that many of us apply similar heuristics to reading comprehension. Computer Science is not the same thing as reading comprehension, but I have to argue that if you're really good at one of these; you're probably potentially good at the other.
I did not get 100%. I actually thought the exam was pretty good, but it tested something like 1% of the skills I use as a working programmer. Skills like API design, software architecture, documenting code, and yes even naming variables, which I think are important to writing quality code.
Do people really struggle with problems like these?
I switched into programming and am low paid, and am doing this right now for interviews. Practicing for whiteboard interviews, or coding tests (real time online and in person).
I re-read the language docs and then did (so far) 20 or so self-timed programming tests. Most of them I did in 5-30 minutes, 1 or 2 I got stuck on and subsequently reviewed. After finishing reading the language docs I will read though the platform docs, and may do some quizzes on that as well. As well as whiteboard high level class creation tests (HAS-A, IS-A).
The tests have a lot of array stuff, String manipulation, using long instead of int, casting from one type to another, doing a lot of stuff with maps and sorting maps by value. It also uses things then can be done quickly and compactly in streams. I do these things once in a while, but now have it memorized and down cold.
Hopefully this all leads to the latter result you mentioned.
Also, what do people think about triplebyte for this?
Can you clarify what this means?
Given the typical CS student body, this basically means Indian, Chinese, and Russian students studying at US universities performed better than those studying at Indian, Chinese, and Russian universities...
Either because US universities teach better or because the best students come to the US for their education.
Once you get down to the average university, the number of foreigners in undergrad classes drops dramatically - it's just too expensive for most foreigners to pay out of state tuition unless it is a prestigious university.
In my undergrad, most of my engineering classes had only one or two foreign students.
1. People are willing to pay a premium for a MS degree. In many of their home countries, a BS is of low value.
2. Funding is much easier to secure for a grad degree - both from US institutions and institutions in their home country. Probably the majority of foreigners are not paying out of pocket.
The same "average" university that had so few foreigners in undergrad were completely dominated by them in their MS program. And I think there was only 1 US person in the PhD program (granted, it was a small department and they had very few PhD students to begin with).
As an example, in 2013 Amazon was only offering an additional $5k salary IIRC for new grads with an MS.
That doesn't say a lot about Indian/Chinese/Russian students in the US in particular, but their data is public at https://osf.io/c78wb/ , so maybe it's possible to get that information, depending on how strongly they anonymized it.
Without being fully absorbed in this culture, you will lack the key thing that makes you go "ah ha! I get it!"
I mean, look at the leadership of a lot of these companies. They're foreign-born, typically from Asia.
How does western philosophy or culture impact software design? I'd argue that how software works is influenced by how computers work, which is influence by how physics works, which is culture-agnostic. You can't possibly say "binary is so Western". So, what, exactly is influenced by the culture / philosophy?
The social animal's default mode is to follow the herd. "The herd" could have even started out as the Texas Sharpshooter's Fallacy (random correlation), but now people are motivated to join in. To assemble. To copy.
I believe that a healthy amount of CS/CE students get into the field due to existing (strong) interest in the field. This is probably especially true of people graduating during the outsourcing era (2000ish-2012), when enrollment plunged due to fears that there would be few software engineering jobs left in the US.
I'd be surprised if people in it for the money rather than interest will perform as well. If parents start pushing their children into the field because it pays well, then schools may lower standards in order to increase graduation rates, thus flooding the market with unemployable people (see: India).
So much for "world domination".
Meanwhile, in the UK CS grads have the highest unemployment among any field, and have maintained that "award" for nearly a decade. That certainly indicates that your educational programs for CS are lacking if you didn't go to a school named Oxford or Cambridge.
There's a lot of ways you can measure which country has the better educational system for a given degree type, but there's very few metrics that indicate anything other than the EU being way behind the US.
Plus the fact that the US dominates the tech sector has more to do with the available capital there than the technical skills of US students (on average)
Like I said it's hard to quantify educational programs, but China and Russia both dominate competitive programming competitions, and have for years. Whereas EU simply doesn't (outside of occasionally wins by Poland and Czech Republic).
lol, you can't judge educational programs by the rate of unemployment! You realize how little sense that makes? The fact that you would even state such a fallacious comment makes me question YOUR educational background. A countries unemployment rate is massively dependent on the economy of the country. You don't say, look at the lack of top tech companies in Russia, it must mean their students are idiots! It has all to do with their governmental policy-making and the sanctions on the country, as well as various historical factors.
The UK has been under government "austerity measures" since the 2008/2009 crash which have massively reduced growth across sectors. The UK actually has a great education system but this doesn't matter if the economy is doing badly and jobs aren't being created.
In fact (I live in California) from all I've seen of the US CS market, US grads are by far the worst in terms of all the nationalities I work with. The fact that the US tech industry is booming has far more to do with the VC world, the booming economy, and the huge amount of people from around the world moving to California, than it has to do with US schools being good at CS. All the top data scientists at my current company are from China!
I'd advise you avoid resorting to personal attacks when you disagree with a non-personal statement, as it really does lead to people ignoring your the premise of your point. Personal attacks when calling out fallacious arguments are particularly off-putting.
That said, you failed to understand the premise because you took one sentence out of context. The point I was making was that Cambridge and Oxford both have top CS programs which can result in jobs outside the UK and EU. Major tech companies routinely nab graduates from those two universities on work visa's in the US.
The importing of UK CS graduates plummets beyond top-tier universities, while China, Russia and India all see massive work visa imports into the US. The UK, outside of Cambridge and Oxford, simply doesn't export a lot of CS graduates, indicating an educational gap (or at the very least, an "interview gap") between those countries and countries with massive work visa exports.
> In fact (I live in California) from all I've seen of the US CS market, US grads are by far the worst in terms of all the nationalities I work with.
Your anecdata isn't interesting or useful.
That's because their CS programs graduate a lot of people that can't program.
It seems reasonable to expect a test written for a specific education system (either implicitly or explicitly) will be biased against others.
On the other hand, it is not that surprising that the US is a the top. It is a pretty big country with a long tradition for high quality education and a lot of funds going into CS departments. However, I will be surprised if this is not changing towards more dominance by India and China, because both countries are focusing a lot of resources in this area.
I'm not trying to denegrate people who compete in programming competitions—they require hard work and talent like any competition, but it is a very specific set of skills that are developed which are different from real-world programming.
https://www.ets.org/mft/about/content/computer_science
Journal paper is here:
Is this study so flawed that the results are meaningless and shouldn't be discussed, or are the results meaningful and the authors will get backlash if they discuss their findings?
I'm curious why they even put the gender graph in the paper. They didn't look at differences in Elite Institutions like they did for countries. They certainly have more data than I do, but I'm left to make my own uninformed conclusion.
[1]:https://www.pnas.org/content/early/2019/03/12/1814646116
>In every country, the men came out ahead; the gap overall was smallest in China, and biggest in the U.S.
But if you are looking at a difference of between gender (0.4) and thinking you'd rather hear them talk about that than the difference between nationality (double that at 0.8) then you clearly are coming to the conversation with a narrative already in mind and are not looking at the data.
I'd also then go on to say in my experience the student who went into a CS program for the stability and potential money "CS-curious" over passion for it might do better on scholastic tests of computer science while being less capable of actually programming well.
(From the article)
> CS departments and CS education in general have a longer history and are much more established in the United States than in China or India especially.
> A lot more is spent per student in the U.S. than in the other three countries.
(From the paper)
> United States have likely similar math and science levels as students in Russia (Implying the difference in CS scores comes from somewhere else)
But the line you quoted was the only "discussion" on the differences in gender in the article. I think there has to be at least something to discuss.Why is the gender gap smallest in China? Why is there a gap at all for people sitting in the same classes? Are there any other metrics (like age?) that had similar differences? Did the size of the gap change when looking at elite institutions?
To be completely fair, there was some discussion in the last paragraph in the paper, but the only hint at what might be causing the difference was that the women in CS aren't high achieving enough. I wouldn't count that as in depth discussion on the cause.
> The gender gap in skills does indicate that more effort is needed to attract higher-achieving female students into CS [...]
Just putting a chart at stating results the end of a paper and an article isn't a discussion. There is a gap between genders about half as large as the gap between countries. Having questions about that doesn't imply a narrative.
Just come out and say it, instead of having this proxy argument over gender.
If it isn't clear: this study can not be used to draw conclusions about racial or gender predispositions. This can only be used to show the status of education on this particular test and for people graduating right now. It doesn't control for how those people got there. If you want to twist it in some other way, that's your prerogative, but you're doing shoddy science.
I am agnostic to the cause of the gender difference, and clearly this study didn't have enough information to give any insights. That's why I find it so unsatisfying that they would put up a chart and mention gender differences if they weren't interested or capable of expanding on the results beyond "[...] more effort is needed to attract higher-achieving female students into CS."
This feels really disingenuous. The questions they brought up are perfectly fine, it's you who seems to be intent on poisoning the discussion by attacking them instead of discussing the issue.
> you'll have to first say that you think that whites are genetically superior to non-white in the world of computer science. After all, doesn't the study show the gap is even greater for nationality?
Nationality is not the same thing as race/ethnicity. At the very least, in the US, a substantial percentage of CS majors are of Asian descent, not European.
Just because they measured the data, does not imply they were trying to explain it. To explain a result would need a very different experimental setup. Every scientist knows this. It's not a conspiracy if a scientist declines to throw darts at a wall on the cause of something. In fact, they specifically say the cause is likely not explainable by the test differences, which were too small.
"The within-country gender gaps in skills are small enough, however, that they may explain little about gender gaps in CS graduates’ labor market outcomes".
In fact, the poster is not "just asking questions". They have a clear opinion, and ignored the researcher's own statements to the contrary.
Here's the mean z-score (and its standard deviation in brackets) aggregated by country, elite status and gender:
China elite male: 0.360 (1.534)
China elite female: -0.338 (1.046)
China non-elite male: -0.730 (0.623)
China non-elite female: -0.791 (0.514)
India elite male: 0.181 (1.304)
India elite female: -0.538 (0.918)
India non-elite male: -0.847 (0.599)
India non-elite female: -0.921 (0.459)
Russia elite male: -0.374 (0.975)
Russia elite female: -0.829 (0.602)
Russia non-elite male: -0.739 (0.594)
Russia non-elite female: -0.897 (0.445)
USA elite male: 0.872 (1.006)
USA elite female: 0.563 (1.174)
USA non-elite male: 0.146 (0.958)
USA non-elite female: -0.248 (0.818)The deltas between genders are quite a bit smaller than those between nationalities, though!
I mean, sure, I guess you could trigger the proverbial SJWs by pointing out the fact that a US woman is about a third of a standard deviation worse than her male classmates. But it's hard to draw firm conclusions about gender when Chinese and Indian men are at an even larger handicap relative to her.
I mean, I gotta flip this around here: why do you feel like we should be focusing on a small subsignal when the data supports a clearer, more defensible, and more hypothesis-rich conclusion about something else?
I am agnostic to the cause of the gender difference, and clearly this study didn't have enough information to give any insights. That's why I find it so unsatisfying that they would put up a chart and mention gender differences if they weren't interested or capable of expanding on the results beyond "[...] more effort is needed to attract higher-achieving female students into CS."
Im assuming that most of the Coursera/udacity/etc combo is ineffective, because the students from other countries must already be consuming them. So the difference is clearly originating from college level courses.
Should everyone just do Abelson and Sussman + Knuth and call it a day ?
Just my opinion, but more doing, less "learning". I've interviewed dozens of Indian candidates at my company, and I've noticed a huge problem: a good 50-60% of them know the jargon and theory, but if you try to probe deeper they don't know what any of it means.
It reminded me strongly of Feynman's account of the Brazilian education system[1]
Basically, I think serious students need to actually build stuff and gain a deep intuition of the concepts they are learning. There seems to be far too much rote memorization of CS concepts and jargon without deep understanding of the why and the how.
It would have far more interesting to see the US stacked up to other English speaking countries. Wonder why they didn’t do that and had the exam translated instead..
https://icpc.baylor.edu/worldfinals/results
Google Summer of Code on the other hand is dominated by Indian Students.
TLDR: If it's a level playing field I don't think US students would be in front of other countries.
Averages in a country don't really say much about the smartest students in that country anyway. It seems likely that the smartest students in China, India and the US are similarly gifted.
If you look at the average high school student in the US, you might conclude that Americans are pretty stupid, but the averages are brought down by the obscene amount of educational inequality in the US. American CS students aren't so affected by this because they mostly come from the "good" schools.
Not sure we're doing anything better than these other countries other than remaining the world's only superpower.
Like making six figures a year for screwing people over at Facebook?
Sorry for being snarky, but lots of jobs in IT businesses in the US tend to be somewhat unhelpful for the society at all and not really providing any benefit in the long term.
In regard of this I'm always somewhat "happy" that we fail in Europe with building "the next Google", "the next Facebook" or whatever.
So you're correct, the EU has failed in building the next Google and Facebook, because the majority of top-tier EU CS grads already work for them.
While you might be happy Europe has no six figure SW jobs, I, not being able afford a decent house in a decent area to raise a family on your average Austrian SW dev salary, am not, and would gladly take a FAANG job any day regardless of how morally questionable it may seem from your McMansion.
My family's well being comes first.
Maybe because of that we're holding values and standards like "do no evil" high, because there are more important things than money.
I can't afford a house in a decent area either, but I'm pretty sure I won't fuck up lives with my job. And I'm still making enough money to support my family and keep everything going.
I'm far from rich and am on a not-even-competetive UK dev salary. I still and pick and choose which companies to work for based on whether they align with my personal principles. It's not like there's only one company to work for.
(If it was hypothesis testing, 1 sigma could be statistically significant depending on sample size. You might be confusing the population variance with the sample mean variance.)
That's exactly what I'm doing - thanks for pointing that out.
1 standard deviation in mean is substantial. That means approximately that 80% of one group exceeds performance of the median of the other group.
There are a few explanations for this
1.) Programming is an English skillset, which the native English-speaking countries have an advantage. Out of the native English speaking countries (US, England, Canada, Australia, and others), US has the biggest economy, the best tech companies that can train students as interns, and the best infrastructure to grow the students. Where as China, while it does have the population advantage, doesn't have the English capability. Chinese government is currently actively engaging in nationalism (maligning foreign brands, censoring foreign cultures, destroying churches, arresting pastors, saying winnie the pooh is an evil foreign influence), thus will have less and less English skillset overtime.
2.) more innovative/risk taking mindsets. This allows a more creative problem-solving skills amongst students. The Chinese education style is regurgitation and there is a culture of copying and cheating amongst students. Rote learning is widely practiced in schools in India as well, which isn't conducive to innovative problem solving.
I think the difference is in the teachers. This difference is most likely originating because of the education given in univs by good professors.
Everyone has access to the online courses/MOOC
That is incorrect. In urban areas in India, schools teach in English but in rural areas it's in Hindi. In urban areas, only if it's a big city, then peer to peer communication is in English. However, in other cities, the only interaction in English is with a teacher; other times is in the native tongue.
And even if the students in big cities are learning mainly in English, the teachers are usually not native English speakers, which means that the grammer/pronounciation suffers. which is what you see with Indian engineers that have migrated to other countries.
https://www.quora.com/What-language-is-used-in-schools-in-In...