selectron

Can you ELI5 this sentence. I don't understand it despite staring at it for a while.

There could be, but why do you assume this is do to racial bias as opposed to the unfortunate fact that black people are more likely to commit violent crime? I assume that arrest rates should be a good proxy for police interaction, this is a logical assumption - if you disagree then you should give a reason of why your assumption is better, not just say that the assumption "could" be wrong. The unfortunate truth is that black people are more likely to commit violent crime in the US. To show racial bias you have to show that police are disproportionately targeting blacks relative to the rate of crime, if you make the false assumption that the rate of crime is equal between black people and white people then your conclusions will be wrong.

http://www.amren.com/news/2015/07/new-doj-statistics-on-race...

Even for something like traffic stops, you need to study whether or not black people are more likely to get pulled over because they are black or because they are more likely to be young and therefore more likely to speed. It isn't enough to just say that black people are disproportionately stopped, you have to show that the rate of speeding is the same to show bias.

http://www.nytimes.com/2002/03/21/nyregion/study-suggests-ra...

A study from the NYT addresses this, and finds that if arrest rates is a good proxy for police interactions (a reasonable assumption) the data supports the theory that the reason black people are more likely to be killed by police is because they are more likely to interact with police:

http://www.nytimes.com/2015/10/18/upshot/police-killings-of-...:

> This in turn suggests that removing police racial bias will have little effect on the killing rate. Suppose each arrest creates an equal risk of shooting for both African-Americans and whites. In that case, with the current arrest rate, 28.9 percent of all those killed by police officers would still be African-American. This is only slightly smaller than the 31.8 percent of killings we actually see, and it is much greater than the 13.2 percent level of African-Americans in the overall population.

One of my biggest frustrations with python for data science is how bad the documentation for matplotlib is. Also the default settings leave a lot to be desired - look at the color map scatter plots to see what I mean. What is with all that white-space around the graph?

As a strong chess player (around 2000 blitz on chess.com) I find it very surprising that time was not the most important factor. I know from experience when you get to under 1 second per move remaining the probability of blundering goes way up.

In regards to the skill-anomalous positions, I'd have to look at the positions to be sure but my guess is that these are end game positions where the most reliable path to victory is not the quickest path to victory. In a winning position an experienced chess player will try to simplify and win reliably over winning quickly. The model might consider this a blunder because it gives up some material advantage, but it doesn't actually change the end result of the game. My guess is that given these positions where a lower skilled player allegedly does better, in reality the better players win a higher percentage of the time.

Take the job - the company offering you the job knows better than you do whether or not you are qualified for that specific job.

This sentiment is not correct - causal explanations do matter. The statistical evidence for global warming is not particularly strong, as temperature data is very noisy. If we didn't have a causal explanation there would not be a scientific consensus behind climate change. That adding greenhouse gases to the atmosphere changes the climate is more like gravity - we have clear physics on why things fall when you drop them, and if we somehow were adding mass to the center of the earth we know what effect this would have.

More generally you are correct, we don't know exactly how much the earth will warm or if there are complicated feedback mechanisms in place that could cause this warming to speed up or reverse course. We can't even reliably say that next year will be hotter than this year (actually it probably will be cooler because this year has been unusually hot).

I can only speak to particle physics, but the main issue is that we can't study the truly interesting problems. Theories such as string theory are basically beyond the reach of experiments. Our current models work very well to describe the universe, but we haven't made that much serious fundamental progress since the 1970s. It takes decades to find new particles which we know must exist (top quark in 1995, Higgs in 2012). This will probably become even more true after the next few years after the LHC collects data at 13 TeV, although of course I could be wrong and something could be found. To study new physics you have to go up an order of magnitude in energy or luminosity, and this scales worse than linearly with cost, so it isn't feasible. Of course it is possible that new technologies emerge, but this isn't a sure thing.

The other problem with physics is that it is really hard to become a professor, and the field forces 90% of bright, dedicated and talented people to go into industry because there is a lack of jobs in physics. We really need permanent positions at labs outside of academia.

Physicists have a better understanding of statistics and better data to work with than pyschologists. There is a very clear causal explanation for why greenhouse gases do what they do. With that said, there is not enough discussion about the cost and effectiveness of proposed solutions to global warming - a lot of the "solutions" are more feel-good measures, which are not worth the cost for what they do. Global warming is a global problem - if the US stopped all of its contributions to climate change the earth would keep warming up.

As a physics grad student I have seen plenty of "negative" results published, and the standard of positive results for instance 3 or even 5 sigma is a much tighter standard than p > 0.05. Science is a big field, the problems in one domain do not necessarily translate into all domains. However there are other fundamental problems with physics as a field.

If the data is truly biased in that it doesn't reflect reality then sure, go ahead and correct for this bias. But make sure you are not introducing your own biases which are wrong. "But is it OK that we’re using extraordinarily biased data about previous arrests to predict future arrests and determine where police are stationed?"

The truth is that men living in certain areas are much more likely to commit violent crime and therefore get arrested than the average person in the United States. This is the sad reality of the world we live in. It isn't wrong to deploy police resources where they are most needed. If the data shows that these areas are no longer hotspots of crime, then we can reroute resources elsewhere.

The algorithm just tries to model the world as it actually is, not as it is supposed to be. The algorithm cares nothing about uncomfortable truths. The algorithm should not be set up and trained to reflect a wrong model of reality, but rather you should consider if you want to use an algorithm based on reality or one based on how you want the world to be.

This proposal shows more concern about being politically correct than actually correct. A truly unbiased system would just make the best analysis possible given the data available. What you propose is to make a biased system in order to give an advantage to one group of people rather than the other group. This type of political correctness fuels resentment.

> Until we have better software we’re unable to build good models from small datasets. (And when I say “small” I mean, not ginormous.)

This is completely wrong - to build a useful model your model just needs to be able to distinguish between the signal and noise in the data. There are decreasing returns to scale with increasing size of your data, as percent statistical uncertainty goes like 1 / sqrt(N).

I agree with this. There has to be some balance - it is not like EU membership lifts the economies that enter it, so it doesn't seem like it should be so bad to leave the EU. I also expect people to overstate the effects - like currency devaluing is both good and bad for the UK while people are acting like it is some kind of disaster. Stock markets also seem to largely be decoupled from the real economies these days.

It looks like the average was around 35%: http://predictwise.com/politics/uk-politics Interestingly the odds went down pretty sharply as the election came closer, but it is not really fair to pick the lowest odds when evaluating how good the predictions were. But even 15% odds are not really all that unlikely, about a 1 in 6 chance. It isn't as though pundits were predicting leave.

The betting markets had somewhere around a 35% of Brexit. So it wasn't that unlikely according to the betting markets.

This site does the calculation: https://electionbettingodds.com/brexit.html

Here is a good site which converts betting odds to probabilities: https://electionbettingodds.com/brexit.html

There is pretty conclusive evidence that taller men are on average more attractive than shorter men. However being more attractive does not necessarily mean you will have more kids. My theory is that people want to marry people of similar attractiveness, so if for very attractive people the pool of potential matches could actually be smaller, in part because attractive people are more choosy. Also in terms of evolution, if you have sex it doesn't matter how attractive your partner was.

You could absolutely learn all the skills and knowledge to be a good data scientist, the real question is if you could get hired. I have been looking into data science and I think the demand for data scientists is not as large as it seems, but it is growing. There are very few junior data science positions. Especially without a PhD or an inside contact it could be challenging to get your foot in the door. My advice would be to check out Kaggle, try out a problem and see if you enjoy it.

As an American, I care much more about the fate of America than I do about the fate of Europe. People are tribal animals.

The startup scene is primarily young, white and male. It isn't all that surprising that Trump has support among the startup scene. American tech workers are also particularly vulnerable to outsourcing and have suffered from the H1B program.

"But he has no serious plan for how to restore economic growth, which is what we actually need." The president can't just wave a magic wand and restore economic growth.

And just because Trump may well be racist, that does not mean he is Hitler. This kind of fear mongering detracts from the conservation.

The whole premise was based on the assumption that meritocracy is bad - my point is that this is a flawed assumption. The proposed solutions were the government could step in and "discourage hiring practices that arbitrarily and counterproductively weed out the less-well-IQ’ed." This is counter to science - it has been shown that intelligence is one of the best predictors of the quality of an employee. Why should the government subsidize stupid people, or interfere in companies hiring the best qualified employees anymore than it already does? The reality is smarter people are on average better employees, which is not at all surprising. This article is using politically correct logic, as opposed to actually correct logic.

A fundamental assumption of the article is that every person is of equal worth. Anyone who objects to this idea is labelled as racist or sexist or whatever-ist. But it is objectively just not true that people are all the same. A murderer is not equal to a doctor. Most people would choose to save a drowning doctor rather than a drowning murderer if forced to pick one. Similarly, if I a rational person had to choose between saving a drowning doctor who is an idiot or saving a drowning doctor who is a geniue, the rational person would save the smarter doctor.

I don't have any problems with a society that is a meritocracy. This article is nonsense. If anyone wonders why so many people don't like political correctness just point them to this article.

Hopefully you at least got paid, so there was some benefit.

Good for them for being transparent even when the company is struggling, but if I was an employee at the company I would be worried. The chart they provide of Buffer bank balance over time seems overly optimistic. It looks like they assume that the employees they fire added no value to the company - they assume the company will generate the same revenue even after cutting 10% of their employees. This is wrong, and would worry me greatly that the founders don't seem to recognize this. Of course I would be worried anyway because laying people off at a start-up is a really bad sign, anyway you try to spin it.

I convert escape to caps lock, I also change : to ; in vim so I don't have to hit shift all the time.

It is interesting how different fields have different terms for statistics concepts. Statistics really should be taught at the high school level, it is far more useful than for instance calculus. I hadn't heard those terms before. In particle physics we have the "look-elsewhere effect" as a synonym for fishing, and discuss local vs global p-values (which might be similar to researcher degrees of freedom).

Thanks, not sure how I missed it.