Hacker News Reader

Out of curiosity, why? Wouldn't they have pretty good proxy data on the demand for different skills?

john_hortonon It shouldn't take 64 lines of code to do something really simple

The reason indentation to imply structure doesn't make sense for datasets is that in data analysis you are constantly subsetting and sorting data at the row level---something you generally don't do to the lines of a compter program.

john_hortonon It shouldn't take 64 lines of code to do something really simple

My point was not that this was absolutely hard, but rather that's way too hard for the totally mundane thing I was trying to accomplish---namely getting public data from a US-funded statistical agency into a form suitable for further statistical analysis (which is the whole reason they make this data public).

Also, I don't see how it's ironic that I used Python---it seems irrelevant. Whitespace in Python has a well-defined, commonly understood purpose; using tabs or spaces to indicate hierarchal data relationships is not at all standard, presumably because it creates messy dependencies across rows of data.

john_hortonon It shouldn't take 64 lines of code to do something really simple

I'm guessing that's the goal, but it's only going to be as good as the inputs provided by the different agencies.

For example, if you look for the time series I was after on data.gov, the "Download CSV" button brings you to...the BLS page, with all the problems I discussed.

http://explore.data.gov/Labor-Force-Employment-and-Earnings/...

john_hortonon The McGurk Effect

Here's the video: http://www.youtube.com/watch?v=G-lN8vWm3m0

john_hortonon 80% of People Quietly Despise Their Lives

According to the Pew Research Center, 89% of people who are not self-employed report being "completely" or "mostly" satisfied with their work; the percentage for the self-employed is even higher, at 95%.

http://pewsocialtrends.org/2009/09/17/take-this-job-and-love...

john_hortonon Saylor offers millions of dollars in bounties for open textbooks

Rather than Saylor decide on what to buy and at what price, I wonder if they could use a kickstarter-like model. If they matched individual donations, they could make their money go further & get some genuine measure of community support/interest in different texts.

john_hortonon Results from Khan Academy experiment

no problem & agreed about what probably happened - my point was more that knowing they had that sample size (n=40), someone could have told them that even if Kahn academy was extraordinarily effective, there was almost no chance they'd detect statistically significant effect. If they knew that, they might have pursued a more qualitative approach rather than have people see the take-away as "no effect."

john_hortonon Results from Khan Academy experiment

In the comments, the authors give the sample size as about 20 in each group. Here's a more sensible graph (error bars to create a 95% CI on the estimated group means):

https://skitch.com/johnjosephhorton/fih62/quartz-2

Obviously, there are some statistical power issues. It's unfortunate that they didn't get someone to help them with a power analysis up front---given how many very tech savvy people are interested in Kahn Academy and want to see it succeed, I think they could have gotten this sort of assistance pretty easily.

john_hortonon Richard Feynman profile in Google Scholar Citations

I think google created a profile for him as a way to illustrate what profiles look like---they mention him in the blog post announcing the change:

http://googlescholar.blogspot.com/2011/07/google-scholar-cit...

john_hortonon Flash game illustrating the Gale-Shapley algorithm

I was impressed by the simplicity as well. I think Gale & Shapley's intellectual contribution was not so much in proposing the algorithm, but in proving that a stable match is alway possible (and that this algorithm generates one such match).

john_hortonon Edward Tufte's "Slopegraphs"

Great article---I once used the "slopegraph" in a paper to show how the types of gov't contracts (e.g., fixed-price versus cost-plus) is different at each step of the litigation process:

https://skitch.com/johnjosephhorton/fjmuc/ssrn-id1094622.pdf...

john_hortonon HBS Student Creates Smartphone App To Detect Malaria

http://whqlibdoc.who.int/hq/2000/WHO_CDS_RBM_2000.14.pdf

From WHO/USAID report on Malaria detection (executive summary):

"In malaria patients, a prompt and accurate diagnosis is the key to effective disease management. The two diagnostic approaches currently used most often, however, do not allow a satisfactory diagnosis of malaria. Clinical diagnosis, the most widely used approach, is unreliable because the symptoms of malaria are very non-speciﬁc. Microscopic diagnosis, the established method for laboratory conﬁrmation of malaria, presents technical and personnel requirements that often cannot be met, particularly in facilities at the periphery of the health care system. In addition, delays in the provision of the microscopy results to the clinician mean that decisions on treatment may be taken without the beneﬁt of the results. Thus, the recent introduction of rapid diagnostic tests (RDTs) for malaria is of considerable interest."

john_hortonon HBS Student Creates Smartphone App To Detect Malaria

http://www.quora.com/How-important-is-accurate-diagnosis-in-...

john_hortonon Ask HN: Do Americans stand a chance on freelance sites?

The "market for lemons" paper was not about the average quality of the market---it was about the selection effects generated when market participants cannot judge quality. Within a online labor market, there are platform-imposed features that reduce information asymmetries & make relative comparisons possible e.g., tests, feedback, verified records of billed hours. The informational problem is much worse without an intermediary, which is why these markets exist.

john_hortonon Which interface is better for users asked to make relative allocations?

That's actually one of the more popular suggestions on the StackExchange question I asked that motivated these prototypes:

http://ux.stackexchange.com/questions/3736/how-to-build-a-bu...

john_hortonon Lean vs VC: How We're Taking on StackOverflow Careers

To me, what's exciting & innovative about SO Careers is its integration w/ the high-quality, high-bandwidth information conveyed by SO reputations---not how SO executes the job board itself.

john_hortonon StackOverflow’s Careers 2.0

sure - it's a simple model, but I don't think the presence of ignorant firms changes the basic insight.

That being said, labor markets are very complex and there's lots of strategic maneuvering going on. Given the indeterminacy of much of game theory (i.e., it depends on how you model the scenario), I'm sure you could be right under the proper assumptions.

But to bring it back to the SO example, it's hard to imagine a scenario where giving high ability workers the ability to truthfully signal that high ability hurts. Low ability workers can be hurt i.e., either I know you're bad from your SO rep, or I start to think "hey - where's your SO profile? you must be bad." In the pre-SO system, the low ability workers were getting an unearned rent (assuming the market existed at all) at the expense of the high ability workers and firms (which is market-killing).

john_hortonon StackOverflow’s Careers 2.0

I think adversarial/zero-sum is the wrong characterization. There are clearly misaligned interests---companies want to pay less; workers want them to pay more. However, if both sides are better off under a match than they would be otherwise (which is why the market exists), then they'll try to match so long as the search costs, frictions and losses due to bad hires (i.e., adverse selection) don't swamp the benefits.

Imagine the worst case job market, which is that anyone can hang out a shingle and say "I'm a highly skilled Python programmer." Suppose 1/2 are good, and add value +1 to their companies, and 1/2 are bad, and "add" -1 to their companies if hired. If there is absolutely no way to tell before hand who is good and who is bad, then no one is getting hired---the expected gain from hiring is zero. We have complete market failure (this is the "market of lemons" example).

Now suppose you throw in some signals---schooling, resumes, interviews etc. if firms can tell w/ probability .8 whether someone is what they say they are. People now get hired, but even the good ones take a haircut b/c firms still wrongly hire bad programmers because the signals available--while better than nothing---are not perfect. I think SO is basically raising .8 to .95 (so to speak). This is valuable to firms and to workers and I can easily imagine a world where firms pay to get to .95 (and good workers pay to be evaluated with the .95 lens).

john_hortonon StackOverflow’s Careers 2.0

What if there was integration with online labor platforms e.g., a "Hire Joel on oDesk/Elance/Freelancer etc." button on profiles? More than once I've asked a question on SO, gotten excellent answers and then wished I could offer one the responders a short job.

full disclosure - I work at oDesk, but I had this idea before I started.

john_hortonon The Unreasonable Effectiveness of Simplicity

I wonder if some of the benefits of simplicity come from the reduced probability of making errors. I know that in my own research, I always try to use simple methods or at least augment more complex methods with graphics and simple methods.

That being said, I think it is useful to know the complex methods, even if you don't use them.

john_hortonon A sequence of lines consecutively traced by 500 individuals

I suppose "De gustibus non disputandum est" applies here. Oh well.

john_hortonon A sequence of lines consecutively traced by 500 individuals

I recommend checking out the other pieces on his site---all very beautiful. Several others also build on the theme of turkers interacting with each other, albeit through a limited channel.

john_hortonon How to game Google Scholar

If you use "appears on Google Scholar" as an indicator for quality, then yes, you'll have problems---but you get the author affiliation, citation tree and journal, which can all be strong, useful signals of quality (esp. if it's your field).

john_hortonon The Cost of Entrenchment: Why CEOs Are Rarely Fired

Agreed - in the short run, it wouldn't have an effect for the reason you highlight (the pool is fixed). I was thinking more about the long run, where a change in CEO probability alters the career choice of people deciding say between going into business or going to medical school, becoming a lawyer etc.

john_hortonon The Cost of Entrenchment: Why CEOs Are Rarely Fired

Interesting. One critique of the paper---the notion that a higher level of equilibrium firing would making being CEO less attractive (and hence lower the talent level) seems wrong, or at least incomplete. First, more firing would make more space at the top, increasing the probability of getting the job in the first place. Second, in a world with more firing, getting fired would be less of a negative signal. Getting canned would still be bad, but my guess is that it would be more like losing an NFL head coaching job e.g., you become an offensive coordinator, coach in college, move to a different team etc.

john_hortonon Beautiful men, women have higher IQs

1) I'd take these findings with a big grain of salt:

http://www.stat.columbia.edu/~gelman/research/published/kana...

2) I'd be very cautious about interpreting causality, even if the results are true. Beauty is influenced by environmental factors (diet, exercise, health care etc.) that are correlated with parental and own-income. Some of these same factors also influence intelligence. There's a decent chance this research boils down to "being poor sucks."

john_hortonon Deficit Reduction

Our country continues to operate a defined benefits plan while most businesses have moved to a defined contribution plan.

A government-run defined contribution plan would almost certainly have to have some kind of guaranteed payment floor to be politically viable, which would encourage serious moral hazard in investing.

I also wonder how many companies can offer defined contribution plans because their employees are also automatically part of a defined benefit plan (i.e., social security).

john_hortonon Solar Panel Maker Moves Work to China

On the bright side, the supposed force driving the move (plunging prices for solar panels) is a good thing.

john_hortonon Curation is the New Search is the New Curation

You'd probably have to have a trusted human-in-the-loop. I.e., if a site gets enough down votes, it would trip some internal procedure to review the site.

I think also if you didn't generate public feedback (i.e., you didn't list the most down voted sites for example) you'd avoid the kind of Beiber-to-DPRK stunt that 4chan would be interested in.