querious

One of our simplest “screening” questions for DS roles at my company is: “your model is 100% accurate. How do you feel?” If the answer is anything other than deep skepticism (Data leakage, trivial dataset etc), it’s a big red flag

These guys are saying something like 17M thefts in 2017, which to me is a bit concerning and almost incredible, even if they’re off by a factor of 2. I mean, it’s not the end of the world if someone assumes my identity but it certainly seems like a monumental hassle to deal with and almost certainly bound to cost several thousand dollars.

https://www.javelinstrategy.com/press-release/identity-fraud...

What I'm psyched about is OpenStreetMaps data queryable with Athena. It's traditionally kind of a pain to convert PBFs to a queryable format.

Apache Airflow defines and runs DAGs in a sane way, IMO. Takes some configuration, but worth it for more complicated projects.

The UK's NHS recently opened up a large amount of its data to Google [0]. In parallel efforts, a company called Nuna is gathering and unifying data from state level Medicaid programs so it can be analyzed similarly [1].

0) https://www.newscientist.com/article/2086454-revealed-google... 1) https://www.nytimes.com/2017/01/09/technology/medicaids-data...

(edit: formatting)

Actually a pretty good way to understand manual transmission. Very cool.

Does anyone know whether this is a degenerative disease or a condition one is born with? It's possible the whole brain is necessary for learning, (including learning how to learn more abstract things), but then can be pruned heavily to move information into denser networks.

Whenever automated transportation becomes feasible, which is on the order of 5-10 years, a good chunk of a multi-trillion dollar market will go to the few players ready to capture it. At this point, Uber is arguably the most promising bet: tons of data, name recognition, long-term vision, talent, and on-point execution. Not to say they can't miss the mark still, but the market size for transportation of people and goods is gigantic, and Uber is dead set on basically being the One to take it all. I can see why it'd be an appealing investment.

Original paper: https://groups.csail.mit.edu/EVO-DesignOpt/groupWebSite/uplo...

Basically how the "Data Science Machine" works: 1) "Feature synthesis": features related to the target are discovered by following foreign keyed relationships in a relational database, automatically generating "deep" queries involving many joins. Aggregates like MIN, MAX, AVG, and STD are automatically calculated to be used as additional features. 2) "Dimensionality Reduction": Truncated SVD reduces feature length 3) "Modeling": Remaining features are clustered and then modeled by Random Forest decision trees with learned hyper parameters.

There's a ton more optimizations, but that's the gist.

The machine can't be evaluated as a stand-alone product, though, because the researchers also mention manually generating features on the problems they tested, without reporting the effect of their own tampering.

They do conclude the system is more of a time-saving device than the general pattern-recognition one that our journalists seem to think it is.

Edited: formatting

Should there be an intermediary the client needs to sign that the summary isn't a perfect representation of the contract? Kidding, of course. Mostly.

Would love feedback and ideas on the concept. The target audience is lab managers in the live sciences field, and we're looking for beta testers.

I think of it as an arms race between traders and regulators. The more complex the security, the longer it will take regulators to pass rules against its misuse, figure out taxation, etc. In the meantime, the inventor stands to make an enormous amount of money.

Also, there's an inherent information asymmetry favoring the inventor.

A) 20% growth in revenue per year is not "flattening out". It's massively exponential growth. B) Google is at the cutting edge of robotics and computer vision. I'd say they are in an excellent position to cash in on the next big market.

The words "Something All Our Own" come to mind (the name of Grant Hill's collection of African American art).

The motivation to defy expectations has been there for a long time, and makes sense on several levels. It's the same drive, I suspect, that led to utterly unique movements like Jazz, hip-hop, and distinctive forms of dance, which really insisted on creating their own modalities and not borrowing from the pre-existing culture.

Some motivations I can think of:

One, asserting independence from a dominant culture that in turns delegitimizes and exploits your own. Why would one choose to blend in with this culture?

Two, demanding legitimacy / highlighting the fact that there are still dire punishments for choosing to embrace one's ethnicity. Hell, name choice is small beans... consider the racism implicit in having your dialect, spoken by millions, deemed unacceptable at work and school by white authority figures everywhere. Having to deal with the fact that no one will take you seriously if you speak the way you do at home, is a big part of the African American experience and must be a constant reminder that you are not accepted by those in charge unless you toe the line.

Three, an attempt at disproving racist forces that wish to deny your intelligence, creative spark, and potential. Self expression was brutally squashed since the beginning of slavery days, and only allowed insofar as it entertained those in power. The desire to actively disregard whether your action pleases the supremacist (who still doles out reward and punishment), and instead celebrate the differences that have normally only meant trouble, is a fully understandable reaction to all this.

On a similar note: anyone know of a tool for generating SQL queries by example?

This is pretty incredible work. There are SO many modules and they all seem to work very robustly and cleanly. Very impressed.

Well, theoretically a brute force universally solves any set of constraints, just takes too long. Intelligence is really only about efficiency and timescales, i.e. the dumbest algorithm would look insanely smart to us if it ran fast enough.

I can't recommend a browser extension like Vimium highly enough. It allows you to browse the web efficiently without a mouse. Also: shortcat for navigating OSX dialog boxes, and Divvy for resizing windows with the keyboard. I hardly touch the gesture pad anymore (external one), and my shoulder thanks me for it.

Python and NLTK :)

The question that comes up for me at this point is whether there is much that is dispensable about humans when it comes to exhibiting intelligent behavior (running on 100 watts, no less). It turned out that for abstracting useful flight dynamics based on birds, there was a rather simple rule: lift > weight. Sure, reducing it to that simple formula may not help you build a machine that maneuvers as well as birds and insects do, but we didn't need that for flight. We just wanted to cross an ocean in less than 6 weeks.

Whether the flight analogy carries on to intelligence, in my mind, depends on how many of our subsystems are 1) indispensable for intelligence, and 2) reasonably computationally reducible.

From neurotransmitters to ganglia cells to hormones and bacteria in the gut, we have found a lot of subsystems that contribute to our abilities to make diverse, everyday decisions that the ideal AI we are discussing would have to make. The cortex actually seems like one of the most orderly and therefore reducible parts of the apparatus. The hormonal system that regulates emotion based decision making may be far more difficult to abstract and less efficient to model. And there are many many other systems. Could it be that without details of those subsystems, our AI behaves in less than optimal ways the same way a human would? How much can we get away with reducing biology to simpler rules while maintaining general intelligence?

It is possible I suppose that all those biological dependencies are merely hampering an ideal algorithm for generalized intelligence that we are only crude approximations of, a powerful and simple algorithm we can finally free of biological constraints, -- but it's too late to get into the probability of that hypothesis! In any case it's not clear to me how that kind of nonhuman intelligence would serve us.