Homunculiheaded

One thing that I haven't seen come up much is the idea that the electric car is going to be essential for a successful robotic car.

Sure robotic drivers will be masters of fuel efficiency, and quiet likely we'll end up sharing one car between many different people when that happens. But Jevon's paradox[0] suggests that we'll actually be using more fuel, not less. And it makes sense as well: driving will be so easy and not constrained by human factors (needing to sleep, eat, boredom with driving, being intoxicated etc) there are millions of reasons to have more cars doing more work.

Even ignoring environmental cost, the fuel cost to the individual would remove a lot of this usefulness (Why not commute 8 hours while you sleep, start your weekend on the beach the moment you wake up? Because it's so expensive!).

I pretty strongly believe that without fully electric cars, we'll never realize the benefits of fully robotic drivers.

[0] http://en.wikipedia.org/wiki/Jevons_paradox

The biggest phrase that I see missing from the private vs government discussion is "monopoly on force".

The Federal government has the authority to create regulations, create laws, collect taxes, send people to your house to enforce those laws, remove/restrict your rights, sentence you to a prison term, send in armed officers to take down civilians groups viewed as dangerous, etc etc all the way to declaring full scale war. And, perhaps most importantly, the government has the authority to coherence third parties to cooperate in information sharing. Google will never be able to force Facebook to hand over their data, but the Federal government can force both of these parties to hand over their data to them.

The catch with having a monopoly on force is that your hands are supposed to be tied by the will of the people. There is a tremendous and intentional asymmetry in power. This necessitates an equally tremendous system of transparency, accountability and oversight.

I think once a day I hear "people willingly give all there data to facebook, why do they care if the NSA is listening", people need to understand what "monopoly on force" truly means.

It's really unfortunate that most people read 1984 as teenagers (I did). I went back to it as an adult, somewhat expecting it to be a let down. What I found was quite the opposite, there's a lot more nuance in that book than would really be absorbed by high school students being forced to read it.

"The object of persecution is persecution. The object of torture is torture. The object of power is power. Now do you begin to understand me?"

You're absolutely right, and I've been intentionally avoiding a lot of mainstream news sources on this so my overall picture is mostly what I've been reading about what people are reporting.

I also don't mean to come across as overly paranoid, I don't think men in suits with dark glasses popped into NYT offices with a pre-written story in hand.

However this is the place in the news cycle where we've gotten over the initial shock, and start wanting to focus our energy on something. This would ideally be the part of the processes where we collectively ask "Now what do we do to make sure the NSA stops this?" But instead the question is "Hey, what can we find if we look through Snowden's laundry".

And the thing that I find really interesting is that Snowden's laundry is very boring. Painting Assange as extremely arrogant and only using Wikileaks to feed is own egomania, while also being a potential rapists feeling from justice is an interesting story. Snowden having no HS degree and doesn't get along with his neighbors? Not so much.

I believe part of the thinking was: the leaker will be found eventually, who do you want to start the story about the leaker? If some government agency were to unveil Snowden's identity they could cast him in whatever light they chose from the very first image (who looks like a defender of freedom in a mugshot?)

Look what happened to Bradley Manning, the first public image of him was his chat confession and a huge amount of deeply personal details that really had nothing to do with the leak.

This reaction is actually scarier to me than the initial surveillance. NPR and NYT are usually sources of very solid journalism, the fact that they would drop the real story for some gossip about Snowden implies that someone is able to successfully exert pressure on these organizations.

The surveillance alone leaves the possibility that one agency has started to go to far, but this more systematic reaction indicates that the trouble is deeper and more wide-spread.

"Big Data" is the buzzword of the day but I think "Big Privacy" will be a bigger trend in the very near future. Right now it's technically possible to put together the resources to give your self a fairly good bit of privacy, just as it's always been possible to write a cron job to do remote backup before Dropbox. However this takes both a bit of technical skill, a good chunk of time and planning, and there's still a chance you'll miss something.

So I believe there's a lot of money to be made in providing instant privacy the same way dropbox provides instant backups. Likewise as statistics + programming = data scientist, statistics + programming + security = privacy scientist

Sadly this also means that privacy will become a commodity and I can easily imagine a tiered system of privacy based on cost (eg 19.99/mo get's you a secure network + encrypted files, 99.99/mo get's you text re-structuring to avoid stylometeric identification.)

just as a point of clarification: "genetic programming" is not the same as a "genetic algorithm". Genetic programming is an area of evolutionary computation in which the AST of a program is created using a genetic algorithm. So while genetic programming is definitely "machine learning" in the truest sense, it's not terribly useful for classification and regression problems (what we usually think of when we talk about ml).

Now genetic algorithms are any algorithm that encodes solutions (typically parameters to a cost function) as a gene and performs optimization through an artificial evolutionary process. Now GAs are incredibly easy to learn how to implement, but it's much harder to figure out when they should be used.

In theory GAs can be used anywhere in this diagram that parameters need to be chosen. However there are two major draw backs to GAs:

1. Cost function is a huge bottle neck 2. In many cases there is a known, better method of optimization.

So for example you might want to use a GA to determine the parameters for training an SVM, however (given you have enough data) this runs into problem 1, since it might take minutes, hours or days to train an SVM. Which is too long and you would have to train 100s or more SVM for each iteration of the GA, with a least 100s of iterations.

Also GAs have been used to find optimal weights in neural nets, however here you run into 2 (and I believe 1 as well), as using backprop usually performs better.

All that said GAs are an amazing tool when you have some cost function you need to optimize that 1.) can be evaluated very quickly, and 2.) is weird, very non linear, or otherwise has no known 'good' solution already.

So finally to your question: most heavily studied areas of ML already have a better optimization algorithm than a GA (hence no GAs on this diagram), however in the real world you never know when you my find some very strange, difficult and poorly studied optimization issue and then GAs can be very useful

"If they pass great if they dont offer them feedback."

I've only personally encountered it once, but if you ask someone to spend a few hours of their free time on your coding challenge you absolutely owe it them to give them feedback, even if you don't like what they've produced.

To make the choice of L1 vs L2 a little more flexible there's also Elastic net regularization [0] which lets you combine both L1 and L2

Also agree with LDA, I've recently been using it for some unsupervised text classification and the topics it generates are impressively sane.

0.http://en.wikipedia.org/wiki/Elastic_net_regularization

The more people get excited about faster ways to hoard bitcoins and eventually make a nice profit (presumably in USD in the end) the dramatically more cynical I become about its future. When the bitcoin discussion was mainly around its use as a currency for things like silkroad, VPNs etc I actually thought it seemed to have a promising future. I really hope I'm wrong, but I have a feeling in 2023 the bitcoin discussion will be relegated to nostalgic jokes about how crazy we used to be.

L1 regularization can also be used for feature selection[0] and hence dimensionality reduction. I've in found practice that the speed of this can compare favorably even to fast implementations[1] of svd.

[0] Feature selection, L1 vs. L2 regularization, and rotational invariance, Andrew Y. Ng. In Proceedings of the Twenty-first International Conference on Machine Learning, 2004. http://ai.stanford.edu/~ang/papers/icml04-l1l2.pdf

[1]"Augmented Implicitly Restarted Lanczos Bidiagonalization Methods", J. Baglama and L. Reichel, SIAM J. Sci. Comput. 2005.

RDF took Prolog clauses, restricted them to triplets, and made their representation dramatically more verbose. Then it removed the reasoning engine part of logic programming and replaced it with OWL which is by default computationally intractable.

I have tried for many, many years to learn to appreciate RDF and semantic technologies in general, but everytime I get involved it screams of a problem desperately looking for a solution, and whose general case of the problem (reasoning about relations) was already solved better decades ago (and with the recent revived interested in logic/relational programming seems likely to improve soon).

If you want to represent a graph structure in your data why not sets of json objects? Sure you end up with an ad hoc vocabulary, but I've never seen a successful project make use of the RDF ideal of massively shared vocabularies.

Cross validation doesn't change the fact that you're trying to predict a non-stationary distribution. Machine learning techniques generally make an assumption that your samples are being drawn from at least a close enough distribution to any future data that may enter the system. With problems like text classification in NLP you can generally make a safe assumption that, though language does change, it changes slowly enough that you don't have to worry. In other cases a model may simply need to be retrained after a certain period of time. Even in text classification real world systems will often incorporate an unsupervised novelty detector as well to indicate when the models may need to be retrained.

Additionally normal cross validation (random or stratified sampling) does not work on time series data since you are in effect cheating by training on events in the future, which your model will not be able to predict. In order to test your model on time series data you would need to hold out a significant chunk of data at the end of the time series.

If you're interested in the general case of predicting the stock market you might enjoy reading up on the Efficient Market Hypothesis[0] which deals quite well with just that question.

http://en.wikipedia.org/wiki/Efficient-market_hypothesis

If you like the Fitzgerald quote, you'd probably also be interested in the idea that it essentially summarizes "Negative Capability"[0] which was originally developed by the poet John Keats.

Particularly relevant I feel in a world of comment thread polemics

0. http://en.wikipedia.org/wiki/Negative_capability

Thanks! This is exactly the sort of thing I've been looking for. I really want to experiment with some deep learning techniques for some problems I have, but the start up cost of "oh yea you have to build all the tools yourself right now" keeps putting me off.

As someone who uses R for just about all of my ML/data analysis needs I'm surprised not to see Theano[0] mentioned. SciPy, SciKit-learn, pandas etc are great and all, but there's not much really different than what you get with R (except of course having it all in a general purpose language). But Theano (plus it's related deep learning tools) really stands out for me as something the R tool chain can't compete with.

I feel like eventually I should become as fluent with SciPy/Scikit-learn/pandas as I am with R, but learning Theano well is much higher on my list.

0. http://deeplearning.net/software/theano/

I think most people considered the spin off into Lucent to be the true end of what Bell Labs was. There hasn't effectively been a Bell Labs since 1997.

I think what the parent is pointing out that P(male, 20-30 | played through the level) is higher than the existing P(male, 20-30 | applied for job)

I've you're looking to build something custom from primitives and you need ie support I would definitely recommend Raphael.js [http://raphaeljs.com/] They claim ie6+ support and in my own experience this has been true. Granted I was just making little dynamic pie charts, but they looked fantastic in IE6 (I'm actually surprised it isn't in more common usage by designers to certain visual bugs in IE6).

Raphael.js is really just a nice way to handle svg, and so it lacks some of the niceties of d3 as far directly tying your visualization to the data. But it's still a great tool.

I've been experimenting with the R/D3 setup as well and so far I really like it. Another thing that's great is that for any sort of traditional data visualization R has you covered, leaving D3 for what it's best at: custom and/or interactive visualization.

Interestingly enough almost all my coding has been in the browser since I've been using Tributary [http://tributary.io/] for D3 prototyping and RStudio Server for R (which is amazing if you haven't tried it)

Although I haven't used it yet, it's worth mentioning Shiny from the RStudio team http://www.rstudio.com/shiny/ which makes it pretty painless (so I hear) to create interactive R visualizations.

"Stealing" in the sense of depriving you access to the documents should be prohibited, but I see no problem if someone requested you immediately copy all of your research.

In fact this is exactly the case with anyone working directly for the federal government, doing research or otherwise. You're told in no uncertain terms that at any moment information on your work laptop can be FOIA'd in which case you will have to immediately hand over the contents of your hard drive (if it has been determined there is a reasonable likelihood that it contains the FOIA'd information). Obviously if you're working on classified state secrets the process is different (also personal information is redacted from the documents). There's no reason research can't be accomplished successfully given the condition that the public has a right to that information at any given moment.

Some things about what you said weren't entirely clear (to me at least), but having read through 1, 3 and some of the relevant parts of the report referenced in 3 here are some comments I'd like to add. For any HNer interested I would say the report is more interesting [0]

If you are between 18-65 and got the flu vaccine this year, eli's comment is correct and it is unlikely that you will get the flu.

The key point behind the articles is that flu vaccines very well might not help reduce flu related mortality. So as an issue of public policy are not as effective as preached. One guess as to why this is, is because the vaccine doesn't produce antibodies well in the at risk population with weakened immune systems.

The rest of the research says what I think most HNers probably know, if you're 18-65 the efficacy of the vaccine is dependent on the strains in the vaccine, and what strain of flu is going around. Even well matched vacines don't always work as well as hoped.

In the general case, if you're 18-65 and healthy, the vaccine is about 60% effective, and in the case of h3n2 is about 88% effective. All of this comes from [0]

[0] http://www.cidrap.umn.edu/cidrap/files/80/ccivi%20report.pdf

I think "around the corner" type predictions generally fall into 2 camps:

1.Problems that we don't know how to solve yet, but we think we are close to based on "similar" problems we have solved.

2.Problems that we have a solution for, but it currently takes an unreasonable amount of time to use these solutions in practice.

Problems in class 1 are like AI in the 1960-70s, everybody thought we were super close to amazing AI based on discoveries we'd had, but these estimates were very wrong.

Problems in class 2 are like nlp and ml work in the 90s and 00s. A rather large chunk of 'wow' ml/nlp we have in applications today were pretty much solved 20 years ago, but there was no sane way to run them, certainly not on your cell phone.

Problems in class 2 are safer bets, there does seems to be consistant increases in processing power, memory, etc. Problems in class 1 are harder to guess because as history has shown, just because a solution seems similar doesn't mean that it actually is (shortest path is solved, longest path is NP-hard, shortest path touching all points once (ie. tsp) is NP-hard).

I think it's safe to say having Watson on our smart phones is right "around the corner" (20-30 years?) saying that we'll create "creative" AI, not so much.

I can't offer much help with many of the other issues you mentioned but if you really want to understand the internals of backbone better I don't think there's really a better path than reading the annotated source [0]. One of backbone's strengths is that it is so minimal, and the source so compact.

0. http://backbonejs.org/docs/backbone.html

King of the Hill is definitely one of the more underrated animated series out there. I actually think it's daring that it doesn't aim for 'gut busting' humor the way most American comedies do, and it's very nuanced in the way it portrays its characters.

On the one hand it's under appreciated, on the other as cartoon with a complex view of american life that isn't going for the "big laughs" I'm surprised it wasn't cancelled in a week.

Here's the resources I've used and found helpful (plus notes) for getting a least a basic grounding in statistics.

Khan Academy + "Head First Statistics"

-No matter what direction you want to go with your learning and before you start getting into street fights over Bayesian vs Frequentists philosophies, "stats 101" is the common base of 'statistics' for: Physicists, Engineers, Sociologists, Doctors, Economists, Journalists and more. Yes there are many "truths" in 'stats 101' that can be questioned, but this is the common language you'll want talking to any of these people.

"What is a p-value anyway?"

-A very quick, non-math heavy read on avoiding magical thinking in classical statistics.

"Data Analysis with Open Source Tools"

-I really love this book because it steps way out of "stats 101" land and gives a lot of really great real world advice and touches on many mathematical tools that can help you understand data. The chapter "What you really need to know about classical statistics" gives a refreshing overview of all the basic stats you've covered and explains many things that seem strange. Also the bibliography/recommended reading list from this book is fantastic.

The biggest thing I've noticed as I've learned more is that statistics is a gigantic field, and can mean extremely different things depending on what field you're in. Obviously learn the advanced statistics relevant to your interests, but when you get a chance listen to how other people think about stats, and keep learning.

For anyone interest in R without a background in statistics: I would highly recommend learning the two in parallel (if not statistics first).

R is first and foremost a language for statistical computing. You really aren't going get much out of it without working on some interesting data/stats problems. Plus for most hacker types I think being able to play with the statistics you're learning about with R can be a great learning aid.

However not only is it beneficial to learn stats with R, it is imho dangerous to learn R without some stats. There's already too much research being published with were 'p-value' means "the thing that t.test() output that I was told needs to be in the paper".

Because R lets you play so freely with stats I find it a great tool to gain greater intuition about certain mathematical principles, but there is a temptation to let the tool do the work and the thinking for you.

One thing I've found somewhat odd is how seriously people take the timeline on coursera courses. To me one of the biggest weaknesses of these courses is the hard time constraints.

For all of the courses I've taken (ML, PGM, NLP, SNA) I've followed roughly the same pattern: follow the course for 50-70% of the material doing all homeworks etc. Take what I've learned and go off do some fun projects. Once I hit a road block return to the course and finish up (usually months after the course is 'officially' done) typically forgoing the homework being I've already created 'homework' for myself.

Compared to my classroom experiences, I've found that this method greatly enhances the benefit of the materials I learn later in the course, as well as allowing me to more strongly reinforce material learned earlier. This is also something that physical classes simply cannot emulate. I do find it a little funny that I've never officially completed a coursera course, despite having covered much of the material very thoroughly. Hopefully as these courses continue we'll start to see them adopt patterns that extend beyond replicating the limitations of a physical classroom online.

Maybe I'm mistaken but I think most of the people interested in becoming 'data scientists' are either currently doing lots of software with an interest in stats, or people doing lots of stats with an interest in software. Given one or the other half of this list is probably already very familiar territory.

I actually found this list encouraging because the things I don't know well on that list are things I'm working on and am aware are holes in my knowledge.

But in the end the reality will always be that the people who are "real" data scientists will be the people that are actually solving real problems whether or not they can check off every bullet point on a check list.