fnbr

I think they mean that Paul Graham et. al launched YC as a small, cheap experiment, and never imagined they would have success like this.

I think this is true, broadly. Biological evolution has a large advantage in that it introduces a large amount of exogenous variation into the system (it just randomly changes DNA via a number of ways, and selects the "best" changes). This can be good as it allows you to do a lot more exploration than with a comparable gradient based approach, as the gradient based approach can't explore entirely new parameters; it can only look at parameters that have already been shown to be good. This is why, for instance, dead neurons are a problem with ReLUs. I don't think (but have not proof) that this would be a problem with evolution.

I think that evolutionary approaches are a lot better at the exploration phase, but weak at exploitation; gradient methods tend to be the opposite.

I think some sort of combination of the two, like Max Jaderberg's "Population based training of neural networks" [1], is the way to go.

[1]: https://deepmind.com/blog/population-based-training-neural-n...

The numbers that I have seen indicate that humans have a fatal accident roughly once per hundred million miles, while we have one fatal accident for a self-driving car, with somewhere around ten million miles driven across all self-driving cars.

I've heard Uber is around 3 million self-driving miles.

So Uber would be 30x worse than humans.

Target's surprisingly sophisticated. They had a booth at NIPS this year, which I wasn't expecting.

I'm so sorry.

No, not at all. A master's does help significantly, but it's possible to get these jobs with any background.

Alphabet's probably accepting stock because they're one of the most risk-loving investors possible.

They've already got a business that's spinning off tons of cash, so they have no need for money, and instead, are pursuing any possible avenue that has the potential of making $$$ (e.g. all of the other bets).

He claims in interviews [1] that he reads everything that goes through there (but obviously doesn't reply).

[1]: https://www.recode.net/2016/6/1/11826718/jeff-bezos-amazon-f...

I agree. It's only as a potential employer that they lose their luster.

What? That's absolutely wild. Is this typical? How does it compare to what, say, Alphabet's treasury does?

Yup. I've had a number of bad experiences (packages not arriving, wrong thing delivered, etc), but every time, Amazon has made it right. They consistently produce a great customer experience.

This is the reason that I plan to either work at Big N companies, where my equity is given to me in RSUs, or start my own company. Working at a startup in any role that's not a founder or an exec seems like a total scam, unless you have an incredible amount of confidence in the founders, and their integrity.

Not sure if you're looking for CV or DL libraries. If you're looking for DL libraries, you can't go wrong with Tensorflow or PyTorch for research/development, and Tensorflow or Caffe2 for deployment. Tensorflow's a bit difficult to learn, but has a lot of great tooling around it, and PyTorch is the opposite. The other frameworks are fine, but don't have the same amount of documentation & beginner resources.

For instance, say I had 2 services, a scraper and then a ML algorithm that classified the scraped data. You only need to run the ML classifier when there's new data. I didn't have anything set up to tell the classifier "hey! There's new data to classify!" Instead, I just ran it every 5 minutes, which was sufficient latency for the project.

That's what I did for a recent side project. It worked really well, although coordinating the different services was a bit of a pain. We ended up using a cronjob to run everything repeatedly, which was inefficient but worked. If we had the time we would have used a proper RPC framework or some sort of queue.

What's missing here is a discussion of compensation. I've seen at conferences how proud senior Google employees (ie Jeff Dean) are of the TPU work that their teams are doing, and I suspect that compensation is following accordingly. I doubt that Intel or similar companies are compensating their people at the same level. Whenever I see people talking about a talent war, I'm always suspicious that they're getting outspent and are unwilling to compete.

Adversarial noise is interesting, but I think it's more of a toy problem that will go away as we switch to more sophisticated models with life-long learning that are able to develop complicated, hierarchical/compositional classifications.

For instance, there was a talk at NIPS this year where the researcher developed virtual stickers that can be added to a photo and cause the image to be misclassified (eg you have a "banana" sticker that you can apply to any picture to make the classifier classify that picture as a banana). The underlying issue, IMO, is that the classifier has only a limited number of categories from which to choose, and can't develop new categories (such as "adversarially perturbed picture"). The modified image does look more like a banana than anything else, and as the classifier never trained on adversarial images as a separate category it doesn't really make sense to make any other classifications.

How would you address your manager?

Based on my understanding of the issue, the main difference between now and then is scale. Pre-iPhone, Apple didn't have the scale to make their own chips economically; now, with the massive success of the iPhone, they're able to do this.

Their major value proposition is that they're able to build software-differentiated hardware, so building their own chips is extremely valuable to them.

It is technically illegal to fire you for discussing this, at least in the US.

I know at Google there's supposedly a big internal salary sharing doc started by Erica Baker, and she wasn't fired for it (although her manager was supposedly quite unhappy).

I'll give you my perspective: it's tough. I tried to go from ML Engineer -> ML Consultant, and while I was able to find work at a reasonable rate, it was difficult, because most businesses don't have technical problems, they have business problems.

For instance, even though the root cause might be high latency, the actual problem is that the high latency is killing your retention.

If the business is sophisticated enough to value technical expertise, often they value it so much that they are willing to hire experts full-time, or they're so small that they can't afford to. So as a technical expert, you end up caught in a weird uncanny valley (in my experience). I think that patio11's advice is, as always, spot on here; the skills needed to become a technical consultant that charges $N/week are way higher than the technical skills needed to become a business consultant at $N/week, for any value of N. That isn't to say that technical skills aren't value, just that it's a tough sell to make, for a number of reasons that patio11, and others, have written at length about.

Currently, I'm employed FT. When I was consulting, typically multiple projects. It's necessary (IME) to do so to guarantee you have a steady flow of work. I'd always be looking for a new project midway through the current one.

It's very doable to find remote consulting/freelance work. I'm in a region of Canada where the goal freelance/consulting rates for software engineers are well below $100/hour, and I could regularly find remote consulting work in the US around the $100/hour mark .

Why would Visa/MC do that? It doesn't seem any different than metering any other service (eg Mailchimp invoicing monthly vs per email).

I'm not saying you're wrong- I just don't understand what leverage the CCs would have.

Exactly. It's also not clear why it's taken VC- seems like it should be bootstrapped somewhere in the Midwest.

What's better than Gurobi? I used it at a previous job and we weren't aware of anything better (but it also seemed slow and not great at scaling).

I'd love to see that research! Could you share some links?

This is basically the stance of every serious practitioner that I've seen (ie the stance shared by everyone who directly works with theory/code to train models, as opposed to those who talk about AI at a high level).

Right now, we have these systems that are effectively ungodly complicated spreadsheets. They're great at a variety of tasks, some of which seem impossible for a non-intelligent entity to perform (neural machine translation is wild to me).

But that's all the systems are- super complicated spreadsheets. There's no way for them to start replicating consciousness without massive advances in the field.

Having said that, there is a road from where we are to intelligence- if we can create a network that performs arbitrary interactions online, and figure out some way to create a positive feedback loop for intelligence, like AlphaGo Zero did with their policy network & MCTS, then we might be able to figure it out. But we're so far away from that that I'm not concerned.

If you're looking to understand the underlying theory behind deep learning, the Deep Learning book by Goodfellow et al. is awesome.

If you're interested in general machine learning, the Elements of Statistical Learning, by Tibsihirani et. al is great; a more applied book is Applied Statistical Learning by the same author. For a more applied view, I'd check out Tensorflow or PyTorch tutorials; there's no good book, as far as I'm aware, because the tech changes so quickly.

I've done a series of videos on how to do deep learning that might be useful; if you're interested, there's a link in my profile.

I suspect that they were using classical ML techniques, and a lot of traditional scientific computing techniques, but not a lot of deep learning, given how new it is, and how slow government operates.

It's also not clear how a lot of the classical ML can be translated into a DL setting- a lot of graph analysis [1] is traditionally solved using matrix factorisation, for instance, which doesn't directly translate into deep learning. You can find different ways of doing it with DL, but it's not as transferable.

[1]: e.g. analysing an individual's social graph. I believe that Facebook uses a lot of classical techniques, like PCA. I've heard that Facebook has been pouring a lot of money into research that uses DL on arbitrary graphs.