fnbr

Not that I'm aware of. I could imagine a few things that you could try that might accomplish this, but I'm not aware of any published literature discussing the efficacy of such work.

You could do the naive thing, which would be to take your architecture and scale each layer size up (e.g. select an architecture on Cifar-10 and then scale it up to work on ImageNet). This is done in practice quite often, and seems to work well, but I'm not aware of any robust research done to validate the effectiveness of this.

I'm not sure about the exact details, but that's my understanding too. It is very exciting. This is directly replacing a week of work (if not more!) of a qualified ML engineer. Additionally, as all of the experiments are cheap (being run on a single CPU) you can run them on the cheap, interruptible, cloud instances, and it's not the end of the world if you need to restart some experiments.

This is also not mature science- there's still a lot of active research being done on AutoML- so there's still a lot of potential for improvement.

(I'm a researcher in ML/DL.)

> Anyone know why they used CPUs instead of GPUs/TPUs?

Cost and resource availability.

They're distributing each computation on a different CPU, not distributing each computation over multiple CPUs.

It would be faster to have each computation run on a CPU + GPU, but that would be very very expensive, and hard to schedule.

GPUs/TPUs are also only faster for sufficiently large networks and sufficiently large batch sizes. There's a large fixed cost to send data to/from the CPU, and for smaller networks, it's often not worth running it on a GPU. No idea if this was the case here.

A recurrent network can with ease.

For a vanilla MLP, you'd need to add in a few previous values (ie you'd predict x_{t+1} = f(x_t, ..., x_{t-N}), where f is your neural network and N is some fixed integer).

Ah, welcome to Alberta :)

A lot of small towns in Alberta only have Catholic hospitals (which are publicly funded, but run by Covenant Health [0], a Catholic institution). They refuse to provide, among other things, assisted suicide or abortion services despite being publicly funded. 4/9 of Edmonton's hospitals are run by Covenant Health.

[0]: https://en.wikipedia.org/wiki/Covenant_Health_(Alberta)

Assisted suicide has been legal in Canada for a few years now, and there haven’t been many major issues (other than Catholic hospitals refusing to comply with the law). Doctors aren't fools, and won't do it in cases that aren't blatantly acceptable. Most of the scenarios where it is desirable are cut and dry, and we have ethics boards/courts to handle the other circumstances.

Yes, absolutely. I'm an AI researcher, and most of my colleagues just use their desktop computers with a standard GPU to do their work. Of course, long-running jobs get put on the cloud, as do large distributed jobs, but those are surprisingly rare.

We just submitted a paper, for instance, that is entirely CPU based, and required running 4 CPUs for a few days to reproduce (and even then, you can reproduce 90% of the paper within minutes on a single machine).

It is capped, at $30k.

Thanks! Let me know if anything's unclear :)

Performing well on the interviews is a skill that you can acquire through practice. If you do 100 Leetcode questions, read through all of Cracking the Coding Interview, and suffer through 30 phone screens, by the end of it, you'll be a hardened interviewee capable of passing an interview anywhere in tech (you can probably get by with much less practice; I'm being purposefully hyperbolic).

Does this mean you'll be good at the job? No. Is this very wasteful? Yes.

Getting interviews, on the other hand, requires you to read the recruiter's mind, and can vary depending on what the recruiter had for breakfast, or if they fought with their significant other that morning. It's much less formulaic.

My background is that of an econometrician (ie quantitative economist), and I now work as a Research Engineer at one of the FAANG research divisions.

I think the advice about getting in as a hardware engineer is solid. At my workplace, there's a ton of need for people working on specialized hardware for DL, and for people working on the software that works with it (optimizing compilers, etc).

If you are looking to break into the software side of DL, the first two thirds of the Deep Learning book [1] contains all the math you need to know to pass the interviews. Then, it's just a matter of getting interviews; I found that I needed professional experience deploying DL/ML to do that. I got that by doing side projects at work. For instance, we had a long standing operations research problem, and I spent some free time at work implementing a RL algorithm to solve it. I didn't get too far, but I was able to talk coherently about the papers involved and about how I planned to conduct the project, which went a long way.

[1]: https://www.deeplearningbook.org/

The joke that I've heard is "performance (review) driven development".

If the way to get promoted is to launch a shiny new product, then your most senior people will be the best at finding shiny new products to launch, even if that's not the right technical decision to make.

(I'm also the partner of someone with severe nut anaphylaxis.)

Can you link me to anything? I was pretty discouraged by the result as the article explicitly says that the trial didn't work on adults:

"The treatment was not effective in the small number of adults enrolled in the study."

A lot of investors won't invest in companies that don't have large offices in the Bay. Investors also generally advise against having remote offices when you have less than 100 people. As a result, there's not really any reason that startups need offices there other than that it can be difficult to get investment if you're based elsewhere.

EDIT: I shouldn't say "the Bay", but rather, "the Bay/LA/NYC/Seattle". I more mean "top tier startups cities" rather than strictly just the Bay, per se.

I would honestly expect it to be more than 10x easier. In Canada, the idea of raising a 2-3 million round is completely foreign, and in my city (Edmonton) the idea is absurd. I strongly suspect you could raise 10x more much easier in the Bay compared to Edmonton, making it worth it to have an office down there, even though the cost of SWEs is ~4x cheaper here.

Yup, they did. They handed them out at their booth.

I agree, it sucks. I definitely would rather have worked with the rest of the team rather than leave as the core idea behind the company was pretty awesome.

Are you in this position currently? Are you trying anything different?

It's tough. I experienced that at a previous job, at a consulting firm, which prized Excel expertise over everything else (eg over data science skills with Python).

I just left, as I knew there was no easy way to argue.

I think you're in a tough spot. Unless you're very senior, and have buy-in from all the higher-ups, it's going to be almost impossible to change the course of the company. Unless you have a specific reason to stay, I'd move on. It'll be a huge battle that you're most likely going to lose, and even if you win, you'll be blamed for all the downsides of the solution you propose. That was my experience.

The Open AI solution uses demonstrations though, which is the article's point, that bare DQN can't solve the games, and something like demonstrations are needed.

At work, we have a bunch of vectorized computations that we run in Tensorflow, as it's a convenient way to get GPU-optimized code, so that is still an option (albeit an awkward one).

You could also use something like CUDA, or OpenGL to do this; there are some Python libraries to do basic numerical work, such as PyCUDA or gnumpy.

Technically, it's not a polynomial as you can have nonlinear activation functions, such as the ReLU function. There's no polynomial that's equal to a network with ReLU activations (although, of course, a sufficiently large polynomial could come arbitrarily close).

I would state that a neural network is a large, complicated, differentiable function, and the beauty of deep learning is that it turns out that by doing optimization that's derived from basic calculus, you can optimize this complicated function to do surprisingly useful things.

To be clear, it's not obvious that the paper you linked is actually accurate. A lot of researchers consider that paper to be deeply flawed, and to show something other than what it claims to.

Disclaimer: I am not a lawyer.

You could probably sue them, though. Have you spoken to a lawyer?

In this case, yes :)

u/stcredzero's example is also a great one, as over-regulation is a massive problem. However, it's not as much of an issue for higher paying jobs, which I think are more relevant when we're discussing high-end software engineers.

Note: I'm discussing over-regulation. There's a very strong case for heart surgeons to be highly regulated. There's a weaker case for having strict regulations for hairdressers.

There's two things called civil law:

1. The legal system [1], based on Roman law, that is common in former French colonies across the world (e.g. Quebec, Louisiana, France).

2. In common law legal systems, the branch of the law that deal with non-criminal matters [2].

[1]: https://en.wikipedia.org/wiki/Civil_law_(legal_system) [2]: https://en.wikipedia.org/wiki/Civil_law_(common_law)

Investment banking is one, another is corporate law. It's incredibly difficult to get a job in either field without a degree from a top school, with top grades. Google hires lots of people with degrees from state schools (or without degrees!). That's not the case at, say, McKinsey. Neither job inherently requires a degree from a top school, but it's almost a prerequisite to get the job.

I would certainly agree with you, but I don't think someone with a master's degree is necessarily an expert. What's important, IMO, is having practical experience in data science, which may or may not come with a graduate degree.

Post-docs, and adjunct professors make O($30k), which is less than what a subway conductor makes, presumably.

Yes, exactly. And most of the time, you can get by with a simple model like a random forest, so you just need to have a few analysts/engineers doing feature engineering, and feeding it into your model.

Almost everyone using more complicated architectures (e.g. neural networks) are adding unnecessary complexity. [1]

[1]: Unless you're doing computer vision.