dennybritz

I do want to release the code, but it may take a couple more weeks. It's currently a mess of research code that's mixed with some other stuff I was working on so I would need to do a cleanup first. See you on the boat!

I think there are some key differences to cycles in Chess. In Chess, the cycles come from your (or your opponents) actions. With minimax you would max (or min) over these actions, so if you pruned away the cycles at some depth your result is not incorrect because the min or max does not change. But here the cycles come from randomness in the game of which chess has none, and you need to take an expectation over the cycles. So you can't just prune them away without changing the result.

A scoring function could definitely help guide the exploration and/or prune the tree, but only at the action nodes, not the environment randomness nodes. Rolling out the full tree more than 1-2 levels would be infeasible because of the randomness in the environment. When you take an action, the randomness can transport you into an exponential number of states, so you have a huge branching factor that is much larger than chess. I think in chess you have a factor of 40ish? Here it's more like ~1000 or ~10,000 depending on the item.

I also wouldn't know how to design a scoring function for this. If you do something simple to take the number of missing modifiers you will end up stuck in bad states. Maybe there is something really clever here that you can do, but I don't know what it is.

If you have an idea how to make tree search work for this I'd love to try it.

Thank you Exile :) I'm glad you enjoyed the read!

Remote Only

Location: Japan (or East Asia in general)

Willing to Relocate: No

Technologies (reverse-chronological order):

    - AI / Deep Learning Research - previously at Google and have published papers. Mostly focused on NLP and RL, but I keep up with other subfields.
    - Infra: Devops, Rust, Go, Kubernetes, Microservices, large-scale systems, all kinds of data stores. Have managed large clusters. Used to be an early Apache Spark engineer and was in a database research group in grad school
    - Worked in HFT-style algotrading for a few hedge funds
    - Worked at multiple early-stage startups, so I can do other things like full-stack web or app development, but I prefer not to do these full-time. But I can help if stuff comes up.

Résumé/CV:

    https://dennybritz.com/about
    https://twitter.com/dennybritz
    http://github.com/dennybritz
    dennybritz [at] gmail

--

Hi! 15+ years of engineering experience, and have been through a lot of technology cycles. I'm in an ok place right now and focusing on research and side projects. I'm not actively looking for work but if there's something at the intersection of my interests I'd love to talk. Not sure myself what that would look like, perhaps something around MLOps, infra/automation, Reinforcement Learning, Algorithmic trading, etc.

Remote Only

Location: Usually Japan/East Asia, but currently in Europe due to COVID

Willing to Relocate: No

Technologies (reverse-chronological order):

  - AI / Deep Learning Research - previously at Google and have published papers. Mostly focused on NLP and RL, but I keep up with other subfields.
  - Infra: Devops, golang, rust, kubernetes, microservices, large-scale systems, all kinds of databases. Have managed large clusters. Used to be an early Apache Spark engineer and was in a database research group in grad school.
  - Briefly worked in algo trading (HFT-style)
  - Worked at multiple early-stage startups, so I can do other things like full-stack web or app development, but I prefer not to do these full-time. But I can help if stuff comes up.

Résumé/CV:

  https://dennybritz.com/about
  https://twitter.com/dennybritz
  http://github.com/dennybritz
  dennybritz [at] gmail

---

Hi! 15+ years of engineering experience, and have been through a lot of technology cycles. I'm in a decent place right now and focusing on research and side projects. I'm not actively looking for work but I figured I would post anyway - who knows what opportunities come along! If there's something at the intersection of my interests I'd love to talk. Not sure myself what that would look like, perhaps something around ML/RL research, academia, trading, or infrastructure.

Yup. The only reason I am starting to write about it now is that I am no longer running the system. You could argue that it's not useful to write about systems that worked in the past, but I would disagree. New systems can work 99% the same way, but get an additional edge from somewhere else, like new data or better models. Most of the engineering will always be the same.

> Just because you have a good forecast doesn't translate into cash. It has to be paired with a trading strategy. This is probably why the author thinks the answer is RL, because coincidentally if you approach this problem with RL, it does the forecasting + strategy.

Exactly, this is one of the nice things about RL. You don't to do a bunch of handwaving to turn your predictions into a strategy.

I think you're comparing apples to oranges here. These funds manage billions of dollars of client money, which forces them into highly liquid markets with scalable strategies. That's quite different from how individuals or smaller prop funds can operate, trading off capacity for higher returns by trading in less liquid markets or with strategies that are "not worth it" for large hedge funds. If you must manage billions of client money then you are right in terms of competition, but as someone who only trades his own capital, you can see a lot higher returns.

A paper with more details can be found here: https://arxiv.org/abs/2006.00979

Remote Only

Location: Usually Japan/East Asia, but currently in Europe due to COVID

Willing to Relocate: No

Technologies (reverse-chronological order):

  - AI / Deep Learning Research - previously at Google and have published papers. Mostly focused on NLP and RL, but I keep up with other subfields.
  - Infra: Devops, golang, rust, kubernetes, microservices, large-scale systems, all kinds of databases. Have managed large clusters. Used to be an early Apache Spark engineer and was in a database research group in grad school.
  - Briefly worked in algo trading (HFT-style)
  - Worked at multiple early-stage startups, so I can do other things like full-stack web or app development, but I prefer not to do these full-time. But I can help if stuff comes up.

Résumé/CV:

  https://www.linkedin.com/in/dennybritz/
  https://twitter.com/dennybritz
  http://github.com/dennybritz
  dennybritz [at] gmail

---

Hi! 15+ years of engineering experience, and have been through a lot of technologies and cycles. I'm in a decent place right now and focusing on research and side projects. I'm not actually looking for work. But I figured I would post anyway - who knows what opportunities come along! If there's something at the intersection of my interests I'd love to talk. Not sure myself what that would look like, perhaps something around ML/RL, research, infra, or possibly trading.

I posted this because it was recommended to me several times in [0], together with several other "computational approaches to Physics" books, and thought it would be interesting to HN users. If you're looking for more books like this, the whole Twitter thread is worth a read. It's full of good recommendations.

[0] https://twitter.com/dennybritz/status/1260137814982787073

Remote Only

Location: Variable. Usually Japan or Asia, but currently in Europe due to COVID

Willing to Relocate: No

Technologies (reverse-chronological order):

  - AI / Deep Learning Research - previously work at Google and have published papers. Mostly focused on NLP and RL, but I keep up with other subfields.
  - Infra: Devops, golang, rust, kubernetes, microservices, large-scale systems, all kinds of databases. Have managed large clusters. Used to be an early Apache Spark contributor and was in a database research group in grad school.
  - Briefly worked in algo trading (HFT-style)
  - Worked at multiple early-stage startups, so I can do other things like full-stack web or app development, but I would prefer not to do these professionally anymore.

Résumé/CV:

  https://www.linkedin.com/in/dennybritz/
  https://twitter.com/dennybritz
  http://github.com/dennybritz
  dennybritz [at] gmail

---

Hi! 15+ years of engineering experience, have been through a lot of technologies and cycles. I'm in a decent place right now focusing on research and side projects and not actually looking for work, but I'm slowly getting bored. I figured I would post anyway - who knows what opportunities come along! If the right thing hits I may be interested. Perhaps something around ML/RL, research, infra, or possibly trading. I'm not sure myself :)

If your goal is to learn, I believe this is THE way to do it. My experience has been that reading about something complex without attempting it first isn't effective. If you are not familiar with the problems you won't understand or appreciate the solutions. It's like listening to a lecture about rocket science without a background in Physics - there is useful information, but you won't be able to process or connect them to anything you know. And that also means you won't retain the information.

Of course there is a limit to this. It wouldn't make sense to intentionally avoid looking up solutions if you are stuck on a specific problem for several hours or days. But IMO it also doesn't make sense to read a whole textbook without trying to implement anything. What has worked well for me is to do it in an iterative fashion. Read when necessary.

You will never know the "generative process" of a paper, which makes it nearly impossible to properly evaluate it, even as an expert reviewer. If you are not a reviewer (or working on a competing paper), you are much better off NOT reading the newest papers. Instead, rely on the best proxy metric of all: Time. Wait until the paper has been battle-tested. See if people on social media are talking about it, what critics are saying, wait for reproduction results and open-source implementations, conference acceptances, and citations and comparisons. These give you a much better idea of the validity of the paper than its content can.

When reading the newest ML papers, I found it useful to not judge them, but instead use them as inspiration. Forget about the results. The paper may contain interesting ideas or viewpoints you didn't consider before, and those are probably much more valuable than the result table.

No opinion on CrossFit, but if you've ever worked in Academia you realize that it's more of a cult and spreader of misinformation than any other establishment. Not always on purpose, but that doesn't matter.

Some reasons that come to mind:

1. Most of the advances do not result in large enough gains to justify them being translated into industry. 99.9% of research papers propose techniques that result in small gains in the optimization metric (accuracy, ROC AUC, BLEU score, etc). However, this comes at the expense of added cost in complexity, more expensive training, model instability, challenges in code maintainability, and so on. For the vast majority of companies, unless you are Google AdWords or Google Translate, a tiny gain in metric X is not worth the costs mentioned above. You're much better off using proven off-the-shelf models that have stood the test of time, are fast to train and easy to maintain. Even if they are 1% worse.

2. Research tends to focus on model improvements and you are not allowed to touch your train/test data. That makes sense as otherwise competing approaches would not be comparable. However, in the real world you have the freedom of collecting more training data, cleaning your data, selecting more appropriate validation/test data, and so on. The vast majority of times, getting better/cleaner/more data beats getting a slightly better model. And it's much easier to implement. So for industry it often makes more sense to focus on that.

3. Metrics optimized in research papers rarely translate into real world business metrics, but many research ideas are overfit to those metrics and/or datasets. For example, translation papers optimize something called BLEU score, but in the real world the thing that matters is user satisfaction and "human evaluations", which cannot easily be optimized in research. Similarly, no business sells "ImageNet recognition accuracy". Research overfits to this metric on this dataset (because that's how papers are evaluated) but it's not obvious that a model doing better on this metric will also do better on some other metric or dataset, even if they are similar. In fact, even datasets that are known to contain errors are still used as-is, because they have always been used.

Prediction Machines | DevOps/Backend/ML/Engineer | Tokyo, Singapore, Bangkok (REMOTE possible) | Competitive salary depending on location

At Prediction Machines, we're applying Deep Learning and Reinforcement Learning techniques to trading in financial, cryptocurrency, sports bedding, and other commercial markets. We're well-funded and have a team of strong researchers, engineers, traders, and a management team with decades of finance background.

We're looking for someone to help build out the infrastructure for a new vertical. Ideally, you would be familiar with cloud services like AWS, Docker,Kubernetes, streaming data infrastructure like Kafka, and modern programming languages like Go, Node, Python etc. Having Machine Learning, Data Science, or Finance knowledge is a plus, but not required.

To minimize time zone differences we're ideally looking for someone in Asia. We have people in Tokyo, Singapore, and Bangkok. Remote work possible for the right candidate.

Please send a brief into and resume directly to "denny.britz@prediction-machines.com"

Prediction Machines | Backend/ML/DevOps Engineer | Tokyo, Singapore, Bangkok (REMOTE possible) | Very competitive salary depending on location

At Prediction Machines, we're applying Deep Learning and Reinforcement Learning techniques to trading in financial, cryptocurrency, bedding, and other commercial markets. We're well-funded and have a team of strong researchers, engineers, traders, and management team with decades of finance background.

We're looking for someone to help build out the infrastructure for a new vertical. Ideally, you would be familiar with cloud services like AWS, Docker and container orchestrations solutions, streaming data infrastructure like Kafka, and modern programming languages like Node, Go, Python etc. Having Machine Learning, Data Science, or Finance knowledge is a plus, but not required.

To minimize time zone differences we're looking for someone in Asia. We have people in Tokyo, Singapore, and Bangkok. Remote work is also possible as long as you're in a nearby time zone.

Please send a brief into and resume directly to "denny.britz@prediction-machines.com"

The other big "hardcoded" constraint is most likely the usage of the bot API. The API itself is complex with lots of functionality and I would assume that researchers were extremely smart about picking out just the right API calls that are needed to make it work. That's very different from and much easier than training an agent based on raw keyboard/mouse input.

I agree. The point I was trying to make is not whether it is impressive or not (it is impressive!), but that the general press hype about "AI breakthroughs that will soon kill us all and need regulation" is over the top and misleading. The probably massive engineering effort that went into this is, to me, more impressive than the algorithmic innovation, if any.

Progress is made with small incremental improvements, including this one, and there have been few real algorithmic "breakthroughts" over the past few years. That's why I think it is important to give some perspective to the hype.

Author here. I agree this is an assumption, but based on my experience it is very unlikely that this is trained on pixels. Training would've been orders of magnitude more expensive. If it really is trained on pixel input I would be shocked and extremely impressed, and parts of post would not apply.

This has been said before by another commenter, but I'll say it again because I think it really is true: Most likely you will never regret taking the year off, but in a few years you probably WILL regret never taking that year off.

> I am worried that I will seem less attractive to future employers when I return

That's not my experience at all. Me and many people I now have huge gaps in their work history, either from travel or startups, and it hasn't hurt anyone.

Also, it's not a black and white decision. Start taking time off and go travel. If you decide it's not for you after a few months just go back early.

This is a good question and we should probably add this to the FAQ. tf.contrib.seq2seq is a low-level library that you can use to build seq2seq models; it is used internally by this project. The key difference here is that the google/seq2seq is an end-to-end pipeline that you can run with your own data and that comes with a lot of bells and whistles.

Yes, these models can applied to a lot of non-NLP tasks. For example, I've seen seq2seq models applied to medical record prediction, program generation, etc. Noise removal seems like a good candidate.

A lot of recent research uses RL to "fine-tune" NLP models. A practical example would be Google's recently announced Machine Translation System (https://arxiv.org/abs/1609.08144). It uses RL to directly optimize BLEU scores on translated sentences.

You'll find similar applications in state-of-the art models for chatbots for example. Though I agree, "widely used" may be somewhat of an overstatement. But it's becoming more common.

On a side note, I actually think RL makes a lot of sense for many NLP problems and it would be super interesting to build a pure RL approach to language modeling or translation. Nobody has managed to do that quite yet.

Another reason may be that this is a pure "resource" post that doesn't make an argument or represents personal opinions that people could easily comment on. It's not a good basis to start a discussion, unlike many other HN posts.

However, I'd appreciate more comments of course ;)

These libraries are not mutually exclusive, they operate on different levels of abstraction. For example, you can use Keras with either Theano or Tensorflow. cuDNN is an interface to the hardware and is used internally by most libraries.

Breaking it down. High-Level Frameworks:

- Caffe is very high-level and almost only used for Convolutional Neural Networks. It doesn't have good support for RNNs or anything else. It has a very good collection of pre-trained models (model zoo).

- Keras is a "wrapper" around Tensorflow or Theano and includes many higher-level abstractions like various types of layers, optimizers, etc. It's typically what I recommend to anyone who wants to get something up and running quickly and doesn't necessarily want to develop novel models.

On the next lower level are Theano and Tensorflow. They are pretty much competing with each other and have a very similar computational model (computational graphs). People/Companies seem to be moving towards Tensorflow, so that what I'd recommend using at this point. Tensorflow recently added several higher-level abstractions (like TF Learn and contrib modules) that are quite similar to those in Keras.

cuDNN is a library for GPU acceleration. It's used by most of these libraries under the hood to speed up computation. You certainly can use cuDNN directly, but unless you're doing low-level research it's probably not necessary.

It's both. Most of the Reinforcement and Deep Learning techniques used in AlphaGo have been around for many years and while there are a couple of novel algorithms many overestimate their impact. It's most a mix & match of techniques that hasn't really been done before. Plus good engineering.

IMO compute power (GPUs) played the bigger role. DeepMind themselves say that training AlphaGo wouldn't have been possible without access to Google's large-scale infrastructure. They've been training it on thousands of machines simultaneously. That's not to be confused with the hardware necessary to play the game, which isn't much and can be done on a single machine. Only the training phase has these extreme hardware requirements.

Shameless plug, but I am curating a Deep Learning / Artificial Intelligence newsletter: http://www.wildml.com/newsletter/

Funnily enough, I just received an email promothing this: http://csplusmh.com/