Many math questions are easy to verify, and it's a classic benchmark for reasoning -> so it's a good hill to climb.
I agree with your meta-point that better benchmarks testing more types of task would be good!
62 karma · joined April 12, 2018
I agree with your meta-point that better benchmarks testing more types of task would be good!
(Also the term “approximate retrieval” is a bad one - reasoning is inherently a process of chaining together associations. What matters is whether the reasoning reaches the right conclusions. Still some way to go, but already very impressive in tasks traditionally considered harbours of human reasoning!)
- The initial use of data is distillation so we’re less bound by question quality (anything that evinces output diversity is good).
- But moving onto RL, we’ll need stronger quality. We have much better things planned both on data filtering and verification!
- Surprisingly, a lot of ML datasets actually look like this when you look under hood. We’re hoping having more eyeballs on it will help improve quality in long run over less transparent status quo!
Our mission is to systemise the world's deep learning knowledge.
We are building a discovery platform and tooling for deep learning that links ideas with implementations, and allows everyone to access and apply the state of the art in AI. We are a team of researchers and engineers from the University of Cambridge, and our founders include an early core developer at Wikipedia and an open source machine learning developer.
We are looking for a full stack developer to help us develop our popular and growing platform. Our tech is built on Python (Django) and React. Additionally we have a number of other roles - including product and advocacy roles.
Learn more here www.atlasml.io or apply at https://angel.co/atlas-ml/jobs
One problem we face is how unmodular deep learning code can be, meaning that adapting someone else's code for your own dataset, your own workflow, your own way of training, and so on, can be a real pain.
We made an open source resource and framework to make it easy to combine and train deep learning models and datasets, with a heavy focus on modularity. Right now we have a few example models and datasets - mainly GANs! - that you can play with on the site.
We invite you to contribute to this new open source resource. We will be also adding a lot of content in the coming weeks. Of course, if you have any feedback that would be great too.
Mantra is a deep learning development kit that manages the various components in an deep learning project, and makes it much easier to do routine tasks like training in the cloud, model monitoring, model benchmarking and more. It works with your favourite deep learning libraries like TensorFlow, PyTorch and Keras.
This is an very early alpha release, so your comments on the general concept and what you like or don't like about it would be awesome!