rasmi

See: https://csvchain.com/#faq

Congratulations on your success! I'm actually familiar with the author's past talks and research, and am not just assuming he's competent because he lists his affiliation with Princeton.

I encourage you to familiarize yourself with the field of socio-technical systems. It is related to (but not the same as) "ML/DL", and it is important to know about if you are doing research in CS. A good place to start is the FAT* conference [0] (which was previously a workshop at NeurIPS).

Regarding manual scoring: The author cites this study [1] and specifically says: "This is a falsifiable claim. Of course, I’m willing to change my mind or add appropriate caveats to the claim if contrary evidence comes to light. But given the evidence so far, this seems the most prudent view." so by all means, do reach out to him with better evidence.

[0]: https://fatconference.org/2019/program.html

[1]: https://arxiv.org/abs/1702.04690

Not to indulge the troll, but Arvind Narayanan is an (associate) professor of CS at Princeton and is one of the foremost researchers in the field on topics of ML/data privacy and ethics [0]. His papers/talks/tweets regularly attract attention on HN [1]. That you're judging the talk based on which conferences the author hasn't published in says more about your ignorance of the STS field than it does about the author's knowledge of the topic. This is top-notch content!

[0] https://scholar.google.com/citations?hl=en&user=0Bi5CMgAAAAJ...

[1] https://hn.algolia.com/?q=random_walker

Code here: https://github.com/google/robotstxt

Here are two more great articles about Transformers:

The Illustrated Transformer (referenced in the parent): http://jalammar.github.io/illustrated-transformer/

The Annotated Transformer: http://nlp.seas.harvard.edu/2018/04/03/attention.html

Source article here: https://www.zdnet.com/article/amazons-cloud-computing-will-s...

For those interested, you can track Beam Python 3 support progress here:

https://issues.apache.org/jira/browse/BEAM-1251

The paper is "ImageNet Classification with Deep Convolutional Neural Networks" by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, and is available here:

https://papers.nips.cc/paper/4824-imagenet-classification-wi...

The search features discussed in the article are now available through Google Cloud Source Repositories: https://cloud.google.com/source-repositories/docs/searching-...

I link these resources often, but they are often relevant! See "The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction" [1] and the Rules of Machine Learning [2]. Another classic: "Machine Learning: The High Interest Credit Card of Technical Debt" [3], and recently added: Responsible AI Practices [4].

[1] https://ai.google/research/pubs/pub46555

[2] https://developers.google.com/machine-learning/rules-of-ml/

[3] https://ai.google/research/pubs/pub43146

[4] https://ai.google/education/responsible-ai-practices

>It is expected that the Department of Electrical Engineering and Computer Science (EECS), the Computer Science and Artificial Intelligence Laboratory (CSAIL), the Institute for Data, Systems, and Society (IDSS), and the MIT Quest for Intelligence will all become part of the new College; other units may join the College.

http://news.mit.edu/2018/faq-mit-stephen-schwarzman-college-...

It is noted in this FAQ that CSAIL will become part of the new college.

http://news.mit.edu/2018/faq-mit-stephen-schwarzman-college-...

An Introduction to Modern Astrophysics by Carroll & Ostlie is the Bible, but it is quite large physically and in scope and better serves as a reference for most people.

For various topics, I would look at:

Introduction to Cosmology by Ryden for cosmology at the undergraduate level.

Cosmology by Weinberg.

The Exoplanet Handbook by Perryman.

An Introduction to Modern Stellar Astrophysics by Carroll & Ostlie.

Particle Astrophysics by Perkins.

Modern Statistical Methods for Astronomy by Feigelson.

Statistics, Data Mining, and Machine Learning in Astronomy by Ivezic.

I can post more in other topics if anyone is interested.

Here's a neat use case analyzing GDELT data: https://blog.gdeltproject.org/peering-into-the-visual-landsc...

Video explanation here: https://www.youtube.com/watch?v=n_pa2AisRUs

Actually, the Google guides were initially published in February (and publicly available before that from Martin Zinkevich since 2016!). But I agree, it's great to see more resources around these best practices.

Hi Vincent, you may want to point your "Best Practices for ML Engineering" link to the non-PDF version here: https://developers.google.com/machine-learning/guides/rules-...

Give "Designing Data-Intensive Applications" a read. It's an excellent book on these topics.

https://dataintensive.net

If you're interested, this project is open source!

https://github.com/tensorflow/models/tree/master/research/as...

https://ai.googleblog.com/2018/03/open-sourcing-hunt-for-exo...

For comparison, here is a look at Waymo's simulation operations: https://www.theatlantic.com/technology/archive/2017/08/insid...

If you're interested in this, be sure to read "The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction" [1] and the Rules of Machine Learning [2].

[1] https://ai.google/research/pubs/pub46555

[2] https://developers.google.com/machine-learning/rules-of-ml/

And if you want to just load generic pre-trained classifiers for a variety of tasks, ml5js (which uses TensorFlow.js under the hood) seems promising!

https://ml5js.org/docs/quick-start

I watched the whole talk, so I heard the bit about the IDE, but I still think there's a really fundamental ability of being able to walk through the "decision-making logic" of your "code" (in this case, model) that wasn't touched upon. For example, suppose your model misclassifies a barrier and a car crashes into it as a result [1]. How do you debug this? You can say, "Well, it's a data-labelling problem" and go get more data on barriers, but in the meantime people have died. Model testing and debugging should be an incredibly high priority for use cases like Tesla's. That means some degree of interpretability, testing edge cases, simulation, anything to find flaws like this before they occur in real life.

See here [2] for an example of production ML testing practices. I wonder how much of this is in place at Tesla? I would argue they should be at the forefront of work like this. Something tells me they aren't.

[1] https://news.ycombinator.com/item?id=17257239

[2] https://ai.google/research/pubs/pub46555

One downside of Software 2.0 as compared to 1.0, at least as of today: it is incredibly hard to debug in the conventional sense. The focus of this talk was mostly on data-labelling challenges. For a company with software as mission-critical as Tesla, I'm disappointed Andrej did not bring up any of the practical challenges around debugging complex models.

These are not revolutionary or even helpful ideas, but they can be convenient or fun. For example:

Some medical applications (just for example, you would need quite a robust model):

* Categorize moles on your skin as cancerous or not.

* Categorize cuts on your skin as infected or not.

* Have the user input certain characteristics (images, temperature data, etc) and give some preliminary diagnosis.

Less interesting/helpful:

* Detect photos of certain foods in your app (e.g. Twitter, Instagram, Yelp) and recommend relevant emoji or hashtags.

* For a note-taking or to-do app, classify notes into categories automatically for the user.

* Suggest actions in your app based on what the user types or does.

* Recommend solutions or help articles from the feedback form in your app. Tag the feedback with a certain sentiment to determine how quickly you should follow up on it.

* Detect your products in images/videos and tag them to make them searchable, and recommend relevant things to the user (e.g. in a beer tracking app [2], detect a certain kind of beer, suggest similar types).

If any of this sounds like it's been done before, it probably has, but it's important to note this is done entirely on device (private!) and with custom labels. The current alternative is to use TensorFlow Lite [1], which is a bit more involved. I'm sure as the field develops we will see more creative (and useful/helpful) applications.

[1] https://www.tensorflow.org/mobile/tflite/

[2] https://untappd.com/

Core ML is bring-your-own-model (or use the built-in classifiers, which have generic labels). This is an easy way to train custom models (for example, on images from your own product catalog).

This looks like a really easy way to integrate your own custom ML models into your app. I imagine this would appeal to a lot of hobbyist developers, especially given how simple it is.

The docs [1] also seem to imply they're using transfer learning from more robust models for their image classifier: "Use at least 10 images per label for the training set, but more is always better." and "Create ML leverages the machine learning infrastructure built in to Apple products like Photos and Siri."

[1] https://developer.apple.com/documentation/create_ml/creating...

To me this doesn't mean "Sharing traditional AI research is dangerous," but rather "Our efforts may be best spent on less popular topics such as safety, policy, and standards research, so you may see us do less traditional AI research." This seems in line with their stated goals around safety.

Great MoMA exhibit here: https://www.moma.org/interactives/exhibitions/2011/AccesstoT...

It's meant to help evaluate the performance of a TensorFlow model after training. In particular, you can define a series of metrics to measure, and different subsets of the data to evaluate on. And it can scale up to evaluate on large datasets since it uses Apache Beam for distributed data processing.

This notebook here shows some helpful cases: https://github.com/tensorflow/model-analysis/blob/master/exa...