Ask HN: What are some of the best papers on AI, ML, DL and their applications? | Hacker News Reader

Ask HN: What are some of the best papers on AI, ML, DL and their applications?

53 pointsnoob_eng8 comments

I was reading the Deep Double Descent paper by OpenAI: https://arxiv.org/abs/1912.02292. The writing is so lucid. Even the structure of the paper is non-conventional. They have a results section before the Related Works section to give sneak peak of what the paper is about. More like telling a story. Even before starting the Introduction there is a self explanatory image spanning the entier two columns.

Can you link to more papers that are written in such style and are easy to read if you have the background knowledge?

View on HN

The paper introducing support vector machines by Cortes & Vapnik was written exceptionally well in my opinion. It tells a part of the story of 60 years of pattern recognition (ML) succinctly from Fisher in 1936 to 1992.

https://link.springer.com/content/pdf/10.1007/bf00994018.pdf

A Mathematical Theory of Communication by Claude Shannon (https://people.math.harvard.edu/~ctm/home/text/others/shanno...)

This one is about compression, reliable communication. While interesting and well-written, I don't think it matches the original request for "papers on AI, ML, ...".

> "I don't think it matches the original request for "papers on AI, ML, ..."."

I feel like most who understand this paper and who also understand AI and ML will disagree.

I had a good quick skim. It has a language model (ok trigrams!) and the cross entropy formula and reasoning for it. On my reading list for sure! We did some information theory at uni but don’t recall all of this stuff, maybe I just forgot.

Research paper on AI doesn't necessarily mean a meme title and chasing a dubious "sota" status on some benchmark. I would say foundational work is more worth reading.

I'm not sure if it's truly a timeless paper, but "Attention is all you need" by Vaswani et al. has been super influential in recent years. Also "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" by Sohl-Dickstein et al. (about Diffusion Models) and "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" by Mildenhall et al. were hugely influential to me.

https://paperswithcode.com

Reply on news.ycombinator.com