Algorithms for Decision Making [pdf] | Hacker News Reader

Algorithms for Decision Making [pdf]

algorithmsbook.com

498 pointsmindcrime50 comments

Mykel Kochenderfer (one of the authors) is my uncle! Very friendly and smart guy. He helps run the Stanford Intelligent Systems Lab (SISL) and I've gotten to see some very neat projects like self-flying drones when visiting.

Could you tell more about self-flying drones. Are they part of some project / program? Is there a website?

He was one of my favorite professors (had him for CS238/AA238)! I still remember some of his project assignments fondly. I got a SISL shirt for being in the top three of the leaderboard on one of them. Inspired me to interview at Lincoln Labs (which I did not land, probably for the best :P)

It's so strange to see my first name attached to someone else!

Also, a related book by the same authors:

https://algorithmsbook.com/optimization/

I really like the way the handle their change log on github:https://github.com/algorithmsbooks/optimization

It's interesting just to go through their comments.

Interesting that here the typical textbook errata is called a "change log"...

It's also the textbook from Stanford course AA228/CS238. Haven't taken it but definitely looks interesting.

https://web.stanford.edu/class/aa228/cgi-bin/wp/

Since it's from MIT I will consider it must be of high quality, saved and will read. Thanks!

I do have a small issues with PDFs like this one, please give me a TOC on the left sidebar for easy navigation.

There is a TOC with the PDF and Chrome's PDF reader shows it in the sidebar. Other PDF readers should do the same.

Thank you, you're correct. Mine showed a list of preview of each page and I noticed there is another option on the left side for TOC now.

May be a little OT but came across this technique called MABs multi armed bandits , related to , finding optimal ways to decide among a sequence of actions and measuring their utility

https://en.m.wikipedia.org/wiki/Multi-armed_bandit

Has applications in a wide variety of fields : optimal design of clinical trials, public policy decision making.

apparently has some relation to the work abhijit banerjee & esther duflo, the nobel prize winning economist couple have done on design of experiments, to measure impact of developmental interventions

Not OT at all, IMO. As it happens, the linked book actually touches on MAB problems explicitly, albeit briefly. And as the book points out, you can formulate a MAB problem as specific form of Markov Decision Process, and MDP's are referenced extensively in the book.

I find MAB's incredibly interesting, and for pretty much the same reason(s) I find the rest of this stuff interesting.

This is not to be confused with Decision problems which is a class of problems that have a boolean output.

Yes, it's unfortunate that two somewhat different fields have such similar sounding names. "Decision Theory" (or "Decision Science") vs "Decision Procedures" (or "Decision Problems"). But what can one do?

If only there was a framework for making choices about actions.

I'd say you would not confuse the State Administration and the Department of Mathematics...

Tbf in a lot of cases you can use decision problems to solve, say, optimization problems with only polynomial slowdown. Hence their appeal.

That's how typesetting should be done.

It jumped out at me as a nice example of the Edward Tufte style (which I'm a big fan of -- I once published a tech report in that style). People have put together style sheets emulating it for LaTeX [1][2], CSS [3], and R Studio Markdown [4], among others.

[1] https://tufte-latex.github.io/tufte-latex/

[2] http://tug.ctan.org/tex-archive/macros/latex/contrib/tufte-l...

[3] https://edwardtufte.github.io/tufte-css/

[4] https://rstudio.github.io/tufte/

The authors open-sourced their setup for Tufte-style books: https://github.com/sisl/tufte_algorithms_book

Do you mean the chosen page layout, the attribution of different classes of information in different specific parts of the page?

It does look quite clear, well thought and well structured.

I think s/he meant the typography, the quality of the drawings and the big right margin allocated for the figure labels and the sidenotes.

Oh how I wish I had this document 7 years ago. I was knee deep in utility curves and felt like I was on the path to madness. I did find a solution, but could not convince anyone that I wasn’t just making it all up!! Oh well…

Just use ML now

Just curious is there an algorithm based "rational style" novel based on people using algorithms? I'd be interested in an entertaining read like that.

Not a book, but I really like the "Moneyball" movie (2013) for that reason. They design an algorithm and use it to manage a team.

It was actually a book first! I enjoyed all of Michael Lewis's books that I read.

The book goes into more details around, well, a lot of things.

I read the book before seeing the movie; as is often the case, you’re better off doing it the other way around so you can better enjoy the film.

No rational style novel but Fortune's Formula was a good non-technical narrative intro to one aspect of this.

In my opinion the book is horrible. It doesn’t provide any in depth details. It functions as a high level survey of the field. It is very superficial and just scratches the surface. It doesn’t describe the presented algorithms in any detail. It’s a very poor version of Daphne Kollers amazing book on graphical models.

These broad survey type classes were the most useful ones in university for me. It's not like I would remember the specifics after the class anyway, but when I run into a problem I now have a huge bag of potential solutions. When I need details, there are often scientific papers or technical reports going in-depth on each of these things.

What's wrong with a survey? Sometimes this is exactly what people want. Witness the popularity of Numerical Recipes, for example.

It's hard to go into detail in "all-in-one" books.

I read the title and introduction, seems like this is a similar book: https://en.wikipedia.org/wiki/Thinking_Strategically - feat. my econ teacher's favorite authors

It's interesting how one algorithm, a 'master algorithm', can presumably subsume all the others in the book, presumably a neural/evolutionary algorithm, that can simply learn/evolve when the other algorithms are useful for decision making/maximizing reward.

The more assumptions you relax, the more general the algorithms become, for example going from immediate reward to delayed reward means going from supervised to reinforcement learning.

The trade-off is the more general algorithms needs many times exponentially more data and compute to come to a similarly good solution.

That's why reinforcement learning has seen so practical few applications relative to supervised learning. There's no free lunch.

That said, as a ML practitioner I would love it if I could just apply a single master algorithm to all problems, but that is likely many years away.

Neural nets only need linearly more data for optimal performance: https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scalin...

At the same time, fine-tuning sample efficiency increases with scale, so at some point you can possibly one-shot learn state and get rid of exponential searches, solving NP-Hard problems with heuristics. Sounds like a free lunch to me. At least if you can afford a net large enough.

It sounds like the more general the algorithm, the more stateful it needs to be before it can be useful. On the other hand, specialized algorithms need less to zero state but have limited applications.

Julia's extensive use including relevant comments is noteworthy. In particular, the use of the mathematical notation of the variables facilitates the understanding of the formulas. At the same time it helps to learn Julia.

From the title alone, I was expecting something related to General Morphological Analysis. [1]

[1] https://www.swemorph.com/ma.html

Judging from that name, I thought that your link would be about something that would allow me to inflect words in various natural languages. ;)

From glancing over the contents, it looks like it's about reinforcement learning. Am I right?

Among other things.

How is the different from mental models? They have a lot of overlap

this book is a wonderful primer on all things related to MDP based algs. That's a very specific subfield of computer science that can work wonders in some domains.

Calculus finally seems useful when reading this.

Welcome newcomer. The path is clearer from now on.

I'd love an audiobook version!

Ouch, site shot down?

Reply on news.ycombinator.com