Hacker News Reader

Tangentially related: It appears that TD ideas pop up in diffusion models, VAEs and neural net training dynamics. Any author/reading advice on links between thermodynamics, information, and neural nets?

View on HN

Since you mention VAEs: you might be interested in Active Inference, which is built on minimizing variational free energy. Same E−TS form as the thermodynamic kind (Helmholtz free energy), so it does fit your thermo/info/NN question. And the negative free energy is equivalent to the evidence lower bound used in generative models like VAEs. Lots of really fascinating connections between all of these topics :)

There's a free 2022 textbook: https://direct.mit.edu/books/oa-monograph/5299/Active-Infere...

And a brand new one (though it's a bit pricey): https://mitpress.mit.edu/9780262050951/fundamentals-of-activ...

Thank you! Parr, Pezzulo & Friston looks like the kind of book I'm after - love cross-disciplinary works. After all, knowledge and nature are continuous. It's humans who like to chop them up into subfields :)

Reply on news.ycombinator.com