HNHacker News
TopNewBestAskShowJobs

_hark

326 karma · joined May 17, 2020

Mathematics @ Oxford
submissionscomments
_hark··on MeshGPT: Generating triangle meshes with decoder-only transformers
NNs are typically continuous/differentiable so you can do gradient-based learning on them. We often want to use some of the structure the NN has learned to represent data efficiently. E.g., we might take a pre-trained GPT-type model, and put a passage of text through it, and instead of getting the next-token prediction probability (which GPT was trained on), we just get a snapshot of some of the activations at some intermediate layer of the network. The idea is that these activations will encode semantically useful information about the input text. Then we might e.g. store a bunch of these activations and use them to do semantic search/lookup to find similar passages of text, or whatever.

Quantized embeddings are just that, but you introduce some discrete structure into the NN, such that the representations there are not continuous. A typical way to do this these days is to learn a codebook VQ-VAE style. Basically, we take some intermediate continuous representation learned in the normal way, and replace it in the forward pass with the nearest "quantized" code from our codebook. It biases the learning since we can't differentiate through it, and we just pretend like we didn't take the quantization step, but it seems to work well. There's a lot more that can be said about why one might want to do this, the value of discrete vs continuous representations, efficiency, modularity, etc...

_hark··on Hired by Google as L4 but rejected by top colleges
https://archive.is/HgqAy
_hark··on Deepmind Alphadev: Faster sorting algorithms discovered using deep RL
Sorry, but this is just incorrect. Go players have gotten stronger over time overall [0][1], and AI discovered many new ideas that all top pros have incorporated into their game-play (idk how to give a source for this, it's just very well known in the Go community that the style of play changed drastically in response to AlphaGo, and absolutely everyone trains with AI these days).

[0]: "The sudden overall increase in agreement in 2016 also reinforces the belief that the introduction of powerful AI opponents has boosted the skills of professional players." https://ai.facebook.com/blog/open-sourcing-new-elf-opengo-bo...

[1]: https://www.goratings.org/en/

_hark··on 4000x Speedup in Reinforcement Learning with Jax
jax.vmap() is all you need?
_hark··on An intutive counterexample to the axiom of choice
These kinds of non-constructive results are almost always present when you invoke choice (when it's actually relevant), and this is a well-known example of the non-intuitiveness of choice. If you don't like this, you're probably a constructivist and may also disagree with the law of the excluded middle. A classic quote/joke on the topic:

"The Axiom of Choice is obviously true, the well-ordering principle obviously false, and who can tell about Zorn's lemma?"

_hark··on AI Learns to Play CS:Go via Large-Scale Behavioural Cloning
This is basically a re-implementation of the recent work from OpenAI[1], where you first collect high-quality observation-action-observation transition tuples, train a model to predict the actions between observations, then use that trained model to label a much larger observation-only observation dataset (e.g. youtube videos of people playing a game).

Finally, you just do dead-simple behavior cloning (predict "expert" action from current observation) on the large labeled dataset. It's still surprising to me how well this works! Behavior cloning has some theoretical issues since it doesn't address the sequential nature of decision problems at all, but apparently with enough scale it can still do fairly well - super cool.

[1]: https://openai.com/blog/vpt/

_hark··on Estimating PaLM's Training Cost
DeepMind to OpenAI and everyone else[1]:

> your hyperparameters are bad and you should feel bad

It's amazing to me that such a big goof was missed by so many for so long. All these multimillion dollar language models and people just took the scaling laws at face value.

[1]: https://arxiv.org/abs/2203.15556

_hark··on USB-C hubs and my slow descent into madness (2021)
Would be great to see a "Hacker News Approved" buyer's guide for various products. Like Wirecutter but written by people who actually know what they're talking about.
_hark··on Making Robots Move Like Animals
This work from Berkeley + Google Brain describes using RL and GANs for transferring motion styles from animals successfully onto robots.

Super neat idea, and love to see ML being used more and more on real robots!

Love the Cost of Transport analysis - naturalistic movement really is very efficient, and good luck getting RL to solve the hard exploration problem of good motion and task performance simultaneously tabula rasa!

It's great to see hand engineered reward functions go the way of the dodo[2] - people are finally coming around to the idea that it was just feature engineering in a difficult-to-reason-about (reward) space! Down with hand specified reward functions! Let imitating nature reign supreme.

What's next, GANs for moral style transfer?[1]

[1] https://imgflip.com/i/6asa47

_hark··on MuZero’s first step from research into the real world
Hmm, I don't think I misunderstood? I get that they're using MuZero to decide the bitrate for equivalent perceptive quality as a function of the content. Sure, once they decide on that using MuZero it's a valid compression and the end-user doesn't have to do something extra... But it's super expensive to run that on the server's end, no? So it ends up being a bit like an asymmetric (in terms of client/server) compute/compression tradeoff, right? And you need to run this for each file you want to compress, hence it's "online".
_hark··on MuZero’s first step from research into the real world
While an interesting use of applied RL, in some sense isn't this just another way to cast the compression/compute tradeoff? I.e. can't we just achieve the same effects by using another compression scheme which trades off local compute for better compression?

Running MuZero online sounds like a fairly computationally expensive prospect...

_hark··on Huge Step Forward in Legged Robotics from ETH
Control policies learned via RL are starting to work in the real world!

Typically policies learned via simulation tend to transfer poorly to the real world (the so-called sim2real gap), so I'm curious to dig into this work to see how they overcame this limitation.

From just watching the video and guessing, it would make sense if noising the belief state (rnn(h,concat(proprio,extero)) + \eps ~ Noise) and learning to condition proprioceptive attention on the belief uncertainty is enough. Very cool work and so exciting to see robotics groups exploiting ML more and more (gate attention + learned belief states here).

_hark··on MuZero: Mastering Go, chess, shogi and Atari without rules
You can make KataGo play moves that keep the score roughly even since it has a trained score head, e.g. kataJigo [1]. This will keep the game even to your level, a nice way to train.

[1] https://github.com/sanderland/katrain#ais

_hark··on Exploration of Generative Art
Sounds interesting, I'll take a look. If you want to email me (grab from my website) so I can ask some questions about the data (if you have expertise here) that would be helpful.
_hark··on Exploration of Generative Art
Wow I had no idea. Thanks for the info and link.

The seismic community has been producing plots like these for decades, but they're just referred to as "wiggle plots", not a great name. Ridgeline seems like the best option going forward!

_hark··on Exploration of Generative Art
What's the horrible meaning of the phrase 'joy division'?

Do people just not like that band anymore?

_hark··on Exploration of Generative Art
Along these lines, I work in geophysics/AI right now and trained a GAN on some seismic data, which people usually visualize with ridgeline/joy plots (literally just realized people call them joyplots because of the joy division cover...). It ended up looking pretty cool: https://brantondemoss.com/seismicdream
_hark··on Love Letter to KataGo or: Go AI past, present, and future
What style of bot did Honinbo Warrior implement?
← PreviousPage 2 of 2