"A good human plus a machine is the best combination" — Kasparov
406 karma · joined February 1, 2021
"A good human plus a machine is the best combination" — Kasparov
https://www.falstad.com/dotproduct/
https://wordsandbuttons.online/interactive_mnemonics_for_dot...
In essence then, not as confusing to the beginner who might even know what a dot product 'is' operationally but not what it 'does'.
So level 1 'it's just a normalized dot product', level 2 more immediately intuitive: 'is arrow 1 pointing in the same direction as arrow 2?' or 'how close is arrow 1's direction to arrow 2's direction?'
Now what's left after that is 'Why is it so? Why did we decide on this in embeddings?'
Author needs to get into Bret Victor. Has no idea how much more fun he could be having.
Programming is a step on the way to access to the state space of information. When we get to that stage, programming will seem like a maze of syntax, that has its own idiosyncrasies that force you into corners or regions in the state space, just like any DAW plugin or 3D tool, or any tool at all that exists.
edit: there's one called "#43 Diffusion (5/1/62)" here: https://www.feynmanlectures.caltech.edu/flptapes.html#restor... so I assume that
I always make this argument:
If a human read all the text GPT read, and you had a conversation with them it would be the most profound conversation you've ever had in your life.
Ecelcticism beyond belief, surprising connections, moving moments would occur nonstop.
We need language models like that.
Instead, our language models are trying to predict an individual instance of a conversation, with the lowest common denominator customer service agent, every time they run (who to his credit, can look things up very well).
And I don't think fine tuning this "tone" in would be the way to go. A better way would be to re-Frankenstein the existing ones architectures or training algorithms to be able to synthesize in this way. No more just predicting the next token.
While they're concerned with safety, I'm much more interested in this as a tool for controllability. Maybe we can finally get rid of the woke customer service tone, and get AI to be more eclectic and informative, and less watered down in its responses.
When a person reads a book, they have an "overall intuition" about it. We need some way to quantify this. Needle in haystack tests feel like a simple test that doesn't go far enough.
The location of the gradient is parametric based on where the found keyword is. So if it's at the bottom, the gradient will be brighter there by default, or if the keyword is at the top, it'll be brighter there.
If there's more than one match in a single preview, the gradient accommodates for that as well. So if your keyword is "potato" and it shows up at the top and bottom, there is no gradient (or it's very small such that there is none)
There's typically enough text to discriminate (even before hover)
When the user hovers, the area that isn't bright gets brighter.