1,684 karma · joined February 14, 2011
“bro like, what if we could price the sum total of human knowledge? That wouldn’t be that much, right?”
https://ravimohan.blogspot.com/2007/04/learning-from-sudoku-...
People had been calling a housing bubble since 2003. You quite literally need the market to do dumb things over a period of time before the bubble pops. Otherwise it’s not a bubble!
Actually making money on a bubble pop is another story and that does rely on timing. pointing at a history of failed market predictions as a gotcha is about as effective now as it would’ve been in 2003-2007 right up until the bubble popped
Right now it’s painfully obvious that there is a bubble, the circular financing is documented, the hype-driven valuations are real, it’s just a waiting game to see who holds the bag
Then you feed a bunch of tokens into a GPU and end up with a distribution of possible next tokens…
Bayesians say that the probabilities represent strength of belief, implying some subjective knowledge or information. It is necessarily subjective in that it requires priors, i.e information the predictor knew before making the prediction. In other words, the LLM has priors from training and is predicting tokens using real knowledge
Frequentists would say that probabilities are simply objective facts - e.g we all agree that the physical property of temperature follows from any molecules matching a particular energy distribution. You’re not predicting anything, there’s just some outcomes that are happening at the expected rate. In other words, the LLM is a stochastic parrot/next token predictor
There’s no reason why you couldn’t write a search tool that e.g combines LSP and grep. Or ast-grep, for that matter. It feels like one of those things we haven’t spent much time investigating because grep is good enough