canjobear

Your model of what AI is good at is wrong. Generative AI is not good at wandering off into novel esoteric abstract corners while maintaining correctness, it is good at things that are close to its training data. I suspect that humans will long outperform AI in the domain of "novel esoteric abstract useless math" whereas AI will outperform humans in the domains of (1) making connections between already-well-understood concepts, things that seem obvious in retrospect but which no human figured out just because of the accidents of what people happened to focus on, and (2) proving things that require long, tedious, intellectually unsatisfying calculations, which would cause a human mathematician to give up for boredom.

I have multiple friends now who used to quote me that factoid, then went to Florida and came back convinced that no, Florida man is real.

Google search already had a huge quality slide before 2022.

As early as last year "AI psychosis" seemed to refer to people going crazy due to talking to ChatGPT too much. That was a useful term for a real phenomenon! Now it seems like it's been taken over to mean "thinking that AI is promising" which is more of a rhetorical bludgeon and less useful as a concept.

PhD students paying tuition would be highly unusual.

Is there a $10 billion "fix everything easily" button you have in mind for homelessness?

China at least has this. The stuff you get at the Ole Supermarket inside a shopping mall is different from the stuff you get from the little store facing the street on the ground floor of your apartment building.

On the contrary, the fact that they exclude free riders is what makes the whole thing possible.

Also there are only 27 variables. Which made it hard for me to write a Snake clone until I figured out how to use Lists.

I think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem.

Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want you to turn on the heater. But the literal semantics is just "the ambient temperature in the room is low" and it has nothing to do with heaters. Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do, often so quickly and effortlessly that we don't notice the complexity of the calculation we did.

Or suppose I say to a bot "tell me how to brew a better cup of coffee". What is encoded in the literal meaning of the language here? Who's to say that "better" means "better tasting" as opposed to "greater quantity per unit input"? Or that by "cup of coffee" I mean the liquid drink, as opposed to a cup full of beans? Or perhaps a cup that is made out of coffee beans? In fact the literal meaning doesn't even make sense, as a "cup" is not something that is brewed, rather it is the coffee that should go into the cup, possibly via an intermediate pot.

If the bot only understands literal language then this kind of query is a complete nonstarter. And yet LLMs can handle these kinds of things easily. If anything they struggle more with understanding language itself than with inferring intent.

You could simulate your own brain in Minecraft. What do you conclude from this?

Behold the actual definition of cross entropy: https://en.wikipedia.org/wiki/Cross-entropy

It's true that the PyTorch API conflates cross entropy and softmax, but they are separate concepts.

They’re not. Cross entropy loss is E[-log q] where q is a probability. You could convert the model outputs x into probabilities using some other function like q = 1/Z x^2, and compute cross entropy loss just fine.

Softmax isn't a loss function. It is used to transform model outputs into positive numbers that sum to 1, so that they can be interpreted as probabilities, and then those numbers are passed into (typically) the cross entropy loss function. I think you mean, which models are trained using some function other than softmax to transform the model outputs. There are a number of alternatives to softmax, such as the ones described here https://www.emergentmind.com/topics/sparsemax

The softmax, after the network has been trained, yields an estimate of the probability in the training data, but it is not that probability itself.

The mechanics of engines was understood at the beginning of the Industrial Revolution, and they were fully reproducible: all of which is true of LLMs today. An LLM is a bunch of floating point numbers and simple operations on them, all of which are fully known.

But the way that steam engines emergently transformed heat into work was not understood at the beginning of the Industrial Revolution. Figuring this out led to an entire new branch of physics, thermodynamics. Figuring out how big next-token predictors give rise to interesting systems is likely to lead to similarly new ideas.

It doesn't seem like eliminating nipples should be any harder than eliminating the uterus...

There clearly is though. You don’t remember every detail of every moment that constitutes the experience.

What’s the evidence Boethius wasn’t Christian? Wikipedia says he was.

What are your OC recs?

Charleston is displayed on the map as a “transitional or anomalous area”…

Link to your work?

> Why use splines or polynomials or haphazardly chosen basis functions if you can just fit (gradient descent) your data or wave functions to the proper computational EML tree?

Because the EML basis makes simple functions (like +) hard to express.

Not to diminish this very cool discovery!

Some will do it, some won’t. The ones that do will be overrepresented in the universe.

It will probably be processed into an edited edition and then published as a book.

At the very least, the bottle I saw on sale at Erewhon was clearly labeled as "not safe for human consumption".

That's exactly what it is. The article is almost entirely misconceived.

In any case, Classical Chinese did the same thing but with 之.

Even more than that: Romanization was the official goal of the Communist party until Stalin talked them out of it!

https://faroutliers.com/2004/04/24/how-stalin-and-the-cultur...

The article is mistaken on a number of linguistic points. For example 们 was used to mark plurals way before some 19th century translation, if you look at things that were written in the vernacular and not in Classical Chinese.