canjobear

There’s definitely a practice effect though. I know people who aren’t used to it will have their eyes glaze over when they read einsum notation.

Only approximately. In the linked article you can see that the color gamut is a horseshoe shape that cannot be fully spanned by a triangle. So there are color percepts such as responses to pure wavelengths that can’t be reproduced as combinations of real primaries. You could invent imaginary primary colors to make it work, hence the XYZ color space.

In any case, the whole point of TFA is that a space is not an adequate characterization of color percepts, because there is no Riemannian geometry that matches the psychophysical data.

For example, say Nate Silver and Andrew Gelman both publish probabilities for the outcomes of all the races in the election in November. After the election results are in, we can’t say any individual probability was right or wrong. But we will be able to say whether Nate Silver or Andrew Gelman was more accurate.

But I think if we take the view that it's not a random variable until we query it, that makes it awkward to talk about how GPT-2 (and similar models) is trained. No one ever draws samples from the model during training, but the whole justification for the cross-entropy-minimizing training procedure is based on thinking about the model as a random variable.

You can apply it other ways. There are lots of interpretations and uses for these concepts. Here's a cool blog post if you want to find out more: https://blog.alexalemi.com/kl-is-all-you-need.html

For example my cat weighs 13 pounds. That seems objective, in the sense that if two people disagree, only one can be right. But the claim is based on my observations. I think your logic leads us to deny that anything is objective.

Consider cross entropy of two distributions H[p, q] = -Σ p_i log q_i. For example maybe p is the real frequency distribution over outcomes from rolling some dice, and q is your belief distribution. You can see the p_i as representing the objective probabilities (sampled by actually rolling the dice) and the q_i as your subjective probabilities. The cross entropy is measuring something like how surprised you are on average when you observe an outcome.

The interesting thing is that H[p, p] <= H[p, q], which means that if your belief distribution is wrong, your cross entropy will be higher than it would be if you had the right beliefs, q=p. This is guaranteed by the concavity of the logarithm. This gives you a way to compare beliefs: whichever q gets the lowest H[p,q] is closer to the truth.

You can even break cross entropy into two parts, corresponding to two kinds of uncertainty: H[p, q] = H[p] + D[q||p]. The first term is the entropy of p and it is the aleatoric uncertainty, the inherent randomness in the phenomenon you are trying to model. The second term is KL divergence and it tells you how much additional uncertainty you have as the result of having wrong beliefs, which you could call epistemic uncertainty.

Another example, if you have an electron in a superposition of half spin-up and half spin-down, then the probability to measure up is objectively 50%.

Another example, GPT-2 is a probability distribution on sequences of integers. You can download this probability distribution. It doesn't represent anyone's beliefs. The distribution has a certain entropy. That entropy is an objective property of the distribution.

The ß sticks around in German because it prevents some ambiguities. For example Busse and Buße are different words with different pronunciations.

Basically German orthography backed itself into a corner because of two rules: a vowel before a doubled consonant is short, and an s before a vowel is pronounced /z/. This leaves you with no clear way to write a long vowel followed by an /s/ sound followed by a vowel: VssV would make the first vowel short, and VsV would keep the first vowel long but the consonant would be pronounced as /z/. The ß solves this problem.

It's hard to find a word that is a truly original coinage not based on any existing words.

Also the “needs fixed” thing is not a Southern dialect property, I associate it more with like Pittsburgh.

> A triangle doesn't have to be filled with a color to be a triangle. Why would you imagine it with any color in it in the first place, not just a wireframe, being transparent inside?

That’s just it: when I imagine a triangle, it is a definite image with a definite color. The visual details like color come automatically for me. I cannot imagine it in the more abstract way you are describing. If it’s a wireframe, the wireframe is gray. It’s always a full, colorful, 3d scene, perhaps vague but definitely with visual detail.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7856239/

But you can also imagine tests based on introspection which are better than just straight up asking how vivid your mental imagery is.

For example: imagine a triangle. What color is it?

For a true aphantasiac, this question is meaningless. It has no color because it’s just an abstract concept. For me it is a white outline on a black background, very vividly.

The extra components would have energy and function as Dark Matter.

Altaic is a little more complicated, but whatever it is, it doesn’t seem to be a linguistic family tree in the same way that Indo-European is.

In principle languages are arbitrary, but in practice they’re describing the same world and the same concepts end up being useful.