canjobear

But what makes humans special is the immortal soul, which Aquinas says animals don’t have. It’s something other than vitalism or metabolism.

They could just append "according to current scientific knowledge" to every sentence. But it would be a waste of time.

Just keep reading the abstract.

> Only universes with very different physical properties can produce some version of this Universe as a simulation. On the other hand, our results show that it is just impossible that this Universe is simulated by a universe sharing the same properties, regardless of technological advancements of the far future.

I met Morris Bart at some kind of party when I was a teenager. He told me that advertising is the most powerful force in the universe.

Chip Forstall takes care of it all!

This makes sense for my path to math. In high school I was bad at math and good at learning languages. Then I started learning Python and realized that it was just like learning a language. Then at some point I realized math notation was just another language for expressing the kinds of things you could express in Python. Now I'm in a job where I do math every day and I read math textbooks for fun.

It helped that Python was meant to resemble natural language. I had learned C++ and Perl before but they never stuck, because I never made the connection to language. Ironically, since Perl was designed by a linguist!

I don't see how it's absurd. The point isn't that languages don't have word order constraints, it's that they're organized in such a way that word order is usually redundant with other things when it comes to expressing meaning. This redundancy is a real and nontrivial property of language which becomes obvious when you analyze it in a statistical and usage-based way, but which isn't so clear if you're focused only on the categorical formal structure.

I wouldn't say NLP as a field was resistant to probabilistic approaches or even neural networks. From maybe 2000-2018 almost all the papers were about using probabilistic methods to figure out word sense disambiguation or parsing or sentiment analysis or whatever. What changed was that these tasks turned out not to be important for the ultimate goal of making language technologies. We thought things like parsing were going to be important because we thought any system that can understand language would have to do so on the basis of the parse tree. But it turns out a gigantic neural network text generator can do nearly anything we ever wanted from a language technology, without dealing with any of the intermediate tasks that used to get so much attention. It's like the whole field got short-circuited.

There's a lot of debate about it. Here's one view: https://arxiv.org/abs/2501.17047

Straight up doesn't work (ChatGPT-o4-mini-high). It's a nothingburger.

Writing blog posts against people who are precious about their coffee is also a lifestyle signal.

> /g/ was the origin of Chain of Thought for AI

Is this documented?

An LLM distribution exists in the physical world, just as much as this comment does. It didn’t exist before the model was trained. It has relation to the physical world: it assigns probabilities to subword units of text. It has commercial value that it wouldn’t have if its objective probability values were different.

That’s the probability to observe a token given the prompt and the seed. The probability assigned to a token given the prompt alone is a separate thing, which is objectively defined independent of any observer and can be found by reading out the model logits.

Suppose you're training a language model by minimizing cross entropy, and the omniscient being is watching. In each step, your model instantiates some probability distribution, whose gradients are computed. That distribution exists, and is not deterministic to the omniscient entity.

> If 2 people are calculating any property and one of them is making a false assumption, they'll end up with a different (false) conclusion.

This implies that there is an objectively true conclusion. The true probability is objective.

Probability is a bunch of numbers that add to 1. Sometimes you can use this to represent subjective beliefs. Sometimes you can use it to represent objectively existing probability distributions. For example, an LLM is a probability distribution on a following token given previous tokens. If two "observers" disagree about an LLM's probability assigned to some token, then only at most one of them can be correct. So the probability is objective.

Those aspects of the book are awesome but I think they run against the themes Ridley Scott was going for. Ridley Scott wanted Deckard to be an android. It's hard to interpret the unicorn scenes otherwise. The androids are shown as cruel due to their lives as slaves, but with human-like longings for life and meaning (like in Roy Baty's final speech about tears in rain). The overall point is to blur the boundary and say the androids can be meaningfully human.

Whereas Mercerism and the animal stuff in the book are all about emphasizing the ways humans are different from the androids. The androids mock Mercerism and they don't care about animals: they are incapable of empathy. They torture people and animals without compunction. The alternate police station scene, where Deckard is tested using a bone marrow test instead of Voigt-Kampff and comes out human, is evidence that he's not an android.

The book is, in my view, one of the few pieces of sci fi media that seriously raises the question "could these apparently human-like machines really be human just like us?" and answers a resounding "no". The androids are psychopaths who are unable to partake of the human experience. Ultimately PKD is concluding that they are meaningfully not human---and, furthermore, some biological homo sapiens who act like them might actually be androids, a theme you can find elsewhere in his essays [1]. To the extent that Deckard's humanity is called into question it's not whether he is physically an android, but if he is psychologically a psychopath because of his job killing androids.

[1] https://sporastudios.org/mark/courses/articles/Dick_the_andr...

Taking notes forces your brain to process the information in a way that just reading or listening doesn’t.

That’s part of why English is mixed.

English is mixed head-initial and head-final, for example objects go after verbs (head-initial) but adjectives go before nouns (head-final). Japanese is strictly head-final.

Lots of work on this kind of thing.

A good starting point is https://en.wikipedia.org/wiki/Combinatory_categorial_grammar

If you're into this stuff, Conquest of Peru (1847) by William Prescott is surprisingly readable and vivid. In case you worry that an old book won't be accurate, basically paragraph is backed up with a footnote giving long primary source quotes in the original Spanish. If you just read the footnotes it's like you're getting a well-paced spliced together first-hand account.

Some things I remember from the book that don't make it into this article:

- The Inca empire was essentially a top-down command economy. Everything down to individual people's jobs was managed centrally from Cuzco.

- Pizarro's translator was from a native tribe that was recently conquered by the Inca, and he hated the Inca himself. It's possible that he translated things in an intentionally adversarial way at crucial junctures, like when the Spanish were deciding to execute Atahualpa.

- The official reason for executing Atahualpa was that, while in captivity, he had sent orders to have his brother Huáscar executed.

- Apparently Atahualpa was amazed to discover that he could write marks on a piece of paper, show it to to Spaniards like de Soto, and they would all be able to read out exactly the same words. He tried the trick with Pizarro but, being illiterate, Pizarro couldn't read out the text. This didn't help with the Pizarro-Atahualpa relationship.

- The account of how the battle at Cajamarca started is a little different. The Spanish sent out a priest who gave a long confusing monologue to Atahualpa about the death of Jesus, the nature of the Trinity, the immaculate conception of Mary, and other deep-cut theology, then handed him a Bible. Atahualpa was basically like "why would you worship a guy who was executed as a criminal" and didn't have any idea what to do with the Bible (or any book). Then the Spanish ambushed him.

This is a terminological difference. Linguists use "grammar" as a technical term for a speaker's implicit knowledge of how their language works. That knowledge could be statistical or rule-based in nature, although most linguistic theories say that it's rule-based. You're using grammars to mean human-produced descriptions of that knowledge.

Wrt reasoning I’ll believe it when I see it. I just tried several variants of “Generate an image of a chess board in which white has played three great moves and black has played two bad moves.” Results are totally nonsensical as always.

> They are simply assuming that there is a brain "embedding" that can be directly compared to the matrix of numerical weights that comprise an LLM's training.

If there were no such structure, then their methods based on aligning neural embeddings with brain "embeddings" (really just vectors of electrode values or voxel activations) would not work.

> They mention a profound difference in the opening paragraph, "Large language models do not depend on symbolic parts of speech or syntactic rules". Human language models very obviously and evidently do. On that basis alone, it can't be valid to just assume that a human "embedding" is equivalent to an LLM "embedding", for input or output.

This feels like "it doesn't work the way I thought it would, so it must be wrong."

I think actually their point here is mistaken for another reason: there's good reason to think that LLMs do end up implicitly representing abstract parts of speech and syntactic rules in their embedding spaces.

> how does a language develop in the brain

Huge question, but there's not really a lot of data to go on. There do seem to be some things that are a lot easier to learn when you are very young.

> why is it easy for me (as someone who grew up in the East) to learn guttural sounds of Arabic

It all has to do with the speech sounds you're exposed to when you're young.

> Does the writing system affect speech, and if yes, then how?

Probably not. Very little if so. The main effect would be preserving old speech forms of the language to some extent, if the writing is phonetically based.

> What are some known neuroscientific tricks (for adults, not kids) for acquiring specific languages?

The most important thing is lots of exposure and practice and also the way you're practicing the language should be as close as possible to how you will actually use the language. There aren't any massively effective shortcuts.

#1 is a textbook that I've taught out of. It's a good summary of decently up-to-date psycholinguistics.

#2 is more of a polemical book, maybe not the best if you want a fair summary.

#3 I've never heard of.

If you’re interested in this topic then you should definitely know the Chomsky/Berwick position, but it’s not the only view and has never been the consensus. For an alternative argument, “The Origin of Human Communication” by Michael Tomasello is good.

> What does the "particle" model bring to the table that a localized (wavelength-sized) wave/vibration could not?

A lot of the article is about this. Start with the section "The Wave Function of Two Particles and a Single Door". The wave packet view can't explain why you don't for example see a "particle" (that is, a dot on a detector) show up simultaneously having gone through two different doors. You have to think about it in terms of a wave in the space of possible joint particle positions.