kovek

What about the investors who put money into those companies and want to get some money out of that? Does the law say that the companies owe them? Something I thought about recently...

Would the loss at the global maxima be much lower that the loss at most local maxima?

I think to learn a language you need to do as many modalities as you can: Speaking, listening, reading, singing, asking questions to native speakers, asking questions to LLMs, writing, flash cards, etc.

The obstacle with LLMs is that they are specifically going at their own uncanny valley pace. I do see how that might be not a problem for some prompts.

I haven't tried a Chinese-trained LLM for learning Mandarin, so I'm not sure which LLM is best..

Try being an English speaker learning Mandarin from Claude

I think learning from another human being live who can understand your confusingly articulated questions and who can adjust their answers for you, another human being, that's great.

Please support /voice for claude!

I think it's easy to lie to consumers. Then, how much agency do they have?

I have only one thing to say....

what

I heard people say this before. I'm wondering, how do you instruct the LLM to generate the tests? Do you tell it the scenarios that would be covered, or do you just tell it to write tests for the code?

I thought it was known since a few years now that if you train models to NOT do certain things, then they start behaving in weird ways…

What if the organization who tries to verify sends a request on an app on the user’s iPhone (or whatever device can do the same), and the user scans their face with FaceID to produce a file send to the organization, which will then send that file to Apple to ask if the file represents the right person? I trust Apple so that works for me.

I’d say it’s possible to have creativity when you’re sitting as well. I like to think that’s it’s all about staying active. Reading, diarying, calling a friendind. All of that.

LLMs are like a search engine that autocompletes. It's a tool.

What negative consequences does being unelected have?

Maybe you can't 100% know what every layer "thinks", if you go through all the layers, you might see a cohesive "thinking" story. So, if there is any information you lose at layer N, you might learn some of it in layer N+1. The masking in the layers is not deterministic so the model can't really consistently lie throughout the layers. It doesn't chose what information we get to inspect. There might be a game of whack-a-mole, but you might get a general sentiment. I think the more layers there are, the more the model itself can hide very nuanced lies (But by that time we'd have a better mind-reading model).

However, I haven't read about it yet. I'm really excited to look into it!

I’ve read recently about natural systems in the book Antifragile. It’s interesting how those systems can become better.

> The core idea is content-dependent selection. For each query, the model selects which parts of the sequence are worth attending to, and computes attention exactly over those positions.

I don't know if this will help for things like understanding code, where the all relevant parts can be the file of 1000 lines that we are analyzing, and where every token is relevant in understanding recursion, loops, function calls, etc.

This sounds like it would be great to do SSA before passing things along to a code model like claude code.

Let me know if I misunderstood

I don’t think triaging is necessarily an easy task

Does thinking about how to offload matter?

10s of GBs? ( 1,000,000 context * 1,000 vector size ) ^ 2 = 1,000,000,000,000,000,000… oh wow.. I must be miscalculating

What about only storing the conversation and then recomputing the embeddings in the cache? Does that cost a lot? Doing a lot of matrix multiplication does not cost dollars of compute, especially on specialized hardware, right?

What if the cache was backed up to cold storage? Instead of having to recompute everything.

Is this similar to send 48656c6c6f2c20686f772061726520796f753f in the prompt? As done here: https://youtu.be/GiaNp0u_swU?si=m7-LZ7EYxJCw0k1-

> You seem to be hinting at the "chemical imbalance" theory of antidepressants, which has been largely debunked

Can you say more?

Models need pre-training and fine tuning. Humans can do online learning.

What if we asked users if they want extra protection? I think that would be nice..

For the tech docs writing, just give me the bullet points and I'll send them to the AI and discuss the bullet points with it.

I was thinking about the ability of representing different kinds of numbers. Imagine that we had a certain CPU that could process algorithms, and the final output of the algorithm is a number. The CPU has a certain number of operations (At least https://en.wikipedia.org/wiki/One-instruction_set_computer). Then, if the algorithm can be described with an integer (since the algorithm can be described with binary), then... can integers describe Real numbers?

What is there to be furious about?

Every word and every hierarchy of words in natural language is understand by LLMs as embeddings (vectors).

Each vector has many many dimensions, and when we train the LLMs, their internal understanding of those vectors sees all sorts of dimensions. A simple way to visualize this is a word's vector being <1, 180, 1, 3, ... > which would all mean a certain value at that dimension. In this example say the dimensions are <gender, height in cm, kindness, social title/job, ...> . In this case, our example LLM could have learned that the example I gave is <Woman, 180, 100% kind, politician, ... >. The vector's undergo some transformation so every dimension is not that discretely clear cut.

In this case, elephant and car both semantically look very similar to vehicles. They basically would have most vectors very similar.

See this article. It shows that once you train an LLM, and you assign an embedding vector for each token, then you can see how the LLM can distinguish the difference between king and queen: man and woman.

https://informatics.ed.ac.uk/news-events/news/news-archive/k...

I think that semantically this question is too similar to the car wash one. Changing subjects from car to elephant and car wash to creek does not change the fact that they are subjects. The embeddings will be similar in that dimension.