mhl47

Love this! Not having spent too much time on understanding the architecture I always struggled to see how transformers get away with compressing all context in a flat vector (e.g. 768 numbers here) between attention and the MLP when computing the current token. But now I think I understand that since you often alternate between attention and MLPs it regularly mixes/queries the information of other tokens into the current computation. Probably common knowledge for everyone that learned about transformers but this made it so much quicker to see.

When we rolled out an ERP in our Brazil subsidiary the whole internal consulting team immediately agreed to leave our brains untouched by this complexity and hire a local consultant team as help. And the local consultant team in turn immediately suggested useing a middleware provider.

If you can touch type try this: Go through they keys once alphabetical 'abcd...' recalling the movement from the letter. Then do it the other way round and move your fingers in spatial sequence to the keys and try to recall the letter.

You will find the former much easier if you did not by chance also memorize the keyboard layout for some reason.

Absolutely can relate. Thats why I would say: Do your own diagrams! At least the text and formatting.

Where I do think it ads value is "AI slop 2". This is somehow even better comprehensible than an average photo.

Maybe there is a bias for action within our moral and legal system. Fundamentally if you can deal with uncertainty correctly or "perfectly" wouldn't more information always be better?

Everybody remember when Zuckerberg told in an Interview in 2024 that human data does not matter that much or more specifically "individual creators or publishers tend to overestimate the value of their specific content". Something along the line RL-Loops are more important.

Hard to square this with that acquisition which seems to be focused on Cursors vast amount of User Data.

At least for now.

This does surprise me, because you'd think that even if they crank up the filter's sensitivity at the expense of specificity, an LLM company wouldn't simply design a filter that triggers on keywords in a completely unrelated context.

First test question: "Is the UV Index a good proxy for when to wear sunglasses." Immediately triggered the safety filter ... oh dear.

What wasn't build with AI is that webpage and it's tiny non-scaling text.

Social Media creates distinctive Filter Bubbles. A dominant LLM company (or multiple aligned ones) create one way of thinking.

Was looking for a precise term for that. Thank you!

Also AI-Linkedin-Bullshit likes to use "just" additionally and it's mostly along the lines of Y being something much more impactful then X.

The knowledge that everything is made out of atoms/molecules however makes it much easier to reason about your environment. And based on this knowledge you also learn algorithms, how to solve problems etc. I dont think its possible to completely separate knowledge from intelligence.

How do you arrive at the statement that a cavemen would have the same intelligence as a human today? Intelligence is surely not usually defined as the cognitive potential at birth but as the current capability. And the knowledge an average human has today through education surely factors into that.

"External Storage" whatever that is can not be the same as continous learning as it does not have the strong connections/capture the interdepencies of knowledge.

That said I think we will see more efforts also on the business side to have models that can help you build a knowledge base in some kind of standardized way that the model is trained to read. Or synthesize some sort on instructions how to navigate your knowledge base.

Currently e.g. Copilot tries to navigate a hot mess of a MS knowledge graph that is very different for each company. And due to its amnesia it has to repeat the discovery in every session. No wonder that does not work. We have to either standardize or store somewhere (model, instructions) how to find information efficiently.

Most beyond products I know don't even contain soy as protein source.

Regardless of what you think about phytooestrogens (which has very little evidence to have negative effects in normal quantities)

The fact that the outputs are probabilities is not important. What is important is how that output is computed.

You could imagine that it is possible to learn certain algorithms/ heuristics that "intelligence" is comprised of. No matter what you output. Training for optimal compression of tasks /taking actions -> could lead to intelligence being the best solution.

This is far from a formal argument but so is the stubborn reiteration off "it's just probabilities" or "it's just compression". Because this "just" thing is getting more an more capable of solving tasks that are surely not in the training data exactly like this.

Worriesome for sure.

However I would say that the cited studies are somewhat outdated already compared e.g. with GPT-5-Thinking doing 2mins of reasoning/search about a medical question. As far as I know Deepseeks search capabilities are not comparable and non of the models in the study spend a comparable amount of compute answering your specific question.

Or there are global events that stress people out .. or their expectations change over time. Not that simple ;)

You are right, just tried even with reference images it can't do it for me. Maybe with some good prompting.

Because in theory I would say that knowledge is something that does not have to be baked in the model but could be added using reference images if the model is capable enough to reason about them.

We are currently working on some christmas puzzle, that are - I would say - a bit more difficult from the visual side. GPT5.1 completely failed at all of them while Gemini 3 solved two till know that I would consider rather impressive.

One was two screenshots of a phone screen with chats that are timestamped and it had to take the nth letter of the mth word based on the timestamp. While the type of riddle could be in the training data the ability to OCR this that well and understand the spatial relation to each object perfectly is something I have not seen from other models yet.

Most people are missing the point here. Testing the GUI/feature more reliable is something that Gemini 3 could unlock (looking at the ScreenSpot-Pro benchmark and its general improvement on visual understanding). At least for the (hobby-)projects I attempted this was really a bottleneck having to always test the GUI after each change as its quite often breaking something.

https://news.ycombinator.com/item?id=44573195 (in the article, search for:"Chat runs really deep")

There was a recent post here about how deeply ingrained the chat interface is in OpenAIs organization. This really doubles down on that, but does anyone really like to interact with so much language instead of visual elements? Also feels horrible that you are supposed to remember a bunch of app names like "zillow" and punch them in the chat. And like an opportunity for them to slowly introduce ads for this apps or "preferential discovery", if you will, as monetization strategy.

Personally I don't hope thats the future.

Currently trying to build #6. Just for private use. My hope is that by throwing a bunch of highly personalized information in a VLM it will provide reasonably first estimates. (E.g. if you see a bowl lentils I will probably have rice below etc.). And then iterate on the main ingredients -> fetch the macros of main ingredients from a DB. If its within 20% that would be enough for me.

I have tried some off-the-shelfe solutions and they currently do not seem to cut it, or are too complex for my use case.

Pretty rich that you are complaining about generalization now where you made the initial statement that the "Car is absolutely essential for driving around small kids no matter the urban density" which doesn't seem to have any limits in scope.

Yes but isn't the cat out of the box already? Don't we have sufficiently strong local models that can be finetuned in various ways to rewrite text/alternate images and thus destroy possible watermarks.

Sure in some cases a model might do some astounding things that always shine through, but I guess the jury still out on these questions.

I think an example would be the two body problem. It stays on an eccentricity. So it does not explore different eccentricities although they can have the same total energy.

(But I just looked that up too because this concept is mostly used/assumes in statistical physics)

This is related to the question whether a system/the universe is ergodic (among other properties changing energy, space).

No it's 100 a week for plus users.

Side note: What is going on with these comments on Mathstodon? From moon landing denials, to insults, allegations that he used AI to write this ... almost all of them are to some capacity insane.