HNHacker News
TopNewBestAskShowJobs

HarHarVeryFunny

7,990 karma · joined July 29, 2017

submissionscomments
HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
The predictive nature of our cortex seems even more direct that that, although perhaps obvious is overstating it. To read about the 6-layer circuit of our cortex, google for "cortical microcircuit". Basically the repeating structure of our cortex (across all regions) is this vertical connectivity pattern ("circuit") between the 6 layers, and also the connections coming into and out of the cortex at the top and bottom of those layers. There are also horizontal connections within layers of the cortex, but these are short-distance/localized likely for coordinating activity with a local region (cortical column).

Our cortex is connected in a loop with our thalamus (cf recent talk of looped transformers!), which can be separated into thalamic core and matrix. Basically our senses feed into the thalamic core, and cortical outputs (predictions) feed into the thalamic matrix. The thalamic connections/loop seem to act a bit like a patch panel connecting different regions of the cortex.

The cortical microcircuit is a real physical circuit - a pattern of synaptic connections between layers and inputs/outputs. Those Transformer circuits are more functional than physical. We know how a Transformer is physically connected, with attention heads etc, and this "circuit" analysis then describes how it functions, e.g. with attention heads in adjacent transformer layers effectively combining to form "induction heads" that copy data across embeddings.

HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
There are certainly a lot of analogies to be found in language - it would be interesting to know how much of this is cultural (e.g. business use of sports analogies) vs universal across cultures. Your "spiraling" example reminds me of one I recently came across as a roman coin collector - the origin of the word cataracts, and also portcullis (latin: cataracta) as being analogical to a waterfall (whose greek name gives us cataract), and the notion of something coming crashing down (even if only an eye covering).

Perhaps the common occurrence of analogies in language is because language is distinct from the things it is referencing, so some degree of abstraction is unavoidable?

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
In the US you can always sue someone in civil court for damages you think they have caused you. For criminal charges you need to have violated the law, but in a civil case it seems it's enough to have suffered monetary or reputational harm which was the other person's fault whether intentional or due to negligence, etc.

IANAL, and I'd be surprised to see any lawsuit come out of this, but you certainly don't need to have "violated the law" to be on the receiving end of a lawsuit.

HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
Generalization is necessary as a way to sense the world for a bunch of reasons:

- The never-same-twice nature of sensory perception

- The need to recognize objects from different directions, distances, partially obscured, etc

- Because the nature of the world is that predictive categories are equally or more important than individual identities, especially to basic survival, and for basic organisms

Rather than "generalization" being unspecific, I'd say that as far as perception goes there is a very concrete realization of generalization in our brain - embeddings and associative recall by partial embeddings (see half a face, recall the whole face).

Analogies don't seem to offer any benefit over prediction in terms of what to focus on - there is always a need to separate foreground (object) from background, and to learn what is instance-specific vs generalizable. Is is irrelevant or coincidence that sabre-tooths are near the water hole, or is this part of a predictive pattern (or analogy of some sort?!) that should usefully be learnt?

When we are not sure about what we are looking at, we often then switch from the gestalt "looks familiar" to looking for specific points of confirmation. Is that car in the parking lot really ours? Does it have our air-freshener hanging from the mirror?

Needing to focus on details can also be part of generalization and exception making - when we recognize that two situations that seem similar are in fact different, and we are trying to isolate what the differences are.

I think that the strongest argument for our brains being built for prediction, not analogy making, are that:

1) Prediction is exactly what is needed to survive and thrive - to take advantage of the external world, and not be killed by it. We directly need to predict. We do not need to make analogies, unless analogies can also be used as a way to predict.

2) Our brain appears to be built for prediction, and prediction gives rise to the very obvious learning signal of prediction failure. One of the lessons of LLMs is confirmation of the power of prediction as all you need to drive learning.

The overall way our perception seems to work is indeed as much by prediction as sensory input - in a way prediction really comes first, with sensory inputs narrowing/correcting the prediction (and when you turn it off then we predict/dream without the constraint of sensory reality). The environment you are in already leads you to predict what might be there, and memory of what specifically was there previously will shape those predictions. Sensory feedback corrects/narrows those predictions, and then - if necessary - we look for specific memorized features to confirm what we are seeing/sensing.

Considerations of evolution, and what would be the sensory and cognitive "needs" of organisms of ever increasing complexity, all the way up to ourselves, can also point to what brain capabilities are older and more fundamental, and would have been there to be refined, adapted and built upon later. Even the most simple organism needs to never-same-twice generalize, but will be stuck being reactive until evolution has either equipped it with "reactive predictions" such as the fly being startled by the fast-moving visual input (predictive of being swatted by the horse's tail), or eventually developing a predictive brain like our own, especially useful to an animal that is a generalist living in an environment that is more difficult to predict.

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
I'm not sure the value of speculating, but since you insist...

If someone said to you "you're going to regret this, mo-fo!", do you really need to suppose they had a specific plan in mind, rather than just intent to intimidate?

If you want to suppose that Brubeck had a concrete plan for how he would ruin Buskmaster's career, then my best guess would be that he was threatening for OpenAI to publish without giving any credit to Buckmaster (who has been working on Euler overall for at least 10 years). Obviously this would be absurd, but no more absurd than what Brubeck was also demanding - that Levant was excluded for any credit and could not have his name on any OpenAI writeup (with this being the exact sticking point that Buckmaster was refusing to accept).

The funny thing is that this threat, as is often the case, seems to say more about the insecurities of the person making the threat than the person they are directing it to.

HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
Agreed, and while subjective experience can be misleading (being a Hofstadian "strange loop" is a strange thing) you'd think that if our entire perceptual and cognitive processes were analogy based, then this would filter through to our everyday experience. You'd expect us to see analogies, not generalities, everywhere... looking at your car engine and seeing a hologram of your failing football team floating over it, perhaps!

It's notable that when we do use analogies there are really two use cases

1) Trying to use them to communicate an alternate or simpler POV

2) To make a joke, by pointing out a ridiculous comparison

The joke case is interesting, because this seems to be the basis of half of humor - breaking expectations (the other half perhaps being taboo breaking?), which really isn't how you'd want an adaptive cognitive system to work. You need to see the predictive regularities in the world, not see absurd similarities that make you laugh.

HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
Well, recognizing (recalling) Pooh on a towel as being the same as a picture of Pooh in book can also just be regarded as generalization, same as recognizing that someone looking left is the same person as them looking right, etc.

Our cortex (all of it - visual, auditory, association. motor..) is remarkably simple and regular - it's six layers of neurons with a specific circuit design, that appears to make it a predictor, with top down predictions meeting bottom up sensory feedback. As noted, it makes a ton of sense that evolution would have equipped us with a predictor (and the more powerful the better). Prediction directly supports reasoning and planning (multi-step what-if prediction).

So, my question for Hofstader, or you, would be given that the brain seems to be literally built for prediction, then what is to be gained by regarding this as analogy making instead? Is there any evidence for this? Even as a philosophical viewpoint, doesn't that make for very convoluted explanations of simple things like reasoning and language? (I explained prediction-based reasoning above in one sentence - the analogy-based explanation takes a whole dense book!).

One could certainly use the language of analogies to describe perception (Pooh's ears on the towel are analogous to his ears in the book, etc), but what is the benefit of doing so, rather than the simpler view of this just being generalization (which is specifically what the animal needs), and where is the evidence in the brain?

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
Just knowing that there had been progress is enough to have an idea that throwing more compute at it might work (OpenAI had previously tried all the Millennium Prize problems with somewhat limited compute and failed).

It's comparable to Magnus Carlson saying that if he wanted to cheat, all he would need would be for someone to tell him to spend more time thinking about a specific move (just a wink would be enough) as an indication that a computer had found something interesting.

It's as-if after OpenAI first failing on Navier-Stokes (which OpenAI had just tweeted about 2 days earlier!), someone winked at them and said "you might want to try a little harder ...".

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
I was referring to the overall pattern of apparently sniffing around for recent mathematical progress then setting the AI on it to see if the problem is now easy enough to solve (if you have the money).

Terrance Tao has lamented this practice as being unhelpful for mathematics, and likely to lead to humans working in private to avoid this.

Tao has also noted that many of these AI math proofs don't really help mathematics (nor does it seem they are intended to), since for many of them the proof was never the point, it was the math expected to be needed to be developed along the way, which the AI solutions don't provide.

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
Brubeck has admitted what he said, but claims he immediately retracted it as a "poor choice of words".

Given Buckmaster's telling, this seems beyond "poor choice of words"... It was a veiled threat, that he then doubled down on with his "If you don’t want me to be nice, then I don’t have to be nice." follow-up.

**

I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”

**

FWIW there are also other people on Twitter, such as this DeepMind researcher, saying this is a pattern for Brubeck.

https://x.com/dheeraj_nagaraj/status/2097266146445774924?s=2...

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
> What sort of guidance do you think is happening in a 10k agent, 320b token, 88 hour run ? AI did this one

If you read the PDF release by Buckmaster, apparently the initial claim from Brubeck was that there as very little human input involved, then as the call progressed more and more people popped up that has been involved with it.

Does this aspect really matter? Not really, other than OpenAI wanting to present this as all the work of their model.

**

https://cims.nyu.edu/~tristanb/statement.pdf

I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used.

I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
OK, good to know (if they can be trusted - Altman clearly is a liar), but it doesn't really change the big picture much.

1) OpenAI by their own admission, only re-tackled Navier-Stokes because they heard it had already been solved (but not yet published). This isn't advancing science or helping the mathematical community, this is just being a dick.

2) OpenAI, specifically Sebastien Brubeck, then threaten to "not be nice" and "ruin the career" of one of the mathematicians whose work they had succeeded in duplicating, unless he agreed (which he refused to do) that his collaborator, an Anthropic employee, was not named. This is not only against mathematical norms of credit assignment, it is also being a pathetic human being.

OpenAI would have you believe this result shows how powerful their mystery better-than-Astra model is, but the reality here is that this model needed 10,000 agents, $20M of compute, and the assistance of a whole team of people at OpenAI, to replicate (then exceed) the work that just took two people, with some academic grants as an AI spending budget to achieve (a few $100K - listed below).

https://cims.nyu.edu/~tristanb/

I'd say advantage humans this time. Better luck next time OpenAI - and if you don't want unfavorable comparisons then maybe choose to work on problems that have not been solved yet, and that humans are NOT making nice progress on.

HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
It's been a long time since I read Hofstader's book "Fluid Concepts and Creative Analogies" that this is presumably based on, so I'll be interested to watch the video as a refresher, later. :) He also wrote a follow-on book, "Surfaces and Essences: Analogy as the Fuel and Fire of Thinking", published after this video was made, that I've never read.

From what I recall, to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process all the way from perception on up, and is the mechanism by which we form object categories in the first place.

As always Hofstader's ideas are interesting, but I can't say I agree with them. It seems that the key evolutionary benefit, and function, of a brain is prediction, which is the superpower that moves us from being stuck in the present to being able to "see" (predict) the future, and therefore from being merely reactive to being able to proactively plan and predict future outcomes (what will the sabre-tooth do, where is the water supply?) based on our experience.

Given the never-same-twice nature of sensory perception, before you can predict you need to be able to generalize/categorize, which I think Hofstader would regarded as analogy making (how is this thing I'm seeing similar to what I've previously seen?), although it seems the actual mechanism involved is embeddings or embedding-like representations where similar inputs have similar representations, and what might more simply be considered as associative recall provides the generalization from view/instance to identity/category.

So, is it really analogies all the way up, or are our perception and cognitive processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations, but also match the evolutionary needs that we would expect to exist?

HarHarVeryFunny··on Douglas Hofstadter: Analogy as the Core of Cognition [video]
Hoftstader's video is pressumably based on his own book "Fluid Concepts and Creative Analogies".
HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
OpenAI said they sicced this agent army on Navier-Stokes on Sept 1st, while only a couple of days earlier OpenAI's Noam Brown happened to reply to a tweet saying that they had already tried to solve all the Millennium Prize problems and failed... So, it seems either the previous attempt didn't have the training to succeed, or was just not given the compute to do so.

Once OpenAI heard that Navier-Stokes was solved, this caused them to immediately revisit the problem and throw a ton of compute at it, apparently using a more (very) recent model than what they had tried before. What we don't know is just how recent this model was, and therefore what it may have been trained on. Buckmaster/Levant had apparently been working towards this for at least a year, and made their "forced" blow-up breakthrough on August 15th.

Presumably any anonymized prompts that are being trained on are part of pre-training, so older, but once OpenAI had heard that Navier-Stokes had been solved and wanted to revisit it, it seems possible they may have done a few weeks of incremental RL training on anything Navier-Stokes adjacent they could come up with, in addition to then throwing unlimited compute at it, now confident that there was something to find.

HarHarVeryFunny··on More questions about whether researchers can trust OpenAI with unpublished math
It seems that in this case OpenAI are suggesting that the researchers whose work they scooped were using OpenAI models with an account setting that allowed OpenAI to train on anonymized prompts.

It seems that Buckmaster and Levant (who is an Anthropic employee) were rather naive in the amount of trust they had in OpenAI, with Buckmaster going so far as to contact OpenAI's Sébastien Bubeck to discuss what they were working on and clarify that contrary to rumor this was a private collab.

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
I'm not joking. Compare to a super-human MCTS system like AlphaGo or Stockfish - once you condense the expertise of your top 10K world experts into a board evaluation or policy function, then the rest is brute force.

Whether this type of agentic swarm approach can be considered closer to MCTS (search), or closer to a less structured GOFAI blackboard type approach (perhaps more like your mathematician retreat) I'm not sure - I don't think they've released any details of the prompt(s) and how these agents were collaborating and building on each others work.

The other part of my easter egg analogy is the direction to "look over there", corresponding to OpenAI specifically asking their hoard of mathematicians to work on Navier-Stokes since they knew it was solvable/determinable, and they certainly had the public work that Buckmaster/Levant were building on as further direction, as well as perhaps their prompts. Unlike Buckmaster/Levant, this wasn't just a couple of humans with a university research grant budget, this was apparently a not-so-small team at OpenAI (says Buckmaster, per a group call he had with OpenAI), with an unlimited budget, so it's hardly surprising (or in the least bit impressive) that they were able to duplicate and surpass their work.

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
Is it really important for engineers, etc?

In reality you can't have things like infinite velocity, so if Navier-Stokes is predicting blowups, then isn't this a problem with Navier-Stokes not handling some edge cases, not a problem with reality that engineers need to be concerned about?

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
> OAI pushed a button and solved NS

Not exactly...

From here:

https://cims.nyu.edu/~tristanb/statement.pdf

"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem".

HarHarVeryFunny··on How An AI math breakthrough ignited a controversy
OpenAI know the exact timelines involved, and what they train on, or do not.

You're right - it is exactly "lawyer speak", rather like Bill Clinton's "I did not have sex with that woman".

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
> What claim would that be ?

Huh? I'm saying there isn't one.

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
With 10,000 agents and $20M of compute this is just brute force search.

It's a bit like telling 10,000 kids there's an easter egg hidden over there, pointing to one corner of your yard (or having "heard a rumor" it was hidden in that corner).

If you have $20M to spend on your problem, then yes, AI brute force search is an option, but unless you know a solution is possible (as OpenAI did here), you may still be wasting your money.

HarHarVeryFunny··on How An AI math breakthrough ignited a controversy
They might, or they might just get out-spent and scooped by OpenAI.

This isn't the way you build trust.

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
You don't need to speculate here, since much of the story is not being disputed.

The recent ground-breaking work on Navier-Stokes "blow-ups" was done over a period of years by mathemtaticians Diego C´ordoba and Luis Martınez-Zoroa.

NYU professor Tristan Buckmaster and Anthropic employee (& mathematician) Levent Alpoge took the above work as a starting point, and over a year with LLM assistance developed a blow-up proof under certain conditions.

Buckmaster: "We used several LLMs throughout: Anthropic’s Claude, OpenAI’s Codex, especially with GPT-5.6 Sol and, more recently, Astra. The latter was only used for writeups and auditing our arguments."

Buckmaster says he thinks that Martınez-Zoroa, whose work this all builds on, deserves the Fields Medal for his work.

OpenAI claim that on Sept 1st they heard a rumor the problem has been solved (which happened on August 15th), and then decided to re-solve it themselves using a 2-week old model, then later reached out to Prof. Buckmaster and Levant to come to some agreement to co-publish.

OpenAI: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models ". In other words, not only did they deliberately choose to tackle a problem they heard had already been solved (in turns out only partially solved), but they may have done so using a model that was aware of the successful way to attack the problem.

It seems there are three potential scandals here:

1) OpenAI by their own admission chose to try to scoop mathematicians who they had heard had already completed a proof

2) OpenAI may have used a model that had seen "de-identified" messages indicating the direction to take

3) An OpenAI employee essentially threatened to "ruin the career" of the NYU professor who had been working on this if he did not cooperate with them

The direct plagiarism possibility, 2), while it should be a warning to anyone using OpenAI's models, doesn't need to be true for OpenAI to have benefited from the researcher's work. It's enough that they heard Navier-Stokes had been solved and could then go out with their swarm of 10,000 agents and $20M of compute to hunt out the latest research and brute force it.

Magnus Carlson once said that if he wanted to cheat all it would take would be for someone to indicate to him (a wink from someone in the audience perhaps) when a position warranted more time to be spent on it (because there was something important to be found if he did). It seems that, at absolute minimum, this is what OpenAI did here, although in context of math this is not cheating - the "wink" was a rumor, originating from who knows where, that a proof existed (but had not yet been published) and therefore there was potential to rush in and scoop rights to publish or co-publish.

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
An Astra sized model takes months to train - they are not saying 2 weeks to train from scratch. The only only interpretation of this "2 weeks" claim that is consistent with reality is that they mean 2 weeks of additional training on top of whatever their starting point was, so it's more like this:

|--- N months of base model training -->|--- X months of post-training -->(Astra?)|-- 2 weeks more training (on Navier-Stokes adjacent material, perhaps)--> this "new" model

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
If there is any truth to this timeline, then presumably it just means an additional 2 weeks of RL training on Astra.

"Twice as capable in mathematics" just means they found some problems that Astra couldn't solve, or make progress on (who knows how they chose to define "capable", or "twice as" for that matter), then put those 2 weeks of training in to focus on those gaps.

At this point, focused on their IPO, the best way to interpret OpenAI press releases is "what is the least this can mean, without being an actual lie". They are not shy - if there was a more impressive claim they could make, they would have made it.

HarHarVeryFunny··on How An AI math breakthrough ignited a controversy
I would think it would have the exact opposite effect.

Why would anyone use OpenAI models for anything commercially valuable, or where secrecy is important, when it appears that if OpenAI "becomes aware" that you are doing so they may try to compete with you?

Not only did OpenAI, by their own admission, rush to re-solve Navier-Stokes once they heard the rumor that it has been solved (the rumor being that it was Anthropic that had done it), but they are leaving the door open ("we cannot rule out that") as to whether the model they used to do it had been trained on the anonymized date from the researchers who's approach they ended up copying.

Terrance Tao has recently said as much for mathematics - that there appears to be a trend (not just this Navier-Stokes incident) of the AI companies going after math problems wherever there is an "rumor" of progress, and that he thinks this may sadly result in breakthrough mathematics being conducted in secret to avoid this.

The rush to steal another researcher's thunder is bad enough, but it also appears that one of OpenAI's employees acted in a very thuggish manner to try to threaten the professor who had been working on this not to publish and to co-operate with their telling of the story.

HarHarVeryFunny··on OpenAI's GPT-6 Astra on ARC-AGI-3
> My point is it's not "gone very far" because he's taking it as far as he can, which isn't very far.

I wouldn't really agree - I'm no fan of LeCun, but the problem with JEPA isn't that it's a bad idea, or can't go very far, but just that it's not much of an idea in the first place!

It's no secret that our brain basically works by prediction, and what we're predicting is necessarily the external world as we perceive it though our own senses, aka latent representations, aka JEPA.

So, you COULD take this unoriginal smidgen of an idea and built it out to a full model of a human/animal brain, whether or not it's LeCun's intention to do so (he seems more interested in just the representational / world model aspect to it), but he certainly hasn't done so yet, nor created any research manifesto indicating that as his intent.

The fact that JEPA implementations to date are using pre-trained Transformers doesn't seem inherent to the approach - one could, with more effort, still predict latent representations (i.e. sensory feedback) but do so using a new real-time learning algorithm based on prediction failure.

LeCun seems more of an academic / research director than a builder, and I would never have put much stock in him being the one to build an animal brain.

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
The NYU professor, Tristan Buckmaster, has now released a statement on this.

https://cims.nyu.edu/~tristanb/statement.pdf

HarHarVeryFunny··on On the Navier–Stokes Millennium Prize Problem
Too late - OpenAI already cured cancer, but they are holding the result for their IPO next year.
← PreviousPage 10 of 34Next →