WhitneyLand

What don’t you like about Claude? I believe the context is larger.

Coincidentally I’ve been using it with xml files recently (iOS storyboard files), and it seems to do pretty well manipulating and refactoring elements as I interact with it.

When will we be able to give it a try?

I’m playing around with similar ideas, sometimes called ensembling techniques.

How much did it cost them?

Using ensemble model estimation resulted in a guess of $400,000.

Can’t imagine when I’ll ever need this, but I love it.

Great work Bahador.

If that’s where we’re going I’ll try to pedant-raise you.

Assuming the cosmological principle is true and the universe is infinite, wouldn’t we be guaranteed an infinite number of Sols? ;)

What’s your rating, have you tried gpt4o?

It’s supposedly good up to about 1300, but aside from that the ability to prompt can make the style of play somewhat tunable for ex aggressive, defensive, etc.

There are some little gems in the article, one emphasizes what a badass Turing was.

As if it wasn’t enough to essentially invent the field of theoretical computer science and save possibly millions of lives in the war, he then decided in 1952 to casually publish in the field of biology.

He had no formal training in biology, and when peer reviewers including the grandson of Charles Darwin commented on his paper “Turing seems to have ignored both sets of comments” lol.

Nevertheless the paper turned out to be groundbreaking and is still considered relevant today. Total badass.

I think they’re different definitions. I don’t believe there’s consensus in all of CS research that the definitions you cited apply to ML. At least not from the papers I’ve read.

Of course if it’s pharmaceutical.

People were alluding to the risks of self-medicating I was referring to that.

My suspicion has always been the majority of adverse reactions have related to improper dosing.

Anyone can have a horrible response with too high of a dose yet with self-medicating theres no sure way to know the amount of medicine you’ve ingested even if weighing.

I was referring to the literature of machine learning and language models specifically, since that’s what the paper is about.

What’s your referencing seems to be more related to symbolic ai / formal logic, and I get that these are related, but it just doesn’t really map neatly onto LLM‘s.

I agree it wasn’t that convincing, moreover the variation wasn’t that dramatic for the large sota models.

Why should they write a paper about the inherent reasoning capabilities for “large” language models and then in the abstract cherrypick a number that’s from a tiny 1B parameter model?

Please provide your precise definitions of “reasoning” and “original”.

There’s no consensus in the literature on what these mean even if you make it more specific by talking about “mathematical reasoning”, so I don’t really understand what opinions like these are based on.

I see a lot of no true Scottsman fallacy going around, even the paper resorts to this as it actually uses phrases like “true reasoning” several times.

I don’t think the paper is very convincing btw, the abstract is kind of click-baity and talks about 65% variation when that was a cherry picked example from a tiny phi model and the SOTA models showed way less variation which was arguably not that interesting.

Both ChatGpt 4.o and Claude failed to answer

“…At what dimension would the red ball extend outside the box?”

If anyone has o1-preview it’d be interesting to hear how well it does on this.

Thank you so much for this.

Whenever people criticize Wolfram the comeback is often, he’s just trying to discuss big ideas that mainstream science won’t talk about. Of course that’s not the reason for the criticism at all and I think your work here shows that it’s totally fine to speculate and get a little philosophical. The results can be interesting and thought provoking.

There’s a difference between big ideas and grandiosity. It also shows big ideas can stay scientifically grounded and don’t require making up corny terminology (Ruliad? lol).

“In 1985 Hinton published a paper ‘How I Intend to Screw Over Physicists by Winning Their Nobel Prize’”

As expected iconoclast Sabine Hossenfelder is quick out of the gate with sarcasm and commentary on this one.

https://youtu.be/dR1ncz-Lozc?si=7kpntqwCzp0HLi02

What a cynical take. Most people working on side projects do so because they find them interesting or useful rather than to just score resume points.

They’re not moving anyone’s goalposts all usage is voluntary.

The fact that HN is mentioned is also confusing. If something had no value beyond paper thin abstraction I doubt we’d be seeing it on the front page with 500 votes.

Even if this project turned out to be less than hoped it seems counterproductive to complain about people creating and exploring. It’s a good thing to see and natural selection will do any sorting needed.

For anyone new to this, the topic has gained interest in recent years alongside the rise of AI. Many ML models project text, images, etc, into an embedding layer in high dimensional space as a vector (an array of numbers). It then becomes possible to numerically compare these vectors to see how closely related they are, in others words to find the semantic similarity.

This nifty feature leads to the need to scale up. For example, given many vectors in a database which one is most similar to my word or image. Applications like this need ways to efficiently scale these kinds of searches and HNSW is one prominent technique to enable this.

I only wish you the best brother.

You seem pretty certain for having only the most rudimentary understanding.

I’m still going to have to disagree. I’d describe the idea that token granularity is the cause of letter counting limitations as a hypothesis, not as something that’s been conclusively shown as far as I’m aware.

I’m not discounting it, or even saying it’s unlikely, but its not hard to imagine other ways it could plausibly be happening.

As a side note when I mentioned training data I was not suggesting anything related to “token metadata”, or to tokens in any regard. Rather, I think it might be possible to instead improve the learning around certain types of counting in a way that could generalize.

I don’t think we’ve yet come to the point where, how an LLM end to end goes from prompt to output is blatantly obvious.

LLMs operate with language at multiple levels of abstraction and tokens are not the only way to have character level knowledge.

For example, nothing excludes pre-training data from directly or indirectly encoding such knowledge.

And of course LLMs famously have emergent properties, for which there’s not yet a precise mechanism to illuminate the results.

We’re dealing with very complex systems that are still relatively poorly understood, and I believe the pool of concepts understood to the point of being blatantly obvious is still small.

The ability to invalidate or critique a solution does not require knowing any part of the solution.

I sometimes wonder what’s more likely, that a towering intellect like Penrose is really advocating such weak conjectures or that he’s messing with us.

I find it fascinating. To me it grounds expectations and motivates discussions about the jagged boundaries of LLM capabilities.

Such an architecture could be implemented, it could use one token per letter, or one token per word, instead of the typical 0.75 per word we see.

The choice just comes with trade-offs in memory usage, compute, and effectiveness of the model in various scenarios.

So what we ended up with was a pragmatic/engineering decision rather than a theoretical or fundamental constraint.

It’s too slow. I can’t stand waiting for builds on what I would call a medium sized project.

I haven’t given 6.0 a full effort yet, but a couple test compiles without any tuning showed only incremental improvements.

The frontier models can all work with both source code and ASTs as a result of their standard training.

Knowing this raises the question, which is better to feed an LLM source code of ASTs?

The answer is really it depends on the use case, there are tradeoffs. For example keeping comments intact possibly gives the model hints to reason better. On the other side, it can be argued that a pure AST has less noise for the model to be confused by.

There are other tradeoffs as well. For example, any analysis relating to coding styles would require the full source code.

What about an autistic person if they could use it for real-time high level cues on the primary emotion being conveyed?

When my son was growing up there were times I wish he didn’t have to struggle or guess as much.

Interesting why would you say that?

There’s not been much opportunity previously to easily fine tune a Gpt 4 class model. I haven’t seen anything written up on this being tried.