Hacker News Reader

> the next task is SO HARD that it doesn't make sense to not use frontier

How does tokenless quantify "so hard"?

> 1 turn to crack versus 100 turns for Deepseek to crack

Interesting definition for a "frontier". What is a "turn" here? Token count? Request count? Context-based?

I've found that MiniMax M3 (a smaller model at 295b) will code up better when DeepSeek v4 Pro (1.6t) will not (and vice versa).

View on HN

Highly encourage you to read the blog post (https://usetokenless.com/blog/building-tokenless). Essentially, we estimate the confidence of a specific model failing or succeeding on a specific task using our own foundation models.

A turn here is a tool call/user input, anything that causes the model to get some new input. We're working on adding Minimax M3 and other models. We think that people have some intuitions about which models are good when--we seek to quantify them scientifically.

It will inspire more confidence if you say your classifier. Saying "your own foundation model" (charitably) suggests marketing hyperbole. If you're fine tuning a LLM to do this, at best it's going to be worse than a frontier model with suitably tuned skills (hence it should just be something in harness, not a service), and I wouldn't refer to that as "your own foundation model."

Reply on news.ycombinator.com