vova_hn2

I was almost sure that they do it client-side for a joke, but if you check the browser console, you can see that they actually make a request! You can even make the same request with curl and it works!

Although, making an HTTP request manually is quite inconvenient, so I'm waiting for Python SDK.

It's "Showing 1 - 25 of 58891 posts" now. HN tells me that your comment was posted 6 minutes ago, which gives us approximately 37.23 posts/second rate.

1. Is the given tok/s estimate for the total node throughput, or is it what you can realistically expect to get? Or is it the worst case scenario throughput if everyone starts to use it simultaneously?

2. What if I try to hog all resources of a node by running some large data processing and making multiple queries in parallel? What if I try to resell the access by charging per token?

Edit: sorry if this comment sounds overly critical. I think that pooling money with other developers to collectively rent a server for LLM inference is a really cool idea. I also thought about it, but haven't found a satisfactory answer to my question number 2, so I decided that it is infeasible in practice.

Yeah, I was also thinking about it A LOT.

We kinda have a little bit of it with some coding harnesses giving model access to LSP, but I think that we can insert this knowledge on a lower level if we find a clever way to somehow utilize it during sampling.

I think that there is a lot of low hanging fruit in this area.

And in general, I think that people try to use LLMs too much to solve problems that can be easily solved by cheaper (computationally), and, more importantly deterministic tools.

For example, back in the day when LLM-assisted coding just became a thing people very often complained about models generating syntactically incorrect code and inventing non-existent library methods.

Well, I, an experienced human programmer, probably would also be making syntax mistakes and inventing non-existent methods if you stripped me of my tools and made me write code in a bare text editor without syntax highlighting.

Thankfully, my IDE would autocomplete real syntax and actually existing library methods for me and immediately give me feedback if I make a mistake anyway. And all of it is achieved using reliable deterministic code without the inherent issues of statistical models.

I think that it is really inefficient to reach for an expensive and unreliable tool when a cheap and reliable tool will do.

I've always thought that it is kinda weird that we spend exactly the same amount of compute to calculate both "fork" tokens and "lock" tokens.

I think that with grammar-aware sampling / constrained decoding [0][1] it is possible to sometimes skip calling the model altogether if only one token is allowed by grammar and just insert it, but I don't think that any of the current, widely used combinations of models/harnesses use it. And it only skips inference in rare edge cases.

I wonder if there is a more general solution that can make models spend more compute on making important choices, while making generation of the "obvious" tokens cheaper and faster.

[0] https://github.com/ggml-org/llama.cpp/blob/master/grammars/R...

[1] https://developers.redhat.com/articles/2025/06/03/structured...

I remember some old sci-fi book or short story (don't remember which one) that had a spaceship with a separate spinning section specifically for a toilet.

You would enter it, activate it, wait until it accelerated to a certain RPM, do the thing, then deactivate it and it would decelerate until it is stationary relative to the rest of the ship again.

I wonder, how expensive it would be to build this for real.

The rotation mechanism could use a flywheel. Let's say, an electric motor spins the toilet section and the flywheel in opposite directions. So that the rest of the ship is not disturbed.

Size and weight are obviously issues, I just wonder how much would be the overhead. I wonder if the real spaceship designers considered this possibility.

> I knew that it was just a minmax with alpha beta pruning and a really good eval function. Now its not.

It is still "just" a minimax with alpha beta pruning, except the eval function is now a neural network. NNUE, to be more specific.

I highly advise anyone who is curious about chess engines, but hasn't heard about NNUE to read about it. I find this technology absolutely fascinating.

The key idea is that a neural network is structured in a way that makes it very cheap to calculate scores for similar positions. This means that during a tree search, each time you advance or backtrack you can update the score efficiently instead of recalculating it from scratch.

Good starting points to read more:

- https://en.wikipedia.org/wiki/Efficiently_updatable_neural_n...

- https://www.chessprogramming.org/NNUE

the name

It is totally fair, but for a lot of average non-tech people, AI == "something you can prompt in a natural language".

I personally prefer to avoid the term altogether in favor of more specific terms, like:

- LLM

- chess engine

- image generation model

etc

Have you tried Chess960?

Genuine question: how to distinguish yourself from the stream of slop?

I am also annoyed by the endless stream of articles and projects related to LLM-assisted coding. Not because I dislike LLM-assisted coding as an idea, but because it's all more of the same (as you said). I think that there are still a lot of low-hanging fruit in improving LLM harnesses that no one is working on because everyone seems to be chasing the latest trends ("agentic", "multiagentic", "skills") without thinking bigger.

But I'm afraid that if I finally invest time and implement some of my ideas on making LLM-assisted coding better (reliable, safer, easier for humans to interpret and understand generated code), I won't be able to gather any feedback. People will simply dismiss it as "yet another slop for creating more slop" and that's it.

What is the way out of this conundrum?

> random LLM-generated statistic-vomit text

I do not understand why this project in particular have set you off.

Their README looks much better than many I've seen on HN:

- no annoying verbosity, that is so prevalent in AI-generated text - not too many buzzwords (they're not saying "agentic" every sentence) - it is very clear what exactly project is supposed to do and why it can be useful

Personally, I upvoted this because I wanted to do something similar for a long time but never got around to it.

Is it weird that I look at most of the offered pairs and think "meh, both are ok, I guess", but do not feel any preference one way or the other?

Like, some fonts look to weird/unusual that I dislike. But most look just fine and I don't really care.

Am I weird? Do I lack taste?

Why would you need powerful models if you give them such mechanical, stifling instructions?

I think that the result would be much better if you told them what exactly do you want in plain text.

> certainly better than Kings just being able to do as they please

That's debatable. In case of a king you always know whom to blame and who has full responsibility. No opportunity to hide behind "well, you voted for this" or "I'm not making the laws, I'm merely enforcing them".

First of all, I'm not advocating for this claim, I'm merely trying to clarify what other people say.

That being said, I don't think that your analogy is valid in this case.

> GitHub and Windows and IDEs need to be open source because they can output FOSS code

They can output FOSS code, but they themselves are not derived from FOSS code.

It can be argued that the weights of a model is derived from training data, because they contain something from the training data (hard to say what exactly: knowledge, ideas, patterns?)

It can also be argued that output is derived from weights.

If we accept both of those claims, then GPL training data -> GPL weighs -> every output is GPL

> If an AI outputs copyrighted code

Again, the issue is not what exactly does AI output, but where it comes from.

I think that the claim that they make is that once a model is "contaminated" with GPL code, every output it ever produces should be considered derived from GPL code, therefore GPL-licensed as well.

> hey just don't try to copy their LLM by distilling it, cause that's "theft"

They can call it whatever they want, but I don't think that it is illegal.

Normal watch + "arc" remove the assumptions about orientation and direction.

I get that this is an art project that is supposed to look cool and not necessarily be practical, but...

For practical purposes just having 3 "arcs" (hours, minutes, seconds) would be much more readable.

The "arc" idea already solves problems of relying on orientation and knowing that it is supposed to go "clockwise" (it is pretty funny, that the only word that we have to describe rotational direction is related to clocks).

> there is a much higher ceiling on your productivity with other tools and their composability

What exactly is the "ceiling" for GUI based IDEs?

All this talk about AI being "too agreeable" makes me worried that they will make it less agreeable, which will basically force me to justify myself to a freaking clanker, while performing actual practical tasks.

For example, I do not want to hear AI "opinion" on technical choices and architectural decisions that I made when using a coding assistant. If I wanted an "opinion" I would explicitly ask it to list pros and cons or list alternative solutions to a problem.

But I f I explicitly ask AI to do X, it should do X, instead of "pushing back" in order to appear less "sycophantic" (which is a term that is used to describe human behavior and is not applicable to a machine).

Go is very verbose.

Last commits in those repos are 5 and 7 years ago.

> opus scores 97.1% when given an actual vision access

Do you have a source for this? I would be very curious to see how top models do with vision.

> Before AI, typing speed didn’t matter much for programming, because everyone needed to take time to think through problems, come up with a good solution, and debug.

And also because a good IDE would autocomplete somewhere between 70% to 90% of the result.

> but I'm definitely faster at talking than typing

I tend to be much more verbose when talking, compared to typing. So, while talking might be technically faster if you measure words/min or symbols/min, I don't think that the speed of actually expressing ideas is higher. Talking just tends to have more repetitions (reiterating the same idea with slightly different wording) and filler words that don't really mean anything.

Cool idea, slightly confusing UI (why after selecting a sound I have to find it again to see the "info"? why can't I just have a button that displays "info" about the currently selected pair?)

Tool calls can have JSON schema enforced on lower level (token sampling). Although, I'm not sure if major providers do it, but I don't see any reason why they wouldn't.

I found this article extremely frustrating to read. Maybe I lack some required prior knowledge and I am not the target audience for this.

> built on the Arm Neoverse platform

What the heck is "Arm Neoverse"? No explanation given, link leads to website in Chinese. Using Firefox translating tool doesn't help much:

> Arm Neoverse delivers the best performance from the cloud to the edge

What? This is just a pile of buzzwords, it doesn't mean anything.

The article doesn't seem to contain any information on how much it costs or any performance benchmarks to compare it with other CPUs. It's all just marketing slop, basically.