Gracana

The eqbench creative writing "slop profiles" do something similar. https://eqbench.com/creative_writing.html

Click the (i) next to the slop score for any model and it will show other models that are similar in terms of their most commonly used words and phrases.

> Literally sat and typed out Monty Python dialogue

Dear god.

The insistence that you must "work alone" and that similarity between submissions will get you flagged would make me really uncomfortable. I think I would be afraid to talk to the TAs lest I get poisoned by their knowledge.

Yeah, I had to go high-end to get one with enough CCDs to make use of the memory channels. I'm curious to know what all-read bandwidth you see when you run intel mlc on your system. My Xeon W5 was doing ~180GB/s after a lot of tuning, which was very disappointing. The Threadripper does ~320GB/s. From what I've heard, Granite Rapids is supposed to have solved the bandwidth problems that ailed Sapphire Rapids, so your numbers ought to be similar.

I wanted to do that, but didn't like any of the PCIe switches / backplanes I found (mostly due to price). Now, it might be worth buying one of these instead of trying to build a fancy PC. https://c-payne.com/products/pcie-gen4-switch-5x-x16-microch...

Wow, somehow I didn't know about this. Now I'm angry, bundling an official Macintosh BASIC on every computer would have been huuuge for young-me.

On the desktop Intel has better memory controllers last time I looked, but it’s not that big a difference. Something in your setup is just broken. Failures will happen with any brand, so I wouldn’t chalk it up to AMD vs Intel… figure out where your problem is and get the faulty part replaced if you can.

I feel you though, regardless of the cause, that is an extremely frustrating place to be.

Zen 5 really is nice. I swapped my RAM from a Xeon W Sapphire Rapids machine into a Threadripper Pro 9000 series machine and I get almost double the memory read performance, plus it's a heck of a lot faster in single and multi core performance, and it runs cooler and quieter. Huge win all around. Aside from the price... (I went from Xeon W5-3435X to TR Pro 9985WX, eep.)

Interesting, thanks for the link. Hopefully this will all become very relevant for me soon!

Because I'm not friends with all 64 players in the lobby.

I don't want kernel level anti-cheat, though. All the games with it have cheaters anyway.

I'll probably manage with Mac OS well enough, but my linux distro comes out of the box with all the latest OSS tooling I'm familiar with, plus a package manager, and it has linux cgroups and namespaces that power the container technologies we all know and love.

If I switch to Mac OS, I have to sort out a package manager and install all the stuff that's missing, and when it comes to containers... they're just linux VMs. I'd happily cut out the weird proprietary middleman if I could.

> And why would you need that to run LLMs?

kokonokko1337 already said it was good enough to run LLMs, presumably RunSet isn't saying the source code is needed to run an inference server.

I think people responded that way because you strongly implied he was an unserious vibecoder who was just fooling around, and you called him suspicious as fuck.

Your post and the ones that followed are a good example of the contrarian dynamic that dang often talks about. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

I think you would be pleasantly surprised by the content of the linked article.

The thing they don't show is the one we really need, especially because model providers can skimp on quality (run lower quantization, lower kv cache precision, etc) to improve their pricing and performance. I agree that it's probably too expensive to keep running the benchmark, but we need some way to hold the providers to a certain standard, otherwise every user has to discover the problems on their own.

Haha, yeah, I mostly remember that they exist and then go look up the actual quote.

Rule of Acquisition number 34: War is good for business.

Rule of Acquisition number 35: Peace is good for business.

I can’t say I really disagree, and as a visitor of the site that’s the solution I would prefer.

It’s done out of necessity, you can read about why here: https://people.kernel.org/monsieuricon/creepy-crawlies

As a user/reader/viewer I absolutely hate Anubis and usually turn around when I see it pop up (at least on my phone where it takes ages to compute), but with stats like that, I get why a site operator would resort to using it.

llama.cpp (but maybe not for reasoning models?) and sillytavern, maybe others... I'm half a country away from my desktop right now so I can't verify much right now.

I had a discussion about this with a friend recently. I was using an LLM with a serial console that it wrote to allow us to share a connection to a Z80 computer running CP/M. I was having trouble with the assembler, it wouldn’t assemble in user 1 of a particular disk, so I asked the LLM for help. It debugged ASM.COM and the BDOS, walked through the source compared to memory dumps, basically banging its head on the wall for an hour or more, until it finally figured out and verified that my disk had a bad block. I never would have put in that much effort to find the root cause of the problem.

Some LLM interfaces allow you to modify and “continue” an agent response. It’s very useful for guidance, including jailbreaking. Need the model to go in a certain direction? Got a refusal that you want to bypass? Just start it off in the appropriate direction and then have it continue from there.

Oh, interesting. I want to say there’s a “preserve thinking” option in mainline llama.cpp, but I’m seeing mixed results in my searches and my in-flight internet is not good enough to dig into it properly. Good to hear you have a solution though, because that is likely something I’ll run into as well.

I’ve noticed dsv4 do that as well, but inconsistently. I thought it was broken at first, but no, it’s just kind of shorthand that it does while thinking.

What’s that about the template breaking the kv cache?

Yeah, of all the parts to leave a out, sorting out a little enclosure is certainly the easiest and most likely to require customization.

I'm surprised that it's pretty much the same as the adapters you can get on aliexpress. Not even a case to protect it from feet in the passenger footwell. Especially compared to the windshield-mount unit, which looks like a very slick product. I suppose they can always improve it if there is interest in this first version.

That’s a good one. And yes, of course I mean its history of usage in computing, not in general.

I wondered how far back the usage of that term went. I was familiar with it in patch, so I did a little digging and found it in the v1.3 (1985!) source by Larry Wall: https://groups.google.com/g/mod.sources/c/xSQM63e39YY

What influence do you have over these companies where your trust or distrust matters?

[edit] This is an awfully pessimistic take. Even if these companies will largely do whatever they like, some of us can at least make choices to use alternatives. That is probably what you were getting at.