redox99

I tried their WebGPU version and it immediately started looping. Yeah "near lossless" my ass. Plus the reasoning that it looped on was clearly wrong and unlike the non quantized 27B

Not everyone uses laptops. I can run Qwen 3.8 27B (which is a REALLY capable model) in the background coding for me while I'm simultaneously browsing the web and playing VALORANT without any performance impact, and that's on a 6 year old GPU.

> But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser.

Then just allow it to be enabled on high end devices? But it must be local only. As hardware advances and people upgrade, more people will be able to turn on the feature.

Targeting byte code or asm instead of high level would be silly for everyday tasks. You blow up the number of tokens, reduce your effective context, and there's just more places for it to make a mistake, which most likely won't be caught by the assembler (unlike compilers).

I'm so happy for the two used 3090s I bought for $500 each after Ethereum mining ended. I even saw them for like $430 at some point lol.

The math is wrong, the tok/s is at least 2x that, at least with MTP and Q8 KV which you should always use. And the default tokens a day is ridiculously low at least for coding.

Having said that, it will never pay for itself. A simpler more absolute math is, if I buy a Mac and use it to sell tokens on OpenRouter, will I make a profit? And the answer is no.

Terminal bench 4 is good largely because it's recent so it hasn't been benchmaxxed yet. It's more of a sysadmin/devops benchmark than a coding benchmark though, but still a decent proxy.

https://artificialanalysis.ai/evaluations/terminalbench-v4-0

They may allocate different number of resources every year based on market conditions but they'll never give up on gaming, that would be extremely silly.

I think in a few decades when there are more humanoid robots than humans we'll likely have skynet, so I'm pessimistic in a way lol.

But in the near future and on a personal level I think we'll need to adapt and pivot but we'll manage.

I think it will be like how a lot of people know how to code in python but have zero understanding of assembly or how a cpu works. As a researcher you'll accept there's this low level stuff that if you want you can dig into but isn't worth your time, like looking at the generated ASM isn't.

You're exaggerating the LLM progress a bit but yeah progress has been crazy. Yet not much has changed right? We mostly have the same jobs, just code wayyyy more than before because things that wouldn't be worth it now are worth it.

10 years from now I have no fucking idea. But I'm sure in the meantime those who leverage AI will do better than those who yell at a cloud.

Of course you're not going to get rich with the kind of software that LLMs can one shot these days. But that kind of software like to-do lists or basic CRUD have been saturated for over a decade, way before LLMs. People overestimate how much you can one shot, yeah a good prompt can get you 90% there but that 10% remaining often takes months of extra work.

Software has always progressed this way, lots of devs back then would work on business websites that have been 99% replaced by wordpress, squarespace and instagram.

I'm sure it's the same with research, you're going to tackle problems that would have not been worth the effort or outright impossible without AI. The old stuff that you'd work for months, yeah that's going to be a prompt away.

This has always happened, way before AI. You'd spend months or years building and growing your business, and then Google would release a feature or product that would kill your business overnight because they can throw way more money at the problem, plus their branding. That's life.

> Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.

Those who think it's me or the machine will fail.

Those who realize how much you can accelerate your research with the help of AI will succeed.

The two times I tried to use fiverr I literally got ghosted.

This is orders of magnitude cheaper and faster than paying for a song to be created from scratch by a human.

Yeah I tried GPT first because that's what I always use, it refused and I didn't even bother trying to trick it, just went straight to Grok.

Can you use Lean to... prove "Lean-fast" is equivalent to Lean?

Btw grok found out in less than ten seconds what place you refer to and likely what software within that company.

Things that wouldn't be worth your time before are trivial these days with LLMs.

You can argue all day whether cars are good or bad. That's not my point. My point is that pretending public transport is equivalent to cars if you have good enough infrastructure is silly. They are different experiences, each with their own tradeoffs. And those tradeoffs matter differently to different people.

Unnecessary is a very strong word. It's also unnecessary to have your own private house, sharing it with other people would be more efficient.

Public transport is not the same as cars. It's silly when people pretend like they are interchangeable and it's just a matter of having "better infrastructure".

With the miles you can establish a confidence interval of the true fatality rate.

Astra is definitely weird. It is more capable than Sol, no doubt about that. There are things sol could simply not solve that Astra breezes through.

However for typical low to medium difficulty code, it will often either overengineer stuff, create massive functions instead of organized code, and just write very hard to read code. It literally looks like minified code. Clearly they trained it to reduce the number of output tokens and in turn the code is often atrocious. I'll keep trying Astra but I might actually go back to 5.6 sol for many tasks if I keep getting these results.

Just use opencode/pi/etc with an open weight model?

Thanks for the reply. I'd suggest you improve your workflow with better tooling like Codex or Claude code, and if you have some kind of weird constraint they'll easily follow an AGENTS.md with that.

Those are really nice numbers. With that t/s, no network latency or queueing it must feel much snappier than cloud models.

Do you... code by copy pasting chatgpt web???

Why not? It makes less mistakes and with subscriptions it's very cheap

Stuff along the lines of implement controller service and tests for the following endpoints:

- list of many endpoints with the JSON they receive and return and description of what they need to achieve

Stuff you could probably do in a single work day if you lock in and enter flow state, but in a typical job takes like half a week. And the vast majority of times the AI one shots it with no bugs, where I would have copy paste errors or dumb stuff I'd need to fix before it's shippable.

Your prompts are probably very underspecified then. Frontier models one shot the majority of my prompts. UI is kind of the exception, there I do have to ask for a lot of tweaks.