randomgermanguy

Very strange article.

I have indeed forgotten how to do long-division, or how to multiply 3-digits numbers in my head. I also forget things without writing them down in a calendar.

How are these bad things ? We have tools for a reason

But a better model almost always comes with a better dollar-per-task-accomplished ratio, compared to worse models.

So not only outpriced, outperformed, but also outregulated.

Idk i wish there was some bright light in the future, but i dont see it

When was FOSS apolitical ?

Definitely something in this realm, they call the models "preview" at a bunch of different points in the paper.

What im really hoping is for a double-punch like with V3 -> R1

> "can I get my coding work actually done today" vs. "this can do customer support chat"

I think you need to define "can get coding work done" for this to make sense. Ive been using GPT-3 back-then for basic scripts, does that count ? Or only Claude-Code ?

I also think this is a false dichotomy, if you look at the Project Vend project or Vending-Bench, customer support etc. is at no means trivial. (Old but great story https://www.businessinsider.com/car-dealership-chevrolet-cha...)

I mean its almost halve a year, i think that counts ?

Only comparing on SOTA scores (ignoring price etc.) is like choosing your daily-driver by looking at who makes the fastest sports-car...

If you found a rare 9000 card with 200+ GB of VRAM, sure

I think the general question is if they'll release it at all, haven't yet read anything stating that they would

One can see the impact of this cultural-wave on people above ~40 pretty heavily.

Hand-in-hand with the whole "Atomkraft ? Nein Danke" campaign. (https://en.wikipedia.org/wiki/Nuclear_Power%3F_No_Thanks)

The major selling point of the tinyboxes is that you're able to run them in your office without any hassle.

I used to own a Dell Poweredge for my home-office, but those fans even on minimal setting kept me up at night

Yes, but in practice land-ownership is only zero sum in places like Europe where every square-kilometer has 300 years of documented ownership etc, or other high-density areas.

The Asia, Africa & the Americas have so much unused space that isn't as inhospitable as central Australia

I think the author might argue, that simply becoming more efficient at creating a rent-seeking mechanism is not beneficial. No matter how well motivated you are to improve your zero-sum game skills, it's still zero-sum.

Or something like that.

You can already buy A100/H100s on eBay. While it might not ever be economical to run these at home (cost of electricity), but it's plenty fun.

Cool idea, but kinda sad that it has to go through a cloud-provider. I feel like there's a possibility with an accelerator-board (Coral TPU or something), to make this into a totally local thing maybe? The longer-waiting time is surely not an issue when considering how many people still use Polaroids.

why would I believe a random account with 1k followers ?

The fact that I have to give them an email for details just feels immediately like a B2B-scam.

Hope they can figure out software, but what im seeing isn't super-promising

I started using https://github.com/kushalpandya/Petrichor a while ago.

Though design is more akin to the default Apple Music app than WinAmp.

Always love seeing a new BMC talk

Made my eyes glaze over only slightly faster than trying to listen to the real Zizek.

Well done!

I guess the question is if it's like the crypto-bubble, where theres no real value left in the end (haven't heard of a good use for those ASICs). Or more like the dot-com bubble where fiber-cable installed is still valuable without pets.com around.

But since I wasn't really around for either of those ... ¯\_ (ツ)_/¯

Alright fair enough.

I dont think this is relevant to the main-point, but it's definitely something I wasn't aware of. I would've thought it might have an impact on like O(100)th token in some negligible way, but glad to learn.

Okay yes, but would you really say that the main part of non-determinism in LLM-usage stems from this ? No its obviously the topk sampling.

I don't think my tech-lead was trying to suggest the floating-point error/non-associativity was the real source.

Topk sampling with temp = 0 should be pretty much deterministic (ignoring floating-point errors)

Oh actually yeah that's true. You have correctly out-nitpicked my nitpick lol.

But at that point i feel like we are getting close to "everything that isn't a perfect Turing-machine is somewhat-stochastic" ;)

Edit: someone corrected me above, it does seem to matter more then I thought

Less then 100 people all-in-all.

Not a joke. I'm still a student, like half a year away from finishing my masters, so switching jobs at this point feels bit risky/early? Maybe im wrong though

Okay thanks for saving my sanity somewhat.

And also just to nitpick/joke:

> More accurately, it is neural networks which are more "stochastic" with their predictions and decisions <...>

I would defend NNs to not even be necessarily stochastic. I had to handwrite weights for NNs in atleast two exams, to fit XOR for example ;)

> machine learning is the sub field of AI.

That's what I tried to explain then as well, and i brought up stuff like path-finding algorithms for route-finding (A*/heuristic-search) as an more old-school AI part, which didn't really land I think.

> Not really stochastic as far as I know. The whole random seed and temperature thing is a bit of a grey area for my full understanding. Let alone the topk, top p, etc. I often just accept what's recommended from the model folks.

I mean LLMs are often treated in stochastic nature, but like ML models aren't usually? Like maybe you have some dropout, but that's usually left out during inference AFAIK. I dont think a Resnet or YOLO is very stochastic, but maybe someone can correct me.

> AI for the most part has been out a couple years.

With this you just mean LLMs right? Because I understand AI to be way more then just LLMs & ML

What do you mean with "building this stuff"? As in building LLMs, or building applications on-top of them.