kolbe

I think the bigger issue is hardware utilization. Meta and Qwen's medium sized models are fantastic and competitive with 6 month old offerings from OpenAI and Claude, but you need to have $2500 of hardware to run it. If you're going to be spinning this 24/7, sure the API costs for OpenAI and Anthropic would be insane, but as far as normal personal usage patterns go, you'd probably spend $20/month. So, either spend $2500 that will depreciate to $1000 in two years or $580 in API costs?

They might be talking about 70B

AMD GPUs are a fantastic deal until you hit a problem. Some models/frameworks it works great. Others, not so much.

If you want to wait until the 5090s come out, you should see a drop in the price of the 30xx and 40xx series. Right now, shopping used, you can get two 3090s or two 4080s in your price range. Conventional wisdom says two 3090s would be better, but this is all highly dependent on what models you want to run. Basically the first requirement is to have enough VRAM to host all of your model on it, and secondarily, the quality of the GPU.

Have a look through Hugging Face to see which models interest you. A rough estimate for the amount of VRAM you need is half the model size plus a couple gigs. So, if using the 70B models interests you, two 4080s wouldn't fit it, but two 3090s would. If you're just interested in the 1B, 3B and 7B models (llama 3B is fantastic), you really don't need much at all. A single 3060 can handle that, and those are not expensive.

The only thing you ever say is baseless criticisms of the intellectual prowess of HN and the media. Either justify these criticisms, accept we are the way we are, or leave.

Sounds like his problem

Just like advertising--if they have earned my attention by saying things I want to read, then they have the right to dilute its quality with whatever else they want, up until it net doesn't interest me anymore. In this case, the jab is tiny and the quality content is bountiful.

It's likely probabilistically linear... like speeding on a street with random traffic lights.

Maybe. Gemini Exp 1121 is blowing my mind. Could be that OpenAI is seeing the context window disadvantage vs Google looming.

I still like ispc, but that's not going to catch on.

You must never have seen Breaking Bad

They did it in the 90s as well, and it worked out fabulously.

"The internet is a thing? Okay, now _everything_ is the internet."

They lean hard into new technologies. That's the nature of VC. Sometimes they're right and sometimes they're wrong.

I think the reality is that it's too costly to put together a process that filters better. This has been deemed "good enough" and they can fire you if the filter didn't work properly.

I don't mind the coding challenges necessarily, but more how they're implemented. I find them to be far too broad relative to the day-to-day, and it only showcases how quickly someone can write unthoughtful code.

For example, it's not uncommon for a challenge in finance to be "build an exchange matching engine that takes orders, modifications and cancels from different threads in two hours." That's totally doable if you just fly through whatever comes to the top of your head, and maybe have an hour left over to test, debug and modify it. But that project (an exchange matching engine) is something that the CME has 10 full time programmers working all the time on. A more realistic 2 hour task is something like "look at edge cases for the order cancel type, and figure out a way to handle malformed inputs." Then you can actually write production quality code for them to judge.

Scalar languages can go a long ways in this exercise. There are a few hiccups in trying to vectorize scalar languages, but see how well clang's latest -O3 models can do on C++.

I would love to see ISPC take off. It's my favorite way of abstracting code in this new computing paradigm. But it doesn't seem to be getting much traction.

I think the future is also going to be less "SIMD" and more "MIMD" (there's probably a better term for that). You even see in AVX512 with things like aggregations (which aren't SIMD), that there's no reason a single op has to be the same exact operation spread over 512/(data size) slots. You can just as easily put popular sets of operations into the tool set.

How does this argument square with them driving a $150k BMW?

Agreed. I gave up on trying to navigate the fraud. I've gone once in the past 15 years because my wife begged me to. I have yet to regret it whatsoever.

I find it less that the fines are trivial than it is fines are levied on the wrong entities. Consultants break the law--fine them personally. Police's officers abuse power--fine them personally. Bankers shirk regulation--fine them personally. Nothing will ever change if the people committing the acts aren't the ones punished.

He's high quality research by comparison to the absolute garbage that exists today--not by say classical Frontline or 60 Minutes standards

I don't think people who are so bitter about not being in a position of power that they will say any hyperbolic thing they can about McKinsey should be trusted on the topic either

If all comments here were eyed with the same cynicism, we'd have no genuine discussions. And maybe we don't.

How did that strategy work out for Intel?

The H100 has 16,000 cuda cores at 1.2ghz. My rough calculation is it can handle 230k concurrent calculations. Whereas a 192 core avx512 chip (assuming it calculates on 16 bit data) can handle 6k concurrent calculations at 4x the frequency. So, about a 10x difference just on compute, not to mention that memory is an even stronger advantage for GPUs.

For numerical PDE solvers, you never do the actual inversion, but the discretization of the system (denoted A) can have significant sensitivity to numerical precision. Look up Condition Number and Preconditioning. In fact, resolving the problems with numerical precision for some A takes as much time as actually solving the system, and it's worth it.

Until my hardware implements f128, even if I had a solid use for it, I cannot sacrifice the speed relative to f64.

I love how the comment I'm responding to literally says "then collect $250k per year risk free after taxes," and then you all pile onto me with downvotes telling me that's he's not just going to invest in treasuries (which is exactly the implication of HIS comment and not mine).

The original comment was premised on them being income-generating assets, which gold and btc are not

And I would argue his "soft power" is greatly diminished as well

Assuming they're 40, how far do you think $250k will go 20-30-40 years from now? It's not a stretch to think dollars could be devalued by 90%, possibly even worthless, within 30 years.

I don't think Gates has much influence on Microsoft these days.