Hacker News Reader

Comment by qzw | Hacker News Reader

I'm sure NVIDIA is a serious pain in the ass to deal with (Linus_middle_finger.gif), but perhaps one of the unspoken reasons is that, as suggested by the rumor mill, there's a serious oversupply problem with next gen GPUs. NVIDIA and AMD placed orders with the foundries based on the ridiculous demand levels they were seeing in the last couple of years, but now demand has cooled off from both crypto (Ethereum merge) and gamers (waiting for new gen), so there's still plenty of current gen stock sitting around. Supposedly they tried to cancel some of the volume with the foundries, but were not allowed to. So maybe EVGA is seeing the writing on the wall and getting out ahead of the game. I'm sure putting up with all of NVIDIA's crap is barely worth it even when making a decent profit. But it's definitely not worth it to EVGA if their cards will just to end up languishing in the bargain bin along with everyone else's.

View on HN

>I'm sure putting up with all of NVIDIA's crap is barely worth it even when making a decent profit

It's illustrative to remember who and what started this: nVidia's practice of intentionally undercutting their manufacturing partners. See, manufacturing partners tend to apply secondary binning of their cards and sell them with an overclock thus delaying their obsolescence for a year or so, and that's bad when you're in the business of selling the chips themselves (while neutral to the secondary manufacturers; you just buy a new GPU from them a year later, who cares).

So it really isn't a surprise that that, combined with artificially limiting supply for the 40-series GPUs (and most of that allocation is likely going to be to, well, themselves), and now the fact that the price that nVidia likely demanded for those GPUs is coming around to bite the secondary manufacturers, meaning that they all bought (or were required to buy) the GPUs at inflated prices, they're leaving the business.

And guess what manufacturer wasn't forced to pay that premium? That's right, nVidia! So they alone have room to drop prices and still make a profit, while making the value proposition even worse for the secondaries (price premiums for overclocking make less sense as they diverge from the base model and those improved coolers are more expensive to make than nVidia's solutions).

EVGA is understandably the first to bow out; they've never dealt with AMD (who have supply problems for other reasons), whose GPUs haven't been the premium product for a while (arguably, ever), and if they say it's no longer profitable even before a primary usecase for GPU compute vanished overnight I'm inclined to believe them.

And sure, there are still going to be secondary manufacturers producing cards, but most of the other manufacturers are either do-all companies (large OEMs more famous for motherboards and laptops), produce AMD cards too (Zotac-Sapphire-Inno3D), or are otherwise diverse enough (PNY) that ramping down production wouldn't kill them as a company...

> they've never dealt with AMD (who have supply problems for other reasons), whose GPUs haven't been the premium product for a while (arguably, ever),

Yep. AMD can't write Windows graphics drivers that are stable and performant to save their lives, their cards use enormous amounts of power compared to similarly performing NVIDIA cards, they're nearly useless for computational stuff because most everyone goes for CUDA, and AMD cheated on benchmarks by overclocking/overvolting their cards and slapping huge heatsinks on them so that they'd last just long enough for a benchmark to run before starting to overheat.

There are only two markets for AMD cards that I see: Hackintoshes (increasingly irrelevant with the shift to the new Apple Silicon and inevitable phase-out of Intel-supporting MacOS releases), Linux systems, and AMD fanboys. All pretty small markets.

AMD is properly fucked in the GPU market now that Apple is no longer using discreet GPUs unless they manage to bring to bear the sort of enginering might they did for Ryzen (where they're also slipping as Intel pushes back.) NVIDIA can carry on with all their datacenter-class ML products and lose a bunch of money on the consumer side. The problem is...if AMD drops out of the GPU market, that leaves NVIDIA and Intel, and reportedly Intel's product is even worse than AMD. If NVIDIA is insufferable now, imagine them when they're the only game in town...

> There are only two markets for AMD cards that I see: Hackintoshes (increasingly irrelevant with the shift to the new Apple Silicon and inevitable phase-out of Intel-supporting MacOS releases), Linux systems, and AMD fanboys. All pretty small markets.

You're forgetting two more, one of which is massive: consoles and high-performance embedded gaming systems (basically, the Steam Deck and other high-end handhelds). Consoles aren't going to be super profitable on a per-unit basis, but AMD is making the chips both Microsoft and Sony are using in their boxes, and they're selling a lot of those boxes (right now, about 36MM since they debuted a little under two years ago). Similarly, the Steam Deck is a pretty damn good first attempt at a cohesive handheld that runs PC titles, and I could see that extending past Valve and into a generalized marketplace (and the fact you can plug in a USB-C cable into a dock and get a real Linux desktop is really, really cool).

and Teslas. Seems like AMD’s play is high performance (somewhat) custom silicon as a service.

I was very specifically speaking about the market for AMD's GPU cards, in the context of a discussion about GPU cards. I'm not being pedantic here.

AMD's embedded GPU unit may do fine, and it would be smart for them to shift their development priority to retain that market share. They lost the desktop/laptop discreet GPU battle years ago.

Replace GPU with CPU and you sound like someone who was very wrong after 2018 came around. Never say never.

I just looked at different benchmarks and the general consensus seems to be that latest Radeon and Nvidia cards are quite similar in performance. Nvidia seems to lead in raytracing and upscaling but in my opinion only the upscaling part is really important. And I bet that all GPU makers will have a decent solution available for that soon.

You are also forgetting another market: HPC. AMD MI200 GPUs are the boss cards to have if you want to do HPC right now.

That's the first time I heard about that, do you have any examples? Google shows me one example of Microsoft using them but that's all I can find.

One I know of is the EU-funded LUMI supercomputer: https://www.lumi-supercomputer.eu/lumis-full-system-architec...

"The AMD MI250X GPU is in a class of its own now and for a long time to come. The technical supremacy and performance per watt were the primary reasons why AMD’s MI250X GPUs were selected for LUMI, explains Pekka Manninen, Director of LUMI Leadership Computing Facility."

Also El Capitan supercomputer, and there are more.

You might look at the current No. 1 on the Top500 list, Frontier at ORNL. https://www.olcf.ornl.gov/frontier/ Arguably the most powerful machine today, delivering >1 Exaflop/s performance.

Aren't all the game consoles running amd?

Yes, and I'm absolutely certain they're kicking themselves for having agreed to do so.

As far as I know, console manufacture comes with supply agreements- you agree to supply X number of processors at Y cost... even if the price of those processors, when fitted to a PCI Express card instead of a PS5 motherboard, would net 3 times the profit.

It's also a nice way to get out of having bad drivers- if developers are making games to the PS5 standard, then any errata becomes their problem more than strictly AMD's...

I doubt they were kicking themselves considering they knew what was coming based on their experience with the last gen consoles.

It's far more nuanced than this, really. Much like Apple funds a lot of TSMC's early R&D for those nodes - it's not just that they're willing to bid more, they are a risk partner in the development of the technology itself - console companies fund a lot of AMD's early-stage graphics development. Without their funding, the Polaris/Vega era of GCN would have seen even less iteration from AMD. And they often also fund custom variations on the general architectures, which helps keep hardware engineers employed. And they also contribute to AMD's wafer purchases, which lets AMD get better deals on the rest of its stuff because they're buying 2x as many wafers as they could use themselves.

The situation with TSMC being super super limited during the pandemic is unusual, and it was significantly amplified by the crypto boom happening at the same time. Normally it's actually a pretty good deal for AMD in a lot of ways.

The console companies tend to be very very conservative in their designs though and letting the console companies set the direction has left AMD significantly behind on many task-specific accelerators like tensor and RT. And tensor is no longer a niche thing, neural accelerators have been everywhere in phones for ages, Intel's laptop chips have had neural accelerators since 10th gen, intel desktop 11th gen had it, AMD's own 7000-series CPUs have neural accelerators, Intel GPUs have it, etc. Different product segments are finding different uses for it, and AMD has been not just left out of the rain but actually is slowing the development of the whole field because of how tardy to market they are. And beyond that they've taken a strongly anti-consumer position against open APIs that allow other brands to utilize hardware accelerators or closed-source libraries.

https://youtu.be/8ve5dDQ6TQE?t=974

Similar problem with RT... AMD's implementation is very lackluster and has been slowing the whole industry down. It's about half the performance of NVIDIA's first-gen implementation. A lot of times it just seems like AMD automatically assumes that NVIDIA is incompetent and wasteful, and there's some big savings to be had, so they'll take whatever NVIDIA does and cut it in half or a third or whatever (which is what they're rumored to do with tensors next generation - it has "accelerator instructions" but less powerful than the full dedicated units on CDNA, and presumably weaker than NVIDIA's dedicated units as well). Not sure if it's that, or if they're just cheap on silicon, especially with the console market's influence...

You're overselling the importance of console chips for AMDs bottom line. Console chips have notoriously low margins due to being custom in nature and cost down to oblivion, making their development an expensive PITA with very low margins.

It's why Intel and Nvidia refused to work with Sony and Microsoft for their console chips and only AMD took the original contract since they were close to bankruptcy at the time so every extra dollar mattered for them.

What I'm saying is that the final sale price isn't the only relevant number here - Sony and MS are paying to develop the graphics architecture in the first place. That's revenue on the frontend that helps develop the chips in the first place. The B2B sales of the console chips that result are lower-margin than the dGPU sales into the broader consumer market, but, it's hundreds of millions of dollars of R&D money that AMD doesn't have to find elsewhere a half decade before the product goes to market. That money has a lot of time-value, because it's available now, not 5 years and 10 billion dollars from now.

Early VC funding rounds are way less than you'll see when your product goes to market too. Applying your accounting philosophy, a couple million bucks surely must be insignificant compared to billions of dollars of sales revenue we'll see later, right? And they want how much equity? What a terrible deal.

Except for the part where you fold years before you get to "later", of course.

And that was a very, very, very real concern with AMD. They were right on the edge for years and years during the Bulldozer years, Ryzen was very much a last-chance move for them and they were obviously still scraping bottom for the first couple years even then. There are numerous factors that all aligned to keep them alive - they would not have been alive without Intel misstepping on 14nm (the early days were very bad) and then 10nm, they would not have been alive without Apple writing huge R&D checks to advance TSMC 7nm and 5nm far enough ahead of the market that GloFo bowed out of the game, they would not have been alive without consoles keeping the lights on while they spent every penny on Zen R&D, and they would not have been alive without their interconnect turning all their IPs into legos and allowing them to break and scale them.

Things were really, really, really touch-and-go for AMD in 2016/2017/2018. I figured they were going under. I have no doubt that console R&D money (along with those other factors) was instrumental in keeping the lights on at RTG.

> even if the price of those processors, when fitted to a PCI Express card instead of a PS5 motherboard, would net 3 times the profit.

The GPUs in Xboxes and Playstations are custom silicon.

True, but they are based on the RDNA arch. Same as the CPU cores are monolithic Zen cores.

It's not like they have any other options. Who else is there? NVIDIA? Microsoft's issues with them shows how precarious that relationship is. I'm sure issues with NVIDIA are why we haven't seen an improved Switch yet and I think there are clear concerns about potential SOCs for a Switch 2. Unless you have to, you don't want to work with NV.

All except Nintendo ;(

RDNA2 APUs are in a similar place as the Tegra X1 was when the Switch first came out; I don't think it's impossible Nintendo switches back to AMD graphics, they have a long history of shipping Radeon in their home consoles all the way back to the GameCube.

> their cards use enormous amounts of power

People on enthusiast forums are now talking about buying 1.2kW+ power supplies for the new generation of nvidia cards. I don't think you can point at one or the other as bad here.

people on enthusiast forums have been sold a bill of goods by a couple attention-seeking twitter stars like kopite7kimi.

remember his rumors about 4090 having a 800-900W TGP? That got wacked way down in recent months by more reputable leakers, and now the number is rumored to be around 450W TBP (more like 375-400W TGP), which is plausible given the size of the die (and much more comparable to the rumored 400W of the dual-die AMD products).

So, off by basically double.

https://hothardware.com/news/geforce-rtx-4090-ti-allegedly-a...

Similarly he had 4080 at 600W and other Shit That Ain't Gonna Happen. Stupid partner boards pushing everything to 2x the official spec? maybe, but as a reference TGP? lol no.

if you believed that 800W was ever a plausible configuration for a consumer product, then I have a bridge to sell you. It was most likely some kind of engineering test bios that turned everything up to 11 at max clocks and he heard about that number and just didn't apply any common sense. Which, why would he when he could get months of attention off it from twitter?

People just race to buy into anything negative that is rumored about NVIDIA products and it's taken as absolute gospel as soon as anyone hears it. And the leakers have realized that and play into it.

I think that the fact it was even plausible to people who are fans of nvidia, and that it is discussed on said forums, means that customers don't have any expectations of nvidia keeping a lid on the power requirements. Both big manufacturers are recommending at least 750W supplies for their current flagships.

As such I still don't think it's fair to characterise just AMD as wasteful power hogs here, the post I was responding to seems to me to be unnecessarily partisan.

> AMD is properly fucked in the GPU market now

AMD GPUs are in XBox and Playstation.

> AMD can't write Windows graphics drivers that are stable and performant to save their lives

When they changed to RDNA they had teething issues. But their drivers are stable now.

> their cards use enormous amounts of power compared to similarly performing NVIDIA cards

Is this a joke? The 6800 XT draws less power than a 3080 12gb while having similar FPS. Ignoring FSR2 or DLSS.

Disclaimer. I've only ever had 1 AMD card (5700XT) and it was rock solid. Currently using a 3070 and 3050Ti.

citation needed.

I've had zero problems with AMD hardware on Windows (Linux kernel, on the other hand ootb still won't boot with this Vega as the amdgpu driver is a disgrace) and other than windows 11 being a complete failure, nobody has reported serious problems.

re benchmarks/power, they scale dynamically and do not have as tight power envelopes as other vendors, so obviously it will use more power [citation needed also].

Their generic GPUs are very fast, but why do they even care when they own the console market, the higher end in car infotainment market, etc?

What? Did you go out of the loop at least 4 years ago?

>AMD can't write Windows graphics drivers that are stable and performant to save their lives

I have an AMD GPU in my Windows PC and it works fine. It's stable. n=1 of course.

The reason I bought it is because I have a linux machine that currently has an Nvidia card. Fuck Nvidia and their linux drivers.

>their cards use enormous amounts of power compared to similarly performing NVIDIA cards

Don't think this is true at all. Eg first link I came across https://www.tomshardware.com/features/graphics-card-power-co...

Holy shit, is this guy being paid by nVidia?

> whose GPUs haven't been the premium product for a while (arguably, ever)

It wasn’t until the nVidia Geforce 285+295 that they unequivocally regained the performance crown (three years after the acquisition). AMD was still competitive, at the flagship level, until about 4 generations ago; so they were a reasonable alternative, if slightly less performant.

It was the constant rehash of GCN cores that screwed them up, with their limited ROP configuration and generally weak Shaders (after the third+ respin).

That lines up with the impression I got. They lose money CURRENTLY on the high end cards, it sure wasn’t going to get any better.

Might as well get out while the getting is good.

Unsurprising, considering the stupid games NVIDIA was playing during the past two years. There are no cards, but vendors aren't allowed to raise prices. Thus creating a cottage industry of zero-value-add scalpers.

It's not shocking that the vendors aren't interested in getting zero of the upsides, but all of the downsides of the bullwhip effect.

I was actually suprised to hear that high end cards are the ones where money is lost. Typically high end products are the ones with by far the biggest margins. They might just not be worth the R&D effort - but in this case I doubt that one is the problem.

To lose money, it seems like the raw chip price differences between mid level and high end cards would need to be even higher than what the MSRP differences are.

But if that's the case, why not just sell midrange cards? Or just a tiny amount of high end ones for advertising purposes.

>To lose money, it seems like the raw chip price differences between mid level and high end cards would need to be even higher than what the MSRP differences are.

NVidia's datacenter products seem to charge for GDDR like it's made from gold. Memory prices may play a role.

> I was actually suprised to hear that high end cards are the ones where money is lost. Typically high end products are the ones with by far the biggest margins.

two things:

first, high-end cards are the ones that miners liked, so, partners really loaded up on orders of those cards. They sold a huge number of high-end cards relative to previous generations. And now the mining market has crashed, and they're stuck with a huge number of high-end cards (relative to actual sustainable demand) both at the vendor (NVIDIA) level and the partner (EVGA, et al) level. And miners are dumping their cards, so you have a huge number of high-end cards flowing back into the market too.

secondly, the card he cites as being "their highest-margin card" is a model that was a bad deal at launch and has seen virtually no price reduction since then. I have actually seen quite a few people commenting on just how little prices of that specific model have dropped, with the 3060 Ti and 3070 pushing downwards (and those are significantly faster than the base 3060). On the AMD side, you can now get a 6700XT (which again, significantly faster) for the same price, which is a performance bracket higher. It's an oddity of that specific model.

so, what he's saying is, an overpriced card that has seen no price reduction while much faster cards crash around it, has high margins - "no shit", as they say. Doesn't mean anybody is really buying it though.

the whole thing is very much "true, from a certain point of view"... like his "wow we're losing money!". Yeah, now that the GPU market is crashing you're losing money... and during those years, the company as a whole lost money because this CEO has been doing all sorts of zany business ventures that ended up in massive losses... the GPU division made money hand-over-fist for the last 2 years, and he's lost money hand-over-fist by trying to break into the enthusiast monitor market and enthusiast motherboards (both extremely support, warranty, and R&D intensive markets) and doing a ton of branding deals where he sticks EVGA logos on hdmi capture devices and pcie sound cards (in 2020, seriously) on products licensed from other vendors (who it turns out were skeezes and EVGA was on the hook for a ton of defective and falsely-marketed products).

https://youtu.be/QsICWRnj-60

he ran the company into the ground and is trying to shift blame to NVIDIA. Yeah, the board-partner thing is predatory, but partners don't really add significant value to the product anymore, and middlemen get squeezed out of business everywhere. Yeah, EVGA is in deep shit, and he's lying when he says they're not going to go under, the writing has been on the wall for months now. The GPU market crashing is almost certainly the last straw for EVGA and they won't be able to pivot to their remaining markets.

But like, he did make a ton of money on GPUs over the last 2 years, he just lost a ton of money on other stuff, and that's not NVIDIA's fault. Partners in general ordered way too much (and again, particularly the high-end stuff) trying to cash in, they make fucktons of margin on it during 2020-2021, and now they are lobbying NVIDIA to cover their downside and buy back all the chips they have left over. This is part of that push, and they know the public generally doesn't like NVIDIA, so it's worth a go.

And it's true that unless NVIDIA buys back the chips from EVGA, they probably go under. I imagine that's not a very pleasant conference room to be in during the negotiations. GN notes the extremely "personal" tenor of this guy's affront. If NVIDIA doesn't cave, his company goes under, and he's toast because EVGA is a pillar of the fucking community. Super tempting to try and go around the negotiations and take your case to the public, try and cash in on anti-NVIDIA sentiment (which is broad and intense), and try to appear to be the good guy.

Partners tried to pull this same stunt in 2018... they ran to tech media framing it as "NVIDIA is forcing us to buy obsolete chips if we want next-gen ones!" and if you read the details, what actually happened is partners wanted to cancel their contractually-agreed orders after mining tanked, and NVIDIA said no, if you do that it's over, we're not giving you 20-series chips if you break your contract on the previous batch.

https://www.techspot.com/news/76103-nvidia-putting-squeeze-a...

I bet NVIDIA wishes they could cancel their contractually-agreed orders from TSMC too. That's not how it works unfortunately. But NVIDIA did knuckle under in 2018 and bought chips back from Gigabyte and perhaps other vendors, and now the board partners are hoping for a repeat. But that was 300k chips, which, while it's a lot, was tractable. The current overstock is like, millions of chips, all in the high end.

That's why both NVIDIA and AMD are launching top-first this generation... the lower chips would compete with those massively-overstocked 30-series cards. So, launch the high-end stuff over the top and let the 30-series inventory burn through.

>first, high-end cards are the ones that miners liked

Last I checked most miners went for 3070s and below, because they're energy efficient. Though when supply ran short they used everything they could get.

And you really didn't get the point about high end cards. EVGA previously served a segment of enthusiast consumers that would spend exorbitant amounts of money for slight performance gains / overclocking sport. But under the new NVIDIA rules you can't take a 3090 and put the highest quality components and beefiest coolers on it, because the price of a 3090 is limited by NVIDIA to be around the range of a FE 3090.

We’ll that blatantly wasn’t the case - the FE’s were under half the market price of other cards for a good portion of the last two years! The markup was coming in somewhere, and if the partners didn't get any of that markup then there's something very wrong.

Actually there probably was a lot wrong all over the place over the last two years, the whole market was insane.

I'm sure some retailers also pumped the price.

Zotac's first-party storefront and other first-party stores were selling 3090s at well over $2k, so if there were caps they were capped at ~50% above FE pricing or higher. Not "around the range of a FE", that doesn't even pass the smell test lol.

TUL Corp is a great example to look at the profit margins of GPU companies, because their revenue is approximately 95% GPUs (they own PowerColor) and they're publicly traded, unlike most of the other examples of GPU-heavy private companies (EVGA is privately owned). I've seen them used for similar analyses of the 2018 mining boom on seeking alpha [0], for example.

https://www.morningstar.com/stocks/roco/6150/financials

So, profit in 2019: -80m. Profit in 2020: 30m. Profit in 2021: 1.08 billion.

So, they made a 36x increase in profits that year. In contrast, topline revenue increased only 2.3x. So they had a massive fucking increase in margins.

People need to stop being so incredibly credulous, do you really think the CEO doesn't have his own angle here?

It's completely amazing, a year ago nobody would have pissed on these CEOs if they were on fire, because everyone knew they were selling straight to mining farms. And now everybody is just taking them at their word that they didn't make any money on GPUs during the mining boom, heck they probably lost money right?

Like gosh if they say they didn't rob the bank I guess we just have to trust them, why would they lie about something like that?

It's been really fascinating watching the "green man bad" field overpower everybody's common sense. I guess the reflexive hatred of NVIDIA is stronger than the hatred of miners and companies that catered to miners. So much so that people will believe obvious lies (or significant misrepresentations) from companies that were blatantly profiteering right in front of our faces for more than a year, as long as it caters to the "green man bad" perspective. It is incredibly frustrating to watch, as someone who considers themself a seeker of truth and insight.

But then, here we are with the "partners didn't really make any money on GPUs during the mining boom" take. Could it just maybe be possible that maybe the CEO is not being fully honest here?

[0] https://seekingalpha.com/article/4181940-asian-aib-read-thro... https://seekingalpha.com/article/4186417-tul-corp-revenue-da...

BTW there's a second take on some of this with Igor's Lab here.

https://www.igorslab.de/en/evga-pulls-the-plug-with-loud-ban...

Basically: other partners have confirmed they think EVGA is just being dramatic and making a statement in general here, that this is kinda just how being a partner is. And that EVGA in general has some factors that make it worse - they're super small and contract out all their assembly, so they take lower margins there, they have a very generous warranty by industry standards, and they've been losing money on other business ventures and the GPU market collapsing makes things tough for the company as a whole.

But they describe last year as a "10x margins year" which fits what I'm talking about with TUL Corp too. They think EVGA just wants to quit on a high note, and quit cleanly during a generational changeover.

Think also a oit the cooling solution, more copper needed, more money.

Their lack of profits from high end video cards also existed in the 2000 series. No doubt worse with the 3000 series, but this is a long term war not a short term battle.

People will pay more for EVGA, because they were the only company that respected their customers. The only company you could actually get a decent RMA from and didn't continually break the law.

But Nvidia says no, there's max cap on prices. Nvidia also competes with their partners with their founders edition series. It's a whole around win-win for Nvidia.

I can't imagine why Nvidia would let one of their most respected partners go. Unless, Nvidia plans to shove out their partners and sell direct as they did with founders edition.

> Unless, Nvidia plans to shove out their partners and sell direct as they did with founders edition.

It’s important to remember that this was a major contribution to the downfall of 3DFX. A good chunk of which was cannibalized by nVidia.

It was a contributing factor, not a major one. Their products were not competitive with Nvidia after 3DFX completed the purchase of STB and started producing their own boards.

Not competitive how? The only card to consistently outperform the 5500 was the GeForce 2 GTS[1], which was a half generation ahead. 3DFX fans would also argue that T-Buffering and other unique technologies made the image quality better.

What killed 3DFX was a combination of no AIB boards to push the architecture past reference; their Sega deal falling through and the delay (and eventual cancellation) of the 6000 as a competitor to the GeForce 2 and Radeon 7000 series’ (later tests of engineering samples showed it outperformed both). The latter being a direct result of the two former problems.

1 - https://www.anandtech.com/show/580/7

Yeah, this is a case where money is inevitably going to be lost and everyone is saying "not it". It's not clear that there's any "fair" way to allocate the losses that will result from massive GPU oversupply.

There is a fair way: it’s the open market.

For what it's worth, the Ethereum merge was planned years ago (but the concrete release date may have been unknown until recently).

Enjoy the GPU glut. I for one am thinking of having fun with Stable Diffusion and the like.

Same here, I can finally get a cheap GPU worker for my weekend project (https://phantasmagoria.stavros.io, if anyone cares).

Have you crunched the numbers on how many users you need to justify renting a VM with an A100 or similar?

From my brief research they seem to be available for about $30 / hour, and can run 50 iterations in roughly 6 seconds.

One of the missing pieces for me is how many parallel requests they can serve, and whether it's feasible to start/stop the machines based on demand (it sounds like some providers don't allow the stopped machines to retain state long enough for this to be practical)

Would be interested in your thoughts, have been tinkering using my laptop GPU but it's dog slow and I have a vague idea for an online game that I might try out if I can summon enough mental energy one weekend

I didn't look for an A100 specifically, but the cheapest GPU spot instance on AWS is around $200/mo, if I remember correctly.

Since this is a hobby project, I probably will never be able to break even on that. I'm thinking of writing a bit of code that will use GPU spot instances to perform the computation, and turn it off when the queue is emptyish.

You don't really need any state on the machines, the worker communicates with a broker, receives a very small bit of JSON containing the parameters, does the work and uploads the image to R2. Then, it sends a message back saying that image is done.

That way, you don't even care if you get interrupted mid-generation, since the work is idempotent. I can just spin up a machine whenever available and tear it down whenever.

It will be nice to try, though I think I'll use my desktop for a while yet due to a lack of users. That one takes around a minute to generate an image, and can't run while I'm using it.

This is essentially what I have in mind - some kind of queue, with a mediator that starts/stops a spot instance using a custom image that has everything installed and configured to start on boot

With regards to state I just meant the image - wouldn't want to have to install everything at each start. Using something like aws or gcp I don't think this would be an issue, but I was window shopping some providers that specialize in GPU VMs and it wasn't clear if they provided equivalent services. Probably easiest to just use aws.

I think if you couldn't make a ready-made image you'd use a Docker container with everything in it instead. Though then I don't know how you'd use spot instances, because AFAIK you want those for the "whenever it's available" startup and can't really be logging into each one to start them up. If they let you run a container, presumably they let you run other commands to set stuff up as well?

This kind of depends on the provider, but it's mostly an implementation detail. The main crux is a broker and a queue, I use Dramatiq (the site is written in Python) and it works great.

With spot instances you just have to be prepared for them to get terminated at random. If you're processing messages off a queue in most cases you get this for free - the ack deadline will expire and the message will be redelivered to another subscriber.

Docker container might be a good shout, though I've not done anything using a GPU from docker before

Yeah exactly, the queue handles this for free. Docker with the Nvidia extension actually works better for me than native, Torch doesn't work on my system but works in Docker.

Finally a sane and useful realization of push messages.

Haha, more of a necessity, but yes, it is very nice to have.

Oddly, I did see an issue yesterday with the push notification arriving multiple times (far apart from each other) even though the server only sent it once.

> Supposedly they tried to cancel some of the volume with the foundries, but were not allowed to.

I would hope so, considering the big chip producers bought out the majority of the supply throughout COVID. Allowing them to bail out on that contract without providing an alternate buyer would set a bad precedent and allow them to monopolize resources whenever they desired.

Who would have imagined that adjusting your production rates over the most volatile and unstable asset AKA Cryptocoins would be a bad idea.

And that's the second time they've done this, after 2018 crypto crash.

third. there was another mining boom back in 2014 when it was bitcoin being gpu-mined. It fucked with AMD's Hawaii/290X launch really badly back then - the cards were unobtanium for months and months which gave NVIDIA an opening to launch Maxwell. Then the prices crashed and miners dumped their inventory, so you had new aftermarket 290Xs being sold for $250-300 to try and compete with mining cards, with used/refurb 290 non-X bottoming out around $175.

I'm sure the idea that infinity cache decreases the relative mining performance was in everyone's minds at AMD when they were designing RDNA2 - by that point they'd had two major launches (vega too) fucked up by mining, and using more cache/less VRAM bandwidth makes your mining performance worse relative to raster, because mining loads aren't cachable.

I mean, the supply shortage was a big problem for people that just wanted to play a game. My GPU died and I had to buy a used RTX Titan (in the 3090 era) just to have anything. (It's great though, I use it for gaming, ray tracing, and machine learning. Still looking forward to the 40xx series though.)

can't they "easily" retool for "AI cards"?

I don't think AIBs like EVGA are allowed to make server cards and the demand for those is not unlimited.

It's not like they are not allowed, it's that no one will sell them actual chips to make them.

There are server cards that use GA102, GA104, etc. which EVGA has a glut of.

It's not the same variant of a chip, though.

A10 uses GA102-890-A1 and there is no retail card on that variant. Variant is very important because Nvidia drivers are...well...not consumer friendly?

For example, Nvidia vGPU only works on certain cards if you ignore ways that break EULA. I don't remember the exact details, but there was also a software lock that prevented using some optimized paths on the consumer's GA102, but used the optimized path for GA102-890-A1.

If Nvidia is great at anything, that would be hostile market segmentation by their drivers.

Nvidia has control to fix that though.

I gave up playing around with AI partly because the drivers are proprietary and not too good.

Only in the past year or two has Nouveau developed well enough that some games are playable. But only on cards not enforcing signed firmware.

How much more anti-consumer can you get after requiring signed firmware? That is when I started my boycott.

Reply on news.ycombinator.com