The GPU shortage is over | Hacker News Reader

The GPU shortage is over

theverge.com

302 pointsmfiguiere253 comments

>Reportedly, one big reason used GPU prices are tanking is because crypto miners are flooding the market with cards that are no longer profitable due to the ongoing crypto crash, where the total market cap of all crypto assets has fallen by two-thirds since its peak of $3 trillion last November. (Miners have been switching away from GPUs for some time, though.)

What to do with all that computing power now. I guess AI.

I'm not an expert but I wouldn't want to buy a second-hand GPU that was stressed by crypto mining.

The damage is actually comparable or lower than normal usage, since they run at a constant temp which is less stressful and tend to be undervolted to get a better hashrate/dollar so the lines shouldn't be eroded much. At least according to that one LinusTechTips test.

Yeah, I don't necessarily agree. I assume many miners didn't run these cards in temperature controlled server rooms, but in places like abandoned warehouses in south-east Asia, with high ambient temps and humidity, which can destroy electrical components.

And even if they did, the capacitors in the VRMs of these cards are full of electrolytic capacitors which do degrade over time due to heat and chemical reactions occurring between the plates and the electrolyte as well as the evaporation of electrolyte, they not only age, but age faster over time as the degraded capacitors are put under increased stress.

Besides, the assumption of running these cards underclocked and undervolted might not be true - if the folks running those obtained the electricity illegally, or incredibly cheaply, it might make more sense to run these cards hotter.

Everything considered, these second hand GPUs are not even that cheap, I saw 5700 XTs going for $250 on ebay, with a comparable brand new 6600 costing about $300.

Edit: The fans also have suffered serious wear and tear and are likely to fail. Considering these are often custom, you might not even be able to replace them.

Thermal cycling such as is typical in a gaming PC is far more damaging than constant temperatures.

Underclock/volting cards to get a consistent higher net yield is standard practice in crypto mining, just as ovectclicking/volting is standard practice in enthusiast gamer circles. The latter being way more damaging for the GPU. Those that obtain the electricity illegally, or incredibly cheaply would not be the ones selling cards now.

As for fans, the things that reduce the fan's lifetime the most is dust buildup and high positive pressure configurations. Both of these are typical for bedroom gamer setups. Miner cards run in cleaner environments and open frames.

I would definetly buy a GPU from a miner over one from a gamer.

  > Thermal cycling such as is typical in a gaming PC is far more damaging than constant temperatures.

the reasoning is probably pretty obvious, but what i heard is the constant expansion and contraction from heating and cooling the chips and connections is what causes the damage im those cases

The GPU core doesn't get stressed much, of course. Memory, however, is another issue - rtx3090 being an absolute disaster (effectively cooling on half of the memory). HBM cards are likely in a similar boat.

The (large) GPU mining operations run at rather high ambient.

HBM GPUs are better off because the memory cores are in the same package as the compute cores. So they get the same amount of cooling love as the compute cores enjoy. At least in theory.

AMD HBM GPUs, the only ones that a PC gamer would buy, are known to suffer from issues with the HBM due to thermal cycling issues

A mining GPU will not experience thermal cycling.

Exactly. Memory would be the main issue.

I don't know anything about component wear patterns, so a question: would a peak constant temp cause less wear than moderate fluctuations? (And is it fast heating/cooling cycles that cause physical stress or is it some other mechanism of action that causes the wear?)

Hot take, but you're not going to get an accurate answer to a question like this online.

There are probably about a dozen people on the entire planet who actually understand how and why GPUs fail, and tens of millions who think that the fact they can build a PC qualifies them to present a regurgitated theory from Reddit as "expert opinion".

> There are probably about a dozen people on the entire planet who actually understand how and why GPUs fail

This seems like an underestimation. I would bet there are mid-level hundreds able to articulate at a precise level at least, and dozens of thousands more who are sufficiently familiar with the design and shortcomings modern GPUs and chips who would be able to give an approximately decent answer from first principles.

That said, despite my guess being four orders of magnitude larger than yours, the odds of actually encountering such a person among the 7 billion available is practically the same.

I think you'd be surprised. There are surprisingly few people who have a bone-deep understanding of much of the technology that modern civilisation relies on to function.

Is bone deep knowledge required to know if fluctuation cause wear due to physical stress of heat/cooling cycles or something else? It seems like something that countless hardware designers would need to know.

Certainly there are many more nuances to wear patterns than that, but that was kind of the basic question I was asking. But as I said originally I was asking from a position of interested though ignorant curiousity, so my assumption that most hardware designers would need to understand this sort is stress might be... discontinuous with the factual nature of the physical & human experiential phenomena relevant to the question. (e.g., stupid assumptions) :)

So if we suddenly had no GPUs what part of "modern civilization" would fail? Crypto markets, so what? Gaming, ok, I play card games on my year 2015 low end phone. And I could continue. Some startup that claims to do AI. (Actually I work for one, but not on the AI side).

Appears to me the effects would be less than the pandemic or the Russian war.

All the stuff that uses GPGPU computation heavily.

- Weather prediction.

- Other aerodynamic and hydrodynamic modeling, used in a lot of industry.

- Image recognition.

- Voice recognition / captioning.

- Machine translation.

- I suppose also stuff like large-scale mechanical modeling, protein folding models, etc.

My first guess would be CAD being a lot harder, which would have a lot of widespread impacts on all sorts of industries.

I mean you are talking about trillions in economic value from those combined industries so while it may not measure up to the pandemic, few things would. And just because you personally prefer card games, a few billion people play video games.

Also having a cheap phone from 2015 is quite an outlier. I absolutely get wanting to moderate tech use and I make full use of the digital wellness limiters to cut down on social media and interruptions, but phones are just so damn useful for just about everything in modern life. Throwing the baby out with the bathwater a bit there, no?

>phones are just so damn useful for just about everything in modern life

Honestly, I'm not sure how true that is. I got rid of my smartphone just before my daughter was born (she's turning 2 this month) and there has never been a single point where I've regretted it.

That's not to say I avoid Android entirely. I've still got a tablet at home for Netflix/Chromecast, and we'll use my wife's phone in the car for Google Maps, but that's about it.

There really is no need to be 100% connected 24/7.

Like, you could also give up electricity, indoor plumbing or motorized transport if you really wanted to and manage to arrange your life so you don't regret it.

You are basically reliant on your wife for accessing any kind of digital service without a website, communicating with anyone, or any computing whatsoever when not at home. For most that is giving up on staggering utility, but I know people who prefer to ride horses still, so different strokes of course.

I don't think you can really compare electricity or indoor plumbing (unobtrusive technology) to smartphones (extremely obtrusive technology).

I still use computers and the internet, including Android devices, at home and work (quite non-trivially, actually - I am my family's sole provider and 100% of that income comes from a mix of online business and remote consulting) and I'll still talk to friends, family and employees via SMS or occasionally voice.

Genuinely, man, as someone who has actually done it, you miss out on absolutely nothing and gain so much.

So "I make my wife do it" is the loophole. Classy.

Please don't cross into personal attack, snark, or name-calling.

https://news.ycombinator.com/newsguidelines.html

Did you actually read the comment you're responding to?

Please don't respond to a bad comment by breaking the site guidelines yourself.

https://news.ycombinator.com/newsguidelines.html

> Also having a cheap phone from 2015 is quite an outlier.

Nobody would deny that, at least not in Europe, North America etc. But I don't think I miss a lot of "modern civilization" by not participating in evey aspect of surveillance capitalism and used mostly apps from F-Droid.

Remember that it was possible to fly to the moon in 1970 with less than a Megabyte of memory. The efficiency of additional resources poured into scientific seems to go down all the time. Not at all sure it is worth ruining the planet by ever increasing resource usage for small gains in knowledge.

Your thought experiment is basically "what if computers could no longer display anything on a screen?"

That is absolutely bigger than the pandemic or the Russian invasion.

Some modern computers use the GPU to display everything. E.g. on Ryzens you don't even get a BIOS screen or text console without a GPU. However that's just design choice because they assume that a GPU would be there anyway. It worked without one for decades.

However, what probably wouldn't work as well without GPUs is video production and consumption. Linux users knew that for years, when GPUs were poorly supported. Doing on the CPU does not give you the same resolution/framerate and requires more energy. Some might claim that watching less Youtube/Tiktok and reading more books or exercise outside instead would be good for "modern civilization".

I can imagine this being true too. How do you come to these conclusions though?

I can believe it by extrapolating from my own domain (which is nothing to do with GPUs), where there are a bunch of people with mid-level knowledge, a very small number of people with deep knowledge, and an utterly tiny population of unique individuals who are the ultimate architects.

I guess it’s a power law distribution like so many other things.

Pure anecdata. :)

I know people who are one-of-a-few experts in technology that is critical to a particular industry, and there's not always good succession plans in place for when they retire.

I could absolutely be wrong in this case - it might be well enough understood that you don't need decades of raw experience to begin to scratch the surface - but at the same time there really are so many advanced technologies held together by one or a few experts.

Failure modes of integrated circuits are taught to students at tech universities. My friends who went to study electronics engineering were taught "how to design a die which dies in X years".

I am not saying this makes them instant experts on GPU failures. On the other hand I also don't believe GPU failures are that special.

...And mix in a boatload of people getting out of crypto mining with cards to offload, and thus a direct financial interest in blowing hot air.

Exactly. Even Nvidia didnt employ competent people up to 2009 :)

https://semiaccurate.com/2009/08/21/nvidia-finally-understan...

>On July 2, 2009, the date being ironically a year after the notorious 8-K that publicly kicked off bumpgate, the company put up a job listing for a “DIRECTOR OF PACKAGE TECHNOLOGY”.

Although I’d guess a fair few would be on HN.

Haven't been an EE for a couple of lifetimes now, but iirc yes - best to constantly run (within normal operating limits), rather than run and stop and restart.

That holds (? used to hold?) true from anything to GPUs to individual capacitors, the repetition of the heat cool cycles is the primary cause of wear.

Holds true for a lot of electrical appliances, even lightbulbs. They always blow when you switch the lights on.

I'm not sure whether constant running is worse for wear and tear vs power cycling but power cycling is typically what precedes a failure.

I expect the main reason incandescents tend to fail when the lights are switched on is because the resistance of the filament is lower when cold, so there's a brief spike in current much higher than steady state as it initially heats up.

The issue is constant running also has thermal cycles, the variance between peaks is just lower, but can be more rapid. (Spikes from particular instructions being more power hungry).

Also in this case we have mechanical components under constant usage (fans), and the effects of long term heat on the thermal compound between the die and cooler plates to consider.

> They always blow when you switch the lights on.

This isn't true at all.

I've never had one fail in a different manner. Modern LED bulbs seem to fail differently. I see lots of power delivery issues before they go

Yeah LED bulbs usually have their power supply go long before the LED itself due to inadequate cooling, LEDs themselves last a long, loooong time if properly driven.

That failure mode is more, it works fine when initially turned on, then starts failing/flickering on and off once it has gotten to temperature.

Back when I was involved with this, the answer was: "it is hard to answer". A number of years ago, I was working with a Global Foundaries fab on a cutting edge process node and they had no idea how to estimate the lifetime degradation due to high temp. The issue was it was unknown on how to accelerate the effects of high-temp for some of the failure mechanisms. I seem to recall hot-carrier injection being one such mechanism, but I don't recall for sure.

If you can't accelerate the failure mechanism in a well defined way, you cannot lab test long-term reliability other than just letting the system run. So if you want to guarantee 10 yrs of lifetime, you need to run it for 10 yrs. Obviously, that doesn't work for most products. Instead you have to rely on models, which, for a new node, might not be proven very well.

Bottom line was this vendor shipped products that nobody could really be sure would last the promised lifetime. This was regarded as a top silicon fab in the world. Back then, finer geometries were much harder because of new failure mechanisms. I don't know state of art today, but I would not be surprised if measuring reliability is still not a really hard problem for new nodes.

Edit: I am just referring to silicon issues, the rest of the PCB has its own set of issues totally separate. Generally however, those are better understood than a new process node.

Heating and cooling, just like what causes cracks in sidewalks. Fast heating and cooling is worse than slow but there is also electromigration which will increasingly become an issue as our process nanometer size decreases. My friend who majored in CS said his teacher recommended always leaving computers on.

Physical stress of constantly expanding and contracting.

The question is by utilizing only a few pathways from core and memory , because the computation never changes aka overusing certain transistors, does the core or memory get damaged?

Ethereum mining uses a lot of vram, perhaps the overuses of the memory damages it?

But they’re put together by amateurs, in non-static safe environments, operated with questionable cooling…

Sounds like exactly how gamers use them too.

Ive seen some posts/tweets that There are cards exiting the crypto market where failed ram chips on them have just been yanked off the board, as crypto just needs cores not lots of vram, i wouldn't trust it unless very, very cheap to lower my risk.

I really wouldn't trust them to have been run with adequate cooling, or to not have been run at higher clockrates, etc.

There are different problems.

Thermal cycling common to gaming uses causes mechanical failures in chips, between chips and the board, and the board - but electrolytic capacitor lifetime drops dramatically as temperature goes up.

Not much thermal cycling in desktop systems seeing mostly "productivity" use that have proper ventilation.

Capacitors have been massively over-specced for PCs for over a decade now. This was only a potential problem ~<2010 for components like GPUs. Maybe still a problem with really cheap power supplies today, but that's not what we're talking about.

optimal running temperatures for gpus during mining is way lower than what most games will pull.

memory has to stay as cool as possible or else the overclock will be unstable.

It's still 24x7 operation vs a few hours here and there. Even a pretty hardcore daily gamer probably only clocks half a dozen hours a day if they're working a day job.

but it's the turning on and off or heating cooling under changing loads that causes the stress, not the constant running

Crypto miners actually often run the cards at low wattages and low heat. Because the top 5% of performance requires like 30% of the energy. So you can reduce your marginal cost (electrical bill) by 30% while only decreasing your revenue (mining) by 5%.

Gamers are the ones destroying GPU's, not professional miners who tune their systems properly.

But they stress the hell out of the VRAM coz mining is limited by memory bandwidth. There are reports on 2nd hand video cards from miners getting rendering artifacts after only a few days of gaming.

It’s not like this memory use actually causes wear and tear though - DRAM doesn’t involve some irreversible process like Flash does. At the end of the day it is still the same slow thermal death - mining just means the RAM is more prone to overheating than the GPU. And yes it’s hard to know after the fact how the RAM has been treated - but if the RAM is properly cooled it should have no wear just due to use.

All transistors wear out the more you use them.

DRAM is pretty durable but the controller, bus pins, and other parts would be the wear/tear in my opinion. Those other bits inside the DRAM could very well wear out.

That transistor wear is going to be on a timescale far longer than at issue here - the silicon aging effects referring to will take place over decades of full time for use at low operating temperature - none of these graphics cards are anywhere near that old.

> low operating temperature

Debatable.

A lot of these miners are keeping their GPU-stations inside of a garage with no air-conditioning (so 90F+ temperatures, maybe 100F+ ambient). These miners are going for the cheapest-of-cheap conditions, they weren't running a datacenter, but a mining operation.

I'm willing to bet that the vast majority of them never paid attention to airflow issues. Closets, garages, maybe even sheds.

While GPU cores are underclocked and undervolted (but not necessarily), the RAM is going to be processing at full tilt as quickly as possible, possibly overclocked and overvolted and therefore at even higher temperatures.

--------

Finally, buying used means having a very ambiguous warranty. You might not have anyone to send the GPUs to. Its generally not a good idea, especially now that new-prices of GPUs is below MSRP.

Honestly, we're not getting a temperature graph of these GPU's life. We're only getting assurances from some cryptobro that they treated their GPUs correctly (and no offense to the cryptobro community, but... there's a lot of lying scammers out there). Its not a trustworthy community at all.

No disagreement on any of this - I would not trust the second hand miner stuff wasn’t treated like shit myself either.

Also the gddr6x that’s used on the higher end ampere cards (3070ti and above) are known to get really hot

I don't think anyone is suggesting you should buy a used one. The fact that crypto miners are dumping them likely indicates that they aren't going to buy the new ones up. It could mean they're just rotating stock, but given crypto prices, more likely means they're giving up and dumping assets. This should free up new stock for the rest of us.

I'd buy a used one if the price were commensurate with the risk. Say 20% retail.

Is there any evidence that chips degrade in such a way? I've got servers that are 8 years old that ran full bore and they still work fine?

> Is there any evidence that chips degrade in such a way?

As far as I can see, the answer is yes[1]:

The relationship between integrated circuit failure rates and time and temperature is a well established fact. The occurrence of these failures is a function which can be represented by the Arrhenius Model. Well validated and predominantly used for accelerated life testing of integrated circuits, the Arrhenius Model assumes the degradation of a performance parameter is linear with time and that MTBF is a function of temperature stress.

However, the dramatic acceleration effect of junction temperature (chip temperature) on failure rate is illustrated in a plot of the above equation for three different activation energies in Figure 2. This graph clearly demonstrates the importance of the relationship of junction temperature to device failure rate. For example, using the 0.99 ev line, a 30° rise in junction temperature, say from 130°C to 160°C, results in a 10 to 1 increase in failure rate.

Wikipedia has an overview of the the Arrhenius equation[2].

Now, as you probably know most complex chips like GPUs and CPUs have built-in thermal management which prevents the junction temperature to rise above some limit, which one would assume is set at a point which reasonably guarantees a decent lifetime.

However, according to this, chips that have experienced less heat should, on average, live longer.

[1]: https://www.ti.com/lit/an/snva509a/snva509a.pdf

[2]: https://en.wikipedia.org/wiki/Arrhenius_equation

I think it's weird how so many people seem to think solid state parts or chips can't fail due to lifetime/heat/use. They don't have obviously moving parts like a fan, but that doesn't mean its materials can't degrade.

They are over-designed enough that failure is quite uncommon for most people.

I've seen enough failed parts that I know this is misleading if not outright incorrect. It happens. We all have so many electronic devices, many of which we keep for a long time, that the likelihood of your average consumer experiencing a chip failure of some kind is fairly high (over a long enough period of time, probably even guaranteed). Yet for some reason people keep painting it as this statistical impossibility. It's simply not that rare.

From my experience pretty much all failures in modern electronics are due either:

  * Dried up electrolytic capacitors

  * Bad solder joints exacerbated by lead free solder.

  * Failed high current transistors (typically regulation mosfets or triacs in appliances)

Failure of low current handling chips is pretty rare.

Not sure about the root cause, but there have been numerous reports of mining gpus ram issues, particularly on the radeon VII line. There, after mining, they're often unable to hold even stock memory clocks.

Interesting - I wonder if they have reverted the gpu bios back to a known good OEM image?

more concerning is the wear/tear on the fan bearings and the thermal interface between heatsink and board will often be dried out and in poor condition.

some gpus are rather hard to get a replacement fan for because it's not like they use an off the shelf modular 60 or 80mm 12vdc fan.

Bah, that's such a minor issue it's not even on the radar. TIM can and should be replaced anyway, fans can be fixed with some automotive grease (not all, sadly). Modding another fan onto it is also an option.

The bigger problem is VRM wear, you never know when it will fail. Said TIM drying out affects the VRM components the most. One failed capacitor or resistor and the whole GPU can get fried (seen it happen many times).

looking on newegg I see lots of factory new 3060, 3070, 3080 selling at pretty regular prices so unless the deal on an ex-mining card was REALLY good (we are not there yet, at price drops in ebay, for when I compare the same NIB newegg price vs used on ebay), I wouldn't buy a used mining card... Maybe if it was like 50% of the new price in a few months from now.

Well, yeah, it'd have to be a great deal.

Is there a good way to spot this sort of thing from a distance when evaluating stuff off eg eBay?

You could try to guess based on dark spots maybe but that's more like gambling. Boards that smelled "burney" worked fine while others just failed - the cause of failure was heat as in most cases the thermal pads weren't touching the heatsink or the component, were missing or were crumbling.

SMD elements can fail either open or closed/shorted, the latter is more likely to cause a worse failure but it's not exclusive. Due to protection circuits elsewhere they can fail without visible marks so it's hard to diagnose.

Thermal pads dry out because of heat first, their quality second, with load coming in last imo. So the fact that it's been used for mining alone doesn't mean much.

The conditions under which a card was used and the time matter more - if only sellers told you the average temperature, case type, ambient temperatures, whether the cards were undervolted/overclocked and how long they actually ran...

The GPU makers aren’t exactly making fans in-house. There is probably a seller on AX with the exact part needed. I’ve had great luck in finding replacement parts for pretty niche hardware.

if the fans aren't sold by newegg and its top ten largest US domestic competitors as standard sized items - i would consider them as rare weird items, because the size, shape and mounting of the blower is often unique to each card. not going to play games with something I have to order on aliexpress and wait 6 weeks for it to maybe show up from china.

Most of them are 95mm fans. You can get replacement fans on ebay for almost any relatively common GPU because most of these fans come from the same places and the companies that make the fans for the GPU manufacturers long ago figured out there was a market for replacement units.

The higher end card companies in some cases will just ship you a replacement fan if your email to them is pleasant. Or sell it to you for a modest price.

I just looked and I can get replacement fans for my specific GPU for less than $20 including shipping.

You can also take off the Orem fans and shroud and zip tie casefans (noctua or arctic are good brands) onto the heatsink

The servers are built and rated for 24/7 operation, unlike consumer GPUs. This includes mechanical as well as power components.

They do, mostly due to thermal expansion and contraction, fan wear, and thermal paste degradation on the card. Even so, this has a very small effect.

Wouldn’t constant usage at a set rate reduce these issues?

not for the fans - they are not designed to run at max speed at all times.

Why not? Embarrassingly-parallel problems (of the kind you'd use a GPU for) are a perfect use-case for enhancing fault-tolerance through redundancy (i.e. running your program on three cards at once and only accepting the output if they all agree.)

The only place this doesn't apply is gaming, where you're trying to get realtime interactive results while economizing on both cost and energy. But for every other use of a GPU, you can't go wrong with "throwing more compute at the problem."

I mean, if the price it right, why not? I wouldn't want to pay anywhere near full price, though.

If the price is like 20% of retail - why not.

If higher or close to retail - why buy used gpu ?

Let miners literally drawn in debt.

Constant-on is typically less harmful to the chips than on-and-off. Electromigration or whatever is less of a risk than solder joints going bad.

Lot of info in this here:

https://www.youtube.com/watch?v=1T0npiqjEWQ

Apparently GDDR6X and HBM memory might degrade from Ethereum mining

>I'm not an expert

This

> where the total market cap of all crypto assets has fallen by two-thirds since its peak of $3 trillion last November.

This is why it's important to keep battering the crypto advocates any time they promote anything linked to proof-of-waste. Cryptocurrency is the "paperclip maximiser" that will consume scarce energy and hardware manufacturing resources, and turn them into "grey goo" which people have been conned into believing is worth something.

>What to do with all that computing power now. I guess AI.

For AI, the lack of memory is pretty limiting.

Having a ton of 8 GB or 10 GB memory GPUs (3070/3080) is annoying and often just not even possible to use effectively without designing specifically to it. If they get really really cheap maybe they'll be some tools or models that are built to leverage scenarios like that, it's not impossible, but for now it's probably just the cheap 3090s (24GB of memory) that will benefit AI.

I picked up a cheap ex-mining 3090 last week. Amazing for a ML hobbyist. Works out so much cheaper over time than paying for cloud GPUs.

A couple weeks of training and I break even.

Where did you shop for used ex-mining cards?

ebay

Feels like distributed model training would be the ultimate yak shaving exercise for training ML models at home. Sure you can do it, the cost of the lower memory cards is super attractive, you'll definitely learn lots of cool things, and, after all the effort you will have made exactly no progress on the original task you set out to do. Kind of like the choice between making a video game or a game engine.

Seems like it'd be a decent time to start some version of a indie "cloud gaming as a service" company similar to Nvidia GeForce Now or Google Stadia.

It perhaps isn't the end of the world if your gaming session errors out due to the service being based on used crypto mining GPU cards - especially as the service subscription cost approaches $0/month.

Nvidia disallows use Geforce on datacenter

Doesn't that violate first sale doctrine?

You can write your own drivers I guess.

Ooh, ooh. T-cool. Let's call the game Nero's Fiddle.

>ai sturtups are flooding the market with cards that are no longer profitable due to the ongoing hype fade, now back to VR news…

I should buy some Fuzzy Logic domain names in case that particular hype cycle comes back around.

What about gaming? People have been waiting to be able to buy affordable GPUs for a while now.

The DIY desktop gaming tower market has got to be pretty small by now.

Margins on pre-builds have been shrinking to almost nothing, I build my own PCs because I see it as a hobby, but I could hardly recommend it nowadays to someone who's looking for a machine, not a new hobby.

There are a lot of people looking to upgrade their prebuilts, for example. It's far cheaper than buying a new system.

You say it like people didn't buy GPUs before crypto. What do you think happened to all those GPUs AMD and nVidia have made every year for the past decade or so?

Computational physics. We have a lot of atom based engineering to work out this decade.

> What to do with all that computing power now. I guess AI.

CEO of Kryptex here. We have an answer:

https://www.kryptex.com/en/rent-cloud-gpus

Big tech would curb their appetite for giant models during down turn as well.

Expect price to dive deeper lol

Collaborative scientific computing!

Halo Infinite is amazing.

Not mentioned in the article: Ethereum will be transitioning to proof of stake in the next few months and the large majority of GPU-based mining will go away entirely. Probably will be some great deals on video cards later this year.

> Ethereum will be transitioning to proof of steak in the next few months

Just in time for the beef shortage!

I'd love to be paid to hoard steaks.

Why? You would never be able to eat them.

White powdery beef!

https://www.youtube.com/watch?v=DZhk4KgY_QU

ETH2 has been 6 months away for 6 years.

Sure, but when asset prices have crashed is probably when resistance to PoS is weakest? So now is the time to spring it upon them.

ETH 2.0 hasn't been held off due to resistance to PoS. It's been held off because there's no ETH 2.0 to ship yet.

There is no meaningful resistance to proof of stake in Ethereum. Users, devs, etc all on board. It has been part of the roadmap for years. Changing consensus is not a light upgrade for a blockchain. The chain needs to maintain 100% uptime in spite of the switch.

welcome to software development

Ethereum transitioning to proof of stake "soon" has been said for a long, long time now. I believe it when I see it.

Fair enough, but none of the following was true until recently:

The production proof-of-stake network has been running in parallel for a year and a half, with 10% of all ETH locked up in it.

What's left is the merge with the rest of the network. They've tested the merge on every combination of five execution clients and five staking clients.

They've done about a dozen large test merges on copies of the live network.

They've done the merge on a large long-running testnet used by applications, block explorers, exchanges, etc. Essentially it was a practice run by the whole ecosystem. There were a few minor glitches but it worked well enough to be a success if it'd been the live network.

There are two major testnets left, and if those merge successfully then the next step is production.

Finally we'll get to see what plutocracy as code looks like

For better or worse, probably about the same as now. Ethereum's stakers have no more governance rights than miners, and their return on capital probably isn't better than miners' since the reward is so much lower.

Governance, no… But transaction selection?

Also same as miners, and censorship resistance is one of Ethereum's explicit design goals.

Some other PoS systems use delegated proof of stake, where most people just delegate their stake to one of a small number of validating nodes, which could easily cooperate to censor transactions. Ethereum supports a very large number of nodes, due to signature aggregation. Right now there are about 400K individual nodes. Entities of course can have more than one node, but the last estimate I saw of the number of individual entities put it at around 5000. (That was at least six months ago.)

There's a countdown and difficulty bomb: https://wenmerge.com/

There has been many.

isnt it doing so since 2019?

First test nets have transitioned to PoS already. It's gonna happen soon.

Did they have a successful test? The last news I caught had 1/3 failures or something like that

They do bi weekly tests with the mainnet state and by now it's getting pretty stable.

I can't say anything about the absolute code quality however I think we are getting to a point where economic and community pressure forces them to take some risks

2015 was the originally announced date I think.

It was just part of the roadmap back then. It wasn't an imminent upgrade for which there are live testnet merges being conducted.

They've merged on the test net, so the August timeline is sounding realistic.

Actually the ideal timeline is for late September

https://wenmerge.com/

I heard that it was the plan from its inception.

This, every single article I have read so far is missing this VERY important piece of information. I am still mining ETH and it is still profitable. I mine with 4 1080s and I do not pay for power. However, if I would have 200+ gpus I would slowly start selling before the prices tank due to the merge. Also more dedicated mining hardware is hitting the market for ETH which is far more efficient then gpu mining.

> it is still profitable

> I do not pay for power

I guess you don't run your own solar farm so I assume someone else is paying that for you without knowing.

To play devils advocate, I indeed have solar panels at home (7kW). I use my graphic cards only when it is sunny. However I don't mine ETH but instead contribute to Folding@Home.

That's still power you could either use to charge batteries or sell back to the grid

Nominally, yes, but the actual economical value of doing so is most likely an order of magnitude below what buying the power would cost (sun shining and grid needing more power is only loosely correlated, and batteries take significant time to pay off). Subsidies have a large impact here though.

>Do not pay for power

Hmmmm, I detect a flaw in you plan....

If anyone does buy a mining-used 3060 or something I would strongly recommend learning how to re-paste the thermal interface between the heatsink and the chips mounted on the PCB. Some good youtube examples out there for people who've never done it before.

a decent syringe of heatsink paste is $7 on newegg or amazon

Does removing and reinstalling a heatsink damage the thermal pads, requiring buying new ones ($10 of thermal pads doesn't last as long as $10 of thermal paste)?

In my opinion, thermal pads are better suited for components that need just that little extra bit of heat dissipation to perform adequately, i.e components that are < 10° C from being able to be passively cooled. Voltage regulators come to mind.

So in that instance, if the thermal pads are reasonably well installed, the components should be just fine, since we're working with much looser tolerances.

When you need active cooling, the manufacturing complexity increases to the point where the convenience doesn't really make sense, since you have to get it right, so thermal paste is the way to go.

Pads are preferred over paste for two main reasons: low assembly cost and ability to fill gaps. Both are very important. If you have an application using a pad, and it's flat enough to replace with (good) paste, you will always see a performance improvement.

I do not ever re-use pads. I have a used pad on my desk at work that shows an indent of the chip that was pressed into its other side... and you can read the part number and lot code laser marked into the chip. If the pad conforms to that but then can't relax back... it's never going to make good contact again. Full stop. (Granted, this was an extremely expensive thick gap-filler pad [TG-A9000, I think] for a particular application, and performance will vary.)

> If you have an application using a pad, and it's flat enough to replace with (good) paste, you will always see a performance improvement.

if the tolerances are tight enough, if the heatsink doesn't even touch the VRM thermal paste won't do nothing.

That's a fair point -- I was assuming contact and already moving on to surface roughness!

thermal pads in general are worse than properly applied paste, if you learn to paste things smoothly and accurately.

there's a reason you won't seen thermal pads as the interface between a cpu's heat spreader and the underside of a skived copper heatsink in a high wattage per socket, dual CPU 1U or 2U server, where the heatsink has to be really efficient.. it'll be a factory applied paste.

I think the question was more surrounding the thermal pads on things like the VRMs. My understanding (though full disclosure, I've never disassembled a GPU myself) is so long as you take care to not tear them, they should be fine.

I don't know that I've ever seen a self adhesive double sided sticky thermal pad that was designed for re-use with re-application of a heatsink, or that I would trust re-using a second time.

For over a month the local microcenter has had hundreds of RTX 3xxx series GPUs, and that's just from the public numbers, which are capped at quantity 25.

Nobody is buying these cards. One, inflation. Two, everyone who really wanted them already bought. Three, people have realized that they can just be careful about graphics quality settings and save themselves hundreds of dollars.

The newer cards are a shitty deal, frankly, at the low end. A new 3060 retail costs twice as much as a used 1070 ti that will perform nearly identically, even have nearly the same power draw (you really have to hand it to NVIDIA to have nearly the same performance per watt with a half decade spread.)

Is DLSS nice? Yeah. Is RTX pretty? Yeah. Is it twice the cost nice/pretty? Nope?

NVIDIA is going to have to trickle the 4000 series cards out so slowly if they want to have any hope of not severely boning their resellers. My guess is that we'll see 4080's first to get the whales, then 4090's to get the whales again...and then the lower cards announced but released from storage in very, very small quantities.

I checked to see if you claims were true (3060 = 1070ti)… and you’re absolutely right!

https://gpu.userbenchmark.com/Compare/Nvidia-RTX-3060-vs-Nvi...

Yeah, there doesn’t seem to be any good reason to upgrade if you aren’t aiming for the highest specs (which is most people). Even ignoring the crypto craze, there wasn’t really that much tangible advance in GPU specs (and even when it did it traded off in terms of size, power consumption, and heat.) Before you were able to buy a GTX 970 for $400 to play all the latest games at the best options and that was enough… and even to this day it’s still a solid entry-level GPU.

Four, some people gave up trying.

True that. I used to upgrade graphics cards every few years, now I've had mine for about 6 or 7 can't even remember. Can't even remember the model, some Nvidia thing. But it plays 90% of games fine, and that's more than a lifetime worth of entertainment. I'll upgrade when my rig dies someday

I'm just hoping 1050TIs become affordable again so I can build a really cheap gaming PC.

The cards I've been watching (radeon pro wx 2100/4100, and RX 400/500 series) are just hitting 2 years ago prices on the used market. They were already used then. I'm just hoping prices settle to something acceptable by the end of the year. It's been crazy watching the price go up for these old models though.

Just looked at prices in Germany... 3090ti is still more than 2k euros, same for Czech Republic (seen one for about 2700 eur few days ago), or Slovakia, very similar in France. So, no - I don't really see prices getting normal. Where are they? How can I get a new 3090ti for 1400 euros in Europe?

> How can I get a new 3090ti for 1400 euros in Europe?

Buy in the US, ship to Europe? Shipping and import tax combined would still seem less.

Not worth it. 1 year warranty is also a major issue

Well, not feasible at all, our customs officers are very well known to hold your mail for random time which can be easily 6 months or more.

What if legitimate travelers entering the country were to bring the items in as part of their checked luggage?

I feel like there's a whole untapped market here where people could rent out X amount of space in their suitcases - with the obvious proviso that the item(s) would need to be completely opened (and maybe independently xrayed) first, to eliminate surprises.

https://www.mindfactory.de/product_info.php/24GB-MSI-GeForce...

1800€ and a very good one. Ofc not yet at 1400 but it is getting there.

The GPU shortage is at the initial stages of being over, largely due to multiple facets of the cryptomining implosion that are right around the corner.

I still can't yet walk into a Best Buy or local electronics store and get one off the shelf, and online prices are still jacked above MSRP.

We'll see real action 6 months to a year from now.

Best Buy has weekly nvidia founders edition drops, on Thursday mornings, and last night they had a 3070fr drop because it doesn’t seem like enough people bought them on Thursday. You might not be able to get one off the shelf but you can order one to pickup.

You totally can buy them off the shelf right now. I saw them a few months ago in a Best Buy in Rancho Cucamonga, just checked the site and I could pick up a 3060 in LA tomorrow.

Here in Australia you pretty much can now, which is good.

Last time used mining cards were selling at 20-30% of msrp so just wait till prices go down to the bottom. Pandemic is over, nobody wants/can play games instead of life anymore.

I volunteer as a coach and we’re facing an enrollment drop due to the pandemic being over. When schools were closed we were considered one of the safer sports due to social distancing. Now, we’re losing kids to indoor online gaming and contact sports.

What sport do you coach?

Youth sailing.

We had kids flying in from all over the world to learn and compete. Other youth sports such as basketball, soccer, football were effectively shutdown.

That’s crazy! Is it an intense competitive program? Our kids are doing youth sailing again this summer (have been doing it for years) and they are missing some weeks because it is completely full.

The program here is very chill and fun, but the kids are still expected to be sailing properly. Racing focused classes are available if desired.

The highlight of the summer was when SailGP was in town and they decided to get a closer look during the practice sessions. They got so close to the boats going 50+ knots that the course organizers went out to yell at them to get away.

Last summer when options were limited we had kids flying in for our summer programs. It’s not intense but you will qualify to join a school or club team in the fall.

This summer because of pent up travel demand for the last 2 years even our locals have decided to travel for summer. Some weeks are fully booked some are dramatically less.

Have you looked at SailGP Inspire? I’m hoping to have a few compete in their programs in the next few years.

A few kids at our club were involved with the Inspire program. I don't know much about it - my kids aren't interested in highly-competitive sailing. I think there is more to it - I think it has aspects that lead to careers in sail racing maintenance/management.

If you can get the kids into the program and they do well, it appears to open a lot of doors.

People have played games because life sucks even before the pandemic.

If your attitude towards life is that it sucks so I'll play computer games instead, it sounds like a negative feedback loop :)

I’m not sure if I’m reading it correctly, but if the reader has the inference / bias that playing games makes life suck, that would be a positive feedback loop (albeit with negative effects).

Negative feedback loops are self correcting. So if games make life better it would indeed be one.

depends! maybe a negative feedback loop that keeps the equilibrium on the negative side

I don't think control theory is equipped to deal with the nuisances of the human psyche x)

Well, negative feedback loops are generally more useful than positive ones. At least in electronics...

> nobody wants/can play games instead of life anymore.

Steam alone is still peaking at 25,000,000 concurrent logged-in users https://steamdb.info/graph/

Right, and how does that number compare to 6 months or a year ago?

If I’m reading the “Lifetime concurrent users on Steam” chart on https://steamdb.info/app/753/graphs/ correctly, it’s a slight increase: today 28.0M, 6mo ago 27.9M, 1y ago 25.2M

Jup, Covid caused it to jump from 19 million pre-pandemic to 24 million on april 2020.

But since then, the numbers increased until march 2022, where it peaked at 29 million and, after a short decline by a million until may 2022, currently stays at 28 million.

Notice that if you look at the all-time stats, the decline in the summer happens every year, so it probably doesn't have anything to do with covid.

I can't tell if you're being facetious or if you think no one games anymore.

OP is probably excluding children and teenagers.

And adults born in the past 30-40 years?

> Pandemic is over

Are you sure? Gov started telling us (in Poland) that it will come back after summer, so pandemic card is still on the table.

It's not actually over, but if your hobby is getting COVID over and over until your immune system blows out you do you.

Is an immune system more like a tire or a muscle when it comes to wear and tear? I’m an engineer so I have zero medical expertise, but I thought that antibodies make you more resilient to a particular disease and getting sick is the usual way to get antibodies.

The particular biology of sars2 is concerning. It has a super antigen, supresses mhc1, and is very fast and stealthy. Long covid is very real. A recent study found vaccination only protects against it by 15%. Another major study found serial reinfection incurs significant risks each infection.

A. Leonardi hypothesizes that naive T cells are protective. Older people have fewer. Hyperstimulation due to antigen will cause the process from naive to effector to proceed and over time deplete that population.

In any case, COVID is slightly better controlled with vaccination to the point where it doesn't outright kill you, but that doesn't mean that with a seatbelt I'm looking to crash my car on the reg. You don't walk away healthier each time.

It's endemic at this point. 2 years in we have vaccines, treatments, and the virus itself has evolved.

Anecdotally over the last couple months most of the people I know have caught it and recovered a week later.

All of us are going to need a lot of therapy to deal with the PTSD from this pandemic. It's very difficult to know how to weigh small but measurable risks. Meanwhile the unseen risks of years of lockdowns will reverberate through generations.

Wait until the recession really hits.

Cards need to be $500 at the top end. That's when the real shortage is over.

Digital Foundry notes that price/transistor isn't really going down anymore, and die shrinks aren't enough to improve performance or price (at least for GPUs). The result is cards that are bigger, more expensive, hotter, and more power hungry. The last two mean more expensive cooling. The Nvidia 3000 series was a nice price improvement compared to the 2000 series, but MSRP at the top end will approach 2k MSRP in the next generation.

It was a price improvement at MSRP but until recently it wasn’t a price improvement at all due to the insane markups and stock shortages.

No, I don't want top end cards to cap out $500.

Imagine being a hobbyist videographer and having to jump to Quadros or professional cards for video editing.

Most of the comments seem focused on the risks of used GPUs.

I’m just loving buying an FE at Best Buy for MSRP.

Wow. I read your comment, went to Best Buy's website, and after a few minutes of waiting in a virtual "line" I bought a RTX 3070 for MSRP ($500).

Finally I can upgrade my 6 year old GTX 1070.

GTX 970 reporting for duty...

Do it.

I made the jump and its so nice having the vram alone for modern titles.

I am now replacing my 8 years old GTX 750Ti with RX 6700 XT.

The shortage is over, but the prices are still not justifiable. Why would anyone pay MSRP for how old these cards are with new product lines being announced this month?

lol

Depends on the card and situation. If 3060ti performance is plenty for you, how much would you expect to save on a 4050? Probably not much, and those cards won't be out until next year sometime.

For a higher end card? Yeah it could go either way.

New product lines are always on the horizon

In some sense, sure, but in this case it means an actual new generation of high-end ("9"/"8"/"7" class) chips likely launching in a few months, not a "refresh" sometime next year.

Perhaps, but nvidia is 23 months into a 24 month release cycle.

Totally agree, been looking for an RX6600xt, but the price is just not justifiable for how old the card is really.

Lets not forgot, its bad news for the Card Manuafacturers if consumers decide to buy the flood of used cards on the market and not new ones. Its really not in their interest for us to be buying them at all.

Why is that?

It's happening https://news.ycombinator.com/item?id=27009221

Lol thanks for sharing.

Nature is healing.

I'm not too worried about any "wear" a used card might have. Any used models I should look into buying? Or is it just recent AMD or Nvidia with lots of ram?

Check this out if you're looking into mining GPUs: https://www.youtube.com/watch?v=1T0npiqjEWQ

I cant recommend the rtx 3090 highly enough. Everything on max settings, including vr games. Some games can even handle 2x render scales. Works great for rendering as well.

It depends on your expected use case. AI? Gaming? Rendering? The priorities change between these different scenarios.

GPU prices, lumber prices, retail surplus and discounts are all indicators that the supply chain disruption contributions to inflation are coming to an end.

We now enter a time of mass surplus combined with high inflation. [Since truck shipping will continue to be super expensive]

HVAC duct is out. I have a remodel that is just sitting idle waiting on a shipment that is TBD.

Let Nvidia and AMD stockpile these overpriced cards with the "ray tracing premium". Even the MSRPs are not justified

Wonderful, so soon I can finally find a new budget GPU that won’t be more expensive than my aging RX 470 (6 years in use now) that turns on the turbines to run Civ V on WQHD :D

I don't think I'd buy a card just yet, unless I could be confident it was new. Don't want to risk buying a card that is already clapped out from mining non stop.

Imagine someone comes with a new dumb idea. Like, let’s run DL on GPUs to generate some unique cool NFTs that could make you rich overnight. Just need to find morons who’d buy it. I don’t think there’s going to be a shortage of those though.

I think the supply of people willing to pay $11,000 for a randomly generated nothing (e.g. NFT) is actually pretty low relative to the full market potential of randomly generated worthless assets.

Just looked up the card I bought a year ago. Same price everywhere

Hardly. Fed is about to pump the market again.

It doesn’t have the political will to do so because inflation is much worse than a recession

What about buying a PS/5 without having to do nonsense?

Sony was selling them last week and it seemed pretty easy. A number of my coworkers got em. Shouldn't be too long before you can just get them at a store hopefully.

Doubt we'll see any change soon. The consoles aren't being bought en masse by miners, but rather by scalpers.

Now the question is 3090 or 4090

4090 if 32GB of VRAM.

4000 series is also a 500-700+W space heater if the rumours are true... Nvidia taking the Intel playbook of shove more power through it to speed it up, it hasn't sublimated yet..

Between electricity prices, and climate change making summers uncomfortable as it is, that's a no go.

Well, you can always go for a Quadro. More expensive, but will give you much better performance per watt. Or, you can always undervolt it, if VRAM is the main thing you are looking for (which for most people buying a 3090, it is).

I also heard 3080 seems to be a good middle grounds in the 3000 series.

I sometimes get feeling we are rather distanced from average consumers when 3080 that is the high-end card in series is considered middle grounds...

I think that's what I'm going to get. I've been reading up on em and the 3090/ti's are either overkill (more than what I'd use) or diminishing returns vs. price. I'm not a graphics heavy creator/machine learning person. My monitor for example is almost 4K but not 4K so that's waste of graphic computing. It's tough because I want the best "just because". But it's a losing battle.

I can buy a graphics cards = The CPU shortage is over.

Damn.

Reply on news.ycombinator.com