Video RAM Transplant Doubles RTX3070 Memory to 16GB | Hacker News Reader

Video RAM Transplant Doubles RTX3070 Memory to 16GB

hackaday.com

151 pointsantman138 comments

So something I've been wondering about for some time. All a GPU really is, is a couple of parts. The GPU chip, memory, PCIE, voltage regulation, and display outputs.

Now, the voltage regulation is pretty darn standard. You can chop up up half of an AMD GPU (the voltagey parts) and Frankenstein them to an NVidia GPU and have that still work [0]. The power transistors and the controllers (responsible for # of phases) are the exact same as in motherboards (they supply the CPU there).

Would it make sense to move the power delivery stuff to a separate, longer-lived part? Perhaps as part of the motherboard, or maybe have the GPU+memory+display stuff be socketable. Making a cheap 400A connector might be somewhat difficult, but not impossible.

[0] https://www.youtube.com/watch?v=4frYxrXD5Gc https://www.youtube.com/watch?v=ewkVNoMbcCc

Intel tried to do the exact opposite, move the VRM to the processor die itself, in Haswell. It did not go well because of the extra heat, but those chips make for great overclock testbeds haha.

Other than that, why would you move it to a separate part? Where? Either you push that extra cost onto the motherboard manufacturers or you need a whole new standard and slot for the VRM? What's the advantage?

These days, everything is moving towards tighter integration (fuck soldered CPUs, RAM and SSDs), so no one is going to do that.

The advantage is being able to replace the GPU, RAM, etc without having to throw away the VRMs.

Those VRMs are not that expensive, and you can't realistically move them far away from the load without constantly under- or over-voltaging the transistors you're trying to power.

Your power company does not deliver 5 volts to your house for your phone charger, nor does it transport 240 VAC more than a few parcels.

We already have 6 and 8 pin additional connectors on graphics cards, so the connector/slot aspect is easily solved.

I think the underlying premise is if you upgrade graphics cards more often or otherwise buy 2-4 graphics cards for every motherboard you buy that moving the VRM to the motherboard (and add costs there), you can still save money and reduce waste.

>We already have 6 and 8 pin additional connectors on graphics cards, so the connector/slot aspect is easily solved.

Those 8 pins are 12V, not 1V. They are spec'd to carry like 12.5A each - make it 15A - bit warmer wires. So around 15W per connector at 1V... and the need is over 300W. So there you have it 20 8-pin connectors.

There is a reason power lanes of the grid are higher voltage - they need a lot less current, so they are efficient. Lower voltage should remain on the PCB with multiple power planes.

To argue alongside your excellent points: it's not just bit warmer wires, but also the underlying voltage drop that would be harmful in such a design, especially in the context of a 1V DC supply.

The power delivery part is cheap; it's the chips and memory that's expensive.

There's a reason why monitors aren't sold as two parts: the display itself and the power circuitry.

Some monitors actually are powered from external adapter. No idea how common it is, but one is sitting on my desk.

Interestingly, it's also the one that turns off when there's some interference from the lights turning off in the next room,...

I guess I'll try re-arranging the wires if that stops the pickup of interference.

> Some monitors actually are powered from external adapter. No idea how common it is, but one is sitting on my desk.

I vaguely remember it was for regulatory reasons. I searched around and it looks like it's because with an external power supply they only need to certify the brick rather than the whole unit https://www.reddit.com/r/sysadmin/comments/24jxfj/anyone_els...

It being thinner is also a bonus.

>Some monitors actually are powered from external adapter. No idea how common it is, but one is sitting on my desk.

Well that's just AC/DC which is similar to the PSU of a computer or laptop. The monitor, itself, has voltage regulation for much lower volts that the one connected to the mains.

> Interestingly, it's also the one that turns off when there's some interference from the lights turning off in the next room,...

My Samsung monitor (actually a TV) does this, with any spike/dip on the mains line (eg the fridge turning on). I assumed this was just poor quality power circuitry.

Is yours a Samsung too by chance?

Both my monitors have an external brick. Unless that is something else?

This is exactly the same as the laptops have a power brick, or desktops have a PSU. It's relatively dumb AC/DC switch mode power supply (say ~90-230 to 18VDC) w/o any major care about ripple (100mV ripple is amazing already) or even exact voltage. The monitors do have more voltage regulation inside.

Yes, that reason is easier UL and CE certification.

I wonder what cost savings would be passed along doing it this way. Also, would PSU be a better (or possible) option? A PSU has the ability to outlast a motherboard in the typical consumer cycle, as a motherboard will only see 1 or 2 GPUs before a system upgrade due to chipset/CPU socket/RAM upgrades

The proposed idea is not viable for so high currents with so little ripple and microseconds transient responses.

>Would it make sense to move the power delivery stuff to a separate, longer-lived part?

Likely no, the voltage regulation is already hard being that close to the chips and memory. A modern vrm for rtx 3090 is 16 phase by 70A stage. The gpus already have multiple power pins and with so high amps the losses in the conductor become non-trivial, hence voltage regulation is even more difficult.

Moving it farther away would make the regulation extremely conservative, overshooting and delivering too high voltage.

Even if all that happens GPUs would still require massive amounts of filtering caps.

I believe transformer based "VRMs" (they are actually unregulated) like these https://www.techspot.com/images2/news/bigimage/2020/05/2020-... is the future as a way more practical way to deal with power supply of few hundred Amp chips.

16 phase buck controllers are greatly, greatly impractical, and the only reason the industry is stuck with them is that nobody was brave enough to do serious research into making responsive magnetic based DC-DC, while everybody knew that such thing should be possible.

It's interesting curiosity how Taiwanese made it an industry trend out of multi-phase bucks because people who were making first PCs in Taiwan had rather subpar electronics engineering education, or were basically self-taught, and designed computer boards by trial end error back in nineties.

> Would it make sense to move the power delivery stuff to a separate, longer-lived part?

That's what your computer's power supply is: A long-lived part that delivers bulk 12V power to be regulated down to local voltages on the spot.

High performance parts will always need local voltage regulation. The VRMs must be physically close to the chip to provide a low inductance path to serve fast current transients. Moving the VRMs too far away from the chip would compromise the power delivery. Running the power through wires or connector pins is significantly worse than running through broad power planes embedded in the PCB.

At these speeds, designers spend a lot of time simulating and analyzing the power delivery network on these PCBs. The power regulation and delivery is fine tuned to support the high speeds required. You can search for "Power Delivery Network Analysis" if you want to learn more.

This is an awesome comment, thank you! I had long wondered myself why GPU power delivery was so complicated.

Figures you have to look at to be scared about in this space is an RTX 3070 can pull 250 watts and the core is 1.2v (I haven’t confirmed this) then the card can be sucking up an average of 200 amps. Which is a lot. Peak currents can be a lot more than that too.

That's a crazy amount of current to pull. I have melted wires or even welded things together with a super-capacitor setup delivering a similar amount.

My 3090 tops out at 400W on a not even very aggressive overclock. And it makes my room HOT when it's pegged. These graphics chips are insanely power hungry and beefy as hell these days.

That reminds me of some hardware teardowns (eg of the new consoles, latest graphics cards and such) where they show how the VRM's are scattered around to balance what they serve as equally as possible (to keep heat spread out?) while still being as close as possible to what they provide power to. Its quite interesting!

A100 uses such "single part" power supply sitting right next to the package.

I believe such chips can be easily unsoldered relatively easily, and reused.

And by the way, refurbished, and recycles parts, and even passives ARE a thing, it's just the part of the industry you never see in manufacturing of goods going to first world countries, but in China you can buy 2nd hand electronics components quite easily.

As an example of a PC power supply doing things slightly differently, the Lenovo P620 has a power supply that connects to the motherboard with a slot connector. I heard that it only supplies one voltage and the rest of the voltages are generated on the board, but I can’t find a reference for that.

https://www.anandtech.com/show/16482/lenovo-thinkstation-p62...

Intel is introducing a new standard where the PSU just supplies 12V: https://m.youtube.com/watch?v=heyGtgdfN7A&vl=en , but it still has cables. The Lenovo design is neat but it would mean the placement of the PSU relative to the motherboard is unchangeable..

I remember seeing some older video cards, and they actually had removable video memory in some slots on the card itself

Yeah, but the memory in something like a Millenium 2 runs at like 1/500th the speed compared to the memory in a modern GPU (like the GDDR6X in the RTX 30 series cranking out 21 GBit/s per pin). Meanwhile the physics stayed the same.

I’ve heard that about SSDs and EEPROMs. They don’t require a 27V or whatever supply anymore because they’ll generate it on-die with charge pumps.

But charge pumps are inefficient (like 25% efficient), take up a lot of die Space.

It would be better to have a switch mode PS generate the voltages required for erasing (effectively required for every write). On a USB key, whatever, but on a 24/7 system or laptop, it can matter.

Maybe it’s a thing now, but it’s very recent:

https://e2e.ti.com/blogs_/b/powerhouse/posts/how-to-speed-ss...

> It would be better to have a switch mode PS generate the voltages required for erasing (effectively required for every write).

Interestingly, chips like simple USB and Ethernet controllers sometimes do feature an on-die switched-mode DC-DC circuitry to simplify the system design. You just connect an inductor and a capacitor to the DC-DC pin, and route its output to the Vcore input, pretty convenient.

Do you mean this [1] kind of charge pump? The article indicates 90-95% efficient, that's a huge gap from 25%

[1] https://en.wikipedia.org/wiki/Charge_pump

Yes, but the keyword here is "sometimes", charge pumps can be efficient only when very specific conditions are met, usually low current and power (but there are exceptions), efficiency drops sharply as current increases. Putting the circuit onto the silicon creates many additional limitations.

The reason why the VRM is tightly integrated with the asic is that there are extremely stringent constraints on the power integrity connecting these two parts. To first order, a decent GPU draws about 1000 amps of current, so even a single milliohm of resistance would absorb the entire voltage supply. Not just the ripple tolerance, the entire power rail! This is before we even get into the ludicrous requirements on transient behavior, inductance, etc, of the connection.

They do make GPUs with a socket for the asic (typically only used in test labs) so it is theoretically possible that you could for example... buy a 2070 carrier and then get nvidia to sell you a 2080 asic to socket into the carrier. But generally the technologies used in GPUs change every generation (memory technology, number of memory channels, display technologies, etc) so you probabably wouldn't be able to keep your carrier more than 1 maybe 2 generations tops.

I think there is a typo in "1000 amps of current."

for desktop GPUs 1000 is bit much, but 600-700A are totally doable under LN2

I think people are conflating watts and amps, watts measure power, amps measure flow. Watts = amps * volts. High power usage with low voltages results in high amperage.

Even so, in this very case the amps are higher than the watts(!) as the voltage is below 1V.

If Vcore is near 1 volt, then the magnitude difference between amps and watts is pretty small for the sort of discussion we are h aving

No, a modern GPU literally draws about 1000 amps of current. For example, a non-overclocked 3090 draws about 500w under load (some of this goes to memory, but that is powered by the motherboard not by the external power connector), at a Vcore of roughly 1.1v which gives a bit over 450 amps--on average! The peak power consumption (as measured on a nanosecond scale being consumed by the asic itself) can easily be double the average power consumption due to the bursty nature of asic power draw.

Remember that it's not just one core - that's what makes a GPU so powerful. My understanding is it's hundreds or thousands of 1.1v cores, which adds up to the 500w.

EDIT - wait, what GPU draws 500w? I can't find one.

Not sure any base cards draw this much. But might be doable via modding:

https://www.notebookcheck.net/The-NVIDIA-GeForce-RTX-3090-ma...

I remember dual Pentium Pro motherboards with socketed power regulator modules, one for each processor.

I highly recommend you to read Linear Technology AN104 - Load Transient Response Testing for Voltage Regulators [0] to understand the staggering challenge in hardware engineering to design an acceptable voltage regulator for digital processors. Even though it describes 2006's tech for a simple embedded system - it's basically a toy today, but makes a good example.

The problem is transient response. The power consumption of digital processors varies wildly. For example, if a processor suddenly switches from an idle to active state, a sudden increase of current consumption makes the voltage to fall, until the regulator is able to react by conducting more power, hopefully fast and smooth enough before the processor crashes, on the scale of microseconds or even nanoseconds. This is not a trivial problem. Inductance is another problem - in any closed circuit, every piece of conductor has the tendency to resist the change of current, which means even if the regulator itself is ideal, if it's too far from the processor, it's physically impossible to react fast enough before the output voltage goes out of control, thus all ASIC requires local regulation, and the end result is extremely sensitive to board layouts and components choices.

This note describes how to build a "simulator" to test such transient currents and analyzes various circuits for this purpose - not very readable and you can just ignore this part. Just read page 1, then start reading page 6 ("Capacitor’s Role in Regulator Response") for some actual tests. Page 8 has a practical example: testing a power supply for Intel P30 Embedded Memory.

> supply tolerances are tight. Figure 26’s error budget shows only 0.1 V allowable excursion from 1.8 V, including all DC and dynamic errors. The LTC1844-1.8 regulator has a 1.75% initial tolerance (31.5 mV), leaving only a 68.5 mV dynamic error allowance.

2020 PC hardware is at least two orders of magnitude more complex than that. A simple example can be Altera Corporation AN 574: Printed Circuit Board Power Delivery Network Design Methodology for an FPGA - every single component connected to the power supply (and the circuit board itself) must be modeled and simulated as a resonant circuit with complicated frequency response and impedance characteristics [1].

[0] https://www.analog.com/media/en/technical-documentation/appl...

[1] https://www.intel.com/content/dam/www/programmable/us/en/pdf...

The last page of the Linear Tech application note is certainly not something I expected to see. Glad to see they're having some fun

> Making a cheap 400A connector might be somewhat difficult, but not impossible.

It will be, I dare you to do.

Inductance will be a problem. GPU makers are already having troubles with inductance VRM directly near the chip.

> Now, the voltage regulation is pretty darn standard.

Saying this, take a look on A100

https://www.techspot.com/images2/news/bigimage/2020/05/2020-...

No VRM in a conventional sense. It's a fancy HF DC-DC with transformator!

It happened the same with TV tuner for computers. Standard Philips tuner and radio chips, and custom additions plus a PLL from the factory.

I think a better approach for enterprise and high end hardware would be socketed capacitor or diode banks. Those are the parts that usually deteriorate or fail anyway.

This way you can keep the components close to where they need to be and still get that modularity you're looking for.

What you will gain in reusability you will more than lose in costs for 00 gauge copper wire, space constraints and interconnect costs, and likely the transients would still kill you.

We already had that, and it was terrible. Socket 5 and some 7 boards shipped with connector for 2.8V VRM. Then then you had server boards with elaborate external VRMs, like in the picture on the wiki https://en.wikipedia.org/wiki/Voltage_regulator_module

Where do you even get video cards right now?

I’ve become oddly paranoid that mine is going to die and I won’t be able to find a (reasonably priced) replacement. And two year old RX 590s are selling on eBay right now for more than I paid for mine. Never imagined that one day video cards would be an appreciating asset.

I bought my son a used 3060 Ti at the beginning of February from Micro Center for $512 + tax. It had been in their display machine before that, and never out of the store.

From a timing perspective, I got totally lucky. I happened to be there twice in one day, and just got lucky that he had dissembled the machine a few minutes before I walked in (he was still putting it together with updated parts when I was at the counter). Prior to that, I had looked everywhere for weeks (it was supposed to be for Christmas!) without any luck.

Patience and luck in my case. And the Micro Center has earned me as a loyal customer.

I really wish I had a micro center near me. I miss them. I also miss the Frys of (late 90's early 2000's).

It's cool though that microcenter is doing the opposite of other vendors and doing an in store only thing for cards.

I wish MicroCenter would work on their customer experience. I go to one in Cambridge, MA to buy IoT supplies and it's never organized (despite signs saying "this area stocked by XYZ"), popular products are behind the service counter and not even in the main area (so you have to ask) and associates rush to stick their associate ID barcode on your merchandise and usually at check out another associate takes it off and puts theirs on. Perverse incentives for sales staff and poor customer experience.

But they often do loss-leaders on popular things to get folks in the store, so...

Wait, has the Cambridge Microcenter had 3080s recently? When do they restock?

I've heard of lines in the morning at other Microcenters but don't know much about the one next to me.

*Patience and luck and a willingness to pay way over MSRP for a display model

Yeah there are ways to get cards, and I'm not gonna fault anyone for interacting with the laws of supply and demand, but it seems a bit disingenuous to heap praise and declare loyalty to MicroCenter for this. A scalper 5 minutes ahead of you would have been given the same "deal" and that card would have been on ebay within the hour and mining ETH inside of a week, just like the rest of that store's inventory.

Linus from Linus Tech Tips said that he has insider information that the problems weren't caused by miners at all, it's just that more people are working from home and need new hardware: https://www.youtube.com/watch?v=3A4yk-P5ukY

Very few people need a GPU for work.

There might be a few, and there might be more gaming, but seriously? If I was a miner and could get RTX 3xxx at 100-150% MSRP, I'd be buying them non-stop.

A 3070 at MSRP pays for itself via mining in, what, 10 weeks? And the risk of it being worth less than MSRP any time soon seems tiny.

TL;DW: GPUs are competing for chip production capacity of 2 producers, who seem to be already in stress because of generally increased demand caused by the current situation, so they can't handle demand even a tick above the previously reserved capacities.

I'm a teacher streaming from home. There's more of us.

Great suggestion! I watched it a few hours ago. Fantastic episode, and I highly recommend anyone interested in GPUs watch it!

> A scalper 5 minutes ahead of you...

I’m not going to argue this point for obvious reasons, but short of the work that say, LLTstore is doing, what should MicroCenter be doing?

You’ve got a valid point, which is probably why all of us gamers are so pissed off - we’re all helpless. It’s so insane that I’m actually considering trying to find a 3080, and then selling his Ti. It would take a bit of work, but if I could find one, it would be worth it, and we might have an extra year of life out of a more powerful GPU.

> what should MicroCenter be doing?

I'm not suggesting MicroCenter should be doing anything other than moving product. If they can mark it up and still sell it, I can't fault them for it. They don't owe me anything. At the same time I don't think it's right to praise them for it which is what my reply was criticizing; they didn't do anyone any favors here.

I had to resort to...gasp...ordering a pre-built system. I first ordered a Razer Tomahawk - which is a NUC coupled with a 3080 Founders Edition. It was about $3300.

The next day I saw I could get an HP Omen which was much more maxed out (including physical space) - including a 10 core processor, 64GB of RAM, etc and a 3090 with 24GB RAM for about the same price.

I feel shameful for having to purchase a pre-built machine, but it was the only way I could get one since my contact at NVIDIA left the company. On the bright side, if I don’t like the machine, I can resell just the GPU and recoup almost the entire costs.

Unsurprisingly, you can no longer order either machine.

> On the bright side, if I don’t like the machine, I can resell just the GPU and recoup almost the entire costs.

Damn, you're right - I couldn't believe that but it seems to be correct:

- Some random (but available) PC that has an RTX 3080 included: CHF 2802.- (Switzerland, shop I usually use, all their HP Omen with RTX seem to be sold out so I could not use that as reference) ( https://www.digitec.ch/en/s1/product/captiva-g25ag-amd-ryzen... )

- RTX 3080 on Amazon.de: EUR 2699.- ( https://www.amazon.de/Gigabyte-GeForce-Grafikkarte-GV-N3080G... )

Incredible times... .

I did the same, but with a 3070. I was going to buy all new parts anyhow, and this saved me a lot of hassle, so it was worth the extra money IMO.

I still didn't like doing it. It's the first pre-built desktop that I've bought ever. And the first one I've owned since middle school. But I was sick of waiting and wouldn't pay a scalper.

Did the same, was on a pre-order waitlist for a 3080 for months before I cracked. But all priced out it was only like $500 more for the prebuilt vs buying all the components myself. Which considering the markup on 3080s on ebay seems like a bargain.

Buying high-end GPUs? Basically don't bother. You can sign up for the Newegg shuffle which happens at least once a week and hope you get picked to buy an RTX 3070 but even those are pretty expensive. People often brag about how they subscribe to notification bots which tell them when Best Buy gets restocked, but for every buyer who got lucky, there were hundreds of people who dropped what they were doing just to be told that the card is now out-of-stock and has been removed from their shopping cart.

For lower end GPUs? They're ridiculously expensive, but if a card was $100 originally and you have to pay 200% the cost due to shortages, it hurts but it's still doable.

Like others have said, it's too bad that you already have a rig, because now is a great time to buy prebuilt PCs. The '/r/buildapcsales' subreddit usually has people sharing the link whenever a decently specced prebuilt PC goes on sale.

you buy them in workstations through your account manager with your friendly tier 1 vendor. ouch I really ought to be doing something to provide something beneficial from 3 decades of corporate purchasing but just slinging this out there skips the pro salesman v pro salesman level of attention required tho it isn't rocket science but neither is rocket science... like the vram upgrade I genuinely badly want to wave my arms in the air shouting "come here I'll fund the smt assembly automation " edit the rest is in my profile now since unreadable in line but details what I still want to do and will if the opportunity to set up shop for people to do like the article with dedicated facilities

In the bay area, Central Computers. They have 3070 in stock for system builds, and recently they have had 3060 3070 and 3090 in stock for standalone purchases. Microcenter and other brick and mortar electronics stores across the country are similar.

Just seconding this. They have had stock fairly regularly, and they have a pretty good notification email for when they will have new items available. I purchased a 3090 from them, along with a few other parts.

Caveats: in my experience the Central Computers guys are kind of dicks. I've had a couple of bad experiences with them before. They also have something like a 10% surcharge over "MSRP" for video cards. OTOH - they actually have stock, nobody else does. Go for it.

I had a friend sell me a NIB 3070 this past week for $850. I think he bought it for $750. MSRP of the founders edition (FE) is $500, but MSRP of all add-in board (AIB) partner cards are $600 to $900. Most US retailers are marking up 20+% above MSRP when they do have them in stock then scalpers use bots to pick them up and resell them on amazon and ebay for 2-3x MSRP. Considering I am using my card to mine ethereum (so the card pays for itself) and gaming otherwise, I think I did okay for myself even though I paid well above what would be considered reasonable pricing in a normal year.

Anyway, if you are reading this and want to buy a GPU from a retailer you have to attack on all surfaces. Scalpers likely write their own bots and spend money to have many accounts running at once (to avoid ban limits from IP and account spam detection). The higher that GPUs are valued (due to scarcity and economic value of mining), the more they will spend and the less likely you will ever get a card.

The first thing you need to do is to make accounts on all AIB sites (such as EVGA or Zotac) then enter their queue for GPUs. They sell some of their cards to people on a FIFO basis and the queue is several months long, but this is the most reliable way to get a GPU in the next two years.

The next thing you need to do is to join a few discords and follow a few twitter accounts (such as stockdrops). Keep your bestbuy, newegg, and amazon sign-ins warm on the device you will be monitoring (likely your phone).

The final thing you can do is run your own open source bot. I only know of fairgame [0]. It's limited to amazon (not a great place to look currently) and you can only check one listing every 3-5 seconds to avoid a ban. I made an account to do this and ran it for a few weeks (with no hits in the MSRP ranges). You can check out the cheat sheet (linked in the readme [1]) to see a decent list of card listings.

0. https://github.com/Hari-Nagarajan/fairgame

1. https://docs.google.com/document/d/14kZ0SNC97DFVRStnrdsJ8xbQ...

I was mainly refreshing the pages of as many vendors I’ve found and overpaid. I wonder if Steam, GOG and Epic could somehow pool and make a “gamer certificate” to skip ahead of scalpers and miners. Nvidia seems to try to brick the drivers of the 3060 to discourage mining but I suppose it’s not that hard to work around.

The Linus Media Group (LinusTechTips etc) is attempting to do something just like this.

Interesting, do you have a link or a name of the initiative to search for?

I have this, the latest update they gave.

https://www.youtube.com/watch?v=30Uvspk0czw

Thanks! I hope there will be more endeavors like this.

I was able to get rtx 3070 last year, but I was refreshing eshop whole day.

The stand-out thing to me is that the RTX 3070 only comes with 8GB of RAM. The GTX 1070 I bought five years ago with an MSRP $100 less than the 3070 (and that I was able to purchase at MSRP) has 8GB of RAM. The GPU I bought five years before that only had 1.28GB and before that 256MB. What the hell has the industry been doing for the last half decade?

Limiting the RAM so AI researchers buy Quadro/Titan cards.

Just like Canon that downgraded their DSLRs to protect their cinema cameras, NVIDIA will loose shares with enthusiasts while AMD (which got solid TF/PyTorch support recently) can finally catch up.

> Just like Canon that downgraded their DSLRs to protect their cinema cameras, NVIDIA will loose shares with enthusiasts while AMD (which got solid TF/PyTorch support recently) can finally catch up.

Is this for real? As in: I'm thinking about starting to do model training with TensorFlow. I can buy an AMD GPU to use for this and not regret it later?

Here's a tutorial : https://medium.com/analytics-vidhya/install-tensorflow-2-for...

I haven't tested myself, but since I have PTSD from Linux CUDA installations I'm glad competition is finally here.

whether or not the drivers work for quick demos/benchmarks is not really what I care about with AMD, tbh, their ML pipeline sw QA is bad, historically the major problem was long-term stability. Segfaulting 12 hours into a run is not good for an ML practitioner's sanity.

I always thought that the marketing jargon "cannibalize your market" just meant "create a kick ass product"

I hate marketing.

You will be happy to know that the RTX 3060 has 12GB of VRAM. I try BERT-Large training on it here:

https://www.naut.ca/blog/2021/03/16/rtx-3060-vs-jetson-agx-f...

True but without comparable offerings like CUDA, it's not really looking bright tbh. Open CL and open vino are years behind virtually speaking with respect to support.

The 1070 had GDDR5 RAM and the 3070 has GDDR6X RAM. Instead of greater capacity, we get greater bandwidth.

https://graphicscardhub.com/wp-content/uploads/2017/01/gddr6...

Thirteen years later and it's less than three times as fast. This is just sad.

I believe the non-RTX (important, confusing with nvidia, as usual) 3060 has 12 GB of ram, which makes it a nice buy for many ML tasks.

non-RTX 3060? I don't think that exists. Got a link to that?

whoops. Thanks for correcting me *non-TI. Can't edit parent post. And also note some non-TIs have 10GB, not 12 (still more than 8GB). Nvidia has successfully confused me =D.

I'm very surprised that Nvidia chips don't have memory size locked in hardware.

They kind of did - there are resistors on the card's PCB to inform the firmware of how much memory is installed and this operation modified them to match the new amount of memory installed.

But beyond that, what would be the reason of spending effort locking this down even more?

Prevent market segmentation bypass.

The effort & equipment required to perform such an upgrade is significant and is very unlikely to be used at scale for defeating market segmentation.

Well, there is a precedent of graphic cards with modded bios being sold.

Shhh.

I can see that happen as an upgrade service on ebay. It just needs to be priced competitively.

I don't see the economics working out. You're going to be competing against the 3090, which has 24GB of memory and costs 3x as much as a 3070 (MSRP), but also 78% more shaders. Just to match the 3090's compute capacity is going to cost you $1000 (or $891 if you amortize over multiple cards). That leaves you $500 for the memory chips, labor, and equipment. According to [1], the memory chips cost $12/GB. You won't be able to reuse the existing memory chips, so that means you'll need to buy 16GB of chips for each of the cards, which works out to $384. At this point there's only $116 left, and I doubt anyone would go for this considering that the 3090 only costs $116 more, but has more memory (24 GB), and has full warranty.

[1] https://www.hardwaretimes.com/doubling-graphics-memory-will-...

3090 is largely sold out despite the ludicrous pricetag

The cost is still going to be high and you risk killing your card. Even if someone offered this kind of service, it would only be taken by a very small number of enthusiasts. So nVidia doesn't care.

If someone is skilled enough to work on surface-mount/BGA components, they likely already know what they are doing. Board level repair vs board level upgrade is a very similar process (to the untrained eye such as mine, at least).

Economic reasons aside — there's historical precedent for Nvidia creating arbitrary limitations in software (and sometimes in dedicated hardware) for their cards.

Famously, many discrete GeForce cards don't play well with virtualized OSes, and there have been some hacky ways to bypass that. This is done because only Quadro cards are blessed by Nvidia to run in "enterprise" environments. Besides this soft lock, there's usually no architectural limitation preventing GeForce cards from working with virtualized OSes if the hypervisor can provide passthrough.

All that to say, it wouldn't be the most shocking news in the world if it turned out they prevent this kind of VRAM modding.

I'm having trouble imagining a world where the cost of this operation is less than the cost difference between a 3070 and an equivalent 16GB card. Buying the ridiculously overpriced 3090 is still going to be cheaper and not come with any of the associated risk (invalid warranty, potential future driver/bios compatibility problems).

In this specific case, I think it's more about if you can do it rather than if it makes sense. It's a fun challenge, and if you can spare the cash, it does inform the community a bit more about how modern Nvidia cards react to this kind of mod.

A big fraction of Hackaday's content is this kind of fun modding that might not make immediate economic sense.

It’s likely that nVidia planned for more RAM on these cards early in the development process. Early rumors pointed to 16GB models of the 3070 and 20GB models of the 3080.

Due to chip shortages, most likely, we got the smaller amount of RAM on production models. This leads to weird situations where the 3060 has more RAM (12GB) than the 3070 (8GB) and 3080 (10GB). nVidia must use RAM in multiples of the bus width, so the only other option was 16GB for the 3070, which is what this person enabled by swapping chips.

nVidia doesn’t care about modders swapping memory chips because it’s not a threat to their sales. I say this as someone who has all of the hot air rework equipment and experience to do this swap: It’s not going to become a common mod. It’s likely not even feasible for someone to offer as a service due to the risk involved in shipping expensive GPUs two ways and then reworking the PCB with hard to source chips that can’t necessarily be guaranteed to work at the exact speeds expected by the drivers.

It’s also possible that this mod will be rendered useless if a future driver update introduces different timing parameters, such as when nVidia releases official 16GB 3070 models.

> It’s not going to become a common mod. It’s likely not even feasible for someone to offer as a service due to the risk involved in shipping expensive GPUs two ways and then reworking the PCB with hard to source chips that can’t necessarily be guaranteed to work at the exact speeds expected by the drivers.

That's too bad. I prefer to buy a top-end graphics card once every 3 or 4 generations. I was up for a refresh with the 20-series, but the value was pretty bad so I waited. Now the 30-series looks great except for one glaring issue (aside from availability): VRAM capacity. With the imminent release of the direct storage API for Windows, I expect the 30-series cards will show their age much faster than previous generations thanks to their relatively small VRAM capacity compared to Radeon and game consoles.

Doesn't that add manufacturing complexity? If you have multiple firmwares it's possible to flash the wrong one at the factory leading to a lot of pain if they get out to the market before this is discovered (your news 16 GB card is only using 8 GB of memory). This current scheme means there is one firmware to rule them all and the configuration of the hardware itself acts as the selector.

Probably because people don't do this at scale. Fermi and Kepler also used laser cuts near the die to disable cores, some people enabled them with a bit of soldering (not always successful because the cores are often actually non-functional). Not sure about newer ones.

Maxwell cards could also be upgraded from 4-6GB to 8GB. This has actually been a cool mod for a long time, earliest I can remember are the ATi Radeon 9x00 and nVidia GeForce 4 Ti cards, people used to double the RAM for fun and profit.

Looks like the VRAM uses a BGA package with quite a lot of contacts. I'm surprised they got 8 of these done well enough, by hand with a heat gun, to work.

I saw someone do a similar act a year or two ago, slightly bigger contact pads. I guess if you get the circuit board fairly level, and you have dead solid hands, you can get the old chip off and leave a little puddle of solder on the contacts (which doesn't spread due to surface tension + the protective coating on the circuit board?).

If you get the new chip straight (and it looks like he added more flux?) then it can bond again. But if you're over caffeinated or the chip otherwise twists, or you use the heat gun too long, then you have an expensive problem.

Gives me anxiety looking at it and the cost/scarcity of an RTX3070 :)

I'd want some kind of jig for lowering it.

It's fairly easy. Heat the board, drown it in flux, heat the part and surface tension will do the rest.

> So did it make a huge difference? That’s difficult to say.

This absolutely would make a difference for many deep learning tasks.

Speaking of chip transplants: There are chip-level flash capacity upgrades for some iPhones that can offer storage amounts that weren't available.

Deep learning's going to love this.

16GB is pretty meh though. I have a 32GB Titan V from 3 years ago which I got for free in a raffle. I'm surprised they're still even doing 8GB cards.

Yeah I have the 3070 and I hate that it has the same amount of ram as a 1080 from 2016 while costing more than a 1080 did at the time.

That sort of feels like complaining your ferrari has the same size gas tank as a Civic. It's not why the car was made.

I was posting in the context of deep learning specifically, per top level comment.

Well yeah, the comment still applies. GeForce is made for gaming, so the bill of materials goes to what improves gaming performance.

It should still do deep learning activities much faster than the 1080 though, doesn't it?

Not relevant if your model doesn't fit in GPU memory.

That's fair, but the 3070 isn't being marketed as a deep learning machine primarily. I don't know why it's the fault of a midrange gaming card that it doesn't fit a deep learning requirement. The 3090, which is much more expensive mind you, is the one you'd want with 24 GB of VRAM.

The size of a car's gas tank has no effect on performance. I think the ram size here does, so you analogy is not applicable

For gaming (which these cards are sold and marketed to), it doesn't make a difference today.

It depends on the games you play. I often run out when playing VR games, so I have to reduce resolution. Even with an 8gb card.

How do you know that's a memory issue? I'd expect that to be less of a concern in VR games and more a requirement to render the exact same duplicate resources at slightly different perspectives, which should not be memory, but pure GPU throughput.

No, it is - in terms of gaming performance, the thing that the 3070 is primarily made for, it's much much faster than the 1080 despite having the same amount of RAM. It's probably also much faster in Machine Learning activities as well.

>The size of a car's gas tank has no effect on performance

It has a dramatic effect on performance when you run out of gas -- a little bit like the brick wall you hit if you run out of memory. The GP'S analogy isn't very accurate, but it's not totally wrong

I guess the analogy works if you could start ripping out random components from the car and put them in your gas tank and burn them as gas

Are you the one of 20 chosen?

https://www.zdnet.com/article/nvidia-reveals-special-32gb-ti...

Yeah. I'm looking to upgrade my 1080, but I'm already running into memory limitations. I was actually considering messaging the guy and seeing if he can make another one like that. The 3080's 10GB isn't very much either, and I really don't want to cool 320 watts (too loud).

You can set a lower limit on the 3090 to be say 250W or even 200W. Performance will suffer, but you'll have full access to VRAM and it'll be quieter.

I've seen 16GB versions of 3080s in laptops. They only run 150W-165W, so they are slower than desktops. But if RAM is your point, they can be found.

WTB this hack for 3090s!

Reply on news.ycombinator.com