monkmartinez

Agreed, it has been very bubbly across the board. However, I think Nvidia (AMD as well) and Ethereum are especially correlated as first and second order effects seem proportional to incentive feedback into their respective ecosystems. Ethereum has been hounded by the 'wastefulness' of mining, which is to say 'proof of stake' wasn't born in a vacuum. Likewise, Nvidia GPU's were being bought at many multiples of msrp in BULK! Call it a perfect storm, call it what you want... but Nvidia has a tough road ahead.

How many people do you think are 'into' ML enough to spend $1k+ on a SOTA gpu and associated hardware? I am slowly getting there having moved from Colab to my own set up...

However... Ethereum enabled/lured many everyday joes into buying 4+ $1k cards due to the incentive structure of; 'buy mining rig then buy a lambo.' Setting up a GPT-NeoX is orders of magnitude more difficult than pointing GPU compute at Nicehash. I really have a hard time thinking ML will have any meaningful uptake in that regard because the incentive structure isn't the same and its much, much harder.

Big cloud seems to be going there own way in regards to compute. GPU's are great for ML, but that doesn't mean they will always hold the crown. TPU's, NVMe storage, and chiplets may find a better path with newer software.

I just don't see how Nvidia really thrives without drastically reducing price (less margin). I don't think they are dead, but they are in big trouble as are many companies.

The article kind of led with Crypto, but could have easily started and ended with Crypto. Full Stop.

Overlay Ethereum-USD and Nvidia charts for the past 5 years and what does that tell you? My take is both Nvidia's stock price and Ethereum-USD price were products of helicopter money. Sure some gamers bought cards, but not on the scale to which miners acquired them.

Analysis of mining hash rate post-merge bears this out[0]. With some estimates being $5 to $20 Billion(USD) in newer GPU's that are no longer "profitable."[1] The 'chip' shortage meant Nvidia could not satisfy demand for their products. I posit no amount of chips would have satisfied demand. They, along with many other companies, made record profits during the pandemic. There was just soooo much 'extra' money flowing around with stimulus, overly generous unemployment, PPP, and more.

I can't tell you how many people were YOLO'ing every cent of their stimulus into Crypto and WSB stonks. It was an enormous cohort of people based on the explosion of Robinhood and Coinbase use mentioned in WSB subreddit. Mooning and Diamond hands FTW, eh?

Mining crypto with gainz from a roulette spin with Robinhood options? Yeah, while we are at it, lets buy 4 mining rigs because mining Ethereum is very, very profitable during this time. The thinking that one can literally turn electricity to heat while making money (even while one sleeps no less) and trade options while one is awake. It may look crazy right now, but the subreddits were non-stop talk like this.

There are mining rigs all over my Facebook marketplace and local craigslist now. Crypto winter is here (for who knows how long) and Nvidia is not going to make those types of returns for the extreme foreseeable future. Reality check 101.

[0]https://twitter.com/WuBlockchain/status/1552837630278385664 [1]https://www.theblock.co/post/168380/whats-left-for-ethereum-...

Or purchase ampere cards with NVLink and pool the memory.

I see it now, thank you for pointing at it. That is not a great decision by Nvidia, IMO.

Personally, I would buy a couple of 3090's with NVLink and call it day if it was for personal use. I would wait and see actual performance before even thinking about Lovelace cards. Nvidia makes all kinds of unsubstantiated claims during these events.

Where are you seeing that, I am unable to find this information.

My understanding may be flawed, but I am under the impression that 2 x GPUs with NVLink will pool memory. It does not change the underlying 2 x GPU architecture in a way that would represent 2x as a single when looking at the system config. Furthermore, I believe the application code needs to have NVLink enabled to take advantage of the pooled memory. Pytorch and Tensorflow have knobs to turn in order to take advantage of NVLink... where programs like DaVinci Resolve may need patching, etc.

$4800 is what the previous RTX A6000 costs in the US after taxes. That is a BIG cost for anyone outside of a professional setting.

You could buy 2 x 3090's with NVLINK for half that. I recently bought a datacenter M40 with 24GB of RAM for $150 on Ebay then added water cooling. The Dell Precision 5810 I bought second hand can now run some really interesting stuff on those 24GB of RAM albeit slower than a 3090. For less than $500 I can have two watercooled M40's that will run almost anything I can throw at them.

I agree there is a way to go... I am just curious if the answer lies in a super computer. I don't think it does, but I am also openly admitting my naivety.

Since you brought up the brain, I will roll with some thoughts. Instead of thinking about a hunk of wet meat that contains a quadrillion synapses... wouldn't it make sense to break those into discrete, specialized and smaller hunks of wet meat of say 10 billion? Then we link those discrete units under a unified command which directs information to the correct specialized core and then the next until some arbitrary stopping point was reached. AKA: An ensemble of models.

My point; It seems that throwing more compute at individual problems has probably reached a kind of apogee in time and cost. PaLM, Bloom, GPT-NeoX and GPT-3 all live in silos. Cool story, but you can't use GPT-3 and PaLM at the same time without an absolute ton of overhead right now. At some point, I think there will be a unifying middleware that can utilize these models in there current form. That is, you will not need to train yet another model to combine the networks of one or more discrete models. Just a hunch.... could be wrong.

Full disclosure; I am not a pro ML'er. I dabble as a hobby and I am trying to learn as much as possible.

Question; Does the innovation need to happen on the hardware side?

I ask because there is a concerted effort, at least for inference, to reduce memory requirements and provide more access to giant LLMs.[0] It seems like everyone for the past few years has just thrown more and more compute at the wall to see what sticks. Where does it stop? 3 or 4 trillion parameter models that cost a $Billion to train? Is there a law of diminishing returns? Have we reached the apogee or will it be the 15 trillion parameter that ate $100Billion?

Accessibility to all manner of models, for knuckle heads like myself, is relatively new. Maybe I should rephrase, accessibility to interesting and FUN models that mere mortals can use and hack on is relatively new. Stable Diffusion is nothing short of spectacular for just having some fun with ML/DL.

My naïve thought is that, similar to most natural systems, smaller and more specialized models that can be "glued" together as a way forward. In my mind this makes more sense than these ridiculously large language models. That is to say, purpose built or fine--tuned models that can communicate seems like a good idea. GPT-NeoX "communicates" with to Bloom and makes a gaggle of small models that can be fine-tuned further. If regular people with a consumer GPU can train/fine-tune a model at home, this happens a lot faster.

[0] https://arxiv.org/abs/2208.07339

Jensen Huang stated that Nvidia was going to make moves to move cards before the upcoming launch of the 40 series in August's earning call: " We implemented programs with our Gaming channel partners to adjust pricing in the channel and to price-position current high-end desktop GPUs as we prepare for a new architecture launch." <- https://www.yahoo.com/video/edited-transcript-nvda-oq-earnin...

Could this be a move in that direction? Will EVGA suddenly come back once they sell out? Is this planned? Consipiracy?

How do AMD gpus compare with Nvidia for deeplearning workloads? Configuration is tough enough on your own hardware with Nvidia. I have zero experience with AMD cards and suspect the challenge would be greater. I do note that I saw support for AMD pretty early on last week when Stable-Diffusion was released.

Nvidia may pull some pricing shenanigans for the rest of 2022 and into 2023 according to this video[1] I watched. I would be thrilled to see some real competition for workloads that are not gaming. Nvidia just dominates so far as I can tell.

[1] https://www.youtube.com/watch?v=15FX4pez1dw

You are much better off running it locally at those speeds. P100 does 13 to 33 seconds a batch in my experience. Cloud to cloud data transfer (Hugginface to Colab) is ridiculously fast tho.

I totally understand the frustration. Hop on the Conda train and don't look back. There is no performance penalty from using Conda for the boring stuff. The only thing it will cost you is more disk space. Otherwise, its an absolute joy to use. You know where everything is if you want to inspect packages, bin files, wheels, etc. It seems like chasing your tail when you install these things from apt, git, curl, pip and brew/choco. To me, I want to see where everything has come from and where it is going on my system. Conda gives me that in spades.

I totally disagree with ditching miniconda. The Colab notebooks that make use of it have been super easy to run and modify. There is documentation everywhere and its very easy to find on SO, and Google. Its a joy to use and really like it for all of my Python workloads. I think of it like a Python VM that just works where ever I place it... so far, haven't been let down.

I really want to put one of these in my Dell Precision workstation, and I share similar concerns. I have an older Quadro in it now and could use a proper upgrade, but I really want to wait for the 4000 series cards due in a few months.

Craft Computing on Youtube has the best information from what I have seen so far. I don't like watching Youtube videos for information like this, but I understand why creators have moved to this medium in general. Linux should be much easier to configure for using the K80 to capacity.

How have you configured Pytorch in the 'setup' section for your card? The hlky/webui (Shout out to Altryne), is configured for lower end GPU that are memory constrained. The knobs that need a twistn' on these DL models feel infinitesimal.

The first question would be; DO YOU HAVE A F**ING fever? If so treat that first, then call your doctor if that fails, then and only then do you call 911.

I have some serious FOMO with Rust... I have been using JS and Python for everything the last few years. Will you please recommend some Primers/Resources/Tutorials for Rust for someone like me? That is, basically a script basher that glues everything together with Python.

> "Plenty of community colleges are run from state funding and have all kinds of adult education shop classes. You can learn to weld, do carpentry, etc."

Once you are done learning the basics which takes about a week, you still have to participate in class and do "class" projects. These classes are not convenient to begin with and access to the shop ends when the class is over. You could buy a small contractors table saw or cheap flux core welder for the price of one class in my area. Not worth the price of admission imo.

Can you share a few, please? I am already out of credits on Midjourney and I don't even feel like I got the hang of it until I was almost out.

Fair point. You would need to spend quite a bit more on a comparable PC if buying new. However, the breath of applications and interchangeability of the PC may render the price point moot.

For example, I just changed the notebook graphics card on a refurbished HP ZBook... I paid $275 for a Gen 3 and upgraded the Graphics card for another $350. It was a lot easier than I thought and this computer is a beast.

It's hard to see experience on the internet. I agree, it's definitely a two way street with regard to fanaticism. The people that "shout" the loudest often drown out the more reasonable responses.

Myself, I have been back and forth between PC (windows) and Apple several times. My priorities have changed over the years and I find PC's fit my requirements to a better extent since 2014. If Apple can bring back a notebook like HP ZBook or Dell Precision I would consider moving. Until then, I will live with the tradeoff's...

I hate it soooo much. FB marketplace is literally the only reason I have a FB account. It was a heaven send when we moved a year ago. I fire sold 50% of our belongings in like a week. Gave away a bunch of stuff and it was just great. I have looked for vehicles, but searching through the scams is nightmare.

You can tell with a simple glance. Most of the time, the picture is just "wrong"... It will look very woody/forest when they are selling in Phoenix. The other give away is the price, as in 50% off or more compared to similar vehicles you are searching for.

I agree. I use the free version to noodle around, but I don't spend a lot of time on the models for a variety of reasons. I may be too used to the feature tree and rollbacks. Also, I am addicted to equations and variables for parametric designing. They have a cool app and I would pay $5 a month for it to be sure, but I am not going to spend $20 per month for it unless it nearly replaces f360, Solidworks, OpenSCAD, FreeCAD... I don't think it beats any of them right now.

TinkerCAD is on the iPad and actually fun to noodle with FYI.

Those look awesome and I did not know they exist. Thank you for sharing!

On another note, I would love a version of Blender for the iPad. Shapr3D is, imo, ridiculously overpriced for what it is. An OSS 3D creation environment with the Apple pencil would be amazing.

Wow... Please, please talk to your PCP about these events and feelings. If you want help sorting through this, may I suggest; https://www.psychologytoday.com/us/therapists/cognitive-beha...

The Capacity variable when looking at the calculus of EV's sucks, quite frankly. On a depreciating asset (supply chain/covid issues notwithstanding) (vehicle)) you are buying something that degrades and depreciates the entire purpose of the asset in the first place. TCO factoring in loss of range over time is probably something you just have to ignore if you really want to buy an EV. Otherwise, most people would just buy a Prius or a Corvette.

> The only limitation is that there's waaaay less Superchargers than gas stations.

Hopefully this changes with time, but it is a GIANT limitation. Skiing around Arizona I have noticed all the chargers have EV's parked in them. I want to buy a used BMW i3, but the convenience factor is still a ways off for me. My Prius gets awesome mileage and I run the heater full blast in the winter and A/C full blast in the summer.

I have half a mind to put a giant DIY battery in the trunk. Wire in a dead man switch of some kind and run on pure 18650's around town... gonna flirt with the idea a bit more. My DIY electric bike has probably given me a bit more confidence in the area than I need, but YOLO!