Nvidia Announces “Titan V” Video Card: GV100 for $3000
anandtech.com
anandtech.com
- it doesn't work with Wayland
- it doesn't work in UEFI mode with CSM enabled. There are motherboards available that won't boot in UEFI mode with CSM disabled without a monitor attached.
- it doesn't work with Secure Boot
- about half of their releases fail to work with at least one of the pieces of software we try to use it with. The current release 384.90-0ubuntu0.16.04.2 in particular crashes our software.
And for some reason Nvidia has the reputation for being good at drivers. FirePro was also really bad, but I haven't had to deal with that for years now...
Some of the software we use is linked against GL to display results when run on the desktop. It doesn't actually use GL when run on the server, but it's still linked against GL so we need it on the server.
So forward looking, one can reasonably expect that AMD's Open Source drivers will be superior. But this only applies to RX 4xx series and newer, and not even to all those cards. (Some cards like RX 550 just don't work right now, despite claimed support on marketing pages).
Don't buy NVIDIA if you don't care about performance of OpenGL/etc.
Intel has indicated that they're working on it, and that's who I'm hoping for right now. If Intel came up with on-chip ML operations that are fast enough and made them work with, say, TensorFlow and PyTorch, I think that researchers would abandon Nvidia pretty much instantly. I've never had to worry about my "Intel driver" breaking.
- nVidia cards work excellently on Linux,
- they are the only choice for serious graphics, GPGPU and ML professionals
- the driver problems are overblown. In comparison to the world AMD has caused me over the years they are angelic.
NVIDIA is a bit behind on things like Wayland, but it has been way ahead of AMD for the last 10+ years on Linux for people running mission-critical software that requires good OpenGL drivers.
Try running with AMD hardware on RHEL/CentOS 6/7 (the OS many major VFX facilities use), you'll have a Bad Time (tm).
I am speaking as someone who has been maintaining software & hardware running Linux making heavy use of NVIDIA hardware for the last ten years.
The NVIDIA driver on macOS on the other hand, now there is a lamentable situation...
Ubuntu and Red Hat will probably have new LTS versions using Wayland coming out in 2018. Which means that 2017 is the time to start testing for them.
And it's not really a glowing endorsement if Nvidia is only the second worst driver on Linux, not the absolute worst.
https://www.phoronix.com/scan.php?page=article&item=amd_cata...
I think AMD has gone above and beyond, especially since 2014's above announcement. I think today's AMD are better than NVIDIA's and they are a much better company for Linux
Is enterprise hardware extremely different?
The proprietery AMD Catalyst™ (fglrx) drivers are known to be bad[2], but AMDGPU[3] is supposed to be the best, when it comes to open source GPU driver support on Linux, based on what I've read.
Better than Nouveau or Nvidia's proprietery blobs. Don't people sometimes buy AMD cards just for its excellent native/open-source driver support?
[1] https://www.phoronix.com/scan.php?page=article&item=nouveau-...
[2] https://www.phoronix.com/scan.php?page=article&item=openclos...
[3] https://help.ubuntu.com/community/AMDGPU-Driver | https://help.ubuntu.com/community/RadeonDriver
I can confirm this being my main reason for going with AMD. They have discrete GPUs which meet my performance requirements, and the Mesa-based drivers perform very well, with very few issues (none on my own workloads). There has been a bit of a gap in display output features, specifically FreeSync support, but that's coming along too, and should be at least of "experimental" quality in Linux 4.15.
A frustrating aspect of the NVIDIA drivers is the poorly thought out update protocol. For our academic lab environment, admins want to run nightly package updates as a general practice to get most network-facing security updates as soon as possible. However, this doesn't mean we expect to logout or reboot every machine every night. Lab users keep odd hours and run jobs at unpredictable crunch times.
In practice, nightly automated updates just work, since most lab apps lack dynamic library loader shenanigans which would get confused by new libraries on disk being swapped into place while an older process is still running older libraries in their memory space.
The update experience is that one day everything works, and then next all commands fail to get an OpenGL context because the NVIDIA-related package update replaced the user-space libraries with ones that are apparently not ABI compatible with the still running kernel module. It seems absurd to me that the kernel interface and user-space library are not designed to allow seamless updates. The new user-space driver should be willing to talk to the previous kernel shim as well as the new one which may have more features. It should not expect a flag-day update for every little periodic update.
The users have had to learn to reboot the workstation and hope that it will resolve the problem. Once in a while, the kernel shim apparently fails to recompile on the latest kernel update, and the reboot does not help. The admins then have to intervene to find some temporary solution such as downgrading the libraries or finding a previous kernel version where the latest kernel module successfully recompiles.
On the other hand, in terms of robustness and reliability on the compute side of things, the NVIDIA drivers are decent. I've only experienced 1-2 hard kernel crashes the last year -- which used to be an at least weekly event 5+ years ago.
I still wish they'd split their driver and contributed upstream parts of it if for nothing else, to make compatibility layers and shims more robust, but knowing NVIDIA I doubt that's on their roadmap for the coming century. :)
I'd bet on seeing open source drivers in the kernel at least within the next 5 years, probably sooner
Why can't CUDA work with nouveau? All the special sauce is in the CUDA libraries, I'm sure all the driver does is init the card and do some resource management.
And good luck getting Nvidia "Optimus" properly configured and running smoothly.
Edit: I didn't originally see that the die size of this chip is nearly double that of the older Titan XP. Perhaps the price isn't as outrageous as I first thought! I'm actually glad that there's a big enough market now to support this kind of ultra high end product. It's certainly a beast and I want one.
Prices as well as the tech itself. $3000 and still just 12GB. I wonder why this isn't in the range of ~64/128GB yet... too expensive for the type of memory?
Low end servers use normal dimm, but typically not many dimms per channel, typically 1-3. But to hit the higher memory configurations you end up putting buffer chips on the dimms which makes them more expensive and also slightly higher latency.
So 500GB/sec to 128GB ram using HBM2 would be difficult, expensive, and likely wouldn't fit in today's GPU form factors... yet.
GPUs generally seem to be mostly hungry for more bandwidth and not that many cases (percentage wise) is 16GB a major limitation. However in those cases check out the power9, the nvlink that x86's uses for GPU<->GPU communication can be used for CPU <-> GPU communication and avoid the high latency/low bandwidth pci-e links.
650 GB/s is pretty impressive.
> should make cloud GPU time cheaper and/or more efficient.
No way. These will be not put in data centers of cloud providers. I'm not even sure many regular data centers will have anything like this -- especially if NVIDIA keeps pushing and blackmailing server vendors to stop selling servers packed with GeForce cards if they don't want to loose big [1,2].
[1] https://www.pcgamesn.com/nvidia-geforce-server [2] https://www.digitimes.com/news/a20171027PD200.html
You could also buy ~4 1080Ti cards, but unless you can't use the FP16 tensor cores, you'd still lose heavily to a single V100.
A price premium that will really stack up when certain cloud providers want to buy some of these in bulk.
Edit: The *999 pricing confused me, it's actually $5000 more.
We're excited to see Volta come down below $5,000 / GPU. We're also looking forward to AMD, Intel, Graphcore, Rex, Bitmain, Vathys.ai, Wave Computing, Tesla or Cerebras providing competitive alternatives.
This all seems really expensive compared to the local PC shops that seem to offer most of the components for less, while only charging ~100 euros for assembly and testing.
Do you guys have some really fancy cooling solution that you forgot to mention on the site?
Perhaps the presence of smart Researchers while the computer is being born will make the machine better at learning, improving tensorflow performance?
GPUS should be general purpose and with Nvidia throwing all their eggs in the ML basket and charging a fortune it might risk alienating their core users (gamers).
It is true that Nvidia could be artificially reducing supply.
Sometime early next year Nvidia will announce the Volta enthusiast cards (GTX 1170, 1180?), priced in line with the current-gen enthusiast cards (GTX 1070, 1080).
If you want the new shiny right now, you can, but as always you can get 85% of the same gaming performance for 25% of the price by waiting a few months.
Not all GPU's need to be good at all things. There are plenty of general-purpose chipsets out there for those that want one, and I'd say there's room in the market for highly specialized GPU's and if those are priced incorrectly, the market will correct it. The irony in your statement is that this is designed to be a cheaper version of a very popular card.
Why is that exactly? This looks to be a great deal given its specs in comparison to the rest of the market, and us ML folks like our application specific GPUs because a specific card will always outperform a generic card and you're not paying for things you don't need. I can't even imagine what a generalist card which works for ML would look like. Do you actually work in this space?
I think it's pretty exciting seeing how we are getting increasingly more specific chips for AI tasks. I wonder what their performance is like compared to GPUs.
Is this because in China you're expected to ignore all certificate warnings?
> I wonder what their performance is like compared to GPUs.
Me too. I tried to find a similar table and found this: https://github.com/tobigithub/tensorflow-deep-learning/wiki/...
25 ms/batch of 128 is 5120 images/second... Is that correct? Am I missing something?
I can access it no problem with Chrome. The price seems to be 589$ per chip.
Google wrote an article some time ago about the their TPUs. There are a few charts at the end of the page: https://cloud.google.com/blog/big-data/2017/05/an-in-depth-l...
I would be quite interested in a comparison between Google's TPUs and this Sophon chip, but it seems it's quite new and they have barely started selling them, so we might have to wait.
> 25 ms/batch of 128 is 5120 images/second... Is that correct? Am I missing something?
I'm not particularly knowledgeable about this, so you will have to wait for someone experienced to chime in.
Just ML? Higher memory bandwidth?
https://pro.radeon.com/en/product/pro-series/radeon-pro-ssg/
They need the patches to be upstream and widely maintained by the developers (and be involved themselves) before people 'at large' will really consider using them. The code is there, but practically doesn't exist for anyone yet, outside of AMD developers and a minuscule amount of AMD users. This is also true of all of Intel's ports of various DL frameworks, too. One thing NVidia has learned is that sticking their engineers directly into the pipeline works -- from everything from game development studios, and now to deep learning frameworks (which have nvidia engineers).
That said, AMD cards do have some excellent advantages like unlocked FP16 that Nvidia does not offer outside of the Tesla series, and bringing card price down would rock. So I'm hopeful that they'll sort all this out and make themselves competitive. I'm hoping to see MIOpen support (and if we want ponies, an upstream ROCm driver) sometime in the next ~year for at least a few frameworks (Caffe2 or PyTorch would be good.) But until then, CUDA, CuDNN and Nvidia technology are really the only name in town for like +90% of people, for better or worse.
Titan V has HMMA, so it most likely also has full hardware FP16. It seems to have a binned V100 chip with 3 functioning HBM2 stacks.
It's part of the 'gamer aesthetic'. If I could speculate, I'd suggest they might intentionally do this to push businesses away from consumer-level stuff.
http://images.bit-tech.net/content_images/2016/06/amd-radeon...
But I agree with you, "gamer" hardware aesthetic is grating. It's as if a F-22 and a Nerf Gun decided to have a child together.
Bottom line is there are lots of factors, most significantly case design and cooling strategies for other components.
Only recently can you avoid it with various manufacturers mesh routers or Ubiquiti units, albeit at low-end business prices.
Wish they'd ship the a version of the GTX line with the design language of the more boxy looking Quadro cards (M4000 etc).
https://www.newegg.com/Product/Product.aspx?Item=9SIA85V4R81...
http://images.dailytech.com/nimage/6446_large_8800GT-512DDR3... https://images.anandtech.com/reviews/video/ATI/2900XT/2900xt... https://images10.newegg.com/NeweggImage/ProductImage/14-102-...
The gold shroud on the Titan V is a bit bling-bling, but at least the shroud design is part of a consistent design language from NVIDIA.
Screw you NVIDIA. I've been vocal about this for a while but this is now getting utterly ridiculous, and frankly insuting to the research community.
I don't mind that NVIDIA is ripping of the big derplearning -- and more recently "data" -- companies (Goog, FB, Baidu, etc.), but this greedy bullshit attitude led to the Tesla cards all of a sudden going from expensive, with a 2-2.5x price jump, to the ridiculously priced. In the meantime, they've been "enabling" research by throwing peanuts at the science community with cancer stunts [1]. However, that was just the facade, the real story was that they'd found the cash-cow and the enthusiastic HPC/sci-comp crowd that helped NVIDIA grow out of their sexy-witch-on-the-front-panel-of-the-GTX company box and step into the "big compute boys' league" is no more than just another means to convert peanuts to marketing material.
After years of complaints that there are no sensibly prices _compute_ cards devs and independent researchers can afford, constant battles with NVIDIA to stop crippling their management tools on GTX cards, to allow cheap cards to be used productively for compute and development, etc. they step up their game and release a $3k "affordable" card for "researchers and scientists". This claim is beyond ridiculous, revolting and, as a researcher (working in HPC on a large FOSS simulation project) this is insulting. Which researchers are they talking about, the ones employed by the big ad, finance, companies? Definitely not me, not us!
Time for us to start showing NVIDIA the middle finger for good! Get porting people, support AMD, test ROCm, their openness and attitude receptive to public feedback and criticism has impressed me recently. At the same time, we need to push NVIDIA too: file bug reports against NVIDIA's shitty OpenCL support, against their consumer card crippling drivers and management tools, and resist their efforts to blackmail OEMs to stop selling servers with GeForce.
[1] https://www.nextplatform.com/2016/11/15/deep-learning-superc...
NVIDIA makes fast computers and sells them for market prices. I'm not sure we need to storm their castle with pitchforks.
Also, please show me how can you use those $800 cards for serious computing/research without potentially running into silly issues and a lot of pain (from not being able to use the "supported" standard OpenCL to running into cooling, fan speed, clocking issues that makes reliable perf tuning a pain, etc.).
It strikes me as pretty awesome that the V100 is now available for only $3k in the Titan V, rather than ~$10k for the Tesla V100. I reckon you need to readjust your expectations.
> and are outraged when reality doesn't meet your expectations.
Untrue. I want an option, any option that allows researchers to develop for the fancy high-end chips -- especially when they claim that they want to and are providing such an option. Computational scientist and HPC researchers who aim to write code for, or porting community codes to machines like DOE Sierra, Summit or future V100-based machines need something that is within the reach of a junior PI, a researcher in countries less well off than the rich western Universities and research institutes.
In contrast, jack up the price this high means that the Facebook et. al will still find it worthwhile, but the grand majority of actual researchers, those that they claim to be enabling and helping with this card will not be able to afford it.
It all too easy to forget how the "democratization" of supercomputing was NVIDIA's slogan very ambitious and cool [1] that has over time turned into a marketing claim [2] that offers little in terms of bottom-up enabling the community.
[1] http://gpgpu.org/static/asplos2008/ASPLOS08-1-intro-overview... [2] http://assets.nvidia.com/nv/tesla/pdf/NVIDIA_Accelerate%20Yo...
A new assistant professor at a medium sized school in North America can afford a few of these.
This is a cheap bit of equipment for a professional researcher.
It’s an expensive bit of equipment for an independent researcher or student, or someone in a poorer part of the world. But why would anyone expect the best kit for hobby prices?
(In case if this all leaves you baffled, e.g. assistant professors will often earn 2-4 TITAN V's worth of money per year in places like Eastern Europe -- and that's still on the map of the "Western world" by most definitions.)
> It’s an expensive bit of equipment for an independent researcher or student, or someone in a poorer part of the world.
Allow me to correct you: that is _much_ (if not most) of the world.
> But why would anyone expect the best kit for hobby prices?
Strawmen. Please read my point again (and see the concrete examples of affordable "kit" that can be used to target the high-end HPC iron).