Nvidia to release documentation on Nouveau project | Hacker News Reader

Nvidia to release documentation on Nouveau project

lists.freedesktop.org

256 pointsthrowaway204892 comments

I think the reality is that most software engineers are starting to get tired of Windows and OSX, and the kind of people nVidia wants to hire don't want to contribute proprietary blobs to Linux when they can just work for the competition on open-source stuff. Thus, nVidia is kind of forced by the market to let the engineers make at least token contributions to Linux.

(This is also why DRM never works; the type of person that could implement it correctly wouldn't want to implement it correctly.)

>the reality is that most software engineers are starting to get tired of Windows and OSX

I agree with you about Windows but note that outside of the U.S. very few software engineers ever even get to work with OS X.

Really? Every developer I know in the UK who doesn't develop for an MS stack is running either Linux or OS X, and that's been the case for the last ten years.

I was in academia in Japan and there cygwin and Windows were the favorite tools.

Some of the people used Linux in VMs, on the other hand, some new students installed XP on their notebooks. In 2012.

And I think this was even a rather progressive lab, I suppose industry only uses Windows.

Japan is a Windows loving country so its hardly representative.

Well actually Silicon Valley is an OS X loving region so its hardly representative.

Indeed, they are both extreme. So, what was your point?

Have you been to a software conference outside of the US? Certainly in Europe it's Mac dominated.

I have.

The majority of developers (in Europe or elsewhere) work for large companies and are not the kind to go to software conferences. Note that this doesn't mean they write CRUD; I know people who do amazing GPGPU stuff that will never see the light of day outside their BigCo's internal use. They are the ones Nvidia should be targeting with their drivers.

I'm yet to meet one such developer who'd use a Mac for work. It's my anecdote vs. yours at this point, though, so I might be wrong about Mac users being a vanishingly small minority in this group (but likely not by a huge margin given how the corporate world is).

Edit: It is also quite possible that not all of the developers who present at conferences with Macs use those at work if their workplace isn't BYOD.

I haven't been to any conferences, but I am employed to write for Windows and Mac OSX. Happily, on a Mac with a Windows VM. In the UK too!

But I think you are correct that the majority of development work is in the MS world for business. At least that is the case if you look for work - everything is .NET

At KTH in Sweden GNU/Linux (Ubuntu) is dominant.

Not the case at all. Travel to Europe and see for yourself. :)

Untrue.

Writing code that goes into proprietary software is like writing a book which can read only your editor. Programming (at least in some areas) is like art, you want the world to see your code, what you've done there. Give me a name of one famous programmer. I bet he is writing free and open source software.

Rich Hickey[0], in addition to making open source software[1], also produces closed source software[2]. Similarly, Joel Spolsky[3] is involved in a fairly high profile[4] closed source project. That's just off the top of my head.

[0] https://github.com/richhickey

[1] http://clojure.org

[2] http://www.datomic.com

[3] http://www.joelonsoftware.com

[4] http://stackoverflow.com

I was under the impression that we admire Joel Spolsky for his management and essay writing skills. Have we seen any of the code he's written directly so we can judge his coding prowess?

Bill Gates, Larry Page, Mark Zuckerberg.

Yes, Larry Page is a good programmer, but I wouldn't say that these people are brilliant coders, they are famous because they are successful entrepreneurs. I was writing about people who are known for being a good programmer. My examples (not in any order): Linus Torvalds, Richard Stallman, Steve Wozniak, Salvatore Sanfilippo (Redis), Jeremy Ashkenas (CoffeeScript, Backbone.js), Daniel J. Bernstein, Alan Cox, Zed Shaw, Igor Sysoev (Nginx), Andrew Tanenbaum, Ken Thompson, Philip Zimmermann...

There are of course many brilliant programmers who are not into open source (like Philip Zimmermann), I don't deny it.

It's easier to be known for being a good programmer if people can read your code, so if you make a list of good programmers you should expect open-source people to be overrepresented.

A few people off the top of my head who are noted for being good programmers but whose code is I think mostly not publicly available: Peter Norvig (though I doubt he's writing much code nowadays), Paul Graham, Sophie Wilson, Charles Simonyi (though it'll be a while before he's forgiven for "Hungarian notation").

(I should add in the interests of fairness that while I was typing that list my brain was angrily trying to remind me of Jamie Zawinski and Don Knuth, both of whom are much more open-source types.)

> It's easier to be known for being a good programmer if people can read your code

Yes, that was my point, I don't say there are no good closed source programmers, just it's much easier to be known/respected as a good programmer by writing free software.

Norvig and Graham both wrote books full of code, with either the book or the code freely available, and AFAIK their books first made their reputations. I'm sure your point is correct that open-source people are overrepresented, those just aren't optimal counterexamples.

Isn't gates considered an excellent coder?

Anyway, John Carmack.

A lot of John Carmack's code is open source. Also, what he used to do at id was not particularly groundbreaking; and the "open sourced" (by talking about novel algorithms for 3D rendering) the groundbreaking stuff at his popular multi-hour-long keynotes. So you can think of Carmack as someone who does open source, albeit in a different way.

I would disagree that Carmack's work at id "was not particularly groundbreaking".

Commander Keen, Wolfenstein, Doom and Quake were all absolutely ground-breaking. Probably ditto for the megatexture stuff in Rage.

Philip Katz of PKZip fame (may he RIP)

Zed Shaw should be on the entrepreneur list.

Think back to Facebook at launch. It was better-looking, more performant and better designed than all its competitors. Zuckerberg isn't just a good businessman.

It's fun to hate on Zuckerberg, because it's Facebook. But the guy produced brilliant stuff that was ahead of it's time.

I can't. I wasn't in college in the US at the time :(. That may have been a bigger factor though...

"famous" != "great". For every famous programmer who cares about recognition and does open-source stuff to get her name out there, there's a great programmer who cares more about getting paid to do awesome things, and isn't particularly concerned with public recognition. There are thousands of astoundingly good programmers who you have never heard of, and many of them are working on closed-source projects.

You, for example, are one of them.

I think that's more to do with closed source software usually being published under the name of a company, rather than the individuals behind it; so there's less opportunity for a closed-source developer to become famous.

John Carmack and Michael Abrash.

This demonstrates complete ignorance of modern DRM and who has worked on it. You could not be more wrong. One word: Tarnovsky

Also, nobody really cares if they work on proprietary code. Such developers are rare.

What I have noticed is that once smart people are exposed to open source programming they have big problems going back to the modus operandi of closed source.

A good middle-ground seems to be doing like github: you keep working on your closed projects using open-source technologies and methodologies.

Huh? Github is not open source. It does not promote open source methodologies or technologies.

No, but it bases it's primary service, and the first half of it's name on an open source project.

Admittedly, it does a lot more than that as well, and I love it for what it is, but it couldn't really exist in it's current form if it weren't for open source.

EDIT: In addition, almost all of the public code on Github is open source, and they have a bunch of open source repositories themselves. https://github.com/github

Gitorious is though, so I always recommend it, even though it doesn't have a lot of the baked in features of github.

GitHub has released a lot of open source code: https://github.com/GitHub

Uh.. what? Github is probably the place to put your code nowadays.

Indeed my point was that Github itself is not open source but uses open source technologies (git and various bots) and methodologies (distributed team, lazy coordination, it ships when it is ready). Have a look at http://zachholman.com/talk/how-github-uses-github-to-build-g... for more information.

I think this open-source-like way of operating makes their great coders happy to work on a closed source project.

BTDT, signed the NDA, and the DRM I know is as good as it realistically can become.

I would say this is either driven by Gabe Newell or Linus Torvalds's middle finger.

Grenades sometimes hit their targets, flowers never miss.

Good thing grenades are on the short list of things that count if they're close!

http://www.brainyquote.com/quotes/quotes/f/frankrobin140160....

I had a teacher who said "horseshoes and hand-grenades." Continuing the "h" sounds, I added hydrogen bombs :)

Flowers fade, but you'll be hard pressed to forget a grenade.

Touché

After Steam announced steambox, and with the massive improvements since Steam for Linux, nVidia would want their drivers to be the best for Linux to sell Linux users on their video cards.

I remember, long ago when the Linux community was much smaller everyone wanted a killer feature so hardware vendors would do something like this. Never thought something like Steam could be such a killer feature.

Gaming is definitely a "killer app" of any platform. Heck, there are devices out there with platforms that only do that, and they are very successful. I'm talking about consoles, obviously.

Microsoft realized this a long time ago, and so did Apple (starting with iOS only). It took Google even more years after that to finally put 2 and 2 together, and realize that the reason why so many still choose iOS over Android is because of gaming.

Same goes for Facebook with Farmville, and so on. So yeah, if you want a very successful platform/OS, definitely focus on gaming as much as you can. It's what brings the "mainstream" in.

> Microsoft realized this a long time ago, and so did Apple (starting with iOS only)

This is a common misconception nowadays, but if you look at the early keynotes Jobs was more dismayed than thrilled by his baby being coopted by games. He resigned himself more than realized it.

It just happens that — on iOS anyway — gaming concerns and Apple concerns align in that both want a good equilibrium between CPU and GPU rather than a CPU-focused device.

Unix was supposedly created to make it easier to develop http://en.wikipedia.org/wiki/Space_Travel_(video_game)

Apple realized that gaming devices could be a killer app. That's why they licensed the Pippin[1] to Bandai[2]. The fact that it didn't go anywhere was mainly due to them not understanding the market.

[1] https://en.wikipedia.org/wiki/Apple_Pippin [2] https://en.wikipedia.org/wiki/Apple_Bandai_Pippin

Hardcore gamers will do ANYTHING for that extra 1 fps.

Linux exposes a lot of options that Windows and OS X will not.

Imagine if these particular gamers switch to linux. They can significantly improve the platform by trying everything they can for that extra performance.

The improvements could have far reaching consequences. Improvements that could possibly help people solving hard problems (HPC). It is not out of the realm of possibility.

gamers grabbing a few extra FPS by switching to linux is nothing new.

That's not the motivation. The motivation is the ease of transition that is finally coming to fruition. Linux used to have larger barriers in place for a layman.

I'm skeptical this is related to steamos/steambox. Nvidia's already working to ensure their proprietary drivers work as well on Linux as Windows and probably considers maintaining one codebase to be far less of a headache.

It's very cool that they're starting to drop docs to support the open drivers, but I suspect it's going to be a long road from these small steps to the open drivers being fully competitive with the closed ones...

Nvidia has been good when it comes to proprietary drivers on Linux, but they've never helped out with open source drivers. Then this happened on the same day as the SteamOS announcement. It's quite a massive coincidence if it is one. Nvidia's refusal to help out with open source drivers has been a controversial issue in the Linux community for at least a decade or so. I'd be very surprised if SteamOS played no role in this.

> they've never helped out with open source drivers

I think that's only true when it comes to 3D drivers. They wrote the 2D nv driver:

http://xorg.freedesktop.org/wiki/nv/

I do remember people complaining it was full of magic numbers to obfuscate what it was doing but it was indeed open source and written by nvidia.

The nv driver exists for the single purpose of getting into a working desktop, so one can install the proprietary driver.

It is not intended as decent 2D-only driver or anything like that.

I don't know think this is a fair assessment; wouldn't having users fall-back on the vesa driver be sufficient for what you are describing? The nv driver supports extensions like xv, so I've always found it was fine for my 2D needs.

That is the reasoning I've heard (read on the ML) from nvidia employees working on said driver.

Yes, I stand corrected on that.

"Nvidia's already working to ensure their proprietary drivers work as well on Linux as Windows and probably considers maintaining one codebase to be far less of a headache."

LOL!

Try getting Optimus working on laptop under Linux. I have a Nvidia video card in my Dell that's basically a paperweight, unless I run Windows.

It's a pain, but at least there's bumblebee. Still gobbles power though...

I wholeheartedly concur. The only piece of kit I had trouble with on a Thinkpad was the Nvidia card, and bumblebee helped a lot here. I haven't really put it to the test though.

I'm in the exact same situation with my Asus laptop.

> After Steam announced steambox, and with the massive improvements since Steam for Linux, nVidia would want their drivers to be the best for Linux to sell Linux users on their video cards.

For a long time, the Nvidia proprietary driver has been the best gpu driver for Linux, not that it has been super good but the competition has been really awful. Only in the recent years there has been any competition if you want a GPU with good 3d features. Intel has had decent 3d only in their latest few generations of integrated GPUs and ATI drivers have ranged from not working at all to barely usable.

That being said, this is a very welcome change and will hopefully make the nouveau drivers better in the long run. In particular, this is good news for the common linux graphics architecture (dri/drm, etc) and Wayland.

Yes

nVidia would be the way to go on professional application on Linux (think 3D Modelling and such)

Also stuff like this http://www.nvidia.co.uk/page/gelato.html

If you want opencl compute performance, though, Intel's haswell parts (especially crystalwell) are basically built for GPGPU, and can deliver huge strides above its price grade. A lot of professional contexts need that compute power, not necessarily opengl performance.

I would be quite surprised if Steam{OS,Box} wouldn't strongly prefer proprietary nVidia drivers, considering the history of co-operation between Valve and nVidia on this front.

After the "Linus flipping off Nvidia" thing, one of the Nvidia people asked what they could do better on a linux mailing list. Matthew Garrett responded, "Move your 3D driver to userspace and use nouveau for command submission." [1] I thought that would be a marked improvement over the status quo; keeping the proprietary stuff out of kernel-space would make things so much easier.

I hadn't seen any possibly-relevant follow-ups until now though. Here's hoping!

[1] http://lists.linux-foundation.org/pipermail/ksummit-2012-dis...

This, again, doesn't really mean anything. Nvidia did the 2D only nv driver for the same reasons - to make their GPUs usable out of the box on Linux. Now instead they are doing documenation that someday will help do the same but using nouveau driver and the community's efforts.

The key areas where the documentation will be needed and have impact - power management and clocking, 3D stuff etc. - Nvidia isn't promising anything on that. If they hired a bunch of OSS developers and promised to release all docs eventually like AMD that would be news.

So no Nvidia - I am still not buying anything you sell.

There have been these baby steps lately, enabling the 2d support on the tegra 2 and now supporting "out of the box" functionality (i guess getting a functioning 2d desktop). I wonder what the relationship is with their stance on 3D support. These aren't really contradicting the proprietary driver monopoly on 3D acceleration that they like to keep.

Could it be they hope to offload the basic driver support to an open source project, especially with the shift towards DRI/DRM, and they plan to move the proprietary 3d support to a blob attached at the mesa layer? I feel like this would be a positive move overall, even if it's not completely ideal.

That's the model we've ended up with with most of the mobile GPU drivers in Android -- open source kernel side managing resources and queueing and proprietary goop in a library in userspace where it can do limited harm (can't crash the kernel, clobber arbitrary memory, etc). Not as nice as a 100% open driver, but a massive improvement over proprietary binary glop in the kernel.

> Not as nice as a 100% open driver, but a massive improvement over proprietary binary glop in the kernel.

FWIW, the most of the Nvidia proprietary driver works in a user space library too. It would be a really bad idea to make your OpenGL implementation and other high-level facilities running in kernel space.

Unfortunately, it's the drivers that are the issue. I have no doubt the open source community could at least maintain nvidia's code for them.

> I feel like this would be a positive move overall

There is one aspect which really could gain from this: the eventual reduction of the harm from the "3/11 rule" fallout.

If and when Nvidia's hardware works acceptably well out of the box on linux systems, there will be less need for annoyed forum postings and endless "help, please" threads. AFAIK Google weighs recently updated pages more than older ones, so the negative PR hits from badly behaving hardware should fizzle and die out in time.

From a purely personal point of view, a unified DRM/KMS access scheme will be nice with Wayland in mind.

This is amazing. Well, not really as amazing anymore from a technical perspective, but the Nouveau-Project, which had a really hard road came into existence because such a move seemed to be impossible, back then.

Other than inertia, and an overrated idea that proprietary drivers give you a "competitive advantage" over your competitors' very different hardware, why aren't more manufacturers making open source firmware for their hardware?

Especially after the NSA scandal, we should really demand open source firmware from hardware manufacturers with every chance we get.

Proprietary drivers and firmware are really two separate issues.

Proprietary drivers usually are the result of licensed code that can't be open sourced or IP concerns (giving away the "secret sauce", patent worries, etc).

Provided it can't be updated behind your back, I don't see proprietary firmware as particularly harmful unto itself -- it'd be nice if it were open and make the device less of a black box, but from the POV of the driver or user, there's not really much difference between a device with an embedded CPU running some firmware and a device implementing the same functionality in the asic.

> Other than inertia, and an overrated idea that proprietary drivers give you a "competitive advantage" over your competitors' very different hardware, why aren't more manufacturers making open source firmware for their hardware?

While this is arguably true for most hardware, GPUs are different from mundane hardware like ethernet adapters. The driver source code will reveal secrets about the hardware architecture, and the architecture changes in every generation. The advantage is not in the software itself, it is not showing the cards you are playing with.

In theory, it would be possible to give out the source code to the video drivers but it would not make sense to give out the source for chips in development. At best, you could get source code dumps after a new chip has rolled out. This is not really open source.

Yes, I want quality open source video drivers but on the other hand I want bleeding edge hardware and competition that drives hardware development.

Disclaimer: I write GPU drivers for living and I'm an open source enthusiast.

I don't see why nVidia can't release a semi open-source driver. For previous generations they can open up everything while the latest generation would come with an pre-compiled blob that can be disabled with a simple compile flag for the purists.

This will allow them to play nice with the kernel, support previous products while taking bug reports and patches, and help game engine designers target 90% of nVidia's user base without signing an NDA.

I'm fine with them releasing the source after the chip is out (why would I need the driver before the chip is out?!) - just not 3 years later, like they're basically doing with Tegra 2 drivers now.

You need the driver before the chip is out so that your favorite distro already supports it when you buy that new GPU card.

Why can't you ship a basic driver with the kernel that is guaranteed to work with all modern video cards, and then have a website download board specific driver when it comes out? (i.e. the current situation on windows anyway)

I just hope nvidia builds wayland support into their proprietary drivers. I mean if nvidia chooses not to support nouveau as their main DRI drivers, thats fine. However the least they can do is make sure they do as much as they can to invest in Linux moving forward and not force users (and app developers) to make painful decisions regarding whether they should use the proprietary drivers or nouveau.

For example if nvidia decides to focus and invest all their effort solely into mir and unity support for their closed drivers, thus giving everybody who backs wayland the shaft in the process, its going to end up being a major problem for developers like myself. Philosophically and pragmatically.

Well, as someone who should understand the value of free software, it is kind of your fault for buying a part from a company who has never supported foss before, and this move is quite a change of pace from them.

If they pull an AMD, except they do it right (ie, just depreciate the proprietary driver for enterprise compute usage and focus entirely on making gallium great) they have my business forever.

I'm thinking of getting a tegra note next month, on the pretense that at least Nvidia provides open source Tegra drivers through a 3rd party so that the platform is hackable. The new Nexus 7 is a disgrace since smart people have to waste their time reverse engineering the gpu in the Snapdragon because Qualcomm is being a dick.

So Nvidia is now doing what AMD is doing with their engagement on freedesktop and kernel recently? Good step but it's not the "breakthrough" like all the news sites are trying to make this up.