Hacker News Reader

In its early days, OpenGL benefited in many ways from being a "forward-looking" API that exposed far more functionality than was implemented by graphics hardware. Software that used those features got faster throughout the 90s with no rewriting needed as that functionality became available on GPUs. (Of course, software like games that wanted to be fast now just avoided those features.) When graphics hardware started to include features that hadn't been anticipated in OpenGL, things got messy and stayed messy.

Now that the feature set of graphics hardware is very stable and uniform across different vendors' hardware, the APIs are needed to solve an entirely different problem from what they were invented to do.

The best TL;DR ever, thank you for that.

About your last paragraph, I would say that that is even more important to optimize and to make low level layers for than graphical game engines (believe it or not :)).

The thing is while gaming is seen as a "performance critical" application, where it is natural to want the maximum performance to be squeezed out of your hardware, Web development is just a behemoth of inefficiency, and we think that that is the norm, an inevitable drawback of developing for the web, an inescapable trait of web applications. Web apps are dozens or even hundreds of times slower than native applications, and we think nothing of it, because it seems to be an inherent characteristic of the web about which we can't do anything.

Your comment made me reconsider that. Maybe it doesn't have to be so. What if there was a Vulkan for the web, instead of the layers upon layers of inefficiency which are all the rage these days? What if I could run a medium complexity web page or app without it bogging my computer several times harder than a comparable native app?

Write your whole webapp in WebGL?

Edit: Seriously, how feasible is that? Creating a GUI framework on top of WebGL and creating an abstraction layer (which the original article advises against) would be a huge performance boost comparing to current webpages, wouldn't it?

It's definitely feasible! A goal for my side project is to give developers a React-like declarative UI with a WebGL rendering backend that abstracts away the graphics programming.

Except that it would all still be done via a layer of javascript (using asm.js may mitigate this)

Native JavaScript performance is not generally what makes web apps slow.

Just because it isn't the bottleneck today doesn't mean it isn't slow.

Someone did this with their WebGL Terrain demo [0]. You can find a detailed post-mortem in the author's blog [1].

[0] http://zephyrosanemos.com/windstorm/current/live-demo.html

[1] http://zephyrosanemos.com/

To be fair, I think the takeaway isn't don't make/use abstraction layers, but don't make a one-size-fits-all abstraction and then hide the lower level stuff. A targeted abstraction that correctly supports the domain you are in is still a good thing. That's what the engine developers end up being here, an abstraction on top of this low level API that gives many devs exactly what they want.

Edit: To be clear, I don't think writing a web rendering engine meant to do the same think as the browser on top of WebGL will be faster in any respect, as really the HTML and CSS are just inputs fed into super optimized low level rendering engines. I don't see how JS could compete in that domain.

Reminds me of MIT's Exokernel- make the lower level interfaces as low-level as possible (while still providing security) and let user-level libraries provide various abstractions suited to different use cases.

No file systems, no network protocols, no processes; just a very thin layer that lets userspace implement those if/how it wants to.

Parts of CSS3 are already faster to implement "by hand" in javascript rather than using the browser's "native" implementation.

HTML/CSS was designed for requirements that don't apply to many modern websites. I would not be surprised if it were possible to implement a rendering engine that performed better for relevant use cases, e.g. perhaps assuming a grid as a first-class part of the layout.

Take a look at what Flipboard did recently, with their whole mobile site built in JS and canvas

http://engineering.flipboard.com/2015/02/mobile-web/

WebGL should never have been invented, at least on current architectures. It's a ticking bomb, precisely for the reasons this articles explored - drivers are complex and buggy black boxes that lie in kernel space so ring 0 access is never too far from reach for many exploits. WebGL has a potential to be exploit vector much more severe than what we've seen with other auxiliary web technologies like JavaScript or Java applets. Why not confine web to (mostly) nicely behaved x86 VMs?

Personally I think browsers will become service layers that expose the underlying OS in controlled and specific ways. Not a completely sandbox, but not a free for all either.

We've started creeping that way with a lot of the newer HTML5 stuff, but we have a long way to go.

Looking at the bug bounty record from Chrome and Mozilla it seems that fuzzing-for-bounties vulnerability research has largely moved away from WebGL. Must be at least partly due to WebGL reaching a reasonable level of robustness. Bounties may be harder to claim since bugs can be hardware/os/driver ver etc dependent, but still there used to be a lot more WebGL bounties dealt out.

Wouldn't that kill accessibility, selectable text, being able to call system services (i.e., on OS X hitting cmd+shift+D to get a dictionary definition of a word), any extensions which modify the page hierarchy, automatic translation/localisation, and so on.

Exactly; bypass this entire service layer maintained by thousands of engineers and try to do it in Javascript?! I think that's probably not a good idea.

Isn't Google's Native Client the Vulkan of the web? Or rather wasn't it, given adoption outside of Chrome is unlikely to ever happen.

http://www.zerovm.org/

Hi, I'm working on a library to allow something like this (rendering web UI's with a WebGL backend).

It's in very early stages. Feedback is very welcome:

https://github.com/davedx/lustro

Good luck. Efficient 2D using OpenGL is very difficult. It will never approach the speed of a well designed 2D API. Microsoft gave a lot of insight about the hard it was to get Direct2D to be competitive with GDI/GDI+.

If you do see any performance benefits it likely will be because you support less features than what a modern day browser supports with HTML/CSS. But in my opinion you could probably achieve faster speeds with Canvas in a similar manner.

> Efficient 2D using OpenGL is very difficult.

It depends on what you are doing. Rendering thousands of sprites with independent rotation and scale is easy. Rendering text is a lot of work, but once you do it you can do all sorts of things with the text. Rendering simple things like bezier curves is very difficult. There are some things in the 2D world that HTML/CSS/GDI/GDI+ are very bad at, compared to OpenGL, and vice versa.

Efficient 2D using OpenGL is very difficult. It will never approach the speed of a well designed 2D API.

All web content on OS X and iOS is rendered using OpenGL. That's not to say it was easy to make efficient, but it definitely outperforms the default CoreGraphics implementation on most content.

I agree it will be very difficult. I'm working with WebGL as my day gig and understand how hard it is, as we're working with embedded devices there. That's actually what motivated me to start this project, so web developers can build fast, rich and interactive UI's on the same level as those you see in current generation video game consoles.

You have to be ambitious, right? :)

(How) do you intend to handle accessibility?

I don't. The target segment is people building rich, interactive video-game-like UI's, for things like mobile/embedded devices.

Though as an aside, I think voice control is very underutilized in modern GUI development...

Please don't. It sounds like you're recreating flash and we all know how annoying flash guis were.

Sounds like the guy is doing entertainment systems where a gamey UI is fine. You don't want your car entertainment system looking like bootstrap.

I hope it works out well. A whole new class of apps and performance might become real.

I just wonder how well text performance will be. Would be awesome to get near native speed with editors like Brackets and Atom.

Yes, text is a big challenge. Got to get maximum efficiency from your sprite batching and triangle strips!

Maybe you're optimizing for the wrong thing?

Let's imagine a world where a browser is just a secure place to execute code. You get vulkan, io, networking, input, camera access in a cross platform way.

So, how do search engines find any info? How do you share links to content? How do you support braille readers? Browser extensions?

I'd argue those things are pretty essential to the web as we know it (well maybe not the extensions) and I might also argue that some of the inefficiencies of the web environment are a direct result of making the decisions that the most important part of the web is a common semi scannable and linkable format, HTML.

The web these days is very clearly diverging into two separate things: a portable application runtime/platform, and the document publishing system it started as. I don't see any way to accomplish a clean split between the two, but it looks increasingly necessary.

The split you speak of has already happened on mobile, that's why we have "apps."

The new medium has been born, but the old one is still being abused, so the split isn't complete and won't be until this no longer happens: http://xkcd.com/1174/

The worst is LinkedIn, which will send you an e-mail with article links it it, and if you click the links it takes you to your browser, which does the thing that xkcd comic describes if you don't have the app installed, but if you do have the app it just launches the app and leaves you at the home screen, and the context of the link you clicked on is gone.

I'm wondering about the "can't zoom" bit. I can't zoom on many websites - because Mobile Safari has sold out control to the web developers, instead of acting as my user agent.

Disabled zoom is one of my pet peeves too. I use this bookmarklet on my iPhone, you may find it useful...

  javascript:document.querySelector('meta%5Bname=viewport%5D').setAttribute('content','width=device-width,initial-scale=1.0,maximum-scale=10.0,user-scalable=1');

Thanks!

> So, how do search engines find any info?

Desktop integrated search.

> How do you share links to content?

Extensions, contracts and intents.

> How do you support braille readers?

OS Accessibility APIs

> Browser extensions?

Plugins

yup. i have started to see the same as well. having a simpler web solution is hard to achieve, over the time, it get messier and messier. i'm now doing small and simple examples in newer language such as go to see if it is any easier there to keep software simple to 1)run 2)maintain 3) code.

> I wonder if/when web developers will reach a similar point of software layer implosion.

Will doing things the "good but hard" way force me to write tons of boilerplate? Because as a (currently iOS) programmer, as soon as I start having to write generic, structural code, I find it difficult to sustain my interest in the project. I want to go from idea to something that works as quickly as humanly possible; having to remake the universe from scratch just because somebody somewhere needed the performance is not what I got into the game for.

(I'm currently working with OpenGL for the first time, and, ugh, all the state management stuff and shader boilerplate is driving me crazy — especially after years of the relative ease of dealing with UIKit! Am I to understand that Vulkan will only make things worse in this regard? I guess it's good for engine devs, but I would not want to code like this in my day-to-day.)

Apparently, a Vulkan "Hello, world!" program takes about 600 lines of code!

https://youtu.be/EUNMrU8uU5M?t=1h33m54s

600 lines of code really isn't that much. At my day job a single file can be 2-6x that long, and there are a lot of files to deal with. Any project that gets beyond "Hey here's a neat little snippet of code that does something" will very quickly begin to dwarf those 600 lines. Also, from the original post talking about the new API and current graphics drivers, every developer is already implicitly using a couple million lines of code by using the OpenGL/DirectX APIs. They're just hidden from the developer in a closed-source binary.

"This begs the question: what about DirectX 12 and Apple's Metal? If they didn't have similar engine developer involvement, that's clearly a point against DX12/Metal support in the long run."

There's several parts to this. DirectX 12 is going to run on Windows 10, and its adoption by developers is probably going to be entirely driven by how much of their target audience can be persuaded to upgrade from Windows 7. If Windows ships with better DirectX 12 drivers than Vulkan drivers (and I would bet that it will), then you'll probably see more AAA games written to DirectX12. Metal, likewise, is the preferred graphics API for reaching iPhone users, which are plentiful and (compared to Android users) lucrative.

The Xbox One is being updated to support DX12 as well, so the major engines will adopt it regardless of whether Windows 10 gets any traction - in fact, Unity and UE4 already have.

We don't know yet whether Apple will adopt Vulkan or not. Vulkan is not just a direct competitor to Metal, but also an evolution of OpenGL ES, and so far Apple has adopted OpenGL ES.

Vulkan is purposely not an evolution of OpenGL ES. It's a clean break from that API and shares nothing in common.

> Many years ago, I briefly worked at NVIDIA on the DirectX driver team (internship).

I've been a full-time GPU engineer at a couple of major companies for about 6 years now. My colleagues at my current company have all been GPU engineers (and video, prior to that) in the 20--30 year range.

While a lot of the technical details are correct, there are a lot of red-flags in this guy's essay. For instance:

> Although AMD and NV have the resources to do it, the smaller IHVs (Intel, PowerVR, Qualcomm, etc)

Intel owns, what? 60? 70 percent of the desktop market? More? PVR, Qualcomm, etc. own 90% of the mobile market? Intel has an enormous number of GPU engineers---far more than AMD, and easily comparable to (or more than) NV.

Now, to OP's statement about DX12 & Metal: obviously they were developed cojointly with AAA game developers. The major title developers are in constant contact with every major IHV and OS vendor (less so for open-source) at all stages with respect to driver- and API- development.

Furthermore, it's not like the major architects and engineers of these driver teams are total tools; these guys live and breathe GPU architecture (and uarch), and are intimately familiar with what would make a good driver.

The impetus for these low level APIs is differentiation and performance. When those couldn't be had from GPU HW, the next place to look is "one up the stack": driver & API. It surprised no one in the industry that Mantle/Vulkan/Metal/DX12 all came out at about the same time: we've all been pushing for this for years.

Thanks for this. It makes sense that DX12 was developed alongside engine devs. As jsheard notes, Unity and Unreal Engine have already announced support for it, and we're already seeing tech demos of Unreal Engine on DX12:

https://www.youtube.com/watch?v=FIk8Q8luWsI

I wouldn't call it a software layer implosion -- most people are going to continue to use abstractions on top of these API's for game development. Unity is to Rails, etc ...

Regarding your last paragraph, the Extensible Web Manifesto was created to encourage new APIs to expose low-level capabilities that can be extended and smoothed-over by libraries, reducing the surface area of privileged browser code and giving web developers more control.

Whether or not the Extensible Web Manifesto is successful or not remains to be seen.

Both are valuable. Plenty of people don't need the full power & debuggability of a difficult high-performance API- but plenty of people do, too.

Much like the simultaneous ubiquity of C/C++ and Python, in their respective spaces.

I wonder if we'll see opengl-old implemented on top of vulkan. Actually one thing I would love to see is vulkan on linux accessible from userspace without any involvement of a window server, so that you can write the rest of the APIs on top of it. Maybe even have graphical console running on it.

There's already work to do this sort of thing on top of GL... (eg: glamour the X driver that use GL for all it's drawing and works on anything mesa and the kernel support)

I don't think anything will stop someone from being able to run vulkan directly on to of libdrm like you can right now with GL. At least for open source drivers like Intel's.

totally unrelated, this has been sort of my experience switching to golang as well.

That last quote really resonates with my experience of software dev in general. "easy" API's generally do it by making a ton of assumptions. Modifying those assumptions becomes a huge pain point and that's if the API even allows you to modify that particular assumption.

Exactly. This is why I like Rich Hickey's Simple Made Easy [1] so much. Basically with easy constructs it becomes harder to build simple systems, even though the simple constructs are harder to learn.

[1]: http://www.infoq.com/presentations/Simple-Made-Easy

yep, I love that talk and I find myself pointing people towards it all the time :)

I love this talk, and Rich Hickey's talks in general, but I think this goes beyond that.

At one point you want full control of the HW, much like you did with game consoles..

On the other you want security: This model must work in a sandboxed (os, process, vm, threads, sharing etc.) environment, along with security checks (oldest one that I remember was making sure vertex index buffers given to the driver/api must not reference invalid memory, something you would make sure is not the case for a console game through tests, but something that the driver/os/etc. must enforce and stop in a non-console game world - PC/OSX/Linux/etc.)

From little I've read on this API, it seems like security is in the hands of the developer, and there doesn't seem to be much OS protection, so most likely I'm missing something... but whatever protection is to be added, definitely would've not been needed in the console world.

Just a rant, I'm not a graphics programmer so it's easy to rant on topics you just scratched the surface...

----

(Not sure why I can't reply to jeremiep below), but thanks for the insight. I was only familiar with one I posted above (and that was back in 1999, back then if my memory serves me well, drawing primitives on Windows NT was slower than 95, because NT had to check all index buffers whether they were not referencing out-of-bounds, while nothing like this was on 98).

Security is actually much easier to implement on the GPU than on the CPU. For the simple reason that GPU code has to be pure in order to get this degree of parallelism. A shader is nothing more than a transform applied to inputs (attributes, uniforms and varyings) in order to give outputs (colors, depth, stencil).

Invalid data would simply cause a GPU task to fail while the other tasks happily continue to be executed. Since they are pure and don't interact with one another there is no need for process isolation or virtualization.

Basically, its easy to sandbox a GPU when the only data it contains are values (no pointers) and pure functions (no shared memory). Even with the simplified model the driver still everything it needs to enforce security.

You are describing GPU from 1990s. Modern GPU is essentially a general purpose computer sitting on the PCIe bus and able to do anything the CPU can. It does not have to run pure functions (e.g. see how it can be used for normal graphics tasks in [1]) and can write any location in the memory it can see. Securing it is as easy/hard as securing a CPU: if you screw up and expose some memory to the GPU it can be owned just like the memory exposed to a CPU task[2].

1. https://software.intel.com/en-us/blogs/2013/07/18/order-inde...

2. http://beta.ivc.no/wiki/index.php/Xbox_360_King_Kong_Shader_...

GPUs these days have MMUs and have address spaces allocated per context. It's implemented internally to the driver though so you don't see it. And it's normally mapped differently, but the point of AMD's HSA stuff is to make the CPU's and GPU's MMU match up.

(To anwser the lack of a reply button:)

This is just hn adding a delay until the reply link appears related to how deeply nested the comment is. The deeper the longer the delay. It's a simple but effective way to prevent flame wars and the likes.

Gah. At our company a bunch of early Rails apps metastasized into financial/billing modules that do DB access using ActiveRecord. With all the convenient pluralization and automatic field/method naming. And are now on the critical path for our revenue. It's a daily reminder to me of how the "easy API" thing can go wrong :-)

> "easy" API's generally do it by making a ton of assumptions.

Well, perhaps a better way of putting it is that "easy" APIs try to model a domain. Sometimes the API designer just nails it, and folks forget eventually take it for granted. The problem is now "solved". For example, the Unix userspace filesystem API (open, close, read, write, etc.) is pretty damn solid for what it is. I admit that I took it for granted until I worked at a place where someone who'd never seen a filesystem API (!!) was tasked with creating one from the ground up. You can guess how that went.

The downside case is where the API's model either never cut it in the first place, or as @wtallis illuminates above with OpenGL, where the model just ages poorly[1].

[1] https://news.ycombinator.com/item?id=9194138

No, it's about the assumptions and it's a mistake to move away from that mindset.

The Unix filesystem api isn't complete, if it were ioctl wouldn't exist. Assumptions all around, and ioctl is the way you adjust those assumptions.

>I wonder if/when web developers will reach a similar point of software layer implosion. "Easy to code against, but a bitch and a half to debug or tune"

for me it was 3 years ago in web. And 6 years ago in Enterprise java.

No idea why you were downvoted. Maybe people percieved it as an implication of "I knew better 3 years ago", but it's not that.

It's hard to imagine someone working with Enterprise Java and thinking "This isn't helping. This isn't making things more reliable".

for the record. I meant "with Enterprise Java and not thinking..."

Personally, Angular broke me. We built an app that was just a table view basically, and it refreshed the data every minute or whatever. For users whose table got to be 100 or more rows, the performance was awful, and there was nothing we could do about it besides rewrite the whole thing a different way.

I still love JS, and coding for the web. But I don't use frameworks now.

So, this suits big engine and AAA studios really well, right, because they can spare that time.

Indies are going to increasingly look at the web and other interfaces because of the easy layers.

This might hold true if you're only looking at the actual design/code side of the lifecycle, but it sounds like a key point of the new APIs is to make new projects more testable, debuggable and optimizable. Given most project spend inordinate amounts of time in the those three activities (and it sounds like games moreso), this should be a win for everyone.

And even if it isn't, the video game industry just got revolutionized by most of the major engines becoming free, so indie devs don't have to drop down to a lower level of abstraction in order to have a chance at competitive performance.

Can we please stop calling Unity/Unreal "free"? Unity still sells a Pro version, and Unreal takes a cut of revenue after a certain amount of $ earned. I'm not arguing against either model, and it's GREAT for hobbyists, but these engines still cost professional studios money.

The salient point is the barrier to entry. The fact that they still have some form of revenue related to the engines doesn't elevate the up-front cost any and the royalties (where they exist) are structured in a way that cannot prevent them from being a viable option for indie studios. The engines will be making the benefits of new graphics APIs properly accessible to indie studios in all the ways that matter; it won't be a AAA-exclusive thing.

(And professional studios don't have to license the ancillary tools and services and source code access of the Pro editions in order to ship a game.)

With all the free engines available these days, the situation for indies has never really been better.

It is already integrated into Source 2, Unity started (they also have Metal support) and Epic also going that way.

It will make it to indie engines that aren't companies funded as well eventually.

This actually gives a ton of power to current engine developers in the market so getting them involved was a big win.

I am a huge fan of OpenGL ES / WebGL (also based on OES) and was happy that we finally had mobile win that battle but with new graphics rendering layers like Metal and optimization for mobile more needs to be done. The driver mess is a big problem and limits innovation as well.