In defense of <canvas> | Hacker News Reader

In defense of <canvas>

holovaty.com

294 pointsadrianh61 comments

I've done a large amount of canvas work over the past years, especially work with improving canvas performance (one of the articles on my site is linked-to by this article). What follows is fairly self-promotional, but relevant to the topic if you're interested in canvas performance.

I've been compiling canvas performance tips for two years now, and have a full-color (syntax highlighting and images) book coming out very soon[1] with its own Canvas performance chapter at about ~40 pages. The book has about ~200 pages total of Canvas tips, oddities and tutorials. These aren't fluff, I don't take pages and pages to re-implement stuff like gravity for a game example. I tried to keep it pertinent and pure.

If you're interested in Canvas, please give it a look[1]. The book will be out in June or July.

And if you're interested in getting help for Canvas, come to StackOverflow! Back when I had free time, if you asked a question on SO about canvas there was a +10% chance that I was the one who answered it. I've been too busy to participate in the last year, but I'm just beginning to come into some free time again these days, and there are several other wonderful people who also hound the canvas tag.

Also, if you're looking for live help with Canvas issues, give the StackOverflow JavaScript room a try[2]. They're very friendly if you are, and always love interesting (read: not jQuery) questions.

[1] http://www.amazon.com/dp/0672336278/

[2] http://chat.stackoverflow.com/rooms/17/javascript

Any chance for an ebook edition? I'm quite interested but don't really want to order a paperback edition from overseas.

Will you release an ebook? It seems interesting and I would pay paperback price for an ebook with no DRM.

I got a lot of technical books and they're a PITA to read and store as they are usually pretty big.

EDIT: Went back to the Amazon link and "Paperback: 700 pages", woah.

It's up to the publisher to do that, I know that there will be a kindle edition, but am not certain of any other formats. I imagine all of them will be DRM'd, sorry.

In the near future I will also have a performance chapter in a different book (written quite differently, and expecting more JS knowledge), but that title hasn't been announced yet. That text will have a DRM-free ebook for it, but you'll have to wait a lot longer.

Lastly I'll have learncanvas.com up at some point this summer, where I will begin posting a lot of canvas performance tips with live examples, among other things.

(The page count surprised me too. This is my first book, and it's been quite the overwhelming experience!)

http://www.informit.com/store/html5-unleashed-9780672336270

Informit sells PDF, mobi and epub versions of all Pearson books. They print "From the library of Your Name" as a footer on every page but there's no additional copy protection as far as I know.

Thanks for sharing this link. For e-books I find PDF a superior format, as it maintains the original design and layout of the book best, while still having all the advantages of an e-book, searchable, etc. Appreciate that informit.com has that option.

Ha! Caught you not mentioning your blog posts while I was adding this comment to my "canvas performance" bookmark list and saw a familiar name.

http://simonsarris.com/blog/tag/performance

Because of the book, I haven't been able to blog about canvas performance in a long time! But very soon there will be a return to (canvas and performance) blogging.

For me learning to use canvas was easy. But I had learned graphics programming when I was a kid messing with SCREEN 13 in QBasic. And a lot of the lessons I picked up then, apply to canvas today.

I found canvas is actually a lot faster than I expected. I made a little graphical Roguelike that was drawing hundreds of sprites at 30fps in all modern browsers easily even on my iPhone (some browsers on some systems are able to maintain 60fps or higher.)

There's certainly plenty of reasons to make native apps. But I was pleasantly surprised by canvas performance, personally.

> For me learning to use canvas was easy. But I had learned graphics programming when I was a kid messing with SCREEN 13 in QBasic. And a lot of the lessons I picked up then, apply to canvas today.

Funny that you mention it, the canvas drawing api seems to be inspired by the QBasic-style drawing apis of the 1980s. Unfortunately, computers have moved on since then and these days we have dedicated silicon for drawing and the "immediate mode" way of drawing things became obsolete.

> I found canvas is actually a lot faster than I expected. I made a little graphical Roguelike that was drawing hundreds of sprites at 30fps in all modern browsers easily even on my iPhone (some browsers on some systems are able to maintain 60fps or higher.)

This is not fast. A typical smartphone these days can do thousands of textured 3d models at 60 fps. Hundreds of sprites is next to nothing on modern standards.

It seems like <canvas> was designed for simplicity, to make it really easy to a few draw lines, boxes and ellipses at the cost of performance.

Good memories! I learned to program making silly games on a crappy computer in QBasic :)

>On Soundslice, we're drawing guitar-chord charts completely on the fly (again, see an example), which is a relatively involved drawing routine -- and it's still near-instant performance. That's across all modern browsers (Chrome, Safari, Firefox and IE 10).

OK, let me get this out of the way. Drawing guitar tabs on the fly is not by a long shot an "involved drawing routine".

If that's his standard for canvas performance, then sure, canvas is plenty of fast for that.

(I don't say it's not a quite complex algorithm to position and determine the guitar tabs: I say that it's not that graphically demanding to draw them. It's easier on the canvas/CPU than even the tamest of platform scrollers for example).

I agree. I'm sincerely impressed by the Soundslice application, it's really awesome how the videos move along with the tab to show how every chord is played, and the UI is fast, responsive and easy to use. That said, I'm also inclined to say this particular application simply falls beyond the threshold where UI performance becomes an issue.

Canvas may be pretty well optimized by now, but it's far from perfect. When I was using it a few months ago, I needed to draw into a temporary context then draw the result into the main canvas, because of unusual composite operations. This worked well on most browsers, but Safari incurred a massive slowdown which, according to Instruments, was largely caused by time spent in memcpy and some sort of surface locking. I'm not sure why memcpying relatively small images on a powerful CPU was using so much time, but a more performant implementation would use render-to-texture and not do a round trip through the CPU, and whatever their implementations other browsers had no such problems. It was slow enough - just drawing a few 300x300 or so images at 60fps! - that on x86 it might well have ended up faster to just construct the frame in JS using array buffers; unfortunately, I was also trying to target Safari for iOS.

Considering that Canvas also has hard limits on what sort of effects you can implement with it, I hope WebGL is supported in Safari and IE soon.

Yeah, my experience roughly matches yours. In practice Firefox, IE and Chrome currently have very fast 'fast paths' for Canvas, but if you hit the slow path - some operation that isn't optimized - you can see a 5x performance hit, or a 50x performance hit. And because Canvas's feature set is so limited your only workaround is usually to do it in software. Composite operations are definitely not consistently optimized across browsers.

It's worse on mobile, where for some reason way less effort has gone into hardware accelerated canvas - there I think you may just be stuck using WebGL.

WebGL support on mobile is pretty rare. I think only a few browsers like Firefox mobile and Opera mobile support it.

Chrome as well. I forget if it has to be enabled in a flag / only on beta builds still, but it does work.

Should you be using canvas directly?

I'm currently writing my 3rd HTML based game, and I'm so tired of using jQuery and DOM directly, so I want to replace it with canvas. Game is a turn-based strategy, so performance is not an issue (there will be some animations, but rare).

I looked at the canvas and then at a lot of 2D libraries that can speed the development and prevent me from reinventing the wheel. I'm overwhelmed with options. From some 20+ I investigated I narrowed the list down to these:

  - CanvasEngine
  - Cocos2D
  - MelonJS
  - Quintus
  - EaselJS

I have no idea which one to pick. Any tips?

I recommend using canvas directly unless you specifically need features offered by a wrapper (for example, Cocos2D provides a scene graph of sorts).

Canvas has enough weird performance problems that you don't need a library adding more on top.

OTOH, if the authors of the library know more about the fast rendering paths on various browsers than you do, they can actually give you better performance than "coding to the metal," so to speak.

I've never actually found a canvas library that delivers consistently better performance, but it is possible. It's difficult because many of the canvas tradeoffs depend on your specific use-case so it's hard for a general library to deliver the best performance. (This is something that is particularly bad about Canvas when compared with other rendering APIs; the way it is specified and implemented makes it harder to work around the performance limitations)

In particular many perf workarounds rely on aggressive caching, and unless you know the workload well it is difficult to create a general-purpose cache that will deliver wins. This is made worse by most modern browsers' particularly low resource ceilings (and in the case of IE, an outright limit on how many canvases you can have)

Games get dull without animation

Well, no tips, but I do have a question:

I'm a web dev who's been looking to start trying game development, particularly with canvas. However, I've hit a huge wall with any type of graphics programming. Have you found any particularly good resources while trying to learn canvas, or graphics programming in general? Or anyone else here? If you have something feel free to email it to mrjordangoldstein at gmail.com if it doesn't seem strictly relevant to this thread—and thanks!

Hey, you should visit http://html5gamedevs.com - we are an active community and I'm sure you will find help there :)

If you want a scene graph, have you evaluated fabric.js? It's a popular canvas library, providing mouse events, and import/export to SVG.

The browser already has a scene graph library, it's called the DOM.

Looks interesting. Thanks.

Caveat worth keeping in mind, since you mention SVG, is that Fabric is still rather picky about the SVG it handles. I was playing around with it for some very simple SVG symbols that rendered fine using built in SVG support in both Chrome, Firefox and rendered to PNG via rsvg, and loaded fine in Inkscape, and had to manually tweak the svg quite a bit before Fabric would handle it. (I don't remember the exact problem, but think it was an issue with stacking transformations including "translate"). I'm sure that will improve over time, but it was a bit of a nuisance.

Other than that, it seems to work great.

I recommend checking out my Breakouts project, which is aimed at answering your question: http://city41.github.io/breakouts. It's the same idea as TodoMVC but for js game engines.

Thanks, I've already seen it. In fact, it was one of the factors that made me narrow the choice. Thank you for making it.

Do you have any plans to include CanvasEngine and/or Cocos2D in there as well?

Yeah, I'd like to add pretty much all engines. Currently I'm not dedicated any cycles to the project as we're in crunch mode getting our game done. But after that I plan to head back to it. First engines on my list are the various cocos2d variants.

For my side/tinker/forever project, I've been using Three.js even though it's actually 2D (for now). Since you can set it to render orthographically, it might be useful (each sprite as a two-tri polygon).

Completely new to game development though, so this may be really really bad advice.

The major problem with the <canvas> vs WebGL is excessive amounts of draw calls. You can't create ship a display list/vertex buffer, and so there's a ton of JS->Native Browser binding calls. They've recently changed this with CanvasRenderingContext2D version 2(adding Path objects). None-the-less, right now, it's much faster to use WebGL and layer over a 2d canvas for text or curves.

I really like this Soundslice canvas app. Why didn't Soundslice use Scalable Vector Graphics? CSS Animations would replace a whole heap of code: the entire requestAnimationFrame callback, for example, would be eliminated. Text rendering would be handled by the browser, instead of taking the performance hit of being rendered to the canvas bitmap every frame, so text rendering caching would not be necessary. Element stacking would be handled by the browser's built-in compositor. You wouldn't have to think about crazy microoptimizations like grouping fillStyles together. SVG can easily be accelerated by the browser, since you're describing intent instead of specific procedures, unlike Canvas' immediate mode drawing. Finally, vector graphics aren't locked to 72dpi, so they look amazing on a MacBook Pro Retina or other device with high DPI (unlike the canvas implementation, which looks pretty chunky!)

Hey -- I didn't spend much time exploring SVG, to be honest. It could very well be faster!

And you're right that it would be much better on retina screens. I've had it on my to-do list to make the canvas implementation take the device pixel ratio into account, but it hasn't been a huge priority.

Here's a little demo I whipped up with just the play head (hover-glow, dragging, and animation).

http://github.johntantalo.com/soundslice-svg/

I don't know the answer here, but it's a solid trend. SVG just seems to get completely neglected. Sure if you're animating things and you're doing so in a world before CSS3 transitions, manipulating hundreds of attributes at 60fps in the DOM won't perform, but we're past that.

These seem like the same sorts of guidelines you want to follow when writing native apps too. It seems fair to treat a <canvas> element like a CALayer.

There's actually a pretty deep connection there. In Firefox, a canvas typically becomes a separate CanvasLayer in the layers stack. The layers implementation in Firefox is very similar to Core Animation—they're composited together using OpenGL (or Direct3D on Windows), animations can happen on the layer tree, scrolling is performed on the layer tree, and so on.

So, all these tip's I've seen focus on improving canvas performance by somehow splitting the drawing into small pieces .. but what if one is working on a drawing app that absolutely needs to redraw a large area most of the time, think sizes like 800x600 or 1024x768.

Any good tips to optimize this kind of use-case ? I am developing an online Geometry Generator (http://GeoKone.NET) that uses a large canvas that is redrawing complex geometry, with paths that almost always change completely when being modified, thus invalidating the "render to image" option almost in all cases, except maybe when moving the formations around.

Good tips though and nice to read about people's experience. Too bad I am using Processing.js so I am kinda stuck with the optimizations Processing.js is using.

A few years ago, my friend Austin Sarner and I were writing a user friendly content publishing system in Objective-J. We designed a majority of the controls and UI components using canvas and found it incredibly fast when the proper caching techniques were used.

One of the components of the application I worked on was a text editor that resembled Apple's Pages software. The text editor was implemented entirely in canvas and would wrap live in response objects intersecting the text. Redrawing the text live was slow but we ended up finding a solution. We cached bitmaps of the characters, lines and paragraphs and only re-rendered areas of the canvas that changed.

The code was open sourced and put on GitHub, if anyone is interested:

https://github.com/austinsarner/Frappuccino/tree/master/Frap...

P.S. There's also a really fast iPhoto-style media browser in the repository that is implemented in canvas. It uses image caching techniques for animation.

I too have developed a pretty "serious" Canvas application that people actually use on a large scale, http://www.luckyvoice.com/sing

While I agree with your suggestions (caching is absolutely a must, as is batched rendering etc), but I think the real issue is the lack of good-quality canvas libraries that have the kind of breadth of functionality that their native counterparts have have. All of these "performance hacks" are certainly not unique to canvas, but what is is unique is the developer (user of these APIs) having to care about them. They should be abstracted away.

Libraries like Paper.js (which I've used and extended a lot) and EaselJS are bridging the gap, but they aren't there yet — either because they aren't feature-complete in the way that they need to be, they're buggy, and/or they don't seem to be getting enough community development (or in some cases, want it).

Canvas may need a lot of sweat to get some performance [1] but there is something it does really well. What strikes me is the lovely API. It just great to use it for charts, procedural UIs, etc... and it also somewhat work with games.

[1] http://www.gamesfrommars.fr/demojsv2

Really? I thought the API was pretty silly compared to for example Flash Drawing API. Why doesn't it have circle(x,y,radius) for example?

because that's better suited for a library?

If you are doing a lot of canvas drawing, you should really check out WebCode (http://www.webcodeapp.com/). It's a drawing app that automatically writes the canvas JavaScript code for you.

I've found canvas to be more than powerful enough for my 2d game http://www.towerstorm.com even with hundreds of animated minions and thousand of bullets moving across the screen. I'm using the impactjs engine which has made things easier as it has a few of these tweaks built in.

Multiple canvas layers is an awesome idea, as things such as the game background never change so it's a waste re-rendering it every frame. will try it out soon and see how it goes.

re: canvas stacking, check out http://kineticjs.com/

Funny how few canvas libs use stacking of interaction layers. Obviously the slowest part is redrawing quickly for a drag/hover operation. Only one charting example that I know of (CytoscapeWeb) drops the interaction elements to an otherwise empty canvas for the interaction to minimize the redraws needed, and then brings them back when done.

Most just seem to render everything, all the time, in the same canvas element and pray that browsers keep getting more optimized. Funny.

There are problems with stacking multiple canvas elements, of course - increased memory usage and significantly increased composition overhead to name two.

the soundslice example might be a bit flawed. It's fast for a canvas app but not even close to "native magic" on my pc using chrome.

I feel like some sort of html luddite, but am I the only one who preferred flash/actionscript3 to canvas/javascript? I get that the web should be open, but beyond the idealism the flash drawing tools were much better. I feel that the criticisms currently levied against flash will come up again with html/js/canvas/svg as apps get more complex.

Hey, thanks for the feedback! What were you doing with Soundslice when it felt slow -- were you zoomed out (such that there were a ton of annotations visible on the page)? Or was it just slow in general, even when it was zoomed in?

it's quite fast actually, I'm very impressed. I think what I was trying to say is that it doesn't feel native at times, but it's difficult to articulate why. Maybe I'm just conditioned to use webapps differently, and pick up on cues like slightly lagging cursors and such.

Ahh, cool -- got it. Thanks!

My guess is that most people who hit a performance wall with canvas are using overly complex vector graphics and not caching bitmaps as often as they could. While things like OpenVG are on the horizon, vector drawing is still a CPU-bound activity most of the time, while bitmap blitting is all GPU. This is the case whether you're using canvas or a native graphics library.

Well, I loaded the Super Mario Bros Soundslice, set it to scrolling, hit play, then put on my ear defenders as the MBP tried to commit fan-suicide. Maybe it's just a Firefoxism but normally only Flash makes the fans go that hard...

The only thing slow about Canvas I've found is when the canvas area needs to be resized : i.e. canvas.height or canvas.width changes. Drawing itself hasnt been an issue for me.

Try changing fonts often on Firefox. That's ludicrously slow. To the point where anyone doing lots of text on a canvas that might need to support Firefox will need to take special measures to get decent performance (caching, lots of caching, and ordering updates to minimize font changes)

canvas is incredibly fast. I think its 100% capable for anythign except 3d games.

You do realize that you can get a 2D and a 3D context from a canvas element, right?

yes, but WebGL seems to be a better solution for 3d. It actually renders into a canvas element, but its does enjoy the same cross-browser support that canvas currently has.

Reply on news.ycombinator.com

In defense of &lt;canvas&gt;

In defense of <canvas>