WebKit switching to Skia for 2d graphics rendering
blogs.igalia.com
blogs.igalia.com
I wish there was a nice and small vector graphics library with GPU acceleration. So far Skia is the only real option, despite its downsides.
I found this:
https://github.com/google/skia-buildbot
Assuming that the Skia maintainers keep that working, it might be easier to build the buildbot and use that to build Skia, than to build Skia directly!
Skia really isn't that hard to use IMO even for GPU accelerated stuff.
o_O unlike GN?
Have you used it much?
I've been using it for years and have no complaints.
There's also the question of "which parts of Skia". If there are five different conceivable ways to implement something in vector graphics, Skia will implement all five, and there will be some sort of hidden obscure configuration setting that Chrome and Android will use to determine which one actually gets used. It's a very unfriendly piece of software to use, honestly.
I'm personally skeptical about GPU acceleration being the answer to 2D rendering for various reasons.
I'm looking forward to Blend2D < https://blend2d.com/ > which is JIT based maturing and being the preferred solution.
You can see some existing benchmarks here:
- https://blend2d.com/performance.html
That's too bad. Is their a successor planned or is Skia the recommended alternative?
If you consider libraries that offer CPU rendering there are basically:
- AGG (CPU only)
- Blend2D (CPU only, GPU planned, but not now)
- Cairo (CPU only)
- Qt's QPainter (CPU only, GPU without anti-aliasing / deprecated)
- Skia (CPU + GPU)
- Tiny Skia (CPU only, not focused on performance)
- GPU only libs (there is many in C++ and Rust)
As an author of Blend2D I hope that it will be a go-to replacement for both AGG and Cairo users. Architecturally, Blend2D should be fine after a 1.0 release as the plan is to offer a stable ABI with 1.0 - And since Blend2D only exports C-API it should be a great choice for users who want to use every cycle and who want their code to work instead of making changes every time the dependency is updated (hello Skia).
At the moment Blend2D focuses on AGG users though, because AGG is much more widespread in commercial applications due to its licensing model and extensibility. However, AGG is really slow especially when rendering to large images (like 4K) so switching from AGG to Blend2D can offer a great performance benefits while avoiding other architectural changes of the application itself.
BTW Blend2D is still under active development. It started as an experiment and historically it only offered great performance on X86 platforms, but that is changing with a new JIT backend, which provides both X86 and AArch64 support and is almost ready for merge. This is good news as it will enable great performance on Apple hardware and also other AArch64 devices, basically covering 99% of the market.
- Canvas Ity (https://github.com/a-e-k/canvas_ity)
It's a tiny single-header C++ library in the style of the STB libraries. My aim was to make it dirt simple to be able to drop into almost any project and get high-quality rendering while providing an API comfortable to those used to <canvas>.
I've been checking out Blend2D every now and then. It seems like a very nice option for the bigger, but faster and more fully-featured end of the spectrum.
(Though for what it's worth, while raw performance isn't my priority, my little library still can hit about 70fps rendering the Postscript Tiger to 733x757 res with a single thread on my 7950x. :-)
BTW for comparison - Blend2D can render SVG tiger in 1.68ms on the same machine (I also have 7950X) so it can provide almost an order of magnitude better performance in this case, which is great I think. But I understand the purpose of your library, sometimes it's nice to have something small :)
NanoVG provided Canvas.Context kind of API in plain C.
AmanithVG is the library on which our SVG renderer: https://www.amanithsvg.com is based. All closed source as now, but things may change in future.
I will do some benchmarks of the current (and next, when the new GPU backend will be ready) version of our libraries against other libraries. Do you know if there are any standard tests (besides the classic post script Tiger)? Maybe we can all agree on a common test set for all vector graphics libs bechmarks?
Regarding benchmarks - I think Tiger is not enough. Tiger is a great benchmark to exercise the rasterizer and stroker, but it doesn't provide enough metrics about anything else. Tt's very important how fast a 2D renderer renders small geometries, be it rectangles or paths. Because when you look at screen most stuff is actually small. That's the main reason why Blend2D benchmarking tool scales the size of geometries from 8x8 to 256x256 pixels to make sure small geometries are rendered fast and covered by benchmarks. When you explore the results you will notice how inefficient other libraries actually are when it comes to this.
https://github.com/thorvg/thorvg#dependencies
AFAIK they have experimental GPU backend but I'm not sure how far they are with it.
I wonder what he’s up to these days?
Update/edit:
Ahh he moved on to Ampere: https://www.linkedin.com/in/carl-worth
Also I was a bad fan: the library had a co-founder too I thought it was a bespoke creation of Carl’s own making.
I remember building a bunch of stuff from source back in the day and a lot of Linux applications had Cairo as a dependency.
> There was an attempt at making Cairo support GPU rendering, which did not work particularly well due to the library being designed around stateful operation based upon the PostScript model—resulting in a convenient and familiar API, great output quality, but hard to retarget and with some particularly slow corner cases. Meanwhile, other web engines have moved more work to the GPU, including 2D rendering, where many operations are considerably faster.
I believe the best way is probably to use Blend2D for rendering glyph bitmaps and then compositing them into the full text on GPU.
Sadly, CPU memory is still plenty slow compared to GPU memory and when you need to copy around 100 MB images (4K RGB float), then that quickly becomes the limiting factor.
At the moment when you render text Blend2D queries each character from the font and then rasterizes all the edges and runs a pipeline to composite them. All these steps are super optimized (there is even a SIMD accelerated TrueType decoder, which I have successfully ported to AArch64 recently), so when you compare this approach against other libraries you still get like 4-5x performance difference in favor of Blend2D, but if you compare this method against cached glyphs Blend2D loses as it has to do much more work per glyph.
So the plan is to use the existing pipeline for glyphs that are larger (let's say 30px+ vertically) and to use caching for glyphs that are smaller, but how it's gonna be cached is currently in research as I don't consider simple glyph caching in a mask a great solution (it cannot be sub-pixel positioned and it cannot be rotated - and if you want that subpixel positioned the cache would have to store each glyph several times).
There is a demo application in blend2d-apps repository that can be used to compare Blend2D text rendering vs Qt, and the caching Qt does is clearly visible in this demo - when the text is smaller Qt renders it differently and characters can "jump" from one pixel to another when the font size is slightly scaled up and down, so Qt glyph caching has its limits and it's not nice when you render animated text, for example. This is a property that I consider very important so that's why I want to design something better than glyph masks that would be simple to calculate on CPU. One additional interesting property of Qt glyph caching is that once you want to render text having a size that was not cached previously, something in Qt takes 5ms to setup, which is insane...
BTW one nice property of Blend2D text rendering is that when you use the multithreaded rendering context the whole text pipeline would run multithreaded as well (all the outline decoding, GSUB/GPOS processing, rasterization, etc...).
Maybe once a year I bite the bullet, do a new Skia build on all the platforms, and then I have to figure out how the C++ API has changed. At least that’s just rote work of fixing compiler errors by looking at the new header files.
Even though it’s a pain in the ass, I still use Skia because it’s got the best combination of performance and features. Sadly Cairo doesn’t quite compete. Skia gives my project a pretty good guarantee that 2D graphics render like in Chrome, and that’s important for this use case.
As a maintainer of a project that includes another popular library from Google, here's why it's difficult to build:
- building requires downloading Googles custom toolchain and build system
- dependencies are huge, so you have lenghty download times even on fast connections
- it usually works if you are using a recent versions of OS and Python, but if you try running the same command in a year or two it might fail because they changed the requirements
- if anything fails you have to dig through multiple levels of abstractions to figure out where it failed and why
- if you want to maintain software for a few years, you'll have to keep fixing the build process because the build will suddenly stop working for unknown reasons once a year or so
Around 5 minutes.
>That's part of the build process.
It's dependent on one's internet speed so I didn't think it made much sense to time it. If it took 20 to 30 minutes to download I would have mentioned it.
The official instructions are https://skia.org/docs/user/build/
If this seems like a big lift, people are going to hate building Chromium.
https://medium.com/@gauravswarankar/flutter-will-use-an-impe...
And to do so onto multiple OS & hardware backends?
There’s no free lunch, Impeller has a different set of trade offs that are a better fit for Flutter.
https://github.com/linebender/vello is written in Rust and already used as the backend for Xilem, a reactive framework for native UI.
https://skia.googlesource.com/skia/+/2e551697dc56/third_part...
My understanding, having not dug into it too much, is that the Skia integration does exist, but isn't enabled by default/any clients at the moment. That is, I don't know that this integration is shipping anywhere.
Vello still has some definite rough edges at the moment, so I'm not sure I'd recommend using it in a production application at the moment. We also don't have a C API, which might rule it out for some cases where you'd be considering Skia.
Yes. So in Sciter I've replaced its build system with relatively simple premake5 script. That replacement took couple of days but was worth it. Premake5 generates human-readable IDE solutions and make files. So you need just a compiler to build the whole thing.
> building it from source takes 20-30 minutes on a modern laptop.
It is not that bad actually. Just tried full rebuild of x64/Windows version:
Whole sciter.dll (HTML/CSS/JS/Graphics) with Skia backend:
sciter.dll build completed at 9:11 AM and took 07:03.415 minutes
sciter.dll build completed at 9:22 AM and took 02:34.412 minutes
> it's constantly changing its APIs
That's very true and is a pain indeed if to change its version frequently. Yet there is no such concept as "Skia version" - just revisions/milestones. It used to be an attempt to make stable plain C API but AFAIR it was removed recently.
Same thing about Google ANGLE that I started to use recently in Sciter.GLX: https://sciter.com/sciter-glx-beta2/
premake5 is a monolithic/portable executable that contains Lua + specific runtime.
Thus it does not rely on installed Python as in GN case as other tools.
Having standard and well known and documented Lua on board benefits the maker a lot. In my opinion any modern build system must include generic and known PL.
And that above is the problem of modern CMake. It started as simple static declarative thing but life forced it to evolve into dynamic programming language with very strange notation and runtime model.
I'm curious if you can expand on how you're using python, and what pain points you have there? I think python 3.9 was one for us but not too bad.
CPU renderer uses a tiny, custom fork of Skia (we only use the path rasterizer and their SSE2, AVX2, NEON backends) and our GPU renderer draws directly on GPU via tessellated paths / hardware MSAA (DX11, DX12, GL, Metal, Vulkan).
We use it in LibreOffice, but its a right pain to update versions and debugging it is.... challenging.
The upside is that the Google Skia team is super friendly and willing to help.
Check out nanovg: https://github.com/memononen/nanovg
- NanoVG
- bgfx with vg-renderer
- Impeller
- Starling
Flutter made a different engine called Impeller[0] which is replacing Skia. Which is a bit surprising as an ignorant outsider. I hope that works out.
Rive (https://rive.app), is a new animation tool that targets multiple platforms including web and their CEO Guido Rosso gave a great interview on School of Motion[1] about how they are building an animation first vector engine. There is a side by side demo at 46:56[2] of Skia, Impeller and Rive.
0: https://docs.flutter.dev/perf/impeller
I did not follow Rive yet (but will do now), but it seems the renderer is MIT licenced, here the wasm/js renderer:
https://github.com/rive-app/rive-wasm/blob/master/LICENSE
Open source renderer would be the requirement for me to invest into it (not gonna fall for flash again).
0: https://www.youtube.com/watch?v=mmW_RbTyj8c
1: https://github.com/linebender/vello
https://thenewstack.io/igalia-the-open-source-powerhouse-you...
Since they are apparently "powerful" enough to decide major direction of WebKit development as evidenced by OP's article, what exactly is their relationship with Apple in this regard? Like who has the final say and who do the day-to-day decisions?
I'm always curious about the politics and power structure/dynamics of these major open source projects, especially the ones backed up by large companies.
To answer your question though. WebKit is Apple's project and they do the majority of contributions. Igalia is the second largest contributor and collaborates with Apple regularly. Within the GTK/WPE ports Igalia controls them.
I always find building these complex stuff from Google a huge pain - and now they have the additional idea of living on the head and not providing actual releases too...
"WebKitGTK and WPEWebKit Switching to Skia for 2D Graphics Rendering"
Can we update the title here also?
"Impeller is a new Flutter rendering engine that the Flutter team claims solves the early-onset jank problem. It is designed as a replacement for Skia, with the goal of enabling better animations and addressing the jank issue, while also potentially providing support for 3D, which was not previously possible with Skia, as it exclusively supports 2D. Unlike Skia, Impeller compiles shaders during the build process instead of at runtime.
In Flutter 3.10, Impeller replaces Skia engine and becomes the primary rendering engine on iOS."
This wouldn't work on most platforms as graphic driver updates can break compatibility with previously compiled shaders.
Metal and CUDA actually let you AOT compile native binaries since they have relatively few hardware targets to support, with a fallback to compiling bytecode to native at runtime for forwards compatibility.
The alternative is to just not have that many shader permutations, and potentially take a performance hit from that. This seems to be the strategy that Impeller is following.
It is long-time known pain point that the skia official doesn't provide a usable C API.
At the very least, the stable interface should probably be C++, mapping between C and C++ is often non-trivial...
And let's be honest, the external API doesn't change awfully fast anyway. A totally dead project would fall behind, but even a tiny amount of work would be able to keep up.
I wondered about the license because they had already problems with LibWebTRC which uses BoringSSL (BSD-License). Skia seems to use the new BSD-License without the advertising clause and is therefore compatible with the GPL.
PS: As valleyer mentions - this affects WebKitGtk not all WebKit ports? At least Skia is usable on MacOS/iOS.
In WebKit, however, that implies that they refactor the Bridge API that is used in between contexts and processes which was an internal API before and broke away often.
So I'd guess they start to do incremental changes on the Web API implementations first before they break too much anywhere else (e.g. sidebars, UIs, widgets, devtools are rendered differently but rely on the very same Bridge API)
Presumably this is partly why it has become so popular, but as someone who's been writing mostly 2D GL/WebGPU apps for a decade I've only briefly considered a Skia as an alternative, but this is mostly out of ignorance.
But the external API is fairly simple. Basically you queue up various drawing commands and at the last minute tell it to execute the queued commands somewhere.
The queue is 'optimized' - for example if you tell it to draw some text and later crop the text out of the image, then no CPU/GPU time will be spent drawing text that won't be visible in the output. You can also draw text small and scale it up later without getting jagged edges as you would if you'd scaled up a raster image.
That ability lets you make tile based renderers, where you queue up all the commands for say a whole webpage, but then draw only a small square of the page like a map tile, and as the user scrolls/pans, you can draw more tiles as needed.
> In December 2023 we made the decision of giving Skia a try internally and see if it would be worth the effort of maintaining the project as a third party module inside WebKit. In just one month we had implemented enough features to be able to run all MotionMark tests. The results in the desktop were quite impressive, getting double the score of MotionMark global result. We still had to do more tests in embedded devices which are the actual target of WPE, but it was clear that, at least in the desktop, with this very initial implementation that was not even optimized (we kept our current architecture that is optimized for CPU rendering) we got much better results.
... for the GTK and WPE ports. Not on Apple platforms.
I guess the Apple platform is CoreGraphics and GTK was Cairo?
(Edit: Read the article. Yes)
https://en.wikipedia.org/wiki/Cairo_(graphics)#Notable_usage
I don't know the full story behind it, but from an outsider's point of view, any open library that pulls that kind of weight for so long should be considered a major feat of engineering.
Looking at https://webkit.org/wpe/, the first design goal is the one that justifies WPE vs other webkit ports: "To provide a no-frills, straight to the point, web runtime for embedded devices."
The other goals, like standards compliant and hardware acceleration, are there to differentiate WPE from non-webkit and ancient-webkit browser engines that people might use on embedded devices.
GTK is sometimes inappropriate for embedded devices that have a unique display stack. Using WPE brings in minimal dependencies and can render to anything you desire.
libwpe is just used for some basic code sharing between the two, GTK is not based on WPE.
Are the capabilities substantially different from Apple webkit?
There are a few big differences like WebRTC support isn't in WebKitGTK releases.
Ah, thanks—I can see how that's the kind of thing you would want to see in a compatibility table!
Why not just import all that code?
Why do I have to increase the font size 150% to read it comfortably?
How can we trust a browser created by developers who seem incapable of creating readable web pages?
They should be capable of dogfooding their own output on the most basic level.
And so it begins...
Question requirements.
You can always find a reason to build rather than buy, but can you reframe your requirements in such a way that you can get away with something off the shelf and then rather spend your resources on the things that you can do uniquely different for your application.
They are a software product fork used by billions, with a team that doesn't get paid to develop on it, with not enough funding to just "buy" a battle tested library which has zero problems; because any bug would literally potentially break the web for years.