Can scrawl-canvas run in a Web Worker and render to an Offscreen Canvas? You'd lose integration with DOM events for interactivity, but you'd be able to perform image processing in a background thread and not block the main renderer thread so that UI elements are responsive.
The majority of the code runs in the main thread; filters use a web worker. OffscreenCanvas currently has less than 70% support (caniuse says), so most of the work is done in regular canvas elements which are created but not added to the DOM, then copied over to the DOM canvas once per RequestAnimationFrame tick