Show HN: Handwriter.ttf – Handwriting Synthesis with Harfbuzz WASM | Hacker News Reader

Show HN: Handwriter.ttf – Handwriting Synthesis with Harfbuzz WASM

github.com

191 pointshsfzxjy53 comments

During the hype of llama.ttf months ago, I was speculating the potential of WASM shaper for even crazier purpose, one that fitter to a font shaper's duty -- to synthesize font at runtime. This project as proof-of-concept implements a synthesizer that generates and rasterizes handwriting-style font, backed by a super-lightweight RNN model (~14MiB).

Open articleView on HN

I bet this is what people felt when JavaScript was first demoed on the web.

In the year 2077, when each font will run it's own virtual machine on WASM containing a "minimal" Ubuntu image, some enlightened blogger will suggest server side rendered fonts as a performance improvement.

JS felt less impressive to me when I first saw it, simply because it couldn't do much. However, I remember the showcase Microsoft put together to show off what CSS could do in IE3 and that was very cool at the time.

I remember that, too, and switched to IE for the next X years

Nice demo.

Without trying to steal the thread, what I would care is actually the opposite direction.

Neither in Swift Playgrounds, nor in any other programming development environment apps for both mobile OSes, have I found a good development experience using pen instead of keyboard.

Given how many of us "program" in paper notebooks, it is quite incredible that besides a couple of research projects done by PhD students, no one cares to actually make it more widespread in a usable way.

I could see something like Apples new calculator drawing app taking this by storm. Writing out your code and getting syntax support and all the things in your ide but with your handwriting?

In the same way that the calculator writes in the answers in your handwriting, it could write in snippets of code in your handwriting.

If it's going to be made though, it has to be e-ink. Go big or go home!

Yeah, something like that.

Yes!

I'm on this route myself, trying various things out at https://github.com/fenollp/reMarkable-tools

Handwriting (in and out) support is very important IMO. Also being able to draw DAGs.

I'd like an e-ink device with high frame rate and HW powerful enough to run some models locally or with good enough connectivity and sensors that e.g. Computer Vision tasks can be offloaded to the users' smartphone.

Feel free to expose your ideas on there :) I welcome Open Source discussion!

I've read the README and watched the video, but I'm still not sure what this is doing? I know it can "synthesize [a] font at runtime", so does that mean it's creating a random handwritten font as you type? But it's not based on the user's handwriting?

Correct

This looks like it'd be quite useful for faking documents more convincingly. Existing handwriting fonts always have tell-tale regularities and there's so few that forensics analysis exposes them easily.

I wonder what makes SIMD an improvement here - in the end it all boils down to TTF bytecode and I don't think this comes with SIMD instructions, right?

wasm has SIMD extension https://github.com/WebAssembly/simd/blob/master/proposals/si... . You can use them even in browsers (e.g. Chrome >= 91)

OP is asking how SIMD is beneficial to this specific application. If there's a small neural network involved then evaluating that is probably a good place to use SIMD.

Yes. The program actually includes an ONNX runtime, which uses SIMD to accelerate NN inference.

but... the program ends up entirely compiled and executed as TTF bytecode in the end right, since it's entirely contained in the TTF font ? And TTF bytecode is only the following instructions : https://developer.apple.com/fonts/TrueType-Reference-Manual/... and I don't see anything related to SIMD in there

No that is not what is happening, HarfBuzz have a experimental Wasm shaper, so the font embeds some wasm code that tells Harfbuzz what to output.

https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-sha...

oh ok, so it can't work in any system that doesn't use harfbuzz.. that's much less interesting than what I originally thought

It’s not TTF bytecode, but WASM bytecode. There’s an experimental version of HarfBuzz that can run this.

I wondered why there's just a video and no demo. But does it require the new wasm features in the browser ttf support?

You can run the demo locally following the Usage part. Currently the WASM shaper feature is not enabled in any browser, so the demo won't work in web pages.

Ah, the wasm association calibrated my brain to seeing a browser window in the demo whereas it's really gedit.

Wtf is happening here what are the inputs and outputs? Hard to tell what this program does for me

The handwriting in fact consists of multiple "tiny black box" glyphs, similar to pixels in traditional rendering. The program takes the text (e.g., "Hello world") as input, and works out the (x, y) locations of these block boxes.

I prefer fonts that can be zoomed.

this is amazing! i'm guessing you can probably get it to antialias without much more work

as the demo video shows, it's probably not something you want to have in between you and the ability to scroll a web page or close a tab. but i guess using harfbuzz now means we're buying into a turing-complete virtual machine running an arbitrary program in order to display a glyph. how seriously crippled are the harfbuzzless rendering paths? i'm assuming opting out of harfbuzz means opting out of arabic, devanagari and other indic scripts, etc.? is there a less out-of-control alternative that doesn't leave two billion people out in the cold?

This is just a project for fun, like llama.ttf.

WASM shaper is still experimental and not yet shipped in and products, not before more limitation carried out as I think. So I'm not too worried about it.

it's super cool!

So you trained a THAT small RNN to make this good handwriting? I'm actually even more impressed at that than at the crazy pipeline turning it into a font in realtime.

I didn't train the model myself. A pretrained model is adopted from another repo [0].

[0]: https://github.com/X-rayLaser/pytorch-handwriting-synthesis-...

Damn, this is impressive!

We built a WebGL text renderer with full CJK support using Harfbuzz for our production whiteboard web app. I thought that was complicated until now.

Is that open sourced? I want to render shakuhachi music notation in the browser trying to replicate the caligraphic style and it's not trivial.

I've been curious about seeing if harfbuzz etc could help create a styled Elianscript font but not really sure where to start.

Fun hack. I bet Alex Graves never in a million years anticipated that his PhD thesis work would be encapsulated in a novelty font.

I don't get it, why there's no link to a GitHub pages website to test the thing?

You can't test it in a browser, since no browser at present is linked against libharfbuzz with WASM shaper. Instead, one can test it with a modified local program such as gedit in the demo.

As convenience, I built a Docker image that packs both the ttf file and the modified gedit together. You can try it out via `make run` as stated in the instructions.

If you just want to run locally. Why not using the native libharfbuzz directly? What's the purpose of WASM here?

WASM is to libharfbuzz (when experimental compile time option is enabled) what javascript is to HTML.

So this is essentially native (albeit experimental) libharfbuzz. WASM is used because its how font files are scripted (when using this experimental version of lubharfbuzz)

Its important to keep in mind that wasm is a general technology and is not just used by web browsers.

Font includes WASM code that harfbuzz executes for shaping: https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-sha...

It's an experimental feature, so it's not available unless explicitly enabled during compilation.

Harfbuzz can be built easily to Wasm using Emscripten.

Interesting artifact of time it would be. Harfbuzz uses https://github.com/bytecodealliance/wasm-micro-runtime to execute wasm, so when compiled it would be wasm runtime running under another wasm runtime.

OP's repository has a Makefile which assumes a docker daemon is available.

The project must be tested in an application linked against libharfbuzz with WASM shaper enabled. Since it's not easy to build a library like this, I make a Docker image which contains both the ttf file and a modified gedit, so that anyone can test the project with a single command.

It visually fails to preserve “handedness.”

Nice demo, for sure a fun app.

Link to llama.ttf, another fun font that abuses HarfBuzz for anyone interested (it runs a language model inside a font): https://fuglede.github.io/llama.ttf/

The llama.ttf video does a pretty good job explaining what the heck is going on

Year 2045: fonts become self-aware and go on a strikethrough.

I'm mad at you for forcing me to laugh at this.

Best comment

:) I sware that smilely just winked at me

This is so cursed, there is so much that begs the question - "why?"...

I love it.

I can't wait for the next beautiful nightmare. Maybe someone should mix font rendering with PDF rendering. Of course, with a LLM doing something in the middle.

The answer is "no why", just for fun :) Though I learned a lot about how to stuff an NN model in a WASM binary and some tricks to optimize performance.

Best reason!

I usually follow with "why not?", but forgot this time. Thank you for this. Again, I love it.

Reply on news.ycombinator.com