Show HN: Handwritten.js – Convert typed text to realistic handwriting | Hacker News Reader

Show HN: Handwritten.js – Convert typed text to realistic handwriting

github.com

255 pointsalias-rahil80 comments

Non-statistical methods of imitating handwriting are always fairly easy to identify as synthetic due to the uniformity of strokes and unnatural kerning (among other things). Even methods which incorporate random variations don't hide these artifacts sufficiently well. In my opinion, the only way to generate synthetic handwriting which is convincingly real is to use statistical methods (i.e. machine learning) to model all variation which is present in real handwriting. I've implemented a neural network in javascript which does exactly this - https://seanvasquez.com/handwriting-generation/. You can play around with it and find a few weaknesses, but in general I find that it can produce handwriting which is indistinguishable from real handwriting.

That's super-cool. Honestly that's what I was expecting the submission to be.

When I looked at the example image in the original submission, the same letters all look basically identical to me -- I can't figure out why it's JavaScript rather than just a font.

Yours actually looks like real handwriting. It's quite slow, though -- like the actual speed it takes to write. Was just wondering if you slowed it down intentionally for effect, or if that's just how long the NN takes?

Also, I'm not familiar with the space, but this feels like something you could commercialize as a plugin for Illustrator or something like that -- particularly if you had a range of sliders to really fine-tune the desired text properties (width, weight, weight variation, etc.).

It's not intentionally slowed down. The sampling process is computationally heavy and you'll notice improved performance when running it on better hardware. There are many algorithmic ways to speed up neural network computation and I'm fairly sure I could speed things up by an order of magnitude or two if I invested some time into it.

You sure you don't have someone from India writing this out on a touchscreen when someone enters the text at the prompt?

(Just kidding)

This is really neat! I love how it looks, and the selection of styles.

A couple of times, using Style #2, it generated a bunch of loops at the end:

Text: "The quick brown fox, etc" Image: https://pasteboard.co/Jn0O17a.png

Still - nifty, and fun to play with. Makes me want to find an application / excuse to use it :)

Here's a "Hello World" with a whole ton of loops as well :)

https://imgur.com/fhG2Fwq

That's a totally acceptable "helloooooooo world"!

The model was trained on a relatively small dataset which has few occurrences of the letter x, almost none of which are in cursive. As you've found, asking the model to write out an x in a cursive style can totally derail the generation process.

Any ELI5 for people who know close to nothing about NN, AI, machine learning etc?

Like, how does one letter it doesn't know how to write well make it skip/mess up the following ones too? I mean, "derail" seems a fitting description to what I'm seeing, but how?

The "handwriting" data for this model is basically the coordinates of a pen. The length of the string representation of the text is very different from the length of the coordinate representation of the text, therefore the model "learns" a window corresponding to when it is drawing the current letter, and when to start the next letter. For these letters, as the model doesn't learn how long this window should be, nor how to transition from it to the next letter, it gets stuck and outputs nonsense.

There must not have been many "z"s either -- "quartz" failed as much as "fox" in my tries. Still, nice work!

I got that to after playing around a bit and then trying that phrase. It gave me a really good laugh, as if AI just went "ah shit, why does everyone enter this stupid phrase eventually?"

It breaks down catastrophically with diacritics https://imgur.com/8U3IKR9

Heh, "¿Cómo estás?" -> "OC"

Wow! How did you do this? Any tutorials or methods that would be helpful to learn how to produce something like that?

From the github project

https://arxiv.org/abs/1308.0850

https://github.com/sjvasquez/handwriting-synthesis

Hi, I like this, and found your project at Github. Could you add licence to your project, please! https://github.com/sjvasquez/handwriting-synthesis

I second this. Please add a license!

Wonderful! And as it generates single stroke SVG exports, I can output to a laser cutter and have it write for me :)

I can't think of a practical use for this, but that's not stopped me wasting the last 15 minutes getting my machine to write out expletives in neater cursive than I could manage...

- ed

Oh, and this is by far the best digital 'handwriting' I've ever seen, despite the occasional glitches and believe me - I've been interested in this sort of thing for decades.

Having played with this a bit - if you can flesh it out a bit, I am sure you can monetize this.

I've been dicking around with it for an hour or two and I'm getting results that - if i mingle outputs - are to my eye completely human in nature.

Honestly, as an ex sort of still designer who still engages with that sort of community, i see a tool like this that makes neat scripty vector paths (and I think that is particularly important) as being super-useful. It outputs unique forms that are individual in nature. I'm thinking all the graphic designers who makes stuff for weddings and invites and whatnot.

Needs work, and I have absolutely no concept of how complex it is to refine things, but what you have here is a helluva start to something I think could be immensely useful to some people.

- minor eds

- ed ed

Can you manage the stroke widths through the cursive forms, thinner when drawn quicker and straighter, fatter when slower around corners, for example?

Does this not work on mobile? I just get random lines then they disappear.

Working on Android chrome for me, a little cludgy but tolerable

just checked again and sure enough it's working now, thanks.

Very interesting. I did notice that with my input of "Hello, this is some hand written text!", the output of the word "text" consistently looked more like "tent" than text, regardless the style or legibility settings.

No matter what style/legibility/text I try the line always just drops as if my computer had a faint [1]

[1]: https://imgur.com/a/IgAV1ef

If I write for example 5 letter "k" in a row (space separator), the technique used to write them varies in a way that irl wouldn't for a real human.I guess you could argue why would someone write 5 k letters. (I won't mention that I observed the same variation when repeating the process using a name of a certain notorious organization that uses 3 k letters)

How would you go about creating a minimal/medium source of input to train this?

Could you create a 'written test' type thing - a couple of pages of selected forms of repeated instances of glyphs and interactons - that would be used as input and to learn a passable subset of one's own handwriting?

This is super cool!

A random FYI: the demo seems to consistently fail on the letter x when using style #9 and typing out:

The quick brown fox jumps over the lazy dog

I tried a few different spots on the legibility slider with similar results..

It draws the first "top left to bottom right" line, but then instead of a "top right to bottom left" it draws another "top left to bottom right" next to it..

Try "fox" in style 2 with maximum legibility. It goes loop-the-loop several times.

Seeing this had a strong uncanny valley like effect on me. Like my brain can't help but imagine a sentient agent writing the text and suddenly becoming psychotic.

hmmm, not necessarily the same output each time

This looks very interesting. I wonder how much data would be required in order to imitate a particular handwriting style? i.e. if I know someone with a lovely penmanship and I can ask that person to create such a reference data set, how much should that person write in order for the network to be able to reproduce the style reliably?

Yours is amazing! By the way it has a big problem with "/" in the default settings. I tried doing a date like "08/20/2020" since that's something I need for signing forms but the slashes mangled it. Dashes worked ok.

This is amazing. It's cool how you can drag the legibility slider up and down while it's writing and it responds live. It looks the most realistic to me when I alternate between high and low legibility, especially with certain styles like #3.

Is there one that instead imitates pressure sensitive input and then follows the same pattern? With a sufficiently large brush and downsizing the image it could look very good. Maybe the lack of artifacts from the brush is the reason for the unnatural look

How much data is needed to train the model to replicate a specific person's handwriting? Could this recreate my hand writing with less than a page of notes in my own handwriting?

It’s giving blank output on iOS or is it because the model takes time to load?

Also, is there any chance we can fine tune the model for each user by taking their handwriting in a predetermined template file?

Works for me on iOS. Did you click the “write” button in the bottom right corner? It seems you have to tap it after any change in text or settings for it to update.

Yup, I did use the write button. It works on my laptop though.

I checked the console and it was indeed throwing an error

"Error: Backend name 'webgl' not found in registry"

Were you using actual Safari or an app’s stripped down UIWebView? If Safari, then I’m guessing you have a content blocker that’s stopping access to WebGL.

The actual browser.

I am only running Pi-Hole on my network but then again if that was a problem it would not have worked on my MacBook either

Thanks for sharing. If you don't mind, did you use Graves 2013 RNN or something recent/more advanced?

Yes, it's mostly based on that. Primary difference is that this model adds an inference model (VAE style), which allows you to a) sample style vectors from a latent distribution b) do so more efficiently than the priming mechanism described in that paper.

Thanks. Great work.

That's a lot of fun to play with. Nicely done.

How realistic is it to have no joined letters? Do many people really write large scale texts letter by letter with space between each?

Sadly, the screenshot looks obviously artificial to me. Like handwriting fonts, I'd be very surprised if anybody thought it was really handwritten.

It's not open source (because it is my handwriting (because my progressive MS means I can't do it manually)), but here's my hack: https://imgur.com/a/frqG0cu

As sjv says above, there are still plenty of tells in mine. So I think this module is very unconvincing.

Maybe I've spent too much time being finicky or am just being braggy (definitely not beyond me, sorry), but 'realistic'?

> How realistic is it to have no joined letters? Do many people really write large scale texts letter by letter with space between each?

Most of the people I know do this. Myself, I have never ever written with joined letters except when required to in school.

I consider it a leftover from the era of fountain pens (which are terrible UX for me as a left-handed writing LTR languages).

Even my signature is based on separate letters (tho blurred together as sigs usually do).

Fascinating. Thanks for responding.

Can I ask what age-group you/they are? And it sounds like you had to write joined-up at school. Why would you stop? Isn't it so much faster?

I went to a snooty high school where fountain pens were compulsory (this was the 80s, not the 50s!). But writing was taught joined-up from pencils in infant school, from when you knew your whole alphabet. It never occurred to me to not join. It would have looked like a very small child's writing. Interesting though that has changed.

My wife is a grade school teacher, and disagrees, but I totally think that teaching handwriting is a waste of time. I wasn't anywhere near as fast writing as typing. I would have been much more productive with a laptop in school.

Anyway. I'm rambling now. Thanks for the response! I stand corrected.

I'm 41 - yeah we had learned both joined and "printed" (as we'd call them) letters, with the intention that for handwriting you'd use joined, but it was only required, for a bit, in lower grades in Croatian classes (ie. that'd be English classes in the US/UK/etc).

That was some 33 years ago but I have an 8yr old now and that part of the curriculum is basically the same.

Both then and now kids only need use pencils for this, thankfully :)

For myself, written joined up is more effort and slower than just individual letters. I don't know if that's due to less practice in it, or due to left handed needing to push the pen(cil) more when writing in that style.

I'm way faster touch typing (tho not on a screen keyboard!). If it were up to me, kids today would learn the easiest handwriting method (I believe that's individual letters) and typing, instead of two ways of writing.

Thanks for sharing your thoughts! Always interesting to find how somr things I take for granted are done/seem elsewhere!

> Do many people really write large scale texts letter by letter with space between each?

As an anecdata, Alfred Wainwright did all his books like that (and justified the text, to boot.)

cf https://www.ramblers.org.uk/-/media/Images/Walk%20magazine/F...

I love his books. Gorgeous.

I think this was more his graphic design style for his books. As you say, the justification shows there's nothing impromptu about it. Here's how he wrote his letters, notes, drafts and so on: https://www.grough.co.uk/lib/img/editorial/wainwrightmanuscr...

I'm not splitting hairs on what 'handwriting' means. I'm fascinated by that era when books with complex text/visual interactions were far cheaper to create by hand. An example of an old magic book I have: https://imgur.com/a/8Bu9ML1 It's hard to beat Wainwright though.

Many times I get university assignments that are compulsory to be submitted as handwritten document. Most of the information needed for the assignment is available on the internet so why not save time? Just make a text document and let handwritten.js do the writing part! example: https://raw.githubusercontent.com/alias-rahil/handwritten.js...

This is insane busywork that I hated in college (xUSSR, 20+ years ago). I would scrutinize the example, though. I don't know how "clean" the documents are expected to be, but the picture is too uniform and too clean - no pen marks, etc.

That is pretty much the US experience also, most of grade and high school was spent copying information from projector slides onto paper. Guess it is an easy way to fill time, read everything off sentence by sentence, wait for everyone to stop writing, cover the sentences not being read so people who fall behind can’t catch up, and speed writers can’t get ahead.

I’m very envious of my son, in his classes they distribute the materials via Google drive and spend the class time in discussion and working out problems.

Writing things down is a known method of studying. I don't envy your son, he should still write to effectively study

Is it though? I've heard very often that lecturing in this fashion is the most inefficient way of transmitting information from a slide onto a notebook without being understood by anyone writing it. If you're processing what you're seeing and summarizing then maybe that works but I think what worked best for me was always asking questions about the material.

He says known method, not known good method.

In our case it created entire "industry" of rewriting lecture notes by hand (some lecturers required it)

Handwriting machines are popular in China for this reason, from what I've heard.

I don’t get this project. This is nothing special, you can very easily spot that every T is identical. You might as well just use a handwritten font.

It applies Cunningham's Law to find sjv‘s project above.

Producing handwriting is a problem a group of people I belong to is facing. We work (free, on our free time), detecting forgeried signatures of a celebrity on eBay (forgeries that can go to thousdans of dollars).

Forgers are not good imitating signatures, but they discovered how to create the celebrity handwriting using an autopen, which means is harder everyday to distinguish real ones from fake ones. Collectors are used to them - and we track eBay sellers, buyers, etc, so we have an idea of who is who, but people who just want to buy a signed book for a birthday present, may end paying thousdans of dollars but something fake that has no value.

I see the importance of projects like this, but in the other hand, I cant help thinking that every piece of tech that imitates human behaviour can be used for bad.

Everything is worth what its purchaser will pay for it. I recommend the Orson Welles film F is for Fake.

But if you think you are buying X and receive Y, that's a scam.

You might consider turning this into an OpenType font including all the random alternates: https://graphicdesign.stackexchange.com/questions/26833/hand...

I like how trivial it is over the sort of machine-learning solutions I envisioned when I clicked into it.

Does it not support a glyph? Just add it. Produces weird output? It's a bug, not a black box issue, just read the code. Want to use it or port it? It's just character substitution, go ahead.

A fun twist on the idea would be to generate an SVG path for each character and then apply some morphing to the path so character strokes are unique.

Also https://news.ycombinator.com/item?id=24170366

I've noticed that I get junk mail now that appears hand written. They go so far as to depress the paper as if with pen in a convincing way. They seem indistinguishable except that they are marketing.

I'm left wondering if there is some new innovation here combined with popularity in some marketing newsletter.

I went looking for the device that could do this and have come up empty-handed. Does anyone know what could create such?

Bulk mail is often produced using a letterpress-style machine. Personalized mail, on the other hand, is usually "handwritten" using a pen plotter (e.g., AxiDraw).

https://axidraw.com/

Even with the Axidraw, it's really hard to produce notes at scale. There are a handful of companies that have built out proprietary machines to do this e.g. https://www.handwrite.io (disclaimer: I'm a cofounder) but as far as I know there aren't any readily available for purchase.

Is this different than a font?

Can a font pick randomly between different versions of the same glyph?

I know a font can define unique ligatures for every character combo, but that would still mean the same combos like "fi" always look the same. Nice thing about this low-tech solution is that it can be trivially scaled up to introduce its own ligatures yet pick from them randomly, just at the expense of chore work (making the images).

Of course, at that point, it would be tempting to start looking for a more general solution.

> Can a font pick randomly between different versions of the same glyph?

In theory, yes, OpenType Contextual Alternates can do this.

See "Introducing variety" on this page: https://www.typenetwork.com/news/article/opentype-at-work-co...

There's a really good write up of this about the font Liza: https://underware.nl/case-studies/random-vs-clever/

Just tried it, font looks great on paper. I don’t have too many uses for this, but I bet it’ll be pretty handy for people who are still addicted to reading notes from printed sheets. Great job.

It would be great if this could output a generated font file, which can then be used on the web etc.

Love it! Kinda remind me of the old Hacker News. I wish there were most post like this one.

Looks good. Where can I use it?

how about being able to use OCR to read real handwriting :) that would be truly amazing to accomplish with high precision

Feel free to checkout our super-realistic type to handwriting generation in greetings cards here: https://wuzci.com/product/we-make-a-great-pear-3/

Reply on news.ycombinator.com