Hyperlinks in Handwriting
handwritten.blog
handwritten.blog
If it were me, I'd go for the <map> tag instead of fancy new CSS!
> image-based websites are massive
With the huge gains we've made in image optimization since <map>s were popular this isn't always true anymore.
I was initially surprised to see it’s not transparent text overlaying the image, but apparently the text recognition is actually that good.
This is a good approach, I think. It's kinda fine-tuned for reMarkable output, as it's mostly pure black or white pixels, but clearly they're fine with that. And the end result works quite well, and seems as easy or easier to implement.
This is all also kinda made moot by there being a link to an excellent transcript at the bottom. Accessibility needs are better covered by a straightforward alternate page than <map> alone.
I can't begin fathom how that could happen. It's not some fancy JS thing, it's a very simple CSS effect:
a:hover {
filter: opacity(0.5) !important;
}
.post a {
background-color: #35c;
mix-blend-mode: screen;
position: absolute;
display: block;
}
Does opacity() combined with mix-blend-mode maybe not work right on your box? I'm not sure why it wouldn't, but you can convince me of just about anything by saying "Firefox does weird things on some GPUs".
But quite seriously, rejoice that you’re using Firefox. It has very few blending issues in general. I’ve found the Chromium family to be vastly buggier in this area, and I’ve learned when doing almost anything with filters and blending to develop with Chromium—previously I developed with Firefox and everything would just work, and then I’d test it in Chromium and discover some important bug or fundamental undocumented limitation in Chromium which meant I had to start again with a different approach, e.g. https://bugs.chromium.org/p/chromium/issues/detail?id=992398 where a perfectly normal combination of things just stopped rendering the page after 8,192 pixels (which isn’t all that much); or for something more recent (earlier this year) look at the source of https://temp.chrismorgan.info/2022-10-03-hand-drawn-with-lin... and see some of the compromises I made for Chromium.
Imagine if you could hyperlink as you handwrote notes, fluidly with the motion or gesture of the pen. As with text editors like emacs or *vim, you'd create new modes of thinking as you interact with a machine at the speed of thought, with the underpinning physical motions being too fast and too easy for conscious attention. Make wormholes from one part of the writing surface to a distant one, recording connections, asides, without conscious effort. No more than you expend on the shapes of letters, or the layout of QWERTY.
Instead, we're kind of stuck on the UX of "ordered sequence of rectangles", with some usability hacks to mitigate the severe limitations of this skeuomorphism.
Surely there must be some astounding low-hanging fruit here, in the UX of epaper handwriting as a tool of exploratory thought.
More generally, with handwritten (or other physically-manifested writings), an indexing and referencing system substitutes for hyperlinks. The Bullet Journal index and references operate in a similar manner. The O'Reilly book UNIX Power Tools offers a print hypertext experience by numbering each individual article and section, and liberally referencing between these. The first edition is superior to later ones in that the text is printed in multiple colours (black and blue) which distinguish the text and navigational elements, much as early Web documents did with the standardisation of blue link colours.
For those who remember such things, the paper and print card catalogues of libraries operated similarly.
To make this work, I need a nice copy/paste method for pen+touch input to input the link, along with nice app switching.
Otherwise, it should be drawing a circle around the marks, right click, add link.
How do you click the links though?
All you would need to control an epaper with gestures is to reserve some subset of the character-glyph space as an in-band control channel. Glyph shapes that are never semantically valid as written text. Shapes that are easy to reliably parse, which are highly "orthogonal" to written text, as well as to other control gestures.
- "How do you click the links though?"
I'd suggest they be interpreted as special "active" regions with a single default action, so any user input works, if it's within the region. Scratch a line over them and they open. My thinking is controlling the machine, and writing, should be the same actions and combine together fluidly: everything is a pen stroke.
(If you're in the same boat as a user, you can work around the issue by resizing the browser window. Well, you can on PC, I'm not sure that's possible on a tablet.)
<img src="xyz" alt="Handwritten post">
I'm with the author in that this is a nifty trick, similar in some way to the map element (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/ma...), but it's one of the least accessible web pages I've ever seen.
- image: handwritten.blog
- image: handwritten post
- link (image): comment on this post
- link (image): read the transcript of this post
- link (image): back to main page
not bad at all. Sure, it would be better if the transcript was just part of the page, but I understand the tradeoff and it's quick enough to get to the transcript.
If you had a screen reader you'd hear literally only:
- Link, image with alt handwritten.blog
- Link, image with alt Comment on this post
- Link, image with alt Read the transcript
- Link, image with alt Back to main page
I think it'd be less confusing if the main body had an alt. It would just be a short FYI that there's also a big image of text and that the transcript reproduces it.
But the transcript would be a lot harder to miss with a screen reader!
It’s possible they’ve improved the technique now, but I’d be a little surprised if they have, given that it amounts to throwing away the entire thing and starting from scratch.
There are a few open-source libraries to convert the .rm format to SVG. None are good for anything but the fineliner. I’ve written the closest one to decent for tools like the pen (though I haven’t got round to publishing it), but it still lacks the texture (pretty much unavoidable) and is thus never going to be excellent. SVG just lacks the necessary primitives.
---
(I've read a bunch about accessibility, but I'm no expert).
I don't know, I actually think this blog is more accessible than most.
You might have missed the transcript link at the bottom.
The only changes I'd make is adding a summary alt to the image saying something like "The handwritten body of the blog post. A transcript is linked below".
(Without any alt the page might be confusingly blank, but as many screen readers poorly support long alt text you don't want to copy the transcript in.)
edit: Yes, it works okayish in screen readers. Accessibility is about far more than screen readers.
May I ask, what other accessibility conditions do you feel this site doesn't meet?
Also - for a non corporate website - why should accessibility be a relevant factor? For a personal project, I don't see why anyone has to make it targeted for every case.
[1]: https://sachachua.com/blog/2022/09/monthly-review-august-202... [2]: https://supernote.com/
It looks like that group is pretty much frozen in 2005-2006, and it's really interesting to see how these blogs were handled back then.
The domain on which I hosted my blog is no longer active, but I've still got all the source code, and it's fun to look back on it as an experiment.
For me, the hardest part was the SEO. I ended up transcribing my handwriting as much as I could, then had the images cover the text.
Jokes aside, yes it is a nightmare. However, you can find a transcript at the bottom of the page, which makes it a little more readable.
I don't think we should discourage people from setting up systems that have this kind of accessibility, even if it's not as "pure" from the perspective of duplicated content.
(The guidelines say it's preferred for everything to conform directly, but they use "artistic integrity" as an example of a situation where a conforming alternate version could be used. I'd say that applies here.)
So we install smoke detectors and put expiration dates on things to detect danger without smell.
You can't make a text description of a guitar solo that's "equal", much less not "separate"; I don't buy that people really think that we therefore shouldn't have music.
Everyone I've heard from thinks a separate site is better than a massive alt though. Many screen readers handle them extremely poorly, and they can't be annotated with html tags to indicate semantics (link, list, etc).
I remember when whiteboard coding interviews still existed, I was shocked to find someone actually writing code by hand on purpose.
Is it what society has come to?
As for my coworker's notes these days, I'd say print has a majority even, but in these days most people use computer documents instead of handwritten anyway.
Perhaps they want it to be readable? :-)
> Is it what society has come to?
Thankfully, yes.
https://www.theatlantic.com/technology/archive/2015/08/ballp...
That said, I do wonder why. The fastest I can write manually is roughly 20-25wpm, and that's already going to produce something far more illegible than this article and be extremely fatiguing, whereas if I'm using a keyboard, I can do 5-7x that for long periods of time without tiring.
In theory you could do it without the plastic sticker on the lookup side, but it seems like you'd use far more resources with little gain (principle gain being you only need one transparent marker; principle loss, you have to search every image for every "marker code" ever created across every part of the image).
My preference has been to use SVG for the links (independent of whether the strokes are done raster or SVG). It’s not fundamentally any more expressive, but it’s much more convenient for a number of techniques like adding background colour or link underlines (I found changing only text colour with mix-blend-mode unsatisfactory), and links that span a line break.
Here’s a snapshot of roughly where I last left this a few months ago. (I’ve had to substitute in an inferior document, as I was working with a lined-paper prose document that I’m not willing to share at this time. It works better with that type of content.) You can see that I’ve leaned into the pagination, styling it more paperily, and also that there’s a fairly significant overload of SVG filters which tend to ruin performance. I’ll pare that down significantly if I go ahead with it. (Still keeping some of the page-edge and maybe link shape roughness, but with a simpler paper texture.)
https://temp.chrismorgan.info/2022-10-03-hand-drawn-with-lin... (almost 800KB because the pen uses a variable-width stroke and my SVG generation for that isn’t particularly optimised because I’ve skimped on some of the fancier trigonometry involved—but I should note that it’s much more accurate than the “lots of short line segments” approach used by every existing .rm-to-.svg library I found).
Also if I go ahead with this, I’ll use the colour support that came in version 2.13 of the reMarkable system software, drawing my hyperlinks in blue from the start and reserving some highlighter colour if I want the background shading. Using distinct colours like this will make identifying and marking up links mechanically straightforward (since all the strokes in a given layer appear in draw order), leaving only the entry of the URL.
—⁂—
One other thing about the presentation on this site: the image scales to the viewport width, which is very, very not good: it means that regular desktop-style zoom (most importantly zoom out) just doesn’t work. The most common desktop screen sizes are getting the equivalent of a body font-size in excess of 50px (three times what is generally reasonable). You should add something like `.post { width: 40em }` which fixes the problem without causing any other trouble.
Right now, it's three steps:
> make newpost NAME='name of this post'
[edit the skeleton until it has what I want in it]
> make rsync
I still don't write as often as I think about it.
The transcript appears to only have 7 steps, there are 9 on this page...so is it a transcript?
Without any effort on my part, the system recognizes handwriting inside an image and lets me copy/paste it like I was working with standard text.
It’s 100% seamless and feels magical.
Have you tried passing the pages through a OCR transcription system like tesseract? You also get this with the text-parsing feature of the reMarkable 2 for free, and I was thinking you could use the [link title] syntax to mark links and have some sort of automatic mapping to the links so you don't have to specify the links "by hand."
Anyhow, great idea. It's a bit small to read on iPhone.
A cool idea would be to convert each word to an individual image and put each of them inside of an inline div to make the article responsive.
My problems boil down to just that flow of the tools sucked. I didn't want to have to attach multiple pages together. Inkscape was honestly a bit clunky to work with and I didn't have any personal tools to generate the full html from the svg.
I might just be inspired to try again though.
A solution could be to OCR the site and overlay the actual recognized text on the site as HTML items that can be hidden but useful for Google index the contents.
I can see the pain and friction that doing that manually would entail though.
I think the problem with doing something like this (besides accessibility) is you can never get the comfortable reading size right because the screen size/shape will never align with the flow of your text.
Or use an SVG.
It unfortunate that the OP didn't turn their writing into SVG, and it's somewhat stunning that turning a webpage into a PNG with a ridiculous resolution results in a _smaller_ webpage size than most webpages with similar functionality today.
The new web: forget HTML, use PNG. \s
My thinking was to allow people to doodle / made little comics on the web. Right now artsy people are confined to exporting to pngs and including it as image.. I was thinking somebody should just grab ipad and start drawing. If they want to write a text they should be able to do with pen, keyboard or voice transcription...