Compressing and enhancing hand-written notes (2016)
mzucker.github.io
mzucker.github.io
(1) Scan document at very high resolution as a JPG and then use a third-party program (like Photoshop or whatever) to re-encode the JPG at your preferred low resolution.
(2) Scan document at your preferred low resolution as a JPG straight away. Don't re-encode afterward.
Intuition says that the results of #1 vs #2 should be identical, or that #1 should be worse because you're doing two passes on source material. But I always get better results with case #1 (i.e., high-res scan and re-encoding afterward) regardless of the type or model of scanner, or whether the scanner does the JPG encoding on-board the device itself or through a Windows/Linux/Mac driver bundled with the scanner.
My theory is that scanner manufacturers are deliberately choosing the JPG encoding profile that gets them the fastest result. They want to brag about pages per minute which is an easily measured metric. Quality of JPG encoding and file size take effort to compare, but everyone understands pages per minute.
If anyone has contrary experience I'd like to hear it. I've been seeing this for years with different document scanners and flatbed scanners -- regardless of how I tweak the scanner's settings, I can always get good quality in a small file by re-encoding afterward.
This is more-or-less correct. The chips in the printers have a lot less power than your CPU, and the algorithms are a lot worse than those in Photoshop.
Would scanners benefit from just using some of the plentiful, cheap, excellent-quality H.264 encoder ASICs—and treating the output as HEIF?
In my own scanning workflow, I scan at 600 or 1200dpi to png, deskew, downscale, and apply a black/white threshold. This is all done with imagemagick: mogrify -deskew 40% -scale 25% -threshold 50%
If I want a PDF after that I'll use img2pdf.
I did some vanilla JS that simulates scaling from high DPI vs sampling directly at a specific DPI and the result does resemble the result of a low resolution scan.
http://chrisbenjaminsen.com/shared/samplerate/
I am aware optics does not work exactly work like this, but it's a reasonable approximation.
Here is why: if you go with (2), then (1) is still done: by the crap firmware in your printer or its driver. It scans at high resolution and then downsamples in some way over which you have no control. It might not even be done with floating-point math.
1. is a somewhat like getting the raw image from a camera: a higher quality source for your own processing.
The worst thing to do is to scan with the lid closed, with a lid that has a white background. This would increase the reflection from the backside of the page.
Very clean results. In some ways it's a smarter version of the "posterize" feature.
Here is a casual job on the first image:
https://i.imgur.com/Sy2rvsU.png
The steps:
1. Duplicate the layer.
2. Gaussian-blur the top layer with big radius, 30+.
3. Put the top layer in "Divide" mode. Now the image is level.
4. Merge the layers together into one.
5. Use Color->Curves to clean away the writing bleeding through from the opposite side of the paper.
6. To approximate the blurred look of Matt Zucker's result, apply Gaussian blur with r=0.8.
Notes:
The unblurred image before step 6 is here: https://i.imgur.com/RbWSUnD.png
Here is approximately the curve used in step 5: https://i.imgur.com/lvfqCNK.png
I suspect Matt worked at a higher resolution; i.e. the posted images are not the original resolution scans or snapshots.
BTW I'm curious how you'd fare on the graph one. I didn't like his results for it. https://github.com/mzucker/noteshrink/blob/master/examples/g...
Note how the grid is completely gone, the Sharpie strokes are fuller and the ghosting around the red ink is gone. (The word "Red" seems to have been written faintly, like with a non-working ball point pen, and then written over properly.)
The thing is, I took a completely different approach here. I won't give a complete step-by-step recipe, but the gist of it is this:
1. Create a copy layer of the image.
2. Optionally level the intensity with the divide trick; I didn't bother.
3. Convert this copy to grayscale.
4. Threshold it to black and white, such that the grid is eliminated, but the writing remains solid.
5. Blur the writing (radius 3-4).
6. Threshold again.
7. Now you have a black and white version that is a bit thicker than the original. TURN THIS INTO A LAYER MASK. An inverted one which passes through the writing, and renders everything else transparent.
8. Apply this mask to the original image. This requires transfering a layer mask between layers.
9. Slide a white background under the masked layer. Now you have the lettering clean on white.
10. Play with simple Color->Brightness-Contrast. I ended up with something like brightness -66, contrast +88.
In the final step, because of the layer mask that is in effect, these controls affect only the writing: the white coming from the unaffected layer below stays white no matter what you do with the contrast and brightness controls.
Why the different approach: I first tried the original approach and the result was good. But I thought you wouldn't like it either. It was similar to Matt's. I did a better job of eliminating the grid, but the writing was less vivid. (Likewise I also preserved the yellow tint of the paper.) I wanted the grid completely gone, with vivid writing. Playing with the intensity transfer curves was not quite doing it; there was poor separation between vanshing the grid while preserving the ink.
This attempt can be seen here: https://imgur.com/a/ldrBN The green writing is particularly unsatisfactory.
In particular my use case is cleaning up pictures of whiteboards, where the brightness from the room is not as constant as a scan, and the approach wouldn't work at all.
Probably an easy PR for that repo though?
It is also more robust than k-means. The author's algo will only work on scanned images. Photographed pages from a book will often have a slight shadow on half the page from the curvature. Blur-divide will clean this up. K-means will think you've used a lot of gray and not figure out that there are multiple background colors.
It's a great file format for space-efficient archiving of scans like that, with a bit of scripted preprocessing.
I think PDF can cover pretty much the same ground with JBig and Jpeg2k. (And I believe archive.org is doing that.) But I don't know of any open source code to do the segmentation / encoding. (You have to split the bitmap from the background for jbig / jpeg encoding.)
Whether that makes the format "widespread" enough for your use case is of course your decision to make.
I never really understood why they were so many lines...
[1]: https://images-na.ssl-images-amazon.com/images/I/815WQQdAHBL...
[0] https://getfrenchbox.com/what-is-french-ruled-paper-seyes-sc...
sudo pip install --upgrade --ignore-installed --install-option '--install-data=/usr/local' numpy==1.9.0
The rest of the options allow pip to soft-override the macOS built-in numpy 1.8.0 which is immutable in the /System/ directory.
Anyway, after I did all that I was able to start playing with the app, I had previously been using a kludge workflow to get a nice output in black and white by using the imagemagick convert -shave option to remove the scanned edges of images, then doing a -depth 1 to force the depth down (which only works well on really clean scans), then I can -trim to clear the framing white pixels and re-center using the -gravity center -extent 5100x6600 to frame the contents centered inside a 600dpi image.
Rough but it works, I was hassling with trying to isolate "spot colors" for another thing, but this might actually do the trick!!!
I totally identify with the need for this. I also want to archive images of notes and whiteboards, and they must be kept small as so far my life fits in google drive and github.
Currently I use Evernote to do this. I don’t use any other functionality in Evernote but the “take photo” action does processing and size reduction very like the blog post.
In your scenario, since you have assigned "scribes" taking the notes, you might be able to streamline the process with a "smart pen."
There are several on the market. The one I got as a hand-me-down from a family member lets you write dozens of pages of notes, then Bluetooth them to a smartphone app that exports to PDF, Box, Google Drive, etc... Or it can actually copy the notes to the app in real time. Combined with a projector, this might be useful for the other students during class.
It's supposed to be able to OCR the notes, too, but I haven't bothered to figure out how. But there's a cool little envelope icon in the corner of each notebook page that if you put a checkmark on, it will automatically e-mail the page to a pre-designated address.
Again, there are several models on the market. Mine retails for about $100. Notebooks come in about 15 different sizes and cost about the same as a regular quality notebook.
Just some thoughts.
I just wish its price would come down. It's still full price from four years ago :| and even getting more expensive because it's so old
However, I prefer the Wifi-only model. I neither need nor want cell service on my tablet. Not only that, but cellular providers end up installing complete garbage for their software.
I recently broke my first device. I bought a new one that was advertised Wifi-only. The received device was branded for Verizon and was literally and completely unusable without a Verizon SIM card. I ended up getting it replaced with a T-Mobile one which... while isn't not completely unusable, there's still tons of crapware from T-Mobile that I cannot uninstall.
It makes me really sad.
I'll have to check out your link at home. Thanks!
Technicalities: 1) the S Pen's sensitivity is much closer to the image. Microsoft Surface, in contrast, feels and appears about 1/4" above the image: that causes me to mis-write a lot since I feel the stylus at a different location from where the drawing is being applied. I don't know about you, but I don't write with my pen/pencil/stylus straight up 90 degrees from the surface being written upon which is the only way I could reliably use a Surface for writing.
2) the Galaxy Note 2014's resolution is higher than the Surfaces I've used even though it's on a smaller form factor. That means that the image simply looks sharper.
3) the S Pen uses a different tech so that the Galaxy Note is able to recognize the difference between my fingers (or my palm) resting on the tablet's surface vs the stylus writing whatever's being written.
4) Both of the Galaxy Note 2014s I've used have had a battery life of about a full week from a full charge. It does depend on how much you use it, of course, but typically it's about 4.5 days, dying right at the tail end of Friday. That's with writing notes for about 2 or 3 hours during every workday. Contrast with Surface 3 and Surface Pro ... one had lasted a day and a half on average while the other lasted about 10 hours.
5) S Note translates my handwriting with about a 90% accuracy. But OneNote translates my handwriting to literal garbage text for about 50% of the time. When it doesn't translate to garbage text, there's of course the common typos from thinking I->l->1 B->0->O->o->D->etc (which S Note also has trouble with, but not as much).
Opinions: I absolutely despite Microsoft's and Apple's OS offerings these days. I don't like Android, either, but it's the least hated of the three. I use Windows 7 at home for games but otherwise Linux the whole way. Linux allows me to actually customize the UI the way I want; when it doesn't, there's the good ole' command line.
I've found the S Note app to be fairly intuitive and straightforward. It's really easy to export notes to PDF or email, eg to share with coworkers. If you can get around OneNote's shitfest version of OCR, I'll admit that the rest of it pretty intuitive. I really didn't care to learn (yet another) different way of doing things though.
If I had _one_ complaint about the Galaxy Note, it's that I have yet to find a way to have it require Active Directory to unlock; so if that's a requirement then ~~your company sucks~~ you're SOL for using it at work. If someone knows of a viable way to login to an Active Directory account on Android, that'd be awesome.
I also have to say that I actively use Google Play Books. Good luck getting that to work on Surface or iOS outside of ~~Chrome~~ a browser; I never could anyway. If I have to log in to a site using a browser and use the browser to use the site instead of using native apps, then why not just use a Chromebook?
1st: I love the size. The 10s are a little too big to cart around in a jacket pocket, and the note phones are too small to be a decent notebook. (I must admit though, I'm rethinking this point, as I used to be a huge moleskin fan)
2nd: The Samsung s-note app from that time is awesome! Honestly, every time samsung comes out with a new pen based tablet or phone, I always check it out. And I'm looking for 2 features: a: the ability to import a pdf file & b: goddamned smart shapes! I suck at drawing, and so I love that on my 8.0 I can draw a shitty square or circle and the software will make it look all ship shape. But they seemed to have removed that feature from newer versions of s-note, and I haven't found a 3rd party app that can do just those 2 things.
Any pointers to an app that will allow me to import pdfs and draw on them using smart shapes would be awesome!
You can remove the background very effectively by dragging a curve with preview.
You almost always need to preview and adjust some parameters, unless you have a template for similar cases.
Anyway very nice writeup and I will add it to my arsenal and give it a closer look later. Could be useful for my document archive+ocr solution.
Edit: too bad seems like it didn't see any activity in the last year
That's not necessarily bad: sometimes a piece of software can be done, or nearly so.
(which doesn't mean that it's a bad project or that I wont use it. it means that I will probably start to work on it)
More you can find at https://medium.com/@mukeshkumar_46704/digital-files-ingestio...
https://docs.opencv.org/3.4.0/d7/d4d/tutorial_py_thresholdin...
It’s also interesting for me to think about how this is a generalization of converting a scan to black and white for clarity :)
It seems to me that the inaccuracies/inefficiencies/errors/whatever you like in using RGB are basically truncated out of existence by the very, very harsh binning that is occurring. I wouldn't expect any visible differences to emerge from any alternate color space.