Show HN: HR code – Designed to be recognized by humans and OCR | Hacker News Reader

Show HN: HR code – Designed to be recognized by humans and OCR

github.com

124 pointsemrehan56 comments

The problem with this is that it's prone to error (doesn't have error correcting bits). Unfortunately, that combined with speed of scanning is really what's key for codes.

I have worked in the space, making some strides in speed & error correction.

Some of my public work is here: https://austingwalters.com/chromatags/

Your best bet is actually an overlay of two codes. A regular image (for humans), plus a code embedded in a color space (see linked post for how to do that).

> The problem with this is that it's prone to error (doesn't have error correcting bits

Doesn't the use of the latin alphabet act a bit like an error correction code here? In contrast to a pure binary code you know that only certain shapes are valid, which should allow you to do some guessing if you don't get a perfect match.

Doesn’t help if a whole cell is missing, whereas with QR codes chunks can be entirely gone without consequence.

Can you give some practical examples of where that would be useful? Honestly curious, don't know much about this space.

Physical sticker exposed to the environment. Scrapes/marks are common, they would render the “HR” code useless, but the QR would be fine. Or at least stay fine for longer. So the trade off becomes: do I post something that will be accessible to everyone, even those without a phone, for a week; or do I post something that will be accessible to everyone with a phone for two weeks. Given almost everyone has a phone, the answer is obvious. (Making up the times, but you get the idea)

A lot of applications centralize their logo in a QR code, obscuring a significant chunk. Yet the code scans fine and there is some nice custom recognizability for which service it is.

Maybe scanning from photos where the code is obscured, bad quality, almost-trashed physical codes.

"Guessing" and "a bit like" are key qualifiers here; error correction codes in QR provide a pretty hard guarantee and won't scan corrupted. With this scheme, who knows?

The whole point of having human readable code is to help avoid being scammed, so having a code embedded that a human cant read but is what actually used for the link defeats the purpose

Also, most likely all URLs would end up coded like https://go.to/abcdef

Edit: just found clicking go.to redirects to flock.com.

What if the data is human readable but the error correction is only machine readable?

There still might be room for scams if the error correction is too clever, maybe it could only be a checksum?

So what happens when someone makes an image that appears like one url but scans as a different url? goDgle.com?

Then that is a failure of the protocol

Thank you for sharing your chroma tags work.

Could the same technique be applied to QR codes? Would it improve QR read times?

At rush hour I have to wait for many people to scan their QR code train tickets, it is surprisingly slow.

Yes, the technique is general. Take a QR code and do the same manipulations and you get 2x scanning speed and 4x redundancy with no change in the algorithm.

I've actually improved it significantly since then, where it's possible to also store 2^8 data in something resembling a QR code, while keeping the same scanning speed as today.

This reminds me of the E-13B font [1] seen on checks since the early 1960s, but in a square format. The goals are similar, produce machine readable and human readable text. There have been attempts to also make a version with alphabetical characters as well in OCR-A [2] in the late 1960s.

[1] - https://en.wikipedia.org/wiki/Magnetic_ink_character_recogni... [2] - https://en.wikipedia.org/wiki/OCR-A

Checks are now often scanned optically, with the MICR line, the courtesy amount field and the legal amount field all being read by software character and handwriting recognition. There is a small security deficit, since a fraudulent check with a non-magnetic MICR line would not be detected.

> handwriting recognition

For decades, I'd write out the amount field as:

          29
    $1234 --
          00

as I was taught that was the correct way to do it. Suddenly, I was getting dunning notices saying I'd only paid $12.34. This happened simultaneously across different accounts. Clearly, the software had changed.

So now I write $1234.29, and everyone is happy.

Since it came up.. I've always wondered how mobile deposit (cell phone photo of a check) is remotely secure?

It's nearly as secure as going to the ATM to deposit the check and most banks are OK with the increased risk because they have extra identifiers (phone/app info) when you 'scan' the check. Which is to say the whole process is not terribly secure.

History wise it came about with the removal (or reduction) of check float when checks began to be processed electronically (turned into an ACH transaction with images of front & back being taken by the submitting bank). In the last TOS I read about check scanning what is actually happening is that your scanning of the check replaces the bank's scanning and they are using extracted data to create the ACH transaction. In the meantime they extend credit to you to match the amount they would normally advance you (varies by account and amount). It's been a while but I recall it being specifically credit as opposed to the usual 'available funds' at an ATM; this seems a distinction without a difference.

In the end - it's not secure or at least not much less secure than going to an ATM to deposit the check. The difference is that the ATM keeps the physical check for some small period of time such that if there is a problem with the ACH the bank might have physical possession of the check to assist in a fraud investigation (not that I think they'd need it as the law enabling electronic clearance I believe made the electronic images equivalent in all ways to the physical check).

It's not. But you're required to keep a copy of the check after you scan it.

This seems to be the worst of both worlds. It's not easy for a human to read a square (compared to a line of text). The pixellated font is also not easily readable compared to a vector font. It's also not easy for machines to read an optical coding with no spatially distributed redundancy.

QR codes and bar codes are brilliant for machines because misreads due to some spurious reflection or spec of dust is mitigated by error correction.

I feel like this problem is already well served by bar codes which have a human readable text representation below them (e.g. serial number stickers).

That said, I can see the security advantage of the computer reading the same representation as a human, although this is probably not the best place to enforce security. As there's no integrity check, there's little guarantee the computer will read what you see though. Maybe linear OCR combined with a barcode checksum would be a better way to achieve these goals.

The problem this is addressing is a code being impenetrable by a human. If your solution is adding a second (human readable) code beneath the machine readable code...you haven't addressed the problem. The user must still trust their reader to parse the code.

QR codes' reconstructability is a major strength that this lacks, but I'd bet there's a way to expand this to include ECC around it, much as QR codes can.

BUT...OCR is quickly advancing, so the need for a specialized code a specialized machine can read will diminish over time anyway.

As I suggested at the end, you could still employ OCR and have a barcode checksum. The checksum would ensure a misread of the human readable text would fail. As long as the checksum was not error-correcting (so could not be engineered to augment a correct OCR read), it doesn't matter that it is incomprehensible to humans, because the OCR is authoritative.

You could also implement the checksum as OCR-able text, although it wouldn't be as dense, and probably wouldn't help human readability.

I think ultimately it should not be trusted that a machine will read what a code appears to be. That should be enforced on the device: "Are you sure you want to visit malware.site?". It's also easy to manipulate computer vision; you can engineer patterns which will read as one thing to humans, but another to machines. In some ways it's better for these codes to not be human readable, such that trust is not misplaced, and the machine is used as the best source of truth.

I can imagine that it would not take long until someone comes up with patterns which are innocent in one orientation and malicious with a ±90 deg. rotation...

Not with QR codes, they read the same from any side

It's interesting, but I agree not easy on human eyes. I think plain old text in a clear font, with an OCR reader than can lift URL's out of any text, would be nearly as effective and gain traction quicker. Feel free to rebut me!

QR codes strangely don’t have error correction, you just have to guess which bits are uncertain and try combinations until it matches the checksum. There is no formula you run which results in the original data.

QR codes use Reed-Solomon codes for error correction, with various levels of error tolerance. This is very commonly used to place logos on top of the code and it still scanning correctly.

Guessing is still error correction of course. If you read the documentation you’ll find that practically Reed-Solomon code can’t correct errors if it doesn’t know which bits are uncertain. In the case of a QR code obviously the bits underneath the logo are uncertain. But if you just flip one bit it becomes much more difficult.

I’m no expert in this but the I’ve read source code for several QR code readers and they don’t have some neat algorithm where you input the bits you read and it calculates the corrected bits.

Perhaps something like that still exists, I’d be happy to see it.

If the app you are using to scan the QR code doesn't ask you for permission before opening the web page that's the problem with the app not the QR code...

Most QR codes are use to open a website and you do get a prompt for that at least on iOS so I don't see an issue there.

I need to ask, is this in response to the discussion in [1]? I see that the initial commit happened after that discussion started.

If so, that’s an impressively fast prototype.

1: https://news.ycombinator.com/item?id=21417433

Yes, I've made this after reading the post. Thanks.

I haven't read that particular comment though. It is obvious that QR codes are a security concern. First, QR code readers are required to ask for permission to open URLs. They might as well not implement it or could have malicious codes for misrepresenting the URL in specific cases. Second, I wouldn't be surprised to see some bugs exploited in QR code scanning code by giving them strange codes to scan. These would not look like valid HR codes but would look like just another QR code.

> QR code readers ... could have malicious codes for misrepresenting the URL in specific cases.

So could an HR code reader. It could just as well, when it scans a google.com HR code, prompt "do you want to go to google.com" and then open "phishing.free.hax"

That specific threat is ridiculous.

In addition, talk of exploiting strange looking QR codes seems silly too. HR code readers could also have bugs in their OCR code which would result in memory corruption or such.

Any type of encoding can have interpreting tools that contain security bugs or malicious behaviour. The only way to remove that is to create something that is not intended to be processed by any tools at all (e.g. a paper that users have to manually copy the text from by hand).

The biggest win with QR codes is their error recovery capability, though. This will just corrupt data if there's errors.

If OCR is required, then why not just put the link in plaintext? Makes it even easier to be read by humans.

That would be the next logical evolution from QR code once CV becomes more accurate for reading plaintext.

Some characters can be easily turned into others - c into o, C into D - by addition. I think it would be a good idea to make priors have an additional pixel or two to the font glyphs to make this impossible. Maybe even touching the border instead of the character.

I wonder if there's a font with the same idea of being easy to OCR. I mean something that one can write full pages of and be easy to read for both the human and the machine.

That's probably https://en.wikipedia.org/wiki/OCR-B - developed in the 1960s. (OCR-A might be more common but I'd say less human readable).

Interesting. I was expecting something with more machine visual aids, like square markers to easily determine the start, size, and angle of the text. I wouldn't be able to guess that this font was designed for OCR. I wonder what design considerations they took. From the look at the applications, maybe they assumed perfect/consistent reading conditions, instead of e.g. a casual photo of the text with little consideration to alignment or using a prepared format.

IMO, the easier and better solution would be for your device to pop up a ”Open <URL>?” alert.

That way, the detection of the URL will be more robust against dirt and damage, and the text can be displayed in a more ergonomic way (a QR code might be 5x5 cm, at two meters height, for example, making reading it by short humans with bad eyes a challenge)

Or are there use cases where the device reading QR codes doesn’t have a display?

User confirmation is a feature in most of the QR code readers. But it’s an optional feature. We are also depending on QR code scanners not to use an ambiguous font for URLs or misrepresent URLs in certain cases maliciously.

By encoding possibly malicious URLs in QR codes and scanning them we try to mitigate the security concerns by having not enforced or even not specified requirements from the QR code clients. Such as a display, user prompt, a fitting typeface in the UI. All this could be optional with a solution like HR codes, with the added benefit of deciding whether to scan a code nor not by simply reading it first.

But making QR code’s readable by users doesn’t protect against that. Whatever the QR code looks like, you have to trust the scanner. It could open “evil.com” every tenth time you use it, roll a die to determine whether to do so, contact its host to ask ”user u scanned QR code q. What URL shall I open?”, run a quick auction for the URL to send the user to, etc.

What _does_ help is just printing the URL in text, and requiring the user to type it in their browser’s address bar (assuming they trust their browser)

The qr code has three distinctive squares on the corners, which makes detection in images accurate. However, the proposed approach is missing that feature. Thats why I wouldn't call it an easy ocr problem, more like scene text detection, which is acctually a hard problem.

This is awesome! Feels like there's a kernel of a very interesting idea here. Could see how this evolves into a successor for QR codes. Looking forward to see where it goes.

The name is bad. "HR code" is immediately associated in my mind with "HR" as in Human Resources. You should pick a better name.

Indeed, I had the same expectation that it was something related to Human Resources, and it's not like the H replaced a word that means "machine" or anything. It could have been "HQR code".

I like the idea. Having symbols is hard because you have to know the generally accepted name for that symbol in order to tell it to someone.

This is pretty awesome! I just read a related thread on HN a few hours ago on this very issue, so big kudos for the time of execution.

Is it possible to make a valid QR code that is human readable by manipulating the size, weight, greyscale?, and positioning of letters?

Wow, I didn't expect this post to have any traction. Many thanks for the diverse feedback.

This is just a PoC of an idea I've created after reading https://news.ycombinator.com/item?id=21417026. But who knows, a human readable QR code alternative could born out of this.

When I looked up for this idea, I have found out http://hrqr.org/ but didn't find it much readable. Thanks to the comments here I came across OCR-A, OCR-B, and MICR just now. From these I've found Westminster typeface: http://luc.devroye.org/fonts-48273.html

Do you guys think using a font inspired by Westminster be a better choice than the prototype we have now?

In any case, the font should be resistant to malicious tampering such as creating an "o" from "c". Manual tampering could be also prevented with trailing checksum images that could be more information dense than the characters.

Error correction concern is the most common among the comments here. Yes, HR code readers would need to have error correction implementations to be reliable. Since each 7x7 grid of 2^49 binary options could only encode one of the 85 valid characters, HR codes could be recognized with even large chunks missing. On the other hand, implementations would be much more complex than the QR code error correction algorithm.

I reduced 3 corner blocks of QR codes to 1. This would make detection of HR codes much harder.

OCR, image recognition from video feed and average phone camera and processor has advanced so much in the past years that I think these technical costs could be favored for a human readable QR code alternative.

> If the app you are using to scan the QR code doesn't ask you for permission before opening the web page that's the problem with the app not the QR code... (wfdctrl's comment)

You're right. If the protocol is not secure enough, then the layer above must be secured enough. But it's better to have the security at the protocol level. Here's the other reply of mine under this thread where I speculate about possible security issues with QR codes: https://news.ycombinator.com/item?id=21424988

I'm open for a better name than "HR code" by the way. It is too silly that makes it recognizable though.

The font and the way to break words is not quite human readable IMO …

I guess the keyword here is "recognizable". The goal isn't to make it extremely readable, but make it both readable by a machine and sufficiently readable by a human.

What problem does this solve that MICR couldn't do better?

Reply on news.ycombinator.com