Hacker News Reader

Comment by noisy_boy | Hacker News Reader

> This is not an OCR problem (as we switched off OCR on purpose), it is a lot worse – patches of the pixel data are randomly replaced in a very subtle and dangerous way: The scanned images look correct at first glance, even though numbers may actually be incorrect.

I have a habit of scanning invoices/statements/manuals etc to pdf and found the above quite alarming. Though I don't use Xerox (my scanner is Epson Perfection V39), I never considered the possibility that a scan of a document can result in altered numbers. How do I know if my scanner could also be affected in case it uses similar underlying software/setup?

View on HN

You should be safe if your scanner uses either JPEG, lossless JBIG2 or other lossless compression.

If however it uses lossy JBIG2 compression, it is quite possible that it suffers from the same problem, due to how lossy JBIG2 compression works.

TFA suggests that Xerox was offering the option of lossless JBIG2 compression but they implemented it incorrectly -- they didn't include the delta blocks that described how individual image blocks differed from canonical patterns.

Which brings up another point: How did Xerox's SW development process take place? Was it done in-house with frequent code reviews or was it outsourced to a low bidder and then thrown back over the wall with no real quality assurance process?

This isn't the kind of issue a code review would catch. Code review will find when the code isn't written to standards or doesn't look maintainable. It isn't going to catch a subtle misuse of a technique that should be valid.

the review should also be reviewing the tests, and you should really have a test if you have a standard.

> the review should also be reviewing the tests, and you should really have a test if you have a standard.

Codes reviews and unit tests aren't a substitute for QA or acceptance testing. It's great when code reviews do catch stuff like this, but that's not going to catch everything. To me this bug points to deficiencies in another part of the process.

One would have to know about this failure mode to write a test for it. Would you think to write this particular test?

I know nothing about this encoding but I would think each implementable feature of the standard should have some tests that you can implement. Maybe some sample data and then process it’s encoded counter part. I bet I am trivializing it but I would hope they are thinking twice about shoring up their process.

That depends on what kind of code review you're doing. Code reviews intended to find bugs are very good at finding bugs that would be hard to find by testing.

Who scans documents as JPEG, though?

PDF is widely more common and there are a million different ways you can write them. Including with JBIG2 images.

PDF files are often made up of JPEGs internally.

If you scan in color/grayscale, it can’t use JBIG2 compression. The machine will likely use JPEG for lossy compression, could result in less crisp scans, but it eliminates this risk.

Or you could use JBIG2 with reasonable settings.

From the original post: "numbers seem to be mangled independently from the compression mode, which makes the issue hard to avoid"

Reasonable settings meaning no compression setting which was mentioned to not have the issue

Thanks would love a 45mb one page PDF

I would love not to send a scanned document with different numbers to my accountant, or to the IRS for that matter.

What the author refers to are the three available compression modes on Xerox scanners, which are "normal", "higher", and "highest". It is not clear what that means exactly, but it is possible that all three of those are lossy compression modes.

In my opinion, the only reasonable mode for JBIG2 is lossless, as the lossy mode of JBIG2 is, in general, prone to these character mangling issues. File sizes for lossless JBIG2 compression are already very low, so I would claim that using lossless is almost always worth it, unless character mangling is explicitly not a problem.

The author was using default settings, I think it's fair to call the defaults reasonable

From the article:

> In the next section, there is a a short manual for you to reproduce the error and see of you are affected.

I wonder if this may be also a problem with Samsung MFPs, as some Xerox consumer models were apparently OEM'd by Samsung (e.g. Samsung SCX-4200 vs. Xerox 3119, or was it the other way around?). Not sure if there's a list which links the Xerox-Samsung similar models.

I guess, if the scanner offers JBIG2 format option, it may need a test at least.

Xerox purchased a number of small desktop and larger machines from Samsung, including the one you mention.

Keep the originals.

Reply on news.ycombinator.com