Hacker News Reader

In the case of this system (a quasi-EDI interface used to move records from a fleet fueling point-of-sale system to the ERP software) those characters were forbidden by the source application. My code would have exploded in a fireball if they had been present, but the specification said they couldn't be.

If it's purely binary data, then you can't.

Otherwise you need to have some sort of escape mechanism, exactly like quoting strings in CSV. In fact, there's an ASCII code "ESC" for entirely that purpose. :)

The problem is that those characters are non-printable, which means if you're just dumping the file out somewhere, you can't see them.

Easy - don't

Same as any separator. Either it's not in acceptable set of non-separator input or there's an escape (that can also escape itself for the literal).