Show HN: Cimbar – File transfer via color barcodes and the Android camera | Hacker News Reader

Show HN: Cimbar – File transfer via color barcodes and the Android camera

github.com

87 pointssz326 comments

Hello all,

You've seen QR codes, maybe Microsoft HCCB [0], maybe jabcode [1] – well, here's a prototype of something new for the pile. :)

I saw txqr [2] a while back and was impressed, but also curious about how much throughput was possible with animated bar codes. I may have gotten carried away in my research into the question.

cimbar is a single and multi-frame color barcode format using reed solomon (for now) and wirehair [3] for error correction. Files are encoded into an animated series of bar codes drawn to the display. Files are decoded by an Android app [4] with its camera pointed at the cimbar code. This works with all antennas off, e.g. in airplane mode, because it's only using the visual data channel.

I ported the encoder to wasm, because I could: https://cimbar.org

Sustained transfer speed is currently on the order of 800 kilobits/second. So it's not very practical for files larger than a couple of MB, unless you have a lot of free time. :)

[0] https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode

[1] https://github.com/jabcode/jabcode

[2] https://github.com/divan/txqr

[3] https://github.com/catid/wirehair

[4] https://github.com/sz3/cfc/releases/latest

As a possible tangent, convert those images to inaudible audio to transfer files over the air: https://github.com/ganny26/awesome-audioqr

(air-gapped computers be damned)

Well, there's diminishing returns in format shifting. The encoded barcode contains various types of quasi-redundant visual information (e.g. error correction codes) to allow decoding to happen, so for audio-based transfer it'd be better to skip the image encode and blast out the file directly.

That said... given the somewhat remarkable way fountain codes work, there's nothing stopping us from having a protocol that uses the audio and the video channels simultaneously for better throughput...

Would it be beneficial to use one channel (either audio or visual) to transmit the information and the other one for responses like acknowledge? So kind of like TCP over two different channels?

Well, as for acks specifically -- cimbar itself doesn't really need them, thanks to fountain codes [0].

But I can imagine a reverse (request?) channel being useful, if it had enough bandwidth for the desired application. :)

As /u/ggerganov notes elsewhere in this thread (with some expertise on the audio side -- I can't claim any), the bandwidth of any audio channel is probably going to be pretty bad.

edit: Notwithstanding how viable of an idea it might be, HTTP over audio+video would be pretty neat. :)

[0] https://en.wikipedia.org/wiki/Fountain_code

I don't think air-gapped audio transmission could ever reach a fast and reliable transmission so that it allows transferring files in reasonable amount of time over reasonable transmitter-receiver distances. It's just too many hardware and physical limitations for this approach.

Having said that, I am actually working on a small library for data-over-sound which can be used for small data chunk transmissions across the room [0].

[0] https://github.com/ggerganov/ggwave

Amazing! Do you know any variants of these which squeeze out the maximum transfer rate over a high quality connection? Thinking about data transfer from a VPN/RDP remote desktop where I want to grab the screen output on the client with very little loss in quality (basically only limited by the chosen video encoder).

Using it to transfer files over a remote desktop connection is like a graphical version of zmodem [0].

[0] https://en.wikipedia.org/wiki/ZMODEM

I don't think I do?

I'm not 100% sure I understand your example. Is it RDP server (remote) -> RDP client (local) -> cell phone?

You might want to have a picture on the main README as in https://github.com/sz3/libcimbar/blob/master/DETAILS.md. Had to fish for a picture...

Probably a good idea. Noted.

Very interesting but couldn't get it to work with my smartphone. Even with a 250Ko image file, I got bored before it finished ! My camera may be too low quality and/or my monitor too small.

A barcode size parameter may be useful (even though it reduces throughput) so that it is easier to scan on all devices.

Multiple sizes isn't a bad idea. I'm not sure I know enough about what sizes would be appropriate though. :)

Transfers getting stuck like that usually is an indication that the decoder can't reliably find the 3+1 corner pattern. I might add a toggle to the decoder app that trades speed for more reliability.

That said, there are some failure modes that are harder to fix: I had a mysteriously slow transfer that was driving me nuts, until I noticed that my mouse cursor was on top of one of the corners.

I'm a HUGE fan of air-gap data transfer, and this is very nearly what I want.

Curious though, can't you make a web app to record the video, too?

Definitely. I had to draw a line somewhere for an initial "does anyone care about this?" release, so there's some rough edges here.

There are a number of loose ends that I'd like to look into down the road, and more flexibility for different use cases is on the list.

edit: I should mention: the main reason the decoder is a native Android app and not a WebAssembly app is that decoding performance is a throughput bottleneck. I wasn't too eager to pay the wasm tax, when I wasn't sure if even native performance would be good enough. As it happens, I now think decode performance is going to be ok -- but it's still a bit of a weak spot in the scheme.

I tried the browser encoder and here is what it looks like:

https://youtu.be/RkpVhW7SuRY

Wonder if someone can decode the .mp4 file from the video. It's not rickroll, I promise ;-)

I like the way you think. :)

Though I think 720p is towards the low end of resolutions this'll work on -- the cimbar code itself is 1024x1024, and while there's some wiggle room (hamming distance between the symbols), in my experience it gets pretty iffy below 900x900.

Also... I should probably update the overlay to auto-hide when a file is selected -- it helps a lot to have the full color range.

Is there a video of this in action?

It's simple to try out by uploading a file via the web page at https://cimbar.org

I have no android device.

I just dragged a file into the site on my desktop computer and it worked. OP said the site uses the wasm implementation so any browser with javascript enabled should be able to encode files.

Decoding needs the Android app.

Hmm... no, but that's a good idea!

Is it really phone CPU performance that limits the bandwidth? I expected it would be the camera.

Right now, it's both.

The decoder has a lot of image processing work to do (intriguingly well-suited for the GPU), and also lots of popcnts. I've optimized it a fair bit, but there's probably some tricks I still need to learn. It turns out that mobile processors don't like heat very much, and blow out their cache much quicker than you'd hope. :)

In the long run, I think your intuition is correct. The hard physics of the camera constraints (exposure time, etc) will put a hard upper bound on FPS+fidelity, and thus bandwidth.

I heard ordinary QR codes can hold up to 3KB of data.

How much data can this one hold?

The simple answer is:

4-color (standard): 7500 bytes

8-color: 8750 bytes

monochrome: 5000 bytes

Those numbers are with the standard, fairly high ECC setting (~20% of the image) that I settled on for video-based transfer. If we want to be more aggressive and use half the error correction:

4-color (standard): 8400 bytes

8-color: 9800 bytes

monochrome: 5600 bytes

Reply on news.ycombinator.com