Realtime Web Messaging over Animated Gifs | Hacker News Reader

Realtime Web Messaging over Animated Gifs

github.com

576 pointsold_sound106 comments

Open articleView on HN

I am convinced HN is the Pinterest of software.

Someone made a coffee table out of old crates? Pin. Someone made a fence out of old wood pallets? Pin.

Someone made a realtime messaging library out of animated gifs? Upvote.

Someone made a logic gate out of crabs? Ignore :(

http://news.ycombinator.com/item?id=3831987

Wow, they found an even messier substrate than water!

... - transitors - vacuum-tubes - water - crabs - ...

Someone made a Turing machine out of Magic: The Gathering cards. http://news.ycombinator.com/item?id=4506865 Even more ignored.

Both got decent upvote counts

They do now :) The MtG post had about half as many upvotes when I posted the link, and I think the crabs one got an even bigger boost.

This made me laugh. Perfect follow-up.

That's the way to get the good stories onto the front page.

I'm kind of OK with that.

It's a hack. HN is for (among other things) news about hacks.

It's not really a hack. It's slowly printing a GIF for a kind of really limited server push. If you want to send someone messages via server-push in an old-school way, use multipart/x-mixed-replace.

A hack would be taking the input of a TV tuner card, converting the video to .gif frames, and printing that out slowly enough to view a slow video over a DSL line to watch TV without a media player.

Maybe not a great enough hack for you, but it's still a hack by definition; using something beyond it's intended purpose.

It's not really a hack. ... A hack would be taking the input of a TV tuner card, ...

No True Scotsman: Hacker Edition?

No, a hack would be doing all that while building an abstraction layer around Google Calculator to act as a primitive CPU to do the GIF encode.

It seems a bit arrogant to say "I don't like this particular article, therefore the community is primitive." An Animated GIF messaging library is certainly of interest to a majority of hackers.

I didn't say anything of the sort.

I simply made an amusing (to me) observation. I had earlier thought Pinterest seemed like MacGyver for people into crafts (at least that's the kind of thing my wife kept getting excited about). That's not to disparage HN or Pinterest or MacGyver for that matter.

As a die hard McGyver fan, I'm offended you'd compare him to a subset of stay at home mom's pinning pictures of other women with flat abs as "motivation." Ok, occasionally they post some pretty cool stuff, but mostly it is the former.

Someone wrote blogspam about how fences made out of old pallets are a bit of flash in the pan? Pin.

> sadly we are in mid September here in the northern hemisphere.

I am in the south and can confirm we are also in September, will report back with news.

/snark

I'm from the south as well ;)

I really hope this statement by the project author was intended to be funny (as opposed to a serious belief that there's another calendar for the southern hemisphere.) I laughed. And then I realized I know people who would find this epiphanic and insightful.

I'm fron Uruguay ;)

Of course if we did add 6 months (mod 12) then we could have hot July (almost) everywhere.

And an australian christmas in June, but it would be worth it!

It's discontinuous. To make this work, stretch the months (add/subtract days) so the tropics only have July August and poles only have Jan/Feb. Adjust holidays appropriately.

And you can add/subtract hours and minutes to days so that the sun rises and sets and the same time all year!

See: Riyadh Solar Time.

We had a white christmas in Australia... in July! ;-)

The world is flat, didn't you know?

Our Canadian office managed to convince a couple of team members in our American office that we ran on a 20 hour clock, with 72 minutes in an hour. That was fun for a week.

Metric time!

As a remarkable coincidence, another hack that Ka-Ping Yee did once was writing a nonstandard clock for his Palm. He spent a summer living 28-hour days: six 28-hour days per normal week, going in and out of phase with the sun once a week. So he needed a way to remember when to go to sleep, when to wake up, when to eat dinner, and so on. When it's 25:00, what do you do? We have a lifetime of intuition built up around clock times on a 24-hour clock.

So he wrote a digital clock that used standard-length seconds, but 70 of them per minute, 60 of those minutes per hour, and 24 of those hours per day. So when it was 10:30:59 by his nonstandard clock, he was at the phase of his nonstandard day that corresponds to 10:30 or so of the standard day --- but the clock would then roll over to 10:30:60, 10:30:61, ... 10:30:69, 10:31:00.

It was very amusing to see people's reactions upon watching the clock for a little while, especially at the time of week that it was more or less in phase with normal time.

I've seen this technique once before... Ka-Ping Yee built a demo that did this back in 1999: http://zesty.ca/chat/

It's an awesome hack, cool to see it being rediscovered/reinvented after so much time!

That's the chat I refer on the CREDITS. Thanks for the link I added you and the link there.

A friend of mine wrote a Citrix ICA client using this idea and a server-side image map, IIRC. It was actually usable.

For a while (they've changed clients a few times but it still might work) the last fallback of LogMeIn remote desktop software was a gif of your desktop in an image map. It was surprisingly usable!

I've always thought of that, but never an animated gif. It seems brilliant!

Also some early webcam/security-cam 'zero install' stream-via-web-pages used this technique, if I recall correctly.

I believe they're all (most?) MJPEG: http://en.wikipedia.org/wiki/Motion_JPEG

Gojomo has it right, the very first streaming cam used a GIF animation that never ended.

This is awesome -- you never know when a solution like this might come in handy.

Wayyyyy back in the day (NS4, IE4 day) I used the width / height of an image the browser polled every few seconds as a transport mechanism... the only other option (refreshing a hidden frame) caused an irritating "page refresh" clicking noise. This was before XMLHttpRequest obviously and was enough bandwidth for our needs. It worked so well that I believe its still being used in production systems.

I haven't looked at the javascript generated in this animated gif solution, but I assume that it does some stuff that wouldn't work in the pre-IE6 browsers. It would be extremely amazing if it did though.

This doesn't use any javascript

Ah, I just realized that this simply prints text out in a gif... it doesn't encode text in the gif and then decode it in the browser into text that javascript could manipulate. Too bad. I wonder if that's possible...

It is possible, as you can now get the raw pixel data of an image by copying it to a canvas and using getImageData.

    <img src="test.gif" id="img">

    <script type=text/javascript>
    var img = document.getElementById("img");
    var canvas = document.createElement("canvas");
    canvas.width = img.width;
    canvas.height = img.height;

    var ctx = canvas.getContext("2d");
    ctx.drawImage(img, 0, 0);

    var imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
    </script>

Then you can do whatever you want with the pixel data.

Unfortunately not on IE6, and this would require both canvas support and CORS. I think most browsers that support that also support true web sockets.

This would be a great step in some kind of rube goldberg inspired website.

Encoding data into images is certainly possible, albeit sometimes a PITA.

Awesome. I did this trick too, many years ago. I stole it from Liveperson I think.

When we were doing Comet in Netscape 4 and IE4 at KnowNow (in 2000, although Rohit, Adam, and Peyman got the technique working in 1999), we used an endless HTML frame, which simply never stopped loading. We'd spit out a <script> tag whenever there was a new message to deliver. There was an annoying clicking noise in IE, but only when the frame did finish loading because you'd lost your network connection or restarted the server.

This eventually got open-sourced as "mod_pubsub".

There you go, Facebook's BigPipe!

First person to turn a 90s animated GIF divider into an actual live progress bar with API wins the Internet.

I did something similar for cheap (insecure) desktop streaming a few years ago.

Roughly, use scrot (or similar screen capturing command line tool) to take a screenshot of the desktop and then encode it into a gif frame. Repeat once per second. Boom, your desktop is now a gif.

The main problem with this approach is that transmitting stuff via gif (low-color bitmaps, remember) is painfully slow even with modern internet.

That said, could probably be very useful in some instances!

It is possible to do some interframe compression with gifs[1], so one can potentially reduce the data sent significantly, especially if one is typing or not changing the whole screen. (And gifs aren't "just" bitmaps, they have some compression[2].)

[1]: "Some economy of data is possible where a frame need only rewrite a portion of the pixels of the display, because the Image Descriptor can define a smaller rectangle to be rescanned instead of the whole image." https://en.wikipedia.org/wiki/Graphics_Interchange_Format#An...

[2]: https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Wel...

No one has mentioned these specific use cases yet...

1) live charts and graphs of server loads

2) interactive maps (instead of loading new images, just append)

3) I'm also thinking of some kind of captcha, where the user waits for the server to show a certain image and then can submit a comment and the server would know which submits were valid based on timestamp... or something.

4) weather, temperature, stocks

5) collaborative drawing applet? (would still require ajax though)

(3) is only secure if no one bothers to write a bot for your site. It would be trivial to do so, however.

I admit that I have only thought about this for a minute or two but I am thinking of something a little more complex than a hard coded image.

More like: Hit submit when a picture of a dog is shown in the slideshow, if its a cat, your comment will be denied. And these images of dogs and cats would be picked randomly by the server from a google image search (...or something). The vulnerability so far is that a bot could just keep trying until it got in but I'm sure a crafty programmer could come up with an attempt limiter.

Still a half-baked idea, but the tech does offer some interesting possibilities. I particularly like that this may work when AJAX/JS is disabled or otherwise not an option.

I worked on CAPTCHAs at Microsoft Research in a past life, and I can tell you, most original forms won't work. In the case of the one you described, I as an attacker can just put it through Tin Eye to find the original, and then do some semantic analysis on the page containing the image, or event just the URL, to figure it out. I have working code for this as a proof of concept for Microsoft's Kitten Auth.

If that is the best attack on my idea, then I'd say that's pretty good! A spammer would have to target the site specifically (for tailor-made script) and parse an animated gif into individual images, then send every image into Tin Eye (which is awesome but the API isn't free right?).

That said, I am a complete amateur at captcha design. Cool research though and I bet a lot of HN'ers would be interested in your PoC if you haven't already submitted it.

Collaborative drawing would be possible without ajax. Just put a 1px image on the page and change its url to encode the drawing commands. Update the animated gif to show the users what was drawn.

So when you read about countries like Venezuela installing proxies in front of Twitter before an upcoming election [1], is there a potential to use this technique to tunnel information into areas that normally would suppress it?

[1] http://orvtech.com/en/general/gobierno-venezolano-elecciones...

They can always use http://priv.ly

This is better, partly because it requires no plugin.

http://fourmilab.ch/javascrypt

It would be much cooler if you were to send the video stream to the gif instead of the booring old messages. Very cool though. Just think if something like this with video was invented back in the days of IE6 , it would have been the skype of its day.

HN's jacquesm did just that, and made a lot of money doing so.

Could you expand on what jacquesm did ? Interested to hear.

    Founder of camarades.com / ww.com, creator of the first 'streaming webcam software'

see: http://news.ycombinator.com/user?id=jacquesm

He was bet a candybar that he couldn't make a gif that never ended. He did it, of course, and that became the first streaming webcam software.

That is extremely clever, this just goes to show that if you know how something works inside and out you can come up with clever hacks.

If realtime messaging is a new and cool thing, what were chat rooms back in the day? not realtime? Am I missing something? I don't remember needing reloading those pages...

All that aside, this is amazing.

A lot of those used a periodically refreshing frame or something similar.

I don't necessarily think they intend "realtime messaging" to be the focus here. We've had obvious IM for years and it didn't exactly lull as far as I know. Definitely cool that someone hacked around to learn the details about some bit of tech to make it do something it wasn't built for. I applaud this attitude.

That said, it seems that since the "digital revolution" we go through these re-inventions of basic ideas and tools quite frequently. We've had "email" since the time multiple people could use a computer; recently, I commented on HN about how someone reinvented server log analysis tools to find click fraud (they were outsourcing analytics and apparently were unaware that web server logs ever existed.)

Chat rooms 10 years ago weren't using web standards like HTML, GIF and Javascript. They were writing either in Java or in Flash.

However MSIE6 also supports XmlHttpRequest (though an Active-X Control), which you can also use for real-time communication. Just setup a XmlHttpRequest and let the request idle on the server-side until new data become available.

Wouldn't that lock up a thread, per request? Things would quickly grind to a halt I'm sure

Depends on your web server. AFAIK, nginx uses a combination of events and threads to handle a ginormous number of simultaneous connections.

See:

    http://en.wikipedia.org/wiki/Asynchronous_I/O

    http://www.slideshare.net/joshzhu/nginx-internals
    
    http://www.aosabook.org/en/nginx.html
    
    http://serverfault.com/questions/86674/why-is-nginx-so-fast

Also, Node.js.

Edit: formatting.

Only if your server code process each request in a separate thread. But that's also true, when using web-sockets or when using the GIF approach. When implementing real-time applications, you always have to write non-blocking asynchronous server code, for example using node.js or Twisted.

I get the humor of this library, but in reality wouldn't you just use HTTP 1.1 chunked transfer encoding instead? According to http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol, that would even work with your IE 2 users.

Would it be possible to use <canvas> for extracting pixel information as binary data?

I was thinking about doing something like that as well. To not send real images but some binary data. Dunno if it will work

> http://ernestdelgado.com/public-tests/gifoncanvas/

Kinda:

> if we add an animated GIF as the first source, every time a drawImage call happens it will render whatever the status of the GIF is at that very second

So if you call drawImage in a (timeout/setinterval) loop, you can get the frame's data, compare to the previous frame's data and if they're different you got a new frame-worth of data.

Considering that requires canvas support (with image-query capabilities) you're probably better off using websockets and the like.

"The awesome image that illustrates this page was given by the internet." That excellent gif is from "Tim and Eric Awesome Show, Great Job!" Just thought I'd give proper credit where it's due.

Does this work on the iphone? I expect this would be a great alternative to socket.io for mobile that doesn't support websockets or flash.

SockJS works basically everywhere.

Combine this with a client side JavaScript based OCR implementation, and you could even send TEXT in real-time.

Imagine the possibilities !!11!1

That's pretty amazing, and it's really too bad that this technique wasn't actually available years before... (or was it?)

It always was actually. I even tried this on IE6 and it works

I wrote the first (only?) GIF decoder for the Commodore 64 back in the day, including displaying animated frames. There was no 'live' delivery of images back then though; you had to download them in-full over FTP or a proprietary file transfer protocol before you started parsing. I suspect the encoding was too slow on most computers at the time to stream the file as it was encoded anyway. But yeah, the basic capability was present back then.

Once the WWW kicked off and GIFs started being served, and especially once cgi scripts started being written, someone could have had this idea and the software and hardware probably could have handled it. So we could have been doing this in the late 90's. I wish I had thought of it...

This is pretty cool. I don't know if I would use this in production, as people with epilepsy usually disable GIFs to protect themselves, so this tech would probably fail (usually people use an extension or set the browser to only load frame 1 of the GIF and stop).

Also, I can't imagine it would play well with screen readers.

Couldn't this be used to add another layer of security to a conversation? If there is a way to generate gifs on the fly that contained what you wished to say, it could be used to mask your message from basic text screening and copy/paste.

Thoughts?

Writing text onto an image is easy. Graphical hit counters are almost as old as the web. The first ones just lined up images with preprinted numbers on them, but as soon as CGI became popular, people had libraries to write the text on top of an image too.

It wouldn't provide any meaningful security; think of how difficult it is to make a CAPTCHA that is both readable for humans and hard to beat for computers. Adding security to a conversation is a solved problem in the technical sense (use encryption such as GPG), the only reason that it's not in widespread use is that people can't be bothered.

How exactly is this a better approach than multipart/x-mixed-replace, which is designed to push new messages from the server to the client in a stream until the server decides to stop?

does it work in email clients?

Question: Could this be used for screen sharing?

You can also use image streams: http://minipenguin.com/?p=647

The original chat system from 1999 from which he took inspiration was written by Ka-Ping Yee, not an anonymous hacker.

Well it was anonymous for me when I published this post today. Then some guy here on HN pointed me to the original link.

there is no client side code or am I missing something?

how do I decode my data from animated gif in javascript?

You don't, it just displays the image.

title is very misleading, on purpose I guess

if flash wasn't dying already I would have asked if this could have been useful to enable flash games or videos on the ipad... but the answer would have probably been "no" anyway

Very clever hack.

hacking at its finest. while real life use cases are debatable the implementation is very, very cool.

make a QR code stream, and decode it in browser.

I'm crying. I always wanted one of these. I'm so happy.

GIF... that's soooo geocities...

Very clever

This is just wrong and needs to die and go away. It's probably fun to read for the 1st minute.

From the example, ;; go and open http://localhost:8081/ in Safari or IE6

IE6, are you serious?

Reply on news.ycombinator.com