Twitter's GIF hack
blog.embed.ly
blog.embed.ly
"A GIF is literally a sequence of independent images squeezed into the same file. An mp4 video can take advantage of all kinds of fancy compression techniques like keyframes and forward-predictive frames."
That's not why the mp4 is smaller. A GIF is not [just] independent images, you can have each frame reuse pixels from previous frames.
The mp4 is smaller because lossy jpeg compression is used on each frame, while the gif stores each frames losslessly.
This has implications from the predictive frames as well - the jpeg is lossy, so two almost similar parts of the images can be considered "the same", while the gif is not, so they must be identical in order to compress that way, and the realities of video noise makes that unlikely.
GIF basically rewrites completely a part of the picture, while the remaining part is reused. And if you consider that a rewrite of a small part of the frame to reuse the remaining part from the previous frame is just displaying a new gif on top of a part of the old one, the you can consider that it is a sequence of independent images. Maybe they don't all have the same size and are not meant to be displayed at the same position, but they are independent in the sense that you don't need a previous or future frame to display one frame.
And I wouldn't refer to GIF as lossless, except maybe if your original animation used only 256 simultaneous colors per frame. You lose information by down-sampling your colors.
GIFs store deltas, true, but they are still raster images.
H.264 on the other hand contains specific techniques for compressing sequential frames, including things like recording only pixel motion for some frames. You can see it in those "corrupt movies" gifs (which on twitter would be mp4s... talk about irony) where you take a clip and remove some keyframes from it - then transformations are applied on the wrong blocks and you get really weird things like someone's head splitting open and stuff flowing into each other.
Additionally keyframes in movies are usually compressed with even more loss than standard jpegs. In most cases only the Y channel is recorded at full resolution and the two colour channels - U and V - are recorded at half.
There is also motion-jpeg which really is literally a sequence of jpeg images.
H.264 also defines such modes, but no consumer hardware decoders support them so they aren't commonly used.
Some years ago there were multiple efforts to produce animated PNGs. There was MNG (Multiple-image Network Graphics, which could contain both PNG and something called JNG) in 2001, and more recently, APNG (Animated PNG) in 2008. The problem here is the spotty support for these formats. AFAIK MNG is effectively dead. I thought APNG was as well (with Firefox being the only browser to support it), but interestingly, the Wikipedia page claims that Safari 8.0 (the upcoming version in OS X 10.10 and iOS 8) supports it. Chrome, Opera, and IE don't, but there is a Canvas-based library for rendering APNG which could presumably be used to polyfill support if you wanted to experiment with APNG.
In any case, the point here is that something like APNG probably has much better compression than GIF while still being lossless (I haven't done any actual comparisons here, but the PNG format itself has support for a lot of fancy compression stuff that GIF doesn't).
---
Reading a bit more, I'm actually really surprised to see that Safari 8.0 apparently supports APNG. It seems that the PNG Group officially rejected APNG back in 2007, which means that libpng will never support APNG.
With a good palette (which most software is terrible at) and proper delta frames (without artificial differences created by blindly applied diffusion) GIFs can be 1/4th of the size you get from an average GIF maker.
Still, it's a terrible video codec.
I think trauma caused by awful plugins like RealPlayer and QuickTime made people stuck thinking that "animated images" and "proper video" are completely different things. But with native video support in browsers IMHO there should be no divide between <video> and GIF at all. It's the same thing, except the latter wastes more bandwidth, needs more memory or CPU (to enable seeking, used when you switch tabs) and can't be hardware-accelerated.
Gif is pretty bad, yes, but somehow I can browse pages with thousands of gifs. I can't browse pages with thousand of embedded videos.
That's true, but it's not limited by compression, only by assumptions/optimizations in the browsers.
For example browsers try very hard top stop decoding GIFs that are not currently visible on the screen. This could be done even better with a properly seekable video codec, but it's just not implemented.
One thing you can do with video that you can’t do with GIF is programmatically describe motion — shift frame to the left, for example. GIF, you’d have to re-draw it all.
I forsee a future where all video clips on the net have been converted back and forth between gif and video so often that they all slowly merge into a single amorphous blob of greyish-brown pixels. Actually, now that I think about it, that would almost certainly be an improvement over the current situation.
Now, all the browsers are pretty good at dealing with video, the codecs are hugely improved (although, there's some headaches with IP) and the plugin is dead.
Nobody ever actually wanted gifs. They want short, small videos that are guaranteed to work in their browser and that they can be pretty sure aren't going to have sound. We can now deliver that.
They want moving images that load quickly. Which has meant short/small/gifs, because most US bandwidth still sucks. And thanks to the mobile explosion it now means short/small/h264.
But I bet that as people become used to 5-10x space savings from h264, they'll quickly respond with videos having 5-10x more data. (larger/longer/higher-res clips)
Also I can't right-click-save-as.
- starts playing automatically
- loops seamlessly
- never has sound
- no logos or buttons like "share" and "embed"
- no scrubber bar on the bottom
- repeated instances of the same gif play back in lockstep
- no frame around it
- pixel-perfect control
It's always been easy to create a looped animation format that combines the feature list above with a better compression scheme. Now that gifs are so popular, someone recognized the need and made one.
IMO, the 256-color dithering was more of a necessary evil for most gif creators, although some took advantage of it. It looks nostalgic on 90s Gourard-shaded untextured computer graphics. But for movie clips, etc, I think many will be glad to get rid of it. I bet we'll see a sect of gif creators who think mp4s are not authentic while most people won't care.
Your last comment reminds me of Alvin Lucier's "I Am Sitting In A Room" (wiki/youtube). Someone repeats that idea with every lossy medium we invent. I've seen jpeg and vhs examples but I can't find the links right now.
- They are dead simple to save (or link to) and drop into your own page/blog/comment.
I'm not sure that's always true, as there some very noticeable problems with GIFs. Most significantly they're often huge, and must be entirely loaded to play properly, which can result in long delays before they can be viewed (usually with no obvious indication to the viewer) and massive resource usage (besides the problems related to size, animated images also usually don't benefit from the sort of acceleration and optimization that videos do). A page with a bunch of embedded GIFs (not at all uncommon), can completely kill a browser.
[There are other problems, of course, e.g. the complete lack of user control over playback, but the size of GIFs seems to be one of the worst.]
- no choice paralysis
- dead simple
- can have sound
- easy to shoop
- drag/drop
- cut/paste
I'm under the impression that the format only specified 256 colors per palette, and that you could use different palettes on a per-frame basis, including frames with 0ms delays between them. and I know some software existed to make this possible, though I don't recall if it made it possible with animation at the same time...
Newer (current) browsers have lowered that to 6ms, and in chrome, 3ms. As yet, there has been no browser that respects the 0ms delay (as in, supposed to display instantaneously, so 2 frames with 0ms delay is supposed to look like 1 frame).
The justification for this is backwards compatibility with incorrectly authored gifs. So .. there you go.
Here's a cap: http://imgur.com/2R84ImD
http://i.imgur.com/NleqJTD.png
Another example of "use bleeding-edge tech or go fuck yourself" from the modern web.
At this point, desktop configurations that can't play mp4s are at risk of being considered "broken."
I'm with you in spirit, but in practice I think we can all understand why people often just go for H.264
The tl;dr is that the support needs to very near universal for this to work, and no one is interested in WebM. Google probably could've forced the issue with YouTube, and they initially claimed they were going to, but they chickened out for some reason. I've heard it's because VP8 didn't live up to expectations and that they'll renew the push when VP9 is done, but whatever the motive, the reality is that if you want HTML 5 video to work, you must use H.264, as even Mozilla has been forced to admit.
As the article points out, the video files are generally much smaller than the source gifs.
Similarly, while there are real impediments to transcoding for many developers, I'm confident that the Twitter engineers in this specific case are capable of building a VP8 pipeline. After all, they built one for H.264.
At that point, achieving universal support is as simple as having two <source> tags inside your <video> tag. It's not difficult at all: http://www.html5rocks.com/en/tutorials/video/basics/#toc-spe...
I think the real solution is to automate this and allow the client to request on-the-fly transcodes to the formats the browser can support, similar to the way some UPnP servers work.
http://imgur.com/Qr3tohD http://imgur.com/Qr3tohD,8okaMlP#1
So... Flash.
This has the added bonus of stealing focus if you click on it, making the "GIF" a UX nightmare.
The upside is that users get a reliable experience. the downside is that laptop Chrome users get a reliably lower battery life, as VP8 isn't typically hardware accelerated.
Just upgrade.
Also lol @ mp4 being "bleeding edge" tech. It certainly is not. Even in the Linux community.
also lol @ your opinion about technology. even in the "linux community"
Anyway, that is why I asked the loaded question about how many users you think there are that browse Twitter.com on their RHEL box? Hint: HN is not a good sample to answer this.
And how is a ~14-year old technology (that has been ubiquitous now for at least 6 years) considered bleeding edge?
If something sucks but is still the standard technique in recent years then it's reasonable to think it should be supported. Similarly, if something is old but great, it's reasonable to think it should be supported. But GIF is old and crappy and there's no reason to expect support for something old and crappy to continue indefinitely.
I'd also note that your other examples have received substantial revisions over the years that have improved them enormously. I'd happily say the same thing about not supporting 1987-era C or filesystems.
https://github.com/MediaCrush/MediaCrush
if you want to try to do this yourself, our code is a good reference.
The player is very unobtrusive and minimalist, and simple things like pronounceable random names probably appeal to the user.
Also, features that are well-known to heavy reddit users (such as easily resizeable videos) only helped with the word-of-mouth.
Gifs have built in slow motion, it's called waiting for the 10mb file to download frame by frame upon initial play-through.
You can encode at 4:4:4 if you want, although obviously you have no control over what twitter does.
I encoded the gif in 420 (subsampling), 422 and 444, having a nice quality with 444 with VLC, I embed the three different videos in a web page and when playing in my browser, they are all as bad as the twitter video! I thought that Firefox was using a standard library to play MP4?
Depending on the webpage you uploaded them, the videos could be re-encoded (or VLC's options could be no-ops and the artifacts hidden when played back in VLC due to better chroma upsampling) hence the problem.
When I said that I embedded the videos in a web page, I wrote a small page from scratch to display them in my browser, locally, without any server. So no re-encoding. I think that VLC just has a better chroma upsampling.
I wrote a simple js lib for detecting the duration of a GIF for each browser. It was a fun weekend project. https://github.com/rfrench/gify
If webp gains more adoption, it's another alternative now that it supports animation.
https://www.evernote.com/shard/s19/sh/e81e070a-4e2e-4367-9eb...
OS X is the last platform where Firefox doesn't support H.264. They're working on it, but it doesn't seem to be landing soon.
Interesting also that 4chan recently added looping, muted WebM videos for similar reasons.
https://stackoverflow.com/questions/3212821/animated-gif-to-...
ffmpeg -i foo.gif -pix_fmt yuv420p -vf crop=floor (in_w/2)*2:floor(in_h/2)*2 foo.mp4
How long has twitter been doing this? Just out of curiousity.
(Yes I know 4chan doesn't convert but there was some talk that 4chan might be a major player in pushing webm forward. Is it even webm or x264? On mobile. Can't check right now.)
gifcat (or gyfcat idk) was before 4chan. github beat gifcat. Basically twitter is doing what gifcat does, which has been doing it for a few years.
I can understand why twitter is saving money on hosting these new "gifs"
cat x.zip >> y.gif and upload would have been nice ;)
Yeah, a 20+ years file format doesn't do compression between frames, who would have thought...
As I wrote in a different comment the actual reason is lossy jpeg compression, not intra frame compression.
Yeah, for pictures jpeg is much better (and smaller) than gif.
Not to mention in several gifs they really reduce the quality to have a smaller file size