Reverse Engineering the GoPro Cineform Codec | Hacker News Reader

Reverse Engineering the GoPro Cineform Codec

medium.com

167 pointsingve25 comments

Pretty cool, I like these kinds of articles.

> An interesting feature where coefficients were pre-aligned to mod-8 widths to presumably allow for SIMD optimisations

I have seen protocols implemented in such a way that when described on paper, they look convoluted, especially how they specify alginment "this will have this much padding, this one will be aligned on this boundary. Little endian encoding for integer values...".

However I quickly realized they basically picked a particular memory layout in a C struct generated on x86-64 architecture, by a particular compler (gcc 4), and particular OS (Linux). I imagine at some point someone implemented it like that, then they went back and wrote into a standard. It would have been nice to at least mention that in the docs footnote ("Encode these as C structs on this combination of arch/compiler/os and cast binary blobs to the struct to serialize/deserialize data in this format)

Platforms (arch+os) usually have ABI documents that all compilers follow. If you get far enough away from C things might vary, but not C.

With C++ on x86, there's a fair amount of effort to try to be compatible, even though the ABI changes pretty quickly.

Elsewise, things can vary quite a lot.

> I imagine at some point someone implemented it like that, then they went back and wrote into a standard. It would have been nice to at least mention that in the docs footnote ("Encode these as C structs on this combination of arch/compiler/os and cast binary blobs to the struct to serialize/deserialize data in this format)

That footnote could cause multiple problems. Someone might just use that C struct and expect the layout to match, which would introduce portability problems (such as if using a different target OS, as Windows and Linux have different struct layout rules). Or, conversely, someone might see that line in a proposed standard and reject it as non-portable, whereas a precise description of memory layout would not come across as such. (In particular, the memory layout of structs by GCC on x86-64 Linux matches the memory layout of structs by any compatible compiler targeting several different 64-bit architectures on several POSIXy platforms.)

> Or, conversely, someone might see that line in a proposed standard and reject it as non-portable, whereas a precise description of memory layout would not come across as such.

Yap that makes sense, I din't think of it that way first but that probably explains it.

> However, real-world files didn’t seem to match a lot of the document which was odd. All of this was buried in the small-print of course: “the core technology behind the GoPro CineForm Codec has been standardized [sic]”. So in reality existing files were not decodable by a VC-5 decoder.

This is why open source is often so much better than open standards (though the latter the was used to help create the former, in this case).

Huge kudos to the author for doing this work, as well as explaining the process.

Not sure what open source would accomplish here - without a standards paper you can't get multiple conformant players for a format, because the company can just constantly move goalposts around as much as they will.

The best way to implement video standards until now has (as pretty much everywhere else) been a good tight specification coupled with a several conforming decoders without any monopolizing the market. It keeps the vendors honest.

> This is why open source is often so much better than open standards (though the latter the was used to help create the former, in this case).

Personally, I prefer both: a standard to give documentation, motivation, and clarification, and an Open Source reference implementation (written for maximum clarity rather than optimization).

Fascinating post. I wonder if anyone has tried to decode the Audible audiobook format - ".aa"?

FFmpeg would seem to already have this: http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavformat/a...

For those who would like to decode an Audible AAX file with the FFmpeg binary itself, you may do so with the following:

    ffmpeg -activation_bytes 1CEB00DA -i test.aax -vn -c:a copy output.mp4

Taken from the docs. (http://ffmpeg.org/ffmpeg-all.html#Audible-AAX)

Is it reversible though?

I've downloaded so much junk from the internet trying to do this but just couldn't and now I find it's so simple! Again, Thanks for sharing it. Who would have thought it would be so simple.

THANK YOU!!! I have been wanting to backup my extensive audible library for years but could never find an easy solution.

edit: I have not tried it yet, but this script seems to extract the key from audible

https://github.com/inAudible-NG/audible-activator/blob/maste...

Spotify's ".file" has been a tough one to crack.

The only work I've seen on breaking Spotify DRM encryption was this[1], back in 2014 and discussed on HN here[2].

[1]: http://moyix.blogspot.com/2014/07/breaking-spotify-drm-with-... [2]:

That is...genius!

Apparently they actually tried unlike Grooveshark. [0] I don't think Grooveshark would have minded if someone cracked their "DRM" even while they were still in business. :)

[0]: http://lgms.nl/blog-1

I'd actually be more interested in the database files that Spotify uses to manage cached tracks on Android - at the moment the UI is terrible, especially when you have 200+ tracks downloaded and are running out of space on your device.

My goal is to write an 'ncdu' application to show what files/albums are using the most space.

Wavelet image compression is itself interesting and quite intuitively simple (if looked at in the right way):

http://www.iquilezles.org/www/articles/wavelet/wavelet.htm

FEC stands for forward error correction

You can edit and delete your comments, no reason to spam one-liners, especially not without connection.

Http://rphlo.github.com/ there is a go pro repo

Interlaced stream of RTP h.264, audio stream, and FEC channel

https://github.com/rphlo/GoProLiveProxy

https://tools.ietf.org/html/rfc6184

Reply on news.ycombinator.com