Image file formats that didn’t make it
tedium.co
tedium.co
It is a really flexible and robust format also. How else are you going to store a floating-point multispectral image of 8 bands and 50.000 x 50.000 pixels, arranged tile-wise for easy cropping?
Why is GeoPNG not a thing? PNG is based on TIFF's design with various chunks allowing all sorts of metadata storage. It'd be pretty easy to translate GeoTIFF's GIS stuff into PNG. Has that been done?
Surprisingly, with PNG, the deflate compression can slow you down, depending on what you are doing with the data. Saving a little bit of IO when you're accessing local data can turn out to be a losing proposition. TGA makes it easier to just shove pixels around, from program to program. TGA has all sorts of weird options which you can ignore, but with PNG, compression is mandatory.
JPEG is a file format from decades ago. Obviously it’s possible to do better today. But it’s not possible to just invent a new format and have it as widely usable as JPEG. PNG did this with GIF but really only because GIF was seriously hobbled by software patents. It’s doubtful the millions of colors and alpha transparency would have won over compatibility otherwise.
At some point that all shifted over to PNG, but macOS support for TIFF remains good to this day.
Source: NeXTStep ;-)
For anyone with curiosity and time to burn, comparing early OS X with NeXT/OpenSTEP in VMs is fun, just to see what changed and what stayed. Some parts are extremely different, while some software seemed to be copy-pasted over, while others clearly had a lot of polish put into them for the consumer market. Fun times!
I still remember trying to explain to a previous employer why I needed Debabelizer to create appropriately-striped TIFFs in addition to Photoshop. In hindsight, the me of today would have said then, "I just need it to do this job, which will make you far more money than it costs to buy a single license." The me of then wasn't quite as savvy.
Several years ago, I attended a talk at an imaging.org conference where the presenter, an archivist from the Smithsonian, described how they used tiff in their work.
also, DNG format (a raw format) it's a derivated from TIFF.
Seeing it on this list makes me think the author didn't dig very deep when writing this article.
That's why I often use tiff to save images initially, and then compress in the background.
I thought I also saw BMP in a lot of games for some reasons (not to mention it's default output for the default image program on the effectively default consumer PC OS :)
[0]: https://gdal.org/drivers/raster/cog.html#general-creation-op...
Even 3D games will sometimes use them in a lot of odd places because it was faster/easier/took fewer cycles to include an uncompressed or simple run-length encoded bitmap than to try to store a smarter image format and need time to decompress it. (Though yes modern GPUs often support image decompression directly for certain texture file formats and that's shifted a lot.)
(Some of that I think is also political/contractual in the way that using MP3 would give some additional types of PC players "free soundtracks" while using other compression formats is a bit of security through obscurity and raises the bar slightly for PC players that don't, for instance, know what Ogg Vorbis even is.)
Do they?
Even if they do, it's typically useless for an application like a game, because the game will need to filter and/or mix that audio in ways that the hardware probably can't handle on its own. (Sure, maybe your hardware can decode one MP3 at a time, but can it decode two dozen MP3s at once, apply a bunch of environmental filters to them, and mix them down to a single stereo output? Probably not.)
The games have to deal with multiple layers of audios at the same time. So imagine one sound file have the dialogue for that specific scene, other sound file have the ambient music to compliment the surrounding, other file have sound effects, etc. So the games have to handle maybe 10 audio files at the same time which can put the strain on the CPU since it need to handle the decoding for compressed format. When that occurs, it eats up the CPU resources when it should be allocated for other stuff. Also don't forget some game have 5.1, 7.1, Dolby variants, etc which can be taxing on the CPU if it have to process multiple compressed audio at the same time.
Do they? Most of "modern audio" is just software decoding/mixing to a pretty not-smart-Digital-to-Analog Converter.
My educated guess is that most compressed audio formats are optimized for size, not for low latency recording/playback.
Also one of the reasons why Opus is so good and is replacing lots of older formats.
When making a quick hack like a toy ray tracer or Mandelbrot generator in C or C++, it's traditional to use .ppm [0] because it's trivial to write out pixel values in ASCII.
That works great for compressed textures like DDS or ASTC.
Unfortunately, the hardware decoders for proper codecs like jpeg or h.264 typically come with APIs prohibitively complicated to use, unless the app is a video player and needs to consume these APIs anyway, despite the complexity.
It's such a simple format. I remember reading about it as a kid, punching in some code to read the header, then the data, then the palette, having an image appear before my eyes. Fixing the bugs in my code made the image appear correct, what a great intro to debugging and seeing the results.
It seemed like magic at the time. Taking this binary data that was gibberish when looking at it with EDIT.COM and making it appear on the VGA screen
https://nambafa.com/tutorials-computer-graphics.php?series=d...
I wrote a PCX reader in QBasic. I can still remember seeing that image of a red rose finally appear in all it its 320x200 pixelated glory.
That had several impacts which scarred a lot of people I've worked with:
1. The commercial implementations were not interoperable and people get VERY nervous when they periodically get images which won't open correctly in one tool or, worse, have defects which one tool hides.
2. Many open source tools either didn't support the format at all or used Jasper, which just didn't get enough support to be fast or compliant.
3. Performance was quite slow for a while and people who weren't starting with Photoshop or native Mac apps (ImageKit uses the Kakadu codec, which has decent performance) were probably hitting Jasper or the Java implementation, both of which were orders of magnitude slower. People will forgive a certain performance hit for better compression but, especially as network bandwidth and storage capacity went up so much, the savings just weren't worth it if you could transfer and decode a JPEG in less time than it took just to decode the JP2.
4. All of that combined to mean browsers never really implemented it, other than Safari which picked it up via the standard image framework. The complexity meant that getting acceptable performance and security would be expensive and the demand just wasn't there.
5. The complexity of the format and low encoding performance meant that many JP2s were not encoded with optimized settings, which reduced the benefits to supporting it.
This makes me a bit sad because technically it was the only format for a long time which supported a wide range of colorspaces, bit depths, etc. and the way you can progressively decode a stream would have been a really neat option for responsive images in HTML — imagine if all your srcset had to do was say “Range-request bytes 1-x for 512x512, 1-y for 1024x1024, …” and the server + CDN could host & cache a single file for every client.
There are certain scenarios where the progressive / tiled decoding can make up for the friction (medical imaging and archival storage in libraries/archives) but they're not widespread enough to establish an entire image format. The open source situation has never been better with OpenJPEG but there just doesn't seem to be a high likelihood of significant increases in demand at this point.
Add in the fact that JPEG2000 showed at best a marginal improvement over JPEG and it's not hard to just stick with the proven technology. Having no good reference implementation is just a nail in the coffin.
If you look at it technically, it'll win on just about every point. Unfortunately for the standard, most of the people involved assumed that was enough to make adoption inevitable. I think it could have gone down a different path if, say, someone had released a high-quality open source version or worked with e.g. Netscape/Mozilla to integrate it into web browsers but that would have basically meant scaling back the older dream of making a profitable business around a single component like an image codec.
Marginal support from tool/browser didn't help boost its popularity, and patent situation as pointed in parent comment just shut the coffin on this otherwise ok format.
At the time vanilla jpeg had compression artifact and had worst compression rate, but was good enough at the time, even now because of its ubiquity the "successors" are still having a hard time dethroning jpeg.
Visually, where JPEG produces ringing artifacts, JPEG2000 produces blurriness. JPEG artifacts look "sparkly" and are often a reasonable replacement for the original texture, JPEG2000 artifacts look offensively dull (my opinion). I think, in theoretical speak, it's called "JPEG better preserves high frequency energy". You should be able to find side by side comparisons using your favorite image search engine.
This is the result of actual experiments with human subjects judging image quality.
I haven't really heard claims that JPEG2000 is better, actually. I remember people saying that it was "supposed" to be better, but not by people who dug into it and made comparisons with human eyeballs.
There were some images that would appear visibly better with JPEG2000, like photographs with those beautiful fields of defocused color. JPEG2000 captured those fields of color with all the smoothness they originally had.
Subjectively I think JPEG2000 is better across the different images, at least at this "medium" size preset.
Here another article [1] I found recently that discussed why Movie theaters use JPEG2000 over popular video format.
[1] https://www.design-reuse.com/articles/4595/digital-cinema-re...
* The industry doesn't want you to see compression artifacts, ever. Digital Cinema has no inter-frame compression, it's encoded as a series of individual jpeg2k images at a pretty reasonable quality.
* If one frame has a glitch, the glitch doesn't propagate over a bunch of frames the way x264 style compression does.
* jpeg2k at too-low a quality doesn't have jpeg/mpeg style square artifacts—instead certain parts of the picture will look slightly blurrier.
* jpeg2k has an interesting property where a 4k image can be encoded (with the proper parameters) such that when decoding you can stop at some point midway through a single frames data and have the equivalent encoded 2k image. Which means you can encode for 4k and ship it to theaters that only have 2k projectors (and that 2k hardware isn't required to render to a 4k framebuffer and then downsample).
* Since it's more modern than jpeg was (and dcinema was standardized before something like webp came along), it has a lower bytes/quality ratio than jpeg. IE, to achieve equivalent quality image from jpeg you'd need a lot more data.
* Support for deeper color depths and a much wider gamut. This is of course limited by what a projector can recreate, but they didn't want to limit to rec 709 or something.
They didn't care about patent encumbrance because they didn't need push this out the consumers—the jpeg2k patent stuff only affects the decoders and encoders, both of which are owned by people who can afford to pay.
JPEG2000 is also sometimes used to encode images embedded in PDFs.
Interesting. I thought PDF normally convert the format into .jbig2 for embedding into the PDF itself.
IMO, the flexibility provided by the container is not worth the development/support effort. It's better to do one or two things really well rather than have hundreds of edge cases.
What this list is missing is WMF/EMF which were basically direct serializations of Windows drawing primitives.
> BMP files are usually not compressed and, therefore, are not well suited for transfer across the Internet
When BMP files were compressed they'd typically be given the extension .RLE.
I use LaTeX for my personal stuff, but employers don't touch such things with 20 foot poles.
The Amiga had bitplane-based graphics. The "interleaved" part was that an image was stored by line first, bitplanes second instead of one whole bitplane after the other. Games often used the "interleaved" graphics layout in memory because it allowed blitting a sprite with a single Blitter operation. (... and EA who made DPaint was a games studio first)
> (list includes BMP, TIFF, TGA)
I remember seeing lots of software using Quicktime PICT files, even on Windows, but I've never seen anyone break it down (Wikipedia does a bit[0]). Did everyone mostly use it as a container for JPEGs?
I've actually toyed with the idea of a PICT to SVG conversion tool. It's potentially useful for viewing certain old documents in the (long obsolete) Apple DocViewer format.
But was expecting MNG but didn't find it. Interesting history with that one, a few browsers implemented it but then removed it. Leading to a lot of geek angst.
https://en.wikipedia.org/wiki/Multiple-image_Network_Graphic...
Was it really just the lack of interest from browser makers preventing people from using it?
And I still see BMP files being used even in modern games. Yeah: PNG is better, but BMP has its uses.
You can write out PNG files with deflate blocks that aren't compressed, so in that sense they are uncompressed PNG files, but all of the complexity of supporting compression is still there in the data format in a way that is not fully avoidable even if you don't actually compress the data.
BMPs on the other hand are simple to write, only a very small portion of the header data is required and then you can dump the image out as its raw pixel data in a way that they are probably already being stored in memory, making it a very easy format to support the writing of and a pretty easy format to support the reading of.
BMP has compression too (RLE and Huffman). PNG is simpler here since you can just drop a DEFLATE library straight in.
Images are usually stored in memory as RGBA in scanline order. You can dump this representation straight to a PNG file (edit: this is wrong, you can't). BMPs are typically written with pixels in packed BGR order, rows going bottom to top. But not always. I think they go top to bottom if you give a negative height, and you can supply masks defining which bits the RGBA come from.
In a couple lines of code with no external library you can create a valid BMP file with just enough header bits to tell the BMP reader how your pixels are arranged and let it deal with the conversion.
Virtually all of the complexity of BMP files is optional and can be ignored by a BMP writer and just let the BMP reader worry about it. This is not true for PNGs where you have to have a significant understanding of the file format to write a PNG file.
Libraries make that not matter so much but are kind of outside of the scope of where this discussion was at the time this came up.
I'm certainly not advocating for BMP over PNG and most people should just include a flexible image handling library that's relevant for their given language and be done with it, but I also understand why even today programmers working in languages like C/C++ will use BMPs just to get a simple and functional export working without dealing with the complexity of using a third party library when their needs for the file format are minimal and internal to their own use and not being exposed to users.
BMP is more complicated, unfortunately. The header structure is more complex. And then there’s a requirement for rows to be 4-bytes aligned, might need to insert padding bytes between the rows.
These days though, I use tend to use the even simpler PPM format. It uses normal RGB byte ordering from top to bottom, and it's simple enough that I can code a basic writer for it from memory:
fprintf(stream, "P6 %d %d 255\n", width, height);
fwrite(image, 1, height * width * 3, stream);
A PPM paletted format would save me a lot of space.
The built-in support for PPM in Preview on Mac OS X makes it extremely useful.
[0] https://en.wikipedia.org/wiki/Netpbm#PAM_graphics_format
[1] https://github.com/nothings/stb/blob/master/stb_image_write....
Sometimes when you are going to use your own format for saving data you could pick the simplest format for your import to accelerate productivity at the cost of diskspace.
In a similar way I use collada for character animations, it gets alot of slack from AAA studios (who all use FBX) but for indie development it's simple enough to be able to implement in a reasonable time (XML so you can see what's what)...
In the same line of thinking, I import static meshes from .obj, sound from .wav and last but not least font from .ttf!
The limiting factor usually is the editor you use to create/manipulate the assets in the first place, always look to the simplest export format they have, for Photoshop TGA is the simplest!
I export everything to my own custom binary formats to be loaded by the engine by customers, and there lz4 is a pretty good/simple RLE compression (RLE is also used by TGA BTW) and then .zip (also very simple!) to wrap it all up...
Keep it simple!
Regarding IFF usage in The Sims, it's not surprising given that EA designed the format. I believe that SimCity 2000 cities and scenarios are also IFF.
Ha ha ha I work in an image format so forgotten it doesn't even make the forgotten list.
Computer Graphics Metafile bitches!
My god how I loathe that thing. It's kept alive in specifications purely by the efforts of a single software company flooding all the WGs with cheerleaders. "No, it's not obsolete, SVG is just a thing used by hackers and programmers"
you poor, lost soul. tell us where it hurt you.
WebP suffers from lack of support in outdated browsers, but that's the only problem I've found with it. AVIF is kind of a silly format, but for some use cases it works quite well.
JPEG XL has very little tool support, isn't (as) free as the alternatives and browser support is practically non-existent. Nobody wants to adopt the format either, because the JPEG group made the same problem Nintendo made when naming the Wii U: JPEG XL sounds like a JPEG file with something special going on inside, while in fact it's actually a completely new format.
It's in Chrome, Firefox, Edge, and Opera behind flags - works great!
https://jpegxl.io/tutorials/facebook/#facebookjxlsupport
> [as of] the 8th of November 2021, all JPEG XL files were accepted ... Facebook offers full JPEG XL support
Had been heavily used on SGIs as their 'native' image file format.
Smells very dead to me.
It seems to happen a lot with codecs (e.g., Xiph's Dalaa tech merging into AV1, or Xiph's CELT and Microsoft's SILK merging into Opus). I see FLIF as the prototype and JPEG XL as the production version.
https://github.com/cloudinary/fuif
But it too got superseded by JPEG XL (.jxl) https://jpegxl.info/
When I read that, I thought it is because it is a vector type. Usually "resolution independent" images, from my understanding, are commonly vector type since it can scale infinitely like SVG. However, I couldn't find information online whatever .FIF is a raster or vector type. I saw one that said it shares similarly to vector images, that didn't confirm anything for me. So I would incline that because .FIF is a vector type, please correct me if I am wrong. It so hard to find this information and wondering if my google-fu is getting rusty.
EDIT: Whelps, it turn out my google-fu is getting rusty. I managed to find the information and I am wrong. It is a raster type[1].
[1] http://fileformats.archiveteam.org/wiki/Fractal_Image_Format
The core insight was great: have one generic lossless simple raster format and make lots of command line image processing tools for it. Then have a bunch of converters. For many years I got a lot of work out of things like xpmtoppm icon.xpm | pnmscale 3 | ppmtogif > icon-big.gif. It helped the format was so simple you could easily create images in C code, too. (These days we use Imagemagick or derivates instead.)
One interesting thing about PNM is very few images were ever stored in that format itself. It was intended mostly as an intermediate format for process.
There was an "object" view which was basically a bunch of still images of an object from different angles. The cursor moved a virtual camera that just displayed the still frame for the appropriate angle.
The "panorama" view was a panoramic image mapped to view angles from a central point. Images could be cubic or spherically mapped IIRC.
Both types supported hyperlinks. So clicking a linked region could load another scene which could have yet more links to more scenes.
The Star Trek Interactive Technical Manual was a famous use of QTVR. It enabled visual fidelity a real-time 3D rendered of the era couldn't deliver. It was made by actually photographing the Star Trek production sets.
Classic MacOS had pretty well-integrated support for QuickDraw 3D Metafiles, they could be dragged and dropped, you could embed 3D objects into a word processing document or presentation or something and manipulate and explore them in there.
Kind of mirrors what Apple is doing with USDZ these days (you can send a USDZ model in iMessage and the recipient can manipulate it right there in chat)
I would create the files in a txt editor and then explore the world-space created via browser.
https://www.cnet.com/tech/services-and-software/silicon-grap...
That was my favorite way to serialize images but I never had the foresight to add enough header or versioning info to make it backward compatible.
Now after 30+ years of programming the first thing I add to any homebrew file format is enough info to make it extensible.
You never get over learning to program on a system with 3.5k ram.
[1]: https://github.com/qubyte/qrcode-png/blob/main/index.js#L6-L...
Each place had a similar hyperlink object to get back to the Parthenon.
I enjoyed it. I probably still have the VRML 2.0 book around somewhere.
I loved the concepts behind VRML, and still do. Unfortunately all of the implementations around it were terrible.
When it was in vogue PCs weren't powerful enough to handle anything but the most trivial models. Dial-up was entirely insufficient to deliver anything but the most trivial models as well.
And this was years after Doom and other 3D shooters introduced wasd movement.
VRMLs biggest problem was that it was just way ahead of its time. Everybody who tried it agreed that 90s PC hardware just wasn't ready for VR, and dialup modems were never going to be adequate. Unfortunately almost everything that has come up to replace it has been proprietary and usually monetized. We've lost that sense of "anybody in the community can be a creator" that defined the early web. It was too hard to monetize.
Being ~18 years away and therefore memory is skewed, may I ask regarding how the implementations were terrible? Honest question.
I don't remember any without awful skeuomorphic buttons and view borders. Their navigation UIs also tended to be terrible. Most I ever tried aped 3D modeling apps using the mouse (a workstation mouse with three buttons) to fly through a scene. But with the typical PC mouse having two buttons (and a Mac with one) you had to use modifier keys to switch between fly and "look". VRML supported world gravity (intensity and a normal) so a viewer program could just navigate like a FPS game but I don't remember any that did.
Another issue I remember but may have just been my PC, because WRL files don't embed any linked content they've got to go fetch every resource referenced like on a web page. On my systems at least this led to a lot of pop-in as I moved through scenes, even locally. On dialup it's was a thousand times worse.
So modulo actual hardware performance, or lack thereof, you had poorly performing software with very alien feeling UI. While that is more than 20 years of time between me trying VRML and now those are things that stand out to me. Maybe none of these problems existed if you were blasting around on an SGI O2 workstation or knew just the right software to use. I had neither so I just stumbled around playing with stuff.
I am so glad that many image viewers/editors still load TGA, my computer has gigabytes of them!
Spec here: http://www.paulbourke.net/dataformats/tga/
>tiff
"The vast majority of the favicons offered up by websites are PNG. 71.6% of <link rel=”icon”> images are PNG. 21.1% of /favicon.ico files are secretly PNGs, including Reddit’s. Strangely, only 96.1% of Apple touch icons are PNG. Presumably the other 4% are broken."