FLAC 1.3.4
xiph.org
xiph.org
That... doesn't sound good ;)
Really cool idea, though more comparable to lossy compression I guess.
There are other worser codecs around, e.g. LDAC (best Bluetooth codec) 660kbps vs 990kbps is very noticable (cant setup the same way to compare 990kbps LDAC vs 128kbps opus)
(Mp3 is way less efficient and depending on the encoder it can have characteristic artifacts. Vorbis was already better, but opus blows that away. Software support for opus is nearly as good as mp3 since recently. We should either stop considering mp3 the default lossy format or hack it into a container format so audio files can have opus inside while carrying the familiar mp3 extension for the 90s folk to be happy about.)
So it's a lossy compression? Why use FLAC then?
Trellis quantization is just a more optimal way to divide numbers - think of it like rounding to nearest instead of down. "optimal" means "optimal rate-distortion tradeoff" and "distortion" means whatever you want it to, but usually it's difference between original and compressed pixels (absolute error/SAD/PSNR).
That can look blurry, because given all alternatives with the same SAD, blurry ones compress more. So psy-rd changes the definition of distortion to add a "has similar amount of noise" factor. That's very far from human optimal (if anything it's SSIM optimal) but it's free detail. Uses the same quantization to get there, though.
Most codecs sacrifice transparency to reach a bitrate and this one does the opposite.
That's what LAME's presets and the Ogg Vorbis quality settings do as well, isn't it?
This preprocessor on the other hand throws away some of least significant bits to save data. This increases the quantization noise but has no other sort of artifacts. The quantization noise can be dithered to fall in the higher frequencies (noise shaping) and is generally not perceptible.
But yes, if you're doing it repeatedly you'd want an un-dithering filter. Noise reduction tends to do this by accident but it helps if you know what the dither shape was.
Same concept just for making images more efficient for PNG
By my calculations, $65 will buy you a 4TB hard drive which can hold over 6,000 hours of uncompressed CD quality PCM. With FLAC you might squeeze 9,000-12,000 hours in. But, is that really worth the bother?
I know it’s not obviously relevant to consumers, but an uncompressed PCM .wav file is a format that any high school coder can figure out completely. FLAC on the other hand has had decades of work put into it. It is an engineering marvel. I just wish people would think simpler than “engineering marvel” when it comes to archival formats when the simplest conceivable format works just about as well… :$
Edit: Many people are bringing up MP3/Ogg/Opus, compressing podcasts, how silly it would be to toss out all forms of audio compression.
To be clear, MP3/Ogg/Opus make perfect sense to me. They are complicated. But, in return you get 8-12x the content at great (not perfect) quality. FLAC on the other hand Is at least as complicated, has a lot less widespread support and has an ROI of 1.5-2x…
As a consumer, I can see that 2x in return for a modest amount of hassle can be an OK deal in constrained situations -like price-gouged mobile storage. As an engineer who has maintained software systems over many years, the complexity of FLAC compared to trivial PCM as an archive format makes me a sad, cranky, old greybeard.
The storage savings are relevant when I consider cached audio on my phone, but... I'm not really sure how relevant.
It's not about the storage, it's about the transfer.
Streaming services are only just beginning to support lossless at all. Improved lossy codecs like Opus provide indistinguishable-from-lossless audio at 128Kb/s, so the formula is store the audio losslessly, then transcode it to lossy (on the fly, even) for streaming.
A weakness in the open source landscape I found is that there's a lack of fast codecs. If you want to encode in real time (eg, streaming speech or an artist playing in the moment), and encode per client for positional audio in a 3D world, then Opus becomes a bottleneck.
FLAC according to some preliminary tests helps by being faster than Opus to encode.
This means codec costs add up fast. You can't just encode once and stream the same thing to a dozen people.
Interesting use-case, got my brain racing off to find silly solutions.
I ask because I know Opus is used quite commonly in real-time applications (e.g. voip), and I remember when researching in the past that it is actually capable of lower inherent latency because of support for smaller frame sizes than some of its competition.
I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU?
> I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU?
Yup! I work on https://vircadia.com/ -- we have to compress audio in real time and every user gets their own mix since it depends on their location in 3D space. It turns out to add up pretty fast, and you can't fit that many people into a cheap VPS.
That's why I'm working on FLAC support for it. If bandwidth is plentiful but CPU resources are lacking it's a good alternative to have.
This code originally came from the High Fidelity company, which made their own codec. It's some piece of black magic that cuts down audio by exactly 1/4th and is amazingly fast. But it's closed source.
It's a lossy codec that's for some reason fixed rate, and shrinks audio by exactly 1/4th. The resulting quality is very good (works perfectly fine for music), and it's much faster than Opus. It retains stereo and high frequencies and sounds just fine. I'm sure there's a tradeoff somewhere, but it's certainly good enough to hold music events without people complaining.
I think it might be possibly related to codecs like AptX.
So far I've not found anything competitive that we could use -- stuff is either closed, or far more CPU intensive.
Usually a fixed compression ratio is a terrible tradeoff, but for streaming it kind-of makes sense.
[0] https://opus-codec.org/docs/opus-tools/opusenc.html [1] > Set encoding computational complexity (0–10, default: 10). Zero gives the fastest encodes but lower quality, while 10 gives the highest quality but slower encoding.
Complexity 0 is about twice as fast as 10. Which isn't bad, but if you want to go really fast it still leaves something to be desired.
I'm still converting to MP3 before adding to my phone for that reason (used to be ogg which I remain convinced sounded better, but player compatibility wasn't there) because 128GB of expanded internal storage isn't much.
As a practical use case, FLAC is the only way I can fit my song library on my phone. Flagship phones today come with 128GB of storage, about 90GB is actually usable. Every additional 128GB is $100 more.
IMO, one of the biggest scams in the mobile space currently.
My old OnePlus 3T flagship from 2016 came with 128GB of storage and it "only" cost €480. Also, best phone I ever owned by far BTW.
Today, nearly 6 years later, Apple and Samsung flagships (I'm staying away from OnePlus nowadays) are charging huge markups for more than the base 128GB of storage, on phones that already cost €1000+, despite flash storage getting significantly cheaper since 2016.
This is beyond insulting, especially since modern flagships also lack microSD expansion.
Any way you slice it it's still price gouging.
No, I already explained above why I consider it price gouging.
Are you 'gouging' your employer by asking for the highest salary you can get away with? Of course not. It's a market price.
They'll do everything they can to push people into using their cloud services. It erodes the concept of ownership when you have to ask for permission from another party to access your own stuff. The more dependent they can make you on them the better. Plus when I listen to an MP3 stored on my phone, there's no opportunity for 3rd parties to track what I'm listening to and how often or to push ads at me.
(I swear there was an episode of The Weeds where they read an ad for a therapy service for dogs.)
I'm in the UK and noticed both kind of happening: an ad for LinkedIn (I think) was localised to use a UK example in several episodes of This American Life during the last year. It was spliced into the programs seamlessly and sounded just like any of their usual sponsorship segments.
I'm presuming they just edited a handful of versions of the program for their largest listener regions and there was no need to do anything on the fly except pick which to serve. Of course I paid little attention to the ad as I was distracted more by it being the first time I'd noticed that kind of localisation.
Your phone doesn't support some kind of SD cards?
I'm inclined to call this The Javascript Mentality, though I don't mean to insult JS users. JS definitely has its place, it's just very much abused, unfortunately.
Edit: Of course websites being slow has nothing to do with storage space, but it has everything to do with wasting resources – whether that's storage space or CPU cycles.
or, a more better term would be 'Atleast it works' mentality.
Even somewhere you don't want to use FLAC because you're worried about the complexity, you should at least throw gzip at your files. PCM is a waste in any kind of computer scenario.
Apple has supported "ALAC" for longer, which unlike FLAC uses only integer math and is therefore less power hungry on mobile devices. You can transcode losslessly between FLAC and ALAC.
You can't just install VLC on an iphone?
Also, the $9 Apple headphone dongle is objectively superior to most audiophile DACs, which isn't surprising since it has a larger R&D budget than that entire industry. (Same goes for Google's IIRC.)
It shouldn't be too hard to build a decent DAC and amp stage, at least for ear buds and IEMs, it's just that many manufacturers (e.g. Samsung / Qualcomm) don't care. Apple sell music and so probably should care more.
https://www.kenrockwell.com/apple/lightning-adapter-audio-qu...
It's still useful to have an amp for very high impedance headphones for electrical reasons, otherwise it can be quiet and lack bass (plus drain the battery faster.)
[1] https://github.com/astoeckel/libfoxenflac/blob/master/foxen/...
…is what I want to say, but IIRC Lagarith actually does use floating point so you have to emulate x87 to decode it.
FLAC allows for metadata. Those .wav files? Nope.
It’s trivial to put a metadata chunk into that. And, trivial to ignore or blindly carry it along if your software doesn’t specifically support it.
There isn't even a de facto standard for metadata with wav files. Most players/software don't even bother at all.
FLAC pretty firmly settled on the ogginfo format and literally everything that supports FLACs use that same metadata format.
But anyway, if you're going to do WAV tagging (and remember software support remains poor) most people are using ID3.
WAV files can have any kind of compressed stream inside. A program that used some ancient API like WinMM ACM could theoretically play anything inside WAV as long as audio driver for that compressed format (now commonly called “codec”) was present. In addition, there are all kinds of standard metadata (like author and comment fields), and players in the '90s had no problem with displaying it in their interface if it was present. Old Windows Media Player had some of these status lines shown by default.
By the way, “driver” was an actual driver, with an .inf file, update of win.ini/system.ini or registry, and probably a reboot required to start using the decoder.
Old SDK tells us to look further into MMREG.H, so let's do just that.
https://github.com/tpn/winddk-8.1/blob/f6e6e4da7d1894536cf1f...
https://github.com/tpn/winddk-8.1/blob/f6e6e4da7d1894536cf1f...
(Last line is quite ironic.)
It's interesting to look how all those early '90s ideas that Universal Tagged Formats would be used to hold anything and everything, and dragged-and-dropped into any and all application completely flopped. Mac had those, Windows had those, Amiga had those. You can say that MP4, an international standard, is actually MOV in disguise, but it doesn't support all those early crazy options that could turn the file into poor man's PowerPoint presentation, poor man's Hypercard, or poor man's FMV game. On the other hand, Windows Media Player did support ASF interactivity like “open the link in a browser at this timestamp”, “show user some text at that timestamp”, and the consequences were so bad that it all had to be killed in mid-00s.
Which is about as good as not having metadata in the first place.
And since people rarely actually code to spec and it’s usual for wav files to not have metadata (and as demonstrated by GP common belief that WAV files can’t have metadata), pessimistically I would absolutely expect choking on wav metadata to be common.
Another fun fact about wav files: they don't necessarily contain LPCM data. The format allows for a variety of encodings, including lossy ones.
I don't know if newer versions still do that. I'd be a little surprised if they don't, given the previous version.
Sounds like the people who are surprised at their end of month AWS bill because they couldn't bother to optimize anything.
FLAC is at least as complex as OGG+gzip. That’s reasonable to deal with because there are libraries with many thousands of hours of effort put into them freely available.
But, trivial is trivial. Anyone who has had to deal with format support rot will sing praises of trivial.
My argument is that it’s a disappointing archive format given that any teenage coder can come within 30-50% of it in an afternoon by doing the most trivial thing possible.
Expectedly, few people here are arguing with me about archives. Instead we’re talking about streaming podcasts over 4G. Even for for high-end music, saying your personal bandwidth ethics/budget necessitates FLAC but not Opus seems more than a bit of a stretch, in my opinion.
If your point is that most users of FLAC would be happier with AAC/Opus etc I agree (and with a more significant savings)
And yeah if you're looking at longer archival times, PCM probably makes more sense. But I think FLAC suffers way less rot than, for example WMA
Anyone who has had to deal with finding and recovering old data should appreciate the ability to fit significantly more on a disk. More redundancy and cheaper storage means more data survives.
> teenage coder [...] in an afternoon
Finding a FLAC decoder is very unlikely to take longer than that, even 50 years from now.
There are lots lossless audio compression. Each with different set of priorities and trade offs. From Compression Ratio, Decoding and Encoding Speed etc. I agree FLAC is a very good codec, especially for consumers. But I am not entirely sure it is a "marvel". Arguably WavePack does somethings better, especially when doing it patent free. ( Sort of irrelevant now at this point in time )
The world has moved to streaming, and it is only a matter of time when lossless becomes a standard. Like both Apple and Spotify are planning. 5G will improves capacity, so by the end of 5G migration sometimes in 2030? This is sort of off topic but HN on one hand dont understand 5G and suggest not upgrading to 5G but on the other hand wants to have more Data for the same price and start doing lossless streaming. Well you cant have both.
While I wish we could research and push the lossless ratio down to 33%, or push 128kbps lossy audio codec to be truly transparent. But much like modern audio codec all research are now going to low latency and real time. Which seems to be a much much harder topic.
I don't stream it; and I don't use it on a phone. If I ever need to do those things, I can convert from FLAC to lossy, knowing that I'm starting with something that is as accurate I can get.
That is, I'm implicitly using FLAC as an archive format. Gzipped PCM would work, except that I can't play that directly in a music player. I don't care about a bit-flip rendering a FLAC unplayable; my collection is backed-up.
Basically, I'm not going to pay for a CD full of bits, and then immediately throw away a lot of those bits.
Having cheap storage is not a valid reason to not to use that space efficiently.
With today's, even last decade's processors, encoding FLAC at ripping speed is not even a CPU saturating task, and decoding is merely a blip on processor's queue. If I can store the same bitstream on a smaller space, why shouldn't I?
If we're not compressing anything, why not forego lossless compression completely? Let's not GZ/XZ our log files. These 20MB files will become 30-40GB, but heck, a 4TB drive is just $65.
Similarly, let's not compress gigabytes, even terabytes of genomic data, scientific outputs and other stuff. They're just ASCII or Unicode strings, a high-schooler can figure these out. Why invest time in lossless compression tools like XZ and ZStd? They just use CPU cycles needlessly.
Thanks for at least arguing against the use case I argued for. Even if you were a bit hyperbolic.
I get that I’m being a cranky old engineer. As someone who has spent a couple decades optimizing and maintaining bespoke high performance file formats, the particular use case of FLAC as an archive format still seems like a bad complexity vs benefit trade.
I’d rather have something that’s trivial for any software to work with everywhere forever. I know FLAC feels like forever. But, so did a lot of dead formats I used to work with.
My dad is very skeptical of compression, not understanding the concept of lossless and thinking it reduces quality to compress it. Even if you explain it in terms of "so instead of writing 1111111 it writes 7x1, look how much less space that takes! But it's the same information in the end." Even NTFS compression is a checkbox he insisted I disable. So it's wav all the way.
Recently had hard drive issues and I copied all data to a new drive at painstaking speeds (the hdd was barely limping). I now have no idea which files might have glitches from broken bits or sectors in them.
I guess a checksummed filesystem or keeping hash sums somewhere (the standard utilities make it pretty easy to verify many files with one simple command) is the solution for him. For me... just use a normal format and save half the space and money. Heck, go one step further and use opus: a healthy young individual can't hear the difference at a good bitrate let alone my "audiophile" "listens to too loud music" old man.
I've played in orchestras and have a respectable system at home, and the biggest differentiator is not the details, but how these details interact and create a bigger, more immersive soundstage. Yes, you can hear subtle sounds of a bow or a cymbal, but the exciting part is how sounds mesh and play with each other.
So, as I age, the excitement of listening the same or new songs with that system doesn't fade away. I still get the same joy, and shiver when I hear that soundstage.
Actually, jitter is really something about audio CDs burned in CD Writers. I still have a Yamaha CRW-F1 CD recorder, and it has a feature to abuse Red Book standard to record audio with bigger pits to ease the CD player's job of tracking the disc. It reduces a 80 minute disc to 66 minutes, but with a higher quality CD like TDK, the sound was noticeably different on the system I aforementioned in this comment.
Currently, that set is fed by a much better CD player, and I'm sure that its tracking is leaps and bounds ahead of the older player, but it was really made a difference back then. I'm not sure with today's electronics, it'd make such a difference anymore.
Agreed. My sound system isn't by any stretch audiophile; and I have the hearing of a 65-year-old. I can't point to things I can hear on a FLAC that I can't hear on an MP3; I hear the same notes, and the same instruments, with much the same tonality. But the former has a presence and vibrancy that is lacking in the latter; when I listen to MP3s, my involvement in the music rapidly tails off.
I suspect that you don't have to be able to hear higher harmonics in isolation, for those higher harmonics to affect what you do hear. That is, even if my hearing range cuts off at 12KHz, I can still tell the difference between an unfiltered sound, and a sound that is low-pass-filtered at 15KHz. The difference seems to be clearest with voices.
There's a reason MP3 is obsolete, though. You probably can't pass an ABX test with high bitrate MP3, but you almost certainly can't with AAC or Opus without extremely critical listening.
If you’re referring to the DAC portion that does make a difference obviously but that doesn’t have to do with the CD itself or tracking.
The two players have 20 years of development apart. First one was a lower cost Sony, and the latter one is an entry level, yet higher end one (Yamaha CD-S300).
On the tracking stability, first one skips if you knock it lightly, and Yamaha doesn't care if you bump into it accidentally. Also, newer electronics can switch much faster, and in turn, creating a clearer eye pattern for DAC to work on [0].
When you used CRW-F1's audio mode, it elongated the pits and lands, so the digital part had more time to switch properly. This created a clearer eye pattern.
A clearer eye pattern allows DAC to create more cleaner signal since it switches and understands the signal much better and allows it to create a more "correct" (or clearer if you pardon the term) analog signal, esp. on higher frequencies.
That said I see no usage for a CD player anymore, really. I can rip a bit-perfect copy, store it in FLAC, and play it pretty much anywhere. The only place I still use CDs sometimes is my car, because it's more convenient than fiddling with my phone sometimes.
OTOH, Listening CDs and vinyls is more of a ritual for me. A good coffee, some good reading material (or nothing), an album I especially like and an hour for myself. That Hi-Fi set also has USB/iPod connectivity over the CD player (and its DAC is a treat for both MP3s and over the iOS streaming) and it has bluetooth connectivity for very lazy times.
So a vintage set with some modern connectivity, and some older formats for enjoying the music for the sake of music.
Digital data on a Red Book CD is encoded with CIRC error correction, which provides an extra parity byte for every 3 data bytes. In normal operation, the recovery of the digital data is 100%. The DAC receives a stream of 1s and 0s, there is no "clearer eye pattern" for it to "interpret". The data is read or it is not. If it is not, the error is corrected. If it is uncorrectable, for example if there is a particularly large scratch, then - and only then - an attempt will be made to "guess" the signal.
Unless the CD is scratched, a 20$ computer optical drive with digital SPDIF output will be completely indistinguishable to a DAC than a drive costing 100x as much.
The "tracking stability" you mention in the newer model is simply a larger buffer, so that it has time to recover the read.
Opus and MP3 are lossy formats. You can’t transcode them to anything else without losing data.
With how cheap storage is these days there’s really no argument against FLAC because you can also transcode it to whatever. Players like Navidrome use ffmpeg to transcode on the fly.
If I put all this effort into ripping CDs and maintaining backups, it’s not going to be in a lossy format that I can’t convert to anything else.
Use ZFS to prevent bit rot.
It was an intentional dramatization to showcase some slippery slope, tbh. Genuinely sorry if that sounded too harsh, or rude. That wasn't the intention.
> I’d rather have something that’s trivial for any software to work with everywhere forever. I know FLAC feels like forever. But, so did a lot of dead formats I used to work with.
I understand the fear and pain of obsolescence in the world of file formats. VQF is the first notable example in my mind, but we need to keep in mind that FLAC is proper FOSS, and it'll live on in one way or another. People still keep Commodore64 kernals, emulators alive. Similarly MiniDisc has a cult following. On the worst case, I can install an old distro to a VM, and convert my music via that VM to something more modern.
I personally don't think FLAC as an "Archival Format". It's more of a bona fide storage format for me. I'm a serious music listener, and I tag my archives ambitiously, and besides being lossless and provides space advantages, it supports proper tagging via ID3 tools and allows me to store a truckload of metadata with my music.
To prevent corruption in my archives, I'd rather store recovery data and take regular backups. There's no other way in my book to keep data safe.
that makes me think of "C compiler" (actually C89 with benign bits of c99/c11) vs "c++ compiler", or "javascript web browser" vs "noscript/basic (x)html browser".
that said, aren't the 30TB hard drives around the corner?
The only answer to my actual question so far indicates that when it gets down to $1 vs 60c, they’ll just say “Whatever. Leave it uncompressed.”
Thus 90% of the discussion has instead been about 4G streaming lossless real time voice and podcasts to mobile devices with price-gouged storage to be listened to through very lossy Bluetooth compression :p
And, 9% has been about how FLAC as a container format did a good job standardizing metadata. But, that has nothing to do with audio encoding within the container :P
(Sometimes they both are.)
Me, because we don't use transparent compression anywhere.
> Actually, who says the disk itself isn't?
I'm not very sure about that, to be honest. SSDs do that to prolong their life, but the saved space doesn't return as more space on FS level. It just returns as endurance, which is more important in my case.
So, disk's transparent compression doesn't mean very much from a sysadmin/OS operations perspective.
Since Fedora 34, a clean install will result in btrfs with zstd compression turned on. (An update of an older install will of course not change the filesystem from underneath the files.)
Other systems we install are configured on many levels, so even if the defaults are BTRFS w/zstd, it might be either known or changed to suit our needs better.
But when you tar.gz your log files you do not loss information.
As a developer I'm working with large amounts of data totalling some dozens of TB compressed. I don't have the option of lossy-compressing it since bit-by-bit reproduction is part of the constraints. I don't want to store it uncompressed since that also increases I/O times. CPU is simply not the bottleneck anywhere along this line. And flac works great. :)
Meanwhile everything bar Apple now supports FLAC natively.
No way in hell we ever switch to WAV.
If software writers choose to not extract metadata from WAVs that's up to them... but I guess it amounts to the same thing if you only want to use software that doesn't read tags in WAVs.
* Most FLAC tools support metadata. For the more "bare" formats that's not the case. When converting to a lossy format, I want my metadata intact.
* FLAC can be played directly on most media players (with metadata).
* The format may be complicated, but the source is open and actively maintained, and there's support everywhere. Using FLAC in your software project is a breeze.
* 2x compression is nothing to sneeze at. My archival collection takes up 1TB on my mirrored, backed up NAS. I like that better than it taking up 2TB (which would be 1/6 of my space). Someday when I look at 1TB the way I look at 100GB today, this won't be a concern anymore.
And, there it is. Thank you.
JPEG2000, for example, never caught on because nobody cares enough about the storage savings, and there were no other compelling features that anyone cared about. But for audio, we want our metadata. Had WAV been built with strong metadata support, FLAC wouldn't have gotten very far. But that ship has sailed.
I'm halfway there, 1TB collection on a 30TB array. When I repurpose a previously used drive, I've got a bad habit of making a multi-TB image in case I forgot to copy anything off, and then forgetting about it. (I'm in the middle of writing some tools to sort through those.)
Still, what would be the advantage to moving to WAV? There is zero experience-overhead to FLAC. This isn't like some proprietary crapware format that becomes hit or miss due to patents or poor licensing. It's a Free format so common players all seamlessly support it. We're talking like 300kB of binary code, in a world where stuffing 300MB CRUD apps at users is normal.
If I want to transcode to MP3 for some bespoke device, then the FLAC compression makes that process quicker via saved IO bandwidth. There's literally no end-user complexity that I would save by moving to WAV. And if I wanted to develop software to read an audio file, then I'd import the FLAC library just as how one would generally import a WAV library. OP might as well advocate for storing audio as textual numbers line by line, so it could be easily processed with awk.
If the argument instead is that raw audio is a simpler format, the answer is that everybody wants some sort of container that carry metadata.
Just because we can doesn't mean we should waste ssd/cpu when there are absolutely no benefits, aside perhaps from saving a few kilobytes of extra decoder code to save multiple megabytes per song.