Yet I still hear people talking about how digitally shot films have an archival problem. I really can't see how any well thought out digital archive wouldn't be significantly safer than film in a can.
One reason for film's superiority as an archival medium is that you can hold a piece of film up to the light and see what it contains, whereas to see a digital archive you need to invent all the technology that leads up to decoding that particular file type.
Disclaimer: I'm not a film archivist, but my partner is, so this is mostly second-hand knowledge backed up with quick google searches.
There have been several different major types of film used throughout the year.
Nitrate film was used from the late 1800s up until about the 1950s. It's highly flammable, and nowadays there are very few cinemas which are equipped to screen it. You need a very secure projection booth which can be well sealed in case of fire. It also burns without needing air, as it produces oxygen as part of it's reaction, which means that it's extremely difficult to douse with water. As it ages, the combustion temperature also lowers, and badly stored films may spontaneously combust. Despite all that, a properly preserved nitrate film looks phenomenal when projected.
That was replaced with acetate film, initially known as "safety stock". It was initially thought to be pretty stable, but within a few years the decay (known as "vinegaring" due to the smell) started to set in. As with nitrate, proper storage can slow down the decay dramatically. This was used from the 50s until the 80s/90s.
So there are these two historic film stocks, which were used in the production, archival, and distribution of every film for most of the last century, and they're both not fit for long term purpose.
In the 80s, a more stable polyester film started to be used. This is the best film archival method available to my knowledge. It's shelf stable, and has proven itself in lab tests. This is what people are comparing digital archives to.
Apart from the glib example at the beginning, I'm not going to try to argue the relative virtues of preserving the content of a film on film vs digital. However, the field appears to have a general acknowledgement of the importance of the medium of creation. So a digitally shot film would probably be preserved digitally (as well as possible) alongside any other medium.
Realistically, unless these nukes are put to intensive use, we are not going to disinvent computers. But your point is certainly valid for codecs. The videos will require to be transcoded multiple times as old codecs become obsolete (who can read real media's .rm files today?)
The transcoding is trivial as far as problems go. We've been dealing with that non-problem for decades persistently and will continue to. The engineering skills needed will continue to exist to deal with it.
DOS 5.0 was released in 1991, over 25 years ago. There is a rather astonishing number of JavaScript based emulators that will let you run it in a browser. Even if the next 5 years of browser development break all of those emulators, getting a 5 year old portable version of Firefox that will run on current hardware isn't difficult.
I don't think decoding video that can be played with VLC today will be a problem in 30 years, and I also think that 30 years after that, all you might need to do is another level of nested virtualization. If we assume that there will be sufficiently well working emulators capable of emulating a 2017 machine in 2047, just like there are such emulators for 1987 machines in 2017, and that at least the most common binaries will somehow be preserved, you should be able to recover most reasonably popular video formats in perpetuity.
Compare that to analog material that rots and requires incredibly delicate handling by experts, while with digital media, you can let anyone try since they can't break anything. And finding a copy of a popular OS and player binary might prove easier than finding a projector/VCR that can handle the ancient format.
Even if you were to recompress every 30 years, the loss will likely be less than from physical degradation of physical media. However, if you have a lot of video in one canonical format (down to using the same encoder and settings, so you don't have to worry about weird edge cases), you can just preserve a copy of the player and every 30 years, you make sure you can either port it to a current OS, or virtualize it efficiently.
The main points are that a timeframe of 30 years is actually quite short. While the moving image on film has only been around for about 120 years, ideally archivists would have film copies that would last far longer than that. Our idea of a computer in 120 years may be very different from what we currently use. More mundanely, what about when the last optical drive is manufactured? We'll suddenly have a finite number of reads left in the world. Or the hard drive connector breaks and it hasn't been manufactured for 30 years. Sure, a special one-off manufacturing run is certainly possible, but could a struggling and under-staffed archive afford to do that for every one of it's thousand hard drives, especially when it may not know exactly what's on every one of them?
I'm know I'm being a devil's advocate here, but I did want to illustrate that every unknown part of this equation is exactly what scares working archivists.
There are two types of digital archiving. One, which you're describing in this line of reasoning, is keeping a digital storage device in cold storage, booting up later, and trying to read the data. I agree that this has significant issues, as digital devices are physical devices, in addition to being digital ones, and not necessarily that reliable at that.
There is also digital archiving in the sense of understanding what the bits represent. I would argue that this sense of the phrase is actually more compelling, because fundamentally digital data is just a bag of bits. While this would be more effort than I'd go to personally, I'd expect any competent digital archivist to store their data in a ZFS-based disk array, SAN, or cloud service like S3, or ideally multiple of the above. (Or futures equivalents of these techniques.) Yes, any given service, technology, or device may come and go. But this is missing the point. The way you achieve continuity is by periodically refreshing the storage so that at any given point in time the data is stored in a modern (for the time) storage device.
There are still important issues to be solved, but the important ones are no longer questions of maintain physical devices, which is key. (E.g. the codec problem.)
The main situation in which this approach does not work out is if you're trying to do archeological-type recovery of a digital storage device that has not been actively maintained. Then you're back in scenario #1. But if we're asking about a situation where someone actively cares about preserving their own data, the situation is not nearly as bad as these arguments make out.
-My chief worry in that regard would be DRM; I have next to no idea how digital content is distributed to theatres, but presumably it is locked down all over.
The transcoding itself is trivial. Decrypting the source material may not be.
EDIT - Note that this assumes knowledge of the DRM algorithm. I admit the brute force approach might fail if the algorithm was secret and no players existed any more.
It's not like there's any real choice these days. YouTube has 300 hours of video added to it every minute.
A modern B+W film, properly fixed and washed, is essentially forever. Metallic silver doesn't really degrade by itself (50+ year lifespan) - and you can use a toner like selenium to convert the metallic silver into silver selenide which is even more stable, easily 100-200 years+ (note that this is generally done with prints but it also works with film).
Color films are generally considerably less permanent because you can't capture color using metallic silver unless you have one negative each for red, green, and blue. You can do that though, the TechniColor "3-strip" process did this and it's actually extremely stable for its time. It was also used for the color series of Magic Lantern glass plates from the Prokudin-Gorsky survey of 1905 [0].
Modern negative films are decent if stored properly, but early color negative films were pretty disastrous in terms of long-term stability.
Color slide films in particular are actually quite decent as long as you store them properly (cold, and out of light). Kodachrome (K-14 process), in particular, was remarkably stable because there is actually no dye in the film itself - it's created by reaction during the (incredibly complex) processing, so (again, if properly washed) there is nothing left to continue reacting and "fade" the image out. Properly processed, Kodachrome does actually approach the stability of B+W film. The only thing that really degrades it is exposure to light (i.e. projecting it or leaving it in a window).
Many early film bases were problematic though, because it was essentially a parallel effort with basic materials-science research in plastics. Nitrocellulose is super flammable and many early theaters burned down as a result, it was replaced with acetate "safety film" which tends to go vinegary, finally everyone settled on polyester. Nowadays you can have a pretty good expectation of the storage properties of our plastics.
Anyway, to make a short story long - you are assuming the question by adding "a well-thought-out digital archive". You can't pick a random instance of film storage and pretend that's representative of a modern, well-thought-out effort at archival. People did all kinds of dumb things in the past - South Carolina's constitution is falling apart because they laminated them back in the 60s and now they're turning to vinegar too [1]. The equivalent in digital terms is a random paper tape from the 60s pulled from someone's attic, or a DVD that was scribbled on with an alcohol-based marker and left on a spindle somewhere. Or someone's VCR bootleg of the first rough-cuts and rotoscope layers of Star Wars (see "Deleted Magic" [2], it's amazing but wow is the quality terrible in some of them).
It's always going to be possible to store some relatively stable physical media, that's the easy part. The problem with digital is you then add additional tasks of loading and interpretation on top. Even if it's stable, how do you get that Quadruplex video tape into your computer? Is there a driver that interprets the filesystem? A codec (ideally open-source)? Is there a sync signal in the medium that might degrade (like VHS)? etc etc. Yes, these are not insoluble problems, but they do add a huge threshold to get over. It might be worth doing to save Star Wars but it's not going to happen for Uncle John's photo collection.
Consider something like the task of encoding a message to an alien race. You have to provide something like a Rosetta Stone to help make sense of your format. Or you could just send a phonograph which can be read by many different methods (laser measurement is the coolest). It's a pretty solid assumption that there will always be basic instruments for physical measurement like lasers and reflectometers, and that's all you need to interpret a basic medium like B+W film or a phonograph. Although I suppose how dare I assume the visual wavelength of an alien species! /s
On the other hand - if you are going at it thoughtfully, i.e. periodically rotating your digital storage medium to whatever the standard is at the time, and you retain sufficient backups - the chances of there ever being a total discontinuity in migration path, or a total loss of knowledge on how to interpret the format is relatively low. The going at it thoughtfully part is the key though. Again, nobody apart from a handful of archival specialists or enthusiasts can really read a random minicomputer tape from the 80s, and that's only 30 years.
I personally started noticing bit-rot on a significant fraction of DVDs that I burned within 5 years (at least 20% of discs). Now I use DVDisaster to encode extra parity/recovery data on a rolling basis - disc X contains a 20% parity file for disc X-1, with the most recent kept on my HDDs, so that I can "chain" backwards through them.
Still though - imagine the kind of digital files that were made in 1995 with MOV files or whatever. Now compare them to the high-resolution 4K scans that are made from the same films. That's really another problem with digital, not just archival - the bits are what they are, you can't go back and try again when technology is better.
[0.0] https://www.flickr.com/photos/prokudin-gorsky/
[0.1] https://www.loc.gov/exhibits/empire/making.html
[0.2] http://www.loc.gov/pictures/collection/prok/
[1] http://www.npr.org/2017/02/21/515410087/an-attempt-to-save-s...
[2] https://www.youtube.com/watch?v=f2r4Nffrc6Y (there is also a Deleted Magic Revisited DVD with better quality and some more footage - finding it is left as an exercise to the reader)
Heh. I've been digitizing a couple handfuls of old VHS tapes that have nostalgia or sentimental value to myself or a friend of mine. You really forget just how poor video from a few decades back was compared to what's available today.
My favorite example here is Star Trek TNG. It was shot on film but immediately transferred to VHS for editing and transmission. The BluRay releases were remastered from the film and the difference in quality is just astounding. Immensely greater color range and resolution. And since everything was done with motion-capture and practical special effects, all that looks amaze-balls too. There are like 2 actual CGI sequences in the entire series (the Crystalline Entity and something else).
It was a massive effort to re-master the whole thing because again, it was cut on VHS. They had a map to the general area where a reel of film was stored but nothing specific ("3rd row of shelves, 4th from the top, halfway across"), and no real list of which takes or shots from the reels were actually used. Only a list of what shots were taken on what day, in most instances, and the shot lists weren't always followed. They literally re-cut the entire thing from scratch. Pretty amazing.
On the flip side - it's going to be a lot tougher to re-master DS9 or Babylon 5, because they were actually shot on tape (VHS?) instead of film, and the effects were done digitally. The model files and stuff are long gone. You could re-create it from scratch, but nobody wants that (see: the Star Trek TOS remasters). Practically speaking, DVD quality is the best it's gonna get there.
But do remember - a hard drive that's sat for 5 years may not even spin up. Spinups and spindowns are rough on hard drives. So is just letting them sit and seize up. A mechanism that depends on moving parts is inherently failure-prone.
SSDs are going to be better in that respect, probably. Although again, you have cases like the Samsung 840 Evo series where the flash cells lose their charge over time and need to be periodically rewritten or else they go corrupt (never fixed, just had a firmware patch to rewrite it in the background).
The funny thing is, I actually think optical media are a pretty good approach for long term stability. It's a lot like film, all you really need is a good image of the disc. Plastic and foil just happen to not be that awesome for long-term storage, if it was diamond or something that would be pretty much perfect. But, I suppose since you need a laser with a particular wavelength it's not really universal...
Anyone know how those "M-Disc" things made of some kind of stone or whatever end up turning out? Any good? Sounded nice from the marketing pitch but of course the salesman's going to tell you it's awesome...
> easily 100-200 years+
I don't understand, is it forever or 200 years?
The videos I uploaded to YouTube 10 years ago of me singing crappy songs into a webcam are plausibly going to exist until the heat death of the universe, whereas it sounds like the best case scenario for B&W film is 200 years.
> I personally started noticing bit-rot on a significant fraction of DVDs
DVDs aren't digital, they are an analog format that we use to store digital data. They are made of plastic and foil. The fact that they degrade is a shot against analog storage. Digital storage is electrical. Like S3. A good digital storage device is a living, self-repairing organism.
> The problem with digital is you then add additional tasks of loading and interpretation on top.
That's where your thinking is being constrained. You're thinking of digital archiving as something you do on top of analog media. That's backwards.
With digital, the only task is loading and interpretation. And dozens of file formats have totally stabilized. A CD packaged with an .ISO image for a Linux computer that can read it is pretty much self-hosted. You could've encoded those images 20 years ago, and they'd still be runnable on EC2 today. And they'll still be runnable on EC2 100 years from now. The fact that some very old formats are difficult to read does not mean we're going to forget how to decode ISO images, or boot Pentium-class virtual machines.
I get it that archivists are obsessed with DVDs and film, and I have a massive amount of respect for people who want to try to protect those physical objects. But the idea that preservation of digital files is somehow equivalent—or even more difficult!— is laughable.
Off topic here, but there is no such thing as the heat death of the Universe. The second princile of thermodynamics (entropy cannot decrease in a closed system) does not hold at cosmic scales. That's because that principle assumes the gravity is negligible compared to the other forces that move the molecules around, which is a perfectly valid approximation for all human-scale systems (such as engines). At cosmic scales gravity becomes dominant and the entropy of closed systems can and does spontaneously decrease. For example a giant gas cloud evolves to become a spinning disc and then a planetary system.
People know vaguely that in a few billion years our Sun will collapse and it will either become a supernova or become a star where the fusion reaction is a higher one (instead of hydrogen+hydrogen->hellium, something like hellium+hellium->carbon). At some point possibly after trillions of years, all the nuclear reaction converge to produce only iron, and from that point on there's no more fusion and fission, and later on the heat death of the universe occurs. Well, even if there are no more nuclear reactions, the universe will not die from reaching a maximum state of entropy (heat death), but may continue to evolve forever, going periodically through states of higher and lower entropy.
Of course, in practice how many people are one head crash away from losing most of their photos stored on a consumer-grade USB drive? And of course, many people are also subject to cloud companies going out of business, computer equipment being stolen, losing access to a site online, etc. And the situation with gaining access to photos and other things if a family member dies or whatever is another whole issue.