Make It Ephemeral: Software Should Decay and Lose Data
lucumr.pocoo.org
lucumr.pocoo.org
I have written here before[1] about the tyranny of "read it later" apps and how liberating it felt to ditch them and move to a "read it now or read it never" (RINORIN) mindset.
Although this articles talks about digital ephemera in different contexts such as shared spaces, wikis, runbooks etc., I believe the underlying idea is the same.
Honestly as a polyglot, only a Go project can run without being touched for 5 years.
The rest of them get vulnerabilities discovered in the packages, which requires you to upgrade, which changes the feature set and takes re-work.
Also design ages, UX paradigms change over time, look at the "web 2.0" look today and the "web 1.0" of before. Skeuomorphic design aged as fast as Garage and Jungle music did in the 90s. This will be the same with flat design and neuomorphic design in 5-10 years, if there is even an UI then.
All in all, software does decay and we lose data every day. Try go through your oldest bookmarks, many sites will be gone. Or run software that used to run on windows 98 or DOS or amiga or classic mac, you need to jump through hoops, the UX will be terrible, that's aging.
Software doesn't need artificial aging, it erodes well all by itself. Even fast moving code in a startup changes so much from week to week, your merges can be difficult. That's like a river bed redefining itself during a rain storm.
As humans go through time, so does software as an extension.
Go CVE archive [1] clearly indicate otherwise. What distinguishes a Go project from any other programming language?
My experience with Go, as user and open source maintainer, is that many libraries just work even if untouched for several years.
There is a culture around building small libraries, with minimal usage of dependencies beyond the standard library. This leads to most CVEs existing in the standard library, which combined with Go's commitment to backward compatibility, makes improving your security posture simple by just upgrading the toolchain.
I don't think it's a dogmatic position either, I don't think the suggestion is to always do this but to do this far more often than it's done now (almost never). And they are pretty explicit that they think this should be a default which can be overriden, and that you should be able to save things indefinitely if you so choose.
- Linear automatically cancels tickets which haven't been touched in 6 months
- Trello would visually "age" cards, making them increasingly yellow and tattered
- Datadog dashboards have a "popularity" score based on usage
Most auto-closed issues in my small team are of the varieties "the PM had a nice idea one day" and "long standing annoyance which we would need to re-architect to solve and we'll get to it one day".
And I don't even necessarily disagree with the CADT post, but... most users are not, for better or worse, so easily dissuaded.
OK. But my desktop reaches uptime of six months very easily (its not common that a security update mandating a reboot happens with Linux).
Once in a while I clean my browser's Download folder. When I do that the very first thing I do, before the mess creeps in everywhere in "proper" directories is to deduplicate the browser's Download folder.
The amount of time I'll have bankingstatement20241103.pdf, bankingstatement20241104-1.pdf (which should be really be "-2" btw instead of that zero-based non-sense but I digress), bankingstatement20241104-2.pdf, etc. in my Download folder is insane.
IMO there is no way to do this programmatically that satisfies all of your users. What the author describes is organizational debt, and the only way to deal with it effectively is to have the organizational will to periodically clean up that debt. It's not easy, and any shortcut comes with it's own drawbacks (see corporate data retention policies that inevitably nuke documentation or context).
Limitations, artificial or not, are not always bad. Walls, for example, can be seen as limitation or protection depending on how it's used.
I would like to play with the idea of a filesystem layer solution that tiers out information based on staleness. The closest I've seen to this is AWS S3 tiers, you've got standard S# at 2.2 cents per GB month, then infrequent, then glacier and some tiers in between.
Software is a tool. Software is infrastructure. Degradation is desirable in neither.
Data has metadata - creation time, access time, etc. - that can power workflows to identify and deal with old data if that is desirable, but in most cases they are not desirable. I haven't looked at the directory full of my wedding pictures in a decade, but I don't want my photo editing software to decide they're irrelevant and delete or hide them.
Having access to old documents and data is often important and beneficial. It's why people look at inscriptions in old books, why they keep photo albums, why finding a forgotten item when moving furniture can be delightful. It's why we're able to identify trends.
Most crucially, it is incredibly difficult to predict in advance which data will be useful in the future. Proactively discarding things when storage is incredibly cheap is a net negative.
When was the last time your mind ran into an out of storage issue and crashed?
You talk about metadata being important, yet more than half of the storage used today isn't used to store books, or medical records or whatever business case, but traffic logs and virtualized package dependencies and shit like that.
And second, it's a graceful decay, you are not completely shut down from storing imporant information for later use. If in that moment something memorable happens, you will remember that.
A video from the 90s that you never read or accessed in that period? Yes, a candidate for deletion, or at least a decay. For example, if it was a video, the transcript might be saved, but the video might be lost. We may also get some written interpretation on the visual elements of the video.
> A video from the 90s that you never read or accessed in that period? Yes, a candidate for deletion, or at least a decay. For example, if it was a video, the transcript might be saved, but the video might be lost. We may also get some written interpretation on the visual elements of the video.
My point is that the video shouldn't decay, either. It might not be playable in modern media software for a variety of reasons, but that doesn't automatically make it a candidate for deletion.
You have to think, in general, that we have finite resources, and the alternative is not remember everything in perfect detail, but forgetting things that you do not choose to forget.
Better to choose what we forget.
And decayed versions wouldn't get the author much mercy from bereaved families.
2- even if you can save the bits and bytes, there's fidelity loss in soft and hard standards, namely video codecs and connector standards. Not sure how planned decay would fit here, but relevant considering that document from the 90s needs is probably on a weird codec in a pre pata drive. Or even worse an analog medium.
Even if you don't admit that it will happen in this generation, it will happen in the next. We have companies that offer free storage like google, the economics can't sustain that forever, youtube videos are already being purged, google drive limited.
The cost of storing forever is unsustainable over the decades.
> Physical objects decay to the fundamental forces of nature, but that is a flaw, not something we should emulate.
That is explicitly called out in the article and I do not agree with you. Useful information is replicated, less useful information tends to die out. That we threw away that concept entirely in the digital world without a good replacement has some pretty strong downsides from my experience.
And if you read what I wrote in more detail you can see that I'm very open to the idea of soft deletes and intentional information hiding.
I think it's often the opposite. Drawing confusing or bad analogies to the past rests on the false assumption that the future resembles the past, and the reason people love old photo albums or forgotten items is usually a kind of nostalgia, not because they hold relevance towards the future.
This is also why forgetting or deliberately wiping the slate clean is fundamentally a good thing, it forces people to rebuild things from first principles without being bogged down in false assumptions carried over from the past, it's why the people who build new things tend to be young rather than old. If past information was so valuable in determining the future most visionaries would be 60. Hanging on to old information is a form of digital hoarding.