Digital Archivists: Protecting Public Data from Erasure
spectrum.ieee.org
spectrum.ieee.org
Internet Archive et al. made noise and promises but told volunteers to stop because they couldn't actually handle the ingest.
https://www.reddit.com/r/Archiveteam/comments/1jbgycm/us_gov...
These folks made a notable effort.
https://webrecorder.net/blog/2025-03-25-govarchive-us-and-mi...
I hate everything about this.
This stuff is very important to talk about so I hope that this submission by rbanffy isn't also flagged.
Climate change is perceived for some reason politically too and not get flagged so often.
30 years ago it was thought collecting every issue of magazines like TV Guide was important. No one even knows what that is anymore.
No one is ever going to look at 99% of this data. In the meantime, send more hard drives for my NAS!!
Case in point: retrocomputing is my hobby. I buy, restore, preserve, and use old computers. Most of them are home computers, because business computers go directly from the office to the recycling facility or the landfill. Unless someone deliberately preserved, say, a Burroughs B-25 desktop, or the similar from Data General, they are gone.
But I can also see why people might want to keep more interesting data, like when the Federal Cheese-Sniffing Agency moved offices back in 1982 and they have meticulous records of the 483 filing cabinets that had to be moved from the original location to their new home in Furrytown, Pennsylvania.
When we were moving out of our apartment there was damage to a door hinge that we never noticed when we moved in but that had definitely been there from the onset of our two years of living in that apartment.
Guess what? I had a photo from the day after we moved in of that door hinge in a state of damage! Not because we took the photo for that intention, but because my daughter was playing in the hallway and my wife snapped a photo and it just happened to capture the damage. Saved me several hundreds of dollars in repair costs from my landlord.
You are right, 99% of the data will never be looked at. But do you know what the 1% is today? I'm guessing you don't.
The government doesn't delete anything. It might be moved or inaccessible to the public but that data is somewhere in perpetuity.
It's one of the most deranged larps I've ever seen, then they pat each other on the back on BlueSky, desperately wanting to be a part of something.
These people envision themselves as folk heroes when what they really need to do is go outside and touch grass.
If the government is democratic and values integrity? Sure.
Otherwise I wouldn't bet on it. My own country's history books and my parents' own life stories have already warned me about how fickle democracy is. No democratic country is free from that fact. Some think "checks and balances" ought to be enough to prevent it, but I wouldn't be so sure.
https://www.police1.com/federal-law-enforcement/national-law...
The insurmountable part of that project would be getting the guide data.
You don’t know what other people will want in the future
There's are sites that stream old content with a old tube tv UI wrapped around the video frame but they don't have all the commercials and they don't follow the old schedules like you suggest.
I've got a friend who has hoarded digitized copies of VHS recordings of old cartoons from that era complete with the commercials, so the content is definitely out there.
Once you format shift, you will always be format shifting.
Keep your originals whenever you can.
But from fire-resistant storage cabinets, to concrete-lined file rooms, to underground archives, the tech to make archives ~99.5% fire-proof is more than a century old. And if you add redundant storage sites for the high-value stuff...
Vs. anything digital is far more vulnerable to digital malice.
Keep originals if you can, but make copies ASAP, as close to lossless as possible. Don't depend on the right hardware being around in the future.
Let’s say an individual posted identifying or incriminating information online, inadvertently or intentionally, in a public place.
Then a third party decides to store it, and possibly make it accessible to others.
If the original self doxxing user then pulled the original dox, but was unable to scrub the rest, would that information still be considered public, or would it be private? Was it ever truly public? Or private for that matter?
In my head, I'm imagining someone early in the morning posting a flyer up on a bulletin board downtown.
Throughout the day many folks walked by and took photos of the flyer with their cell phone.
At the end of the day, the original person came back and removed the flyer.
IMO, at the time that the folks took the photo of the flyer, that flyer was public information. It remains public information even after the flyer is removed[0].
This isn't a great analogy of mine, and has plenty of holes, but was interesting to me after I read your comment. I know it was in the context of doxxing, but I think it's pretty interesting philosophically.
I think something similar applies to photos taken of other people in public spaces. Both the person who took the photo and the subject of the photo are no longer in that physical public space, but the actions took place within that space.
I think something similar applies to digital "public spaces". But what does a public space even mean in the context of walled gardens[1], etc.
[0] you then run into the question of what happens if someone posts non-public information, publicly? [1] are digital walled garden communities that different from physical communities that gate access, whether free or paid. Whether information shared within those contexts are public or private is an interesting thread as well.
The tricky part is dealing with inadvertent or malicious (i.e. some other party), posting of private information to a public space. That's really hard to deal with on multiple levels.
For one, the archives would retain the information and scrubbing it is effectively impossible.
Secondly, legitimate things which should remain public (i.e. were posted publicly, are of public interest, etc.) can be argued to have been inadvertently or maliciously posted. So you need some way to moderate and create rulings for each individual case, which quickly becomes untenable due to the sheer volume of information being posted and the inordinate amount of time required to investigate vs. post.
Though given the space in general and some of the people involved it all should be audited very carefully.