Dead USB Drives Are Fine: Building a Reliable Sneakernet
changelog.complete.org
changelog.complete.org
Sometimes I wish for a return a that feeling of preciousness, instead of incomprehensible amounts of data carelessly shoved down wider and wider tubes.
For my own sneakernet-like uses, I turn to git-annex. See use case “The Nomad” here: https://git-annex.branchable.com/
Record your neice blowing off the candles at her birthday party, and the next day, her mom asks you for the original (high quality) video file, so she can create a party-video... the file is shot on a phone in 4K and uses 300MB of space.
Mail? Nope, too big. Chat platforms? Too big. Cloud upload? You have to share a link, and some services (ahem, skype) actually open those links when you send them... who knows, they might even save the files there. Also, do you really wan't personal videos in the cloud? HTTP/(S)FTP servers too much of a pain to set up for a one time need. We atleast had DCC on irc, now not even that.
So a USB flash drive it is... and a car ride.
We actually made a great solution for this in the 90s: mailservs, inspired by Usenet. We made AOL bots that would take a file, chunk it into 1.5MB sections, and send each via email attachment. Then a client side program would download all the attachments and reassemble them. We downloaded entire movies this way. After download, the mailbox was emptied; if it wasn't, the next transfer would fail.
So people just need mailserv programs to send files, and download their mail regularly.
For example: what is storage? A router stores packets temporarily; is that storage? A DNS resolver caches records; is that storage? A browser stores cookies; is that storage? A chat or email server & client may both store messages in different states for different purposes. What storage should be shared and shouldn't be, in what specific circumstances? Who is allowed to read and write to which thing at which time in which circumstance? All of that will affect your abstraction.
It's sort of a shame that so much of the new web3 storage work intrinsically links storage to a blockchain, instead providing a blockchain integration as one of many authentication/provisioning models.
Let's think of the simplest possible implementation of that.
A todo app wants to write to 'your storage'. What are your options for storage? If it's some thing like Google Drive, that is a proprietary interface, so right off the bat, you are now implementing vendor-specific things, so an abstraction is not worth much. You could make some kind of "JavaScript Framework For Storage", aka a js library that has 50 different proprietary implementations, but that would only work for javascript apps. Which, if you only code in JavaScript, is fine, but if you want to support some other application written in another language, now somebody has to maintain those 50 different proprietary implementations in another language too. That's just not sustainable.
If instead you want to create one standard web API to access storage through, even if you could define exactly what it should do, you now need to get Google to implement that one standard web API. But why would they? They already have their own Google Drive API which works perfectly well for their own purposes. You would have to show them some significant business advantage to throwing away all the money and code they've sunk into their own API (not to mention that all their customers and their apps have sunk into it) and build and adopt this new API. (That assumes you could even get them to agree to whatever standard API you created, as they may want a half dozen extra features that have nothing to do with storage)
By modeling the whole system, you can quickly see all the weird quirky problems that you will run into trying to make this API and get it adopted. It's not impossible, but it's a much larger problem space than you imagine at first.
Tasks like this or even punters(html heading tag denial of service through instant message lol) were what got me interested in programming in middle school, no training or education, just hacking around trying to figure things out in a pirated VB3.0. The community was great with private chatrooms full of users testing and helping others, trying out tools. People shared libraries to learn from, use, adapt. Anyone remember the popular old and terribly named genocide.bas? I'd like to get my hands on some of these old proggies and code if they're still out there.. Or even just the cartoon sheep program which would run around the desktop interacting with window borders.
edit: the way I'd solve the rest is just by using a torrent magnet link. The problem with that is that you might have to teach someone how to open a port for UDP traffic, but after they've done it, sending files of any size to them becomes trivial.
With that said, better infrastructure (IPv6 + better connections) can make P2P very feasible in the future, hopefully. Or software that defaults to local connections if it's possible (so if we can find the device via a private IP, use that connection instead).
And doesn't Dropbox work for the given example? That's the mass-market productized/paid/marketed/surveilled solution.
I think GP's point is that this churn indicates that it's not a "solved problem". It's clearly solvable, but whoever runs these services doesn't find a revenue stream. It should probably be provided by your ISP just like email and usenet once were, but the cultural expectation isn't there. Maybe they get shut down because people use them for bad things? I don't know.
I think you're really just wanting some big name or trustworthy player like Dropbox to come out with a similar service. It's not going to happen--like I said by design wormhole and similar tech doesn't send or store data on any central service. There's no value for some big company to give people this service, they're literaly just burning bandwidth to shuttle invisible (to them) bits and data. They can't extract anything like advertising targeting, revenue, etc. from the service. It will always exist as some weird self-hosted thing.
And of course a lot of time spent copying - that second floppy drive made my life so much easier !
Shame it arrived so late in the game, because it was really an ideal match for floppies. Five disks of data, two... three... four? disks of parity, use 'em only if you need 'em.
Dear Friend,
Please tell my story.
Sincerely,
Dave Koresh
But the FBI raid of the Branch Davidian compound in 1993 caused... unrest in certain pockets of the American politisphere, primarily on the right but groups like the ACLU also got involved, and Janet Reno was mocked on SNL for her role in the incident. Nobody liked the Branch Davidians, they were a weird-ass cult preparing for an apocalypse that never came except maybe in a parrot sense[0], but there were concerns that the FBI (and potentially the ATF before) acted too aggressively, overstepping legitimate law-enforcement bounds and violating the Davidians' rights, and that caused a lot of political introspection: on our status as a freedom-respecting republic, the competence of our federal law-enforcement apparatus, what is the threshold beyond which the government legitimately could or should take action against weird-ass prepper cults. So there was a lot of philosophy and politics stuff in there, often in the form of USENET postings and other 90s internet copypasta, that dealt tangentially with those issues.
It felt... kind of weird and awesome to have and to read that stuff. Cyberpunk. Here were thoughts that people felt not quite safe transmitting or discussing openly, so they were copied and distributed as floppy-disk samizdat. It was a mysterious object, alarming at sight because it suggested that Koresh might still be alive somewhere (and the shortening of "David" to "Dave" made him seem... humbled?), whose contents held even deeper and more thought-provoking mysteries.
[0] "I heard tell once of a Jefferson City lawyer who had a parrot that would wake him each morning crying out 'today's the day the world shall end as scripture has foretold'. And one day, the lawyer shot him for the sake of peace and quiet I presume, thus fulfilling, for the bird at least, his prophecy." -- Daniel Day Lewis as Lincoln in Lincoln (2012).
I proceeded to walk a floppy over to his house 2 miles away.
I'd love for it to have some kind of expiry on it after which I need to fsck it.
There is "git annex expire" (https://git-annex.branchable.com/git-annex-expire/).
Edit: Appears at least one of the files added is called “ “.DS_Store” and is still around:
https://appleinsider.com/articles/22/02/19/google-drive-user...
Expect Mac software does fuck all to prevent it "leaking", so pretty much every zip from a Mac user contains one of these in each folder. Only case we had desktop.ini do the same was with Dropbox.
It can be toggled to avoid it on removable devices when they're connected to Macs. It's benign, though I guess annoying to some non-Mac users.
I wish it were built-in.
If this were handled at file system level, the FS should pretend all disks are online to let me list files and directories. When I want to access an offline file or mkdir or perform any write operation on offline disks, it would raise an error and tell me which disk to load into the server. Then I try again.
If NNCP doesn't do that, is there and Linux file system able to do it? I googled but I didn't find much.
This website been around since at least 2010, but number of the drops listed were physically removed, but not delisted from the site:
Do think though as a social experiment they’re fun as long as you’re only using throwaway systems/devices.
Beyond that, personally would not suggest damaging property, but instead affixing USB to padlock like this:
All and all, basically same as geocaching [1] — which obviously could be dangerous if someone wanted to be malicious, but never heard of that happening; have heard of malicious USB dead drops though.
If I ever grow some more software clue, I'll be trying to unify all of the above into a single interface. Wish me luck.
_____________
Toosheh being a “satellite filecasting technology deployed in Iran and the Middle East that uses common satellite equipment to deliver digital content without relying on access to the Internet” per Wikipedia:
That said, unlike cheap and replaceable USB sticks, they’re easy to find using RF analysis. As such, unless it’s physically secured, likely quickly be stolen. Also likely require a power source and weather proofing.
I think many people still do it like that. There are 3.7 B people in the World without internet.
Good times
This is really amazing stuff. Years ago, I was heavily interested in scuttlebutt [1]. I'm interested in grid down and "occasionally connected" communications. I particularly liked the island to island sailing analogy.
SSB has a few downsides - mainly that your client needs to download full logs. You can't just request the last 30 days. They call this out as well that your first sync could take over an hour and consume several gigs of data.
I'm just today learning about NNCP, but I've used FidoNet and UUCP in the past, so the concept is pretty familiar to me. In my mental model, I would be less interested in sneakernet than standalone wifi hotspots that one could connect to and exchange data, possibly combined with an AREDN style mesh[2].
[1]: https://staltz.com/an-off-grid-social-network.html [2]: https://arednmesh.readthedocs.io/en/latest/arednGettingStart...
Yggdrasil can also run as an overlay network atop standard Internet (IPv4 or IPv6), or both. It will opportunistically find peers on a local broadcast domain and find routes to other networks over the standard Internet if need be.
Feel free to drop me a note if you like; I had a very similar experience with SSB. NNCP, while it has a bit of a learning curve, Just Works. It processes thousands of packets for me every day (hourly ZFS snapshot backups for every filesystem I have), some of which are huge, and it Just Works.
It would be ultra cool, IMHO, to fly SSB and NNCP traffic between "islands", whatever form those take. For bonus points, do it with pigeons, which have the advantage of operating in a GPS-denied environment.
I don't mind the full sync for SSB, I mind the lack of any progress indicator while it's doing so. But that seems like an easy UI tweak and maybe they've addressed it since I last played with it. What I haven't figured out is whether a brand-new user can sync from another existing user on the same island, or if they need a real internet connection for that first bit.
― Andrew S. Tannenbaum
> NNCP is to UUCP what ssh is to telnet; NNCP is an Encrypted, authenticated, onion-routed version of UUCP!
_____
* Core issue I have never been able to workout is how to know given the potential for firmware hack how to make sure only the known trusted data makes it out of the isolated processing system to the trusted system; open looking at anything though as long as it open source, doesn’t have to solve that specific issue.
https://en.wikipedia.org/wiki/USB_Killer
And interesting, always knew high voltage was a threat, but never thought of capacitor being used, since it wouldn’t require an independent power source.
In most cases a chain of cheap hubs would work too.
Commonly two types of isolation data and power chipsets. For a dead drop, you would want a cheap voltage isolator set to trip/blow a fuse at 5 volts DC — with no data isolator.
So what do you do if someone puts 120 volts on the data lines?
You very much want to isolate both. Which is precisely the function of the chipsets I mentioned.
No, no, no.
A USB data-line isolator does not block data flow. Its entire purpose is to alow data to be communicated across a galvanically-isolated gap. It uses magnetic (transformer) or optical isolators to do that.
It blocks POWER flow, while passing data. Look up the ADUM3160 datasheet.
Besides Filespooler, any other options?