Pingfs: A filesystem where data is stored as ICMP Echo packets (2016) | Hacker News Reader

Pingfs: A filesystem where data is stored as ICMP Echo packets (2016)

github.com

119 pointssigna1144 comments

This is exactly what I needed - not a lot of people are doing eventually inconsistent databases yet. If theres a network partition, shits probably fucked anyways so its best to throw away all your data.

Unfortunately the metadata (list of files) is stored in memory, not network. Not quite as unreliable as I would like.

Sounds like your looking for something like MangoDb https://github.com/dcramer/mangodb

Note: joke assumes parent poster was somewhat humorous

nsafs might be another good choice: https://github.com/freedomtools/nsafs

That's innovative!

I didn't realize there were so many of these jokefs projects.

It's like the software dev equivalent of those silly domain jokes:

https://helenkellersimulator.org/

http://motherfuckingwebsite.com/

http://thingsididlastnight.com/

I wrote a joke filesystem that does nothing but provide 1x1 PNG files[0]

(I was bored and asked for dumb filesystem ideas on IRC)

0. https://github.com/sector-f/colorfs

Another good one: https://github.com/philipl/pifs

Obligatory: "Tokutek claims to improve MongoDB performance 20x. It is unclear if this also means losing 20x as many documents."

Eventually inconsistent databases could save a lot of the effort developers faced with databases without inconsistency guarantees put into implementing eventual inconsistency at the application level.

Just had a quick look to see the general flow.

Only looked at the read path, which is roughly:

* filesystem receives read

* blocks on a semaphore (https://github.com/yarrick/pingfs/blob/master/chunk.c#L141)

* network thread wakes up semaphore when ICMP reply containing data is received

* filesystem thread copies data into new buffer to return from syscall

* filesystem relinquishes chunk back to network thread, which wakes up, sends it back to be echoed on the network via ICMP, and frees the in-memory buffer (https://github.com/yarrick/pingfs/blob/master/chunk.c#L107)

How silly :)

This reminds me of delay line memory, storing information in the echo of a sound: https://en.wikipedia.org/wiki/Delay_line_memory . LEO had long tubes of mercury for this, not sure I would have wanted to work in that office.

Same principle, but using slabs of glass, was used in PAL analog color television until digital memory got cheap enough to implement the delay line.

I think very few bits here are actually stored in-flight on the copper/fiber/radio at any time. Most of them are probably stored in various buffers of networking equipment (which are ordinary semiconductor RAM). It would be interesting to see someone do an actual breakdown for, say, a transatlantic link.

My gut feeling says that only geostationary satellite links have any significant number of bits in-flight.

Very simplified, but assuming a 5000 km cable and 300 000 km/s speed of light (is slightly slower in fiber), light needs 0.01666... seconds for that. At 10 Gbit/s (fairly slow), that'd mean ~ 166 Mbit or ~ 20 Megabyte in flight. (I hope I didn't forget a factor 1000 somewhere)

Your calculation makes sense. However, for those 20 MB in flight, how much data is stored in buffers? I think that could be approximated by comparing ping time with speed of light.

e.g. if a 5000 km long link at 10 Gbit/s has latency of 0.0166 s, then you have zero buffering. If you have latency of 0.2 s, then around 2 Gbit of data must be stored in buffers somewhere, making 20 MB in flight only 1% of all data stored in such a ping filesystem.

How about the earth-moon-earth path for 768,000km?

EDIT: For the EME path that's around 3 gigs in flight

We've found a viable way to a space elevator: Sell it as a delay-line storage device.

In a small toroidally shaped universe, your radio is also your memory. All emissions eventually return to their origin, but you can reorder memory transmissions by moving while the data is in flight.

Actually, Engineering Research Associates (predecessor of Univac) built a CPU with mercury delay line memory using acoustic transducers to create a shift register. It did bit-serial binary arithmetic. You could get two logic gates or one latch out of a dual pentode, so a bit-serial ALU could fit easily in a single rack.

I'm not familiar with the overall architecture, though, unfortunately. I do remember having to do a most-significant-bit first, bit-serial, two's complement adder as a homework assignment that was inspire by the machine.

This is the perfect transport for GNU Terry Pratchett.

http://www.gnuterrypratchett.com/

Previous discussions: https://hn.algolia.com/?query=Pingfs%20points%3E3&sort=byDat....

Reminds me a little of PingTunnel :) https://www.cs.uit.no/~daniels/PingTunnel

The author of PingFS is actually the same as iodine (IP over DNS: http://code.kryo.se/iodine/).

Prior art https://www.shysecurity.com/post/20120401-PingFS

Of course the gut reaction to this is "why?" and the obvious answer is "because I can"... that's fine, but there's been plenty of filesystems built on protocols or things that weren't really supposed to be filesystems.

I remember way back when gmail first announced giving away 1GB of storage, there were a bunch of hacky things to turn SMTP into a real filesystem.

So, I get the answer to "why make this weird filesystem", but I'd really like to know: why is this weird filesystem, weirder or more intriguing than the others that have come before it.

This needs IPO.

You mean ICO, right?

Want to buy some ICMPCoins?

Ping Coin or i'm out!

Can't wait for the ICO.

redundancy == broadcast ping

Cool .. all we need now is a host waaaaay out in space, and we can just use space itself to store all our data (not that we aren't already) ..

not exactly new: https://hn.algolia.com/?query=pingfs&sort=byPopularity&prefi...

Linux is the only intended target OS, portability is not a goal.

Goodbye then.

You can fork it and port it to something like dokany to make it work in Windows.

Based on Annatar's profile, he'd like it to work on FreeBSD.

illumos and it’s the programmer’s “Linux only” attitude which irritates me.

I get your sentiment in general but... you're upset because a tool for storing files in IMCP pings isn't available on your OS? Poor you, where else are you going to keep your files!

No, I’m irritated because of the programmer’s attitude, which is quite systemic in programming and has been for a while.

Well, it's not merited, and you're being rude. I'm saying this as someone who references the POSIX standard several times a week to ensure portability. This kind of tool is a toy project and is by its nature highly dependent on non-portable interfaces.

If you’re against GNU and Linux like I am then it is very much merited.

I utterly fail to see how being honest and direct by expressing my irritation is rude and am both insulted and appalled by what I perceive to be the Anglo-Saxon definition of rude hoisted upon me. I could just as easily turn this around and say that your lack of consideration for what is a different value system is just as rude. The entire argument around “rudeness” is nonsense, I’m not at all sorry to write.

Buddy, I hate GNU and Linux as much as you do. More, probably. You and I are on the same side here. You can either continue to attempt to extract yourself from me into a special class of person for which your behavior is supposedly acceptable, or take a good look at yourself and decide to be better in the future.

I’m not doing anything wrong.

wosh

Reply on news.ycombinator.com