Protocols that have not changed in the last 30 years
dmitryelj.medium.com
dmitryelj.medium.com
MIDI-over-USB has been popular for a while, but we're seeing a resurgence of "classical" MIDI over serial with the current synthesizer renaissance.
And of course it's all the same on the software side.
Note on:
There are obviously a number of 2.0 features that are much more advanced than 1.0 (MIDI is primitive). But MTS is actually pretty good and not a hack.
I'm looking at the so-called "1.0 Specification" circa 1995, and it's document version 4.2.
https://gemini.circumlunar.space/
It's still obviously very small, but Gemini space is growing.
The Gemini protocol is 13 months old.
Some day, someone explained this allows for a client(with low bandwith) to setup a transfer between 2 servers(high bandwith) with the data not moving through the client. I've never seen this usage and don't know any ftp package supporting this, but it's pretty neat nevertheless.
Simpler answer: because Firewalls didn't exist at the time!
* https://en.wikipedia.org/wiki/Lftp
* http://lftp.yar.ru/features.html
See also:
* https://en.wikipedia.org/wiki/Comparison_of_FTP_client_softw...
> The original specification for the File Transfer Protocol was written by Abhay Bhushan and published as RFC 114 on 16 April 1971. Until 1980, FTP ran on NCP, the predecessor of TCP/IP.[2] The protocol was later replaced by a TCP/IP version, RFC 765 (June 1980) and RFC 959 (October 1985), the current specification.
* https://en.wikipedia.org/wiki/File_Transfer_Protocol
Unlike TCP or UDP, NCP was not duplex:
> NCP preceded the Transmission Control Protocol (TCP) as a transport layer protocol used during the early ARPANET. NCP was a simplex protocol that utilized two port addresses, establishing two connections, for two-way communications. An odd and an even port were reserved for each application layer application or protocol. The standardization of TCP and UDP reduced the need for the use of two simplex ports for each application down to one duplex port.[1]
* https://en.wikipedia.org/wiki/Network_Control_Program
So the original transport layer that FTP relied on necessitated two ports, and when the move to a newer transport layer occurred the use of two ports was carried over for simplicity's sake. And several decades later that design still exists.
> FTP needs two ports (one for sending and one for receiving) because it was originally designed to operate on Network Control Program (NCP), which was a simplex protocol that utilized two port addresses, establishing two connections, for two-way communications. An odd and an even port were reserved for each application layer application or protocol.
* https://en.wikipedia.org/wiki/File_Transfer_Protocol
Per RFC 114, responses from the server are considered a data type: when a user sends a command to the server, sometimes the response is a status update ('MKDIR successful'†) and sometimes the response is the requested file.
The data structure of the response has fields to say what's coming back on the data channel for the requested transaction.
† MKDIR was not a command, just using it as an modern example.
There were specific people whose main role was that they had access to one source server and one or more target servers. Without spending more than a few kB of their own bandwidth they could transfer files between big servers easily.
I also remember talking to some people back in the day who regularly did that supposedly. Members of some minor warez group who uploaded new releases to one server and then used FTP to directly transfer files from one server to another to create mirrors. Then retransfer from the two servers that now had it to even more servers for some exponential growth.
Unfortunately, this assumption has failed the test of time, as the "peer to peer" nature of computer networks has been fundamentally degraded due to widespread adoption of a more strict client-server model. Today, it is mostly not true that any two internet hosts can intiate connections with each other, and while seldom phrased this way the reality is so deeply embedded in our understanding of the internet that protocols designed under this assumption, while common, seem absurd. There are numerous reasons for this that range from security concerns to IPv4 exhaustion to the simple practicalities of the way that computer networks entered widespread usage.
We should all wonder: is it a good thing that we have abandoned the peer to peer nature of networks?
NAT’s might go away under IPv6 but firewall policies with “default deny inbound connections” will almost certainly not.
This possibly surprising fact, that at the core of the P2P distributed system there is a centralized client-server service that provides the initial discovery information to begin participating, is pretty much universal to P2P systems because of the limitations the internet imposes. Removing this requirement would require a significantly more P2P-architected internet that allowed, for example, participation in arbitrary multicast groups from residential ISPs. There are both theoretical and practical (but mostly practical) challenges to doing this.
In other words, the way the internet works, the only real way to begin to participate in a P2P internet protocol is to connect to a centralized service to provide initial peer hints. This is commonly referred to as an "introducer" or "bootstrap node" or "peer helper" or other terms depending on the P2P system. Hypercore and the Hyperswarm P2P network it relies on are actually unusually advanced in how much they minimize the need for an introducer, to the extent that a Hyperswarm node can completely forget about the introducer (it could go down) after initial setup and it will probably continue to work okay although there is a probabilistically small risk of a split-brain scenario. Older P2P protocols like Bittorrent and Tor have a greater dependency on centralized introducers and cannot operate without them even after initial startup (with caveats such as Bittorrent's support for Kademlia DHT, the same underlying DHT protocol used by Hyperswarm, which is in use by some Bittorrent nodes but not all for various reasons).
For hypercore specifically the introducers are bootstrap1.hyperdht.org through bootstrap3.hyperdht.org. These are basically hardcoded into your client.
Is there a “centralised” server under Bitcoin? Or is it you can bootstrapping even manually then the peer to peer network can go ahead to find more peer …
The problem of “centralisation” is not due to the need of finding peer. It is in the edge. You control the isp you control all. See China.
How to break this edge issue … very hard.
Yes, Bitcoin clients use a semi-hardcoded list of "seed peers" that they look up via DNS and connect to first. Peer discovery proceeds from there using P2P methods. One example is seed.bitcoin.sipa.be, there are several set in the source tree of bitcoin core. Each of these "seed" DNS names provides a list of records that are known good Bitcoin nodes. This DNS method replaced Bitcoin's original method, dating back to Nakamoto, which was to connect to a hardcoded IRC server and then join a hardcoded channel where Bitcoin nodes advertised their presence. There is no real theoretical advantage to the DNS method but it is simpler and DNS is far less likely to be blocked than IRC.
As in most distributed/P2P systems, the Bitcoin DNS seeds are somewhat integrity sensitive. Normally the Bitcoin network cannot enter a "split-brain" state (e.g. two independent and equally valid blockchains) because there are a large number of nodes which are strongly interconnected, preventing any substantial number of Bitcoin nodes being unaware of blocks that other nodes are aware of (in actuality Bitcoin enters a "split-brain" state on a regular basis, but as long as nodes are aware of both "valid" blockchains they have an agreed upon convention to select a single blockchain head as valid). But this is only true of nodes which are already participating. When a new Bitcoin node starts for the first time, it has no way to discover any other nodes besides the DNS seeds. In theory, if the DNS seeds were malicious, they could provide a list of nodes which were complicit in an attack by intentionally not forwarding any information about some blocks or advertisements of nodes which are aware of those blocks. In other words, in practice the cost of a sybil attack is actually reduced to the number of nodes directly advertised by the DNS seeds, but only for new users and only if the DNS seeds are complicit. In practice the former is a massive limitation and the Bitcoin project allows only trusted individuals to operate DNS seeds in order to mitigate the latter, so the issue is not that big.
The problem is that there actually is no decentralized way to initially discover peers on the internet. Imagine if you wanted to implement a peer-to-peer information flow over the telephone system. If you have a list of telephone numbers of participants you can come up with all kinds of schemes. But what about when you very first join? You don't know any numbers to try. Randomly calling every phone number in order isn't feasible. You need to have some kind of "point of contact" to first discover other peers or you simply have no one to call. There are various terms for this, I like "introducer" but Bitcoin uses "seed" as do some others. Every decentralized system that operates over the internet has a semi-centralized introducer mechanism somewhere deep down.
There are several theoretical mechanisms such as multicast groups, but none of these actually work across the internet. They do usually work across local networks (and are used by things like Bonjour), but in the majority of cases when a user wants to bootstrap a P2P system there is no other peer on the same local network.
I can't agree that it has failed the test of time. Peer to peer is still the superior solution to nearly every problem.
What happened is that the Internet became commercialized and now profit runs everything. Centralized solutions are better at: injecting advertising, tracking users, requiring subcriptions, ensuring lock-in and preventing interoperability. None of these are good qualities for anyone involved, except the company making the profit. But these companies now rule the internet, so they hate peer to peer.
Just to point out, it was realized. In the early days everyone ran their services (SMTP, NNTP, later HTTP, finger, telnet, etc, etc) locally in a 100% distributed fashion.
It's just that then the commercial interests showed up and ruined everything with centralization, ads, spam, tracking.
Getting acces to the internet (AKA, getting an IP block and routes to connect to others) has always been sort of centralized, mainly because ip blocks need to be handed out from a single source of truth to prevent ip address collisions.
Still, the peer to peer nature of interconnecting still exists, there is no hierachical information source which defines routes across the internet. This is all decentralized and done by doing peering between parties with BGP (And other routing protocols).
Doing this yourself is possible, although rather hard and niche for a consumer. But getting an AS number is doable, and getting an IPV6 block is even easier. IPv4 is hard to come by though..
Yes, this is basically all an accident of history, but so is everything else about computing. There's no real sense in which the original "network of peers" IP concept has been successful since IP spread beyond the realm of the National Science Foundation. It is possible to build systems which are far more resistant to these problems but it's difficult to begin with, and on top of that adoption would require turning the battleship that is the internet around. We're sort of stuck with the situation we found ourselves in, which is that we build peer-to-peer system by piling a few hacks on top of IP to enable peer-to-peer application layer usage.
When Microsoft made Windows Firewall opt-out with Windows XP SP2, it caused SOOOOOOO many headaches. On one hand, no more rogue porn pop-ups during work! On the other hand, nobody knew what a firewall was and why trying to use Skype or whatever was such a PITA.
It’s a bit of a shame that one simply cannot use the Internet without a firewall of some kind (unless you really like getting cryptoransomed in, like, 30 seconds)
This was a paid product, though?
Sendmail ran locally and I'd send/receive email to/from the world directly on my workstation. Same for FTP, HTTP, etc. That's how the Internet was originally intended to operate.
What's a firewall? By that I mean, firewall wasn't a thing when FTP was designed.
That said, eternal-september.org is a great way to get back into usenet.
Just do it and post a reference implementation of a server and a client (assuming this is the architecture that you want to use), and the protocol specifications somewhere.
First: The parent was talking about FTPS, which is the FTP protocol spoken over TLS, similar to how HTTPS can be HTTP/1.1 spoken over TLS
Second: SFTP is not FTPS, and it also isn't FTP spoken over SSH. Instead. SFTP is a protocol specifically for SSH which, like FTP, is concerned with file transfers, but matches what its authors actually wanted from file transfers in the modern era, rather the concerns FTP had decades ago.
at what point did I mention that FTPS is SFTP??
my comment regarding SFTP was in reference to the parents comment
>but I left FTP for ssh long ago and never looked back.
but you're correct that >SFTP is a protocol specifically for SSH
I had also understood your comment to mean that SFTP is "just FTP tunneled through SSH" (i.e. FTPS) rather than a redesigned protocol.
It is quite fast, however, compared to SFTP/SSH.
As my username will attest to, 300 bps modems were very common in the 80s. I got my first one in 1985.
My first modem was 110. 300 was exciting, and 1200 was outrageous.
That, and reverse engineering GORILLA.BAS :-)
These days I use irssi, so am completely out of touch with mIRC.
Why the “Pfft”?
ircII scripting felt vaguely C inspired, and mIRC's scripting definitely felt foreign to the UNIX/IRC culture it got attached to. I never got seriously into scripting mIRC, and the mental model might have changed, but draft 1 of adding scripting into mIRC definitely had that Rasmus Lerdorf "I don't know how to make a language at all, I'm just making something work" feel, whereas ircII scripting felt vaguely coherent from a C-ish background.
Very old impressions, and mIRC scripting might've gotten good in later versions. The earliest versions of mIRC scripting definitely felt very weird.
Side note: I found the Rasmus analogy particularly relatable. I’ve been building in PHP for a little over 20 years now, and despite all the hate it gets around these parts, I remain a fan.
I don't think any of that entails that FTP is somehow failing to fulfil its contract. Implementing transport encryption is not within its scope. It has to be run over a secure channel, which can easily be done with FTPS.
That said, it is an interesting retro blast from the past. As someone who cut their teeth during the BBS and early days of the web, it always brings back nostalgia. Computers were just more fun back then IMO.
FTP is amazing on so many perspective. About a month ago, I crawled the entire IPv4 range looking for anonymous FTP and found about 200k of them. As I wanted to publish the results and provide a way to get through those servers, I ended up building crawlers but quickly stopped as the sheer amount of movies, series, music I found would have been more problems than it's worth.
Lists of the contents of Anonymous FTP servers have been produced since at least the 1980s.
And in fact, they still exist:
http://fy.chalmers.se/OLDUSERS/ivi/ftpsites2.html
http://www.diam.unige.it/informatica/documentazione/httpd_do...
Back in the day, you'd download updated lists to find the stuff you were looking for. Because that was how you found stuff. Although there wasn't a lot of video or music out there on the public anon FTP servers.
That was before Archie[0] and Veronica[1] and, of course, websites.
Apparently, there was a discussion[2] about this here back in 2016, as well.
I have fond memories of wuarchive.wustl.edu, simtel20.army.mil[3] and a bunch of other FTP sites I can't remember 30+ years later.
A good topic. Thanks for posting it. Perhaps you should publish your list, or stand up an Archie server for it. I'd use it.
[0] https://en.wikipedia.org/wiki/Archie_(search_engine)
[1] https://en.wikipedia.org/wiki/Veronica_(search_engine)
For instance, BGP can be used as an mac learning mechanism (EVPN), It can be used to communicate MPLS LSP's, it can even be used for source routing.
Disappointed that NTP is not listed as it may well be the oldest continuously running protocol.
Telnet text based muds are one of my favorite things we used to have. You’d telnet to a host and play a text based game where you’re trying to make progress walking through a world solving puzzles.
I worked for a major Usenet provider for 20+ years.
The mechanisms for posting large binary files essentially created a separate application layer on top of the protocol. uuencode was replaced with yEnc, and the applications built in parity checking to handle the provider problems with missing articles. Article subjects were co-opted in semi-standard ways to allow applications to combine individual articles into files, but eventually, they just started posting files of unique message IDs (NZBs) to websites or specific groups, completely bypassing the need to even look at article lists. That ended up being a requirement anyway, because some groups got so big, readers could not realistically download lists of new articles anyway.
That is still the norm. Massive files are still split across 1000s of posts, and the most popular binary groups are now "dump" groups, where the only thing common about the posts is that they are binary files of some kind. What's crazy is that even though Usenet readership has declined, Usenet posting volumes have continued a very steady increase.
It suffers from all the same problems that UUEncode did, and that MIME had already solved. Using magic words in the body text as delineators, splitting parts using the subject line, it's terrible.
It’s one of the best ways to get new files today and been around since 1988.
It's the commercialism and hierarchicalism that ruins everything.
None of the protocols discussed in TFA have anything to do with the web. In my opinion, www was created with a strong server client model, and was not as "democratic" as some of the other protocols prevalent at that time.
First I want to make browsing web apps feel as good as browsing an app. Then I want to make it better. To do that I want to 'translate' the HTML into a common structure and create UI that can be similar across websites and rendered in a clean, easy to navigate way. Not sure if that makes sense. Join our Discord if you want to chat more about it (https://join.wapps.app)
I can see javascript,python updating every few years and once or twice a decade for HTTP,SMTP,SSH,Rust,C++. But that's just my opinion, what I stand firmly on is avoidance of extremes in either end.
More importantly though, I think JS has really benefitted from small improvement scopes: there's only a few new features that really matter ever year. That, along with clear roadmaps leading up to a release have made adopting new features easy.
Gopher is starting to look pretty good.
finger south_pole@graph.no
Or, if you want the weather in Boston, Massachusetts, try:
finger boston@graph.no
I think it was Carmack who introduced me to that protocol through a dev talk, if my memory is correct.
Obviously one thing you can do is, for every resources spin up a new HTTP connection to the remote server. These connections have no relationship to each other, so that's a bunch of extra overhead but nevertheless your browser today will probably try to do this several times per server. Still that's pretty inadequate, surely we can do better?
So, OK, HTTP/1.1 also has keep-alive. We can re-use connections, ask for a resource, wait until that's delivered, then fetch another one.
But notice this is strictly sequential. If you're a Unix person it's natural to try the next idea for HTTP, pipelining. Don't wait until things are delivered, just ask for everything you want, surely the remote server will provide one at a time, reading your next request, serving that, then the next one and so on, right?
But for 1990s server programmers excited by threads this doesn't work. Their server has one thread gathering requests, and for each request it spawns a thread to provide the answer. If I try to pileline my HTTP requests I get muddled nonsense back from their server. Sometimes. Is that forbidden by the HTTP specification. Sure. Did it happen anyway? Yes. So your browser does not use pipelining in HTTP/1.1
Also, this suffers head-of-line blocking. If I request CreditCardStatement19960402.pdf and you must send a tape robot to retrieve that, too bad, even though I also wanted bankname.jpeg and hideousfont.css I can't retrieve those because I'm waiting for CreditCardStatement19960402.pdf. The network is idle while we wait for the robot.
Google's servers and Google's browser spoke SPDY back in the day, which fixes a bunch of this. That doesn't exist any more because HTTP/2 was standardised and has the same fixes plus other stuff, some of which proved very useful, other things not so much. Only one way to find out.
... <file1, bytes 2048-4096> <file2, bytes 0-125> ...
The server has to order critical stuff first on a pipelined connection to reduce latency but this approach can easily send stuff the client requested and will request, while collecting data serverside by any number of threads.
If you just meant, "Why didn't we ignore how the HTTP/1.1 protocol is defined and do something else?", that's not what a protocol is, you're not going to build a successful application without agreement about the protocol.
A client of mine would like to start blogging, and I was hoping to recommend the write.as free tier. I'd be interested to hear what other alternatives folks here recommend, or a write.as invite link from a paid user if anyone has one to spare.
https://medium.com/interaction-reimagined/regular-expression...
https://scribe.rip/interaction-reimagined/regular-expression...
- Monopolistic platforms
- Annoying advertising
- Creepy tracking
- Walled gardens of content
- Needless SPA-ification
- Engagement optimization
- Gamification
- Dark patterns
The only things we've really lost are MIDIs, <blink> / <marquee>, and Flash. And for some reason people don't like bedazzled mouse pointers anymore.
For me Medium seems to be the only acceptable blogging platform, but neither do I use it myself as I don't blog, nor do I visit any non-tech/-science content on that site.
But I do use a very strict setting in uBlock Origin, so maybe that's causing the site "to behave". And I only visit it on the desktop.
To authors everywhere: Stop writing on medium.
Good ole Google Ads on a WordPress blog seems fine historically.
At least substack loads fast…
writing is a power. even though literacy seem down - you can use this to make your point. not just to convince people, but to spark discussions that allow us to try to chip away to get to the underlying truth of the matter.
face it - most of the authors on medium aren't really insightful or entertaining enough to get paid.
there is plenty of room in society for open mic nights
Though I would not expect some piles of money unless you have an army of invested readers.
Here is a list: https://en.wikipedia.org/wiki/Modem#Evolution_of_dial-up_spe...