Fun with Kermit and ZMODEM over SSH
cambus.net
cambus.net
ooooooo whooooooo OOOOOOOOO AAAAAAAAAAAAAAA !!!!!!!!!!1!! qcjqrjrcorRC!!!
WHAT THE HECK WAS THAT? Did I just do megabytes per second? Holy shit. Am I high or something? What is this hardware? What is going on here?
No... wait... is that the end? Am I done here? No! No, I want to transfer more! More! Megabytes per second! Gigabytes per second! I can see it now, I want it, I want more, please, let's transfer another file, come on man, I want to ride again, please ple
EXECUTION COMPLETED
$I also don't know about data transfer rates but if you want to emulate the really old stuff in DOSBox, you'll need to carefully examine pages like https://www.dosbox.com/wiki/Performance because there are plenty of games that do a divide-by-zero crash when they're trying to time the CPU for running at the desired speed.
“The timing calibration code in the Network Driver Interface Specification (NDIS) driver causes a divide by zero if the CPU runs at 2.2 GHz or faster. This problem does not occur with CPUs that run at 2.1 GHz or slower.”
It was patched for Windows 98 (312108USA8.EXE), but '95 was left without an official fix due to its age: http://lonecrusader.x10host.com/fix95cpu.html
$ ls -lh /usr/bin/cp
-rwxr-xr-x 1 root root 123K Apr 3 19:00 /usr/bin/cp
Before the C64 I used a ZX81 and before that a ZX80 - they had roughly 1 kilobyte of RAM. The ZX81 had a RAM pack upgrade which added 16Kb. It was a bit wobbly and required a strip of Bluetac to stop it crashing the computer when nudged. An uncle of mine calls "bloody luxury" on that - he learned programming with punched cards - probably ForTran.
This laptop's kernel is 12M and initramfs is 14M and a fallback image of 70M. Add to that a microcode image (7M).
You'll be glad to hear that Kermit weighs in at a svelte 26K (I've just installed it from the Arch AUR)
Busybox is 521K and udevadm is 572K. ext4.ko is 2.4M and libsystemd-shared-253.3-3.so is 3.5M. usr/lib has quite a lot of file in it eg libc.so.6 at 1.9M.
Funny part about Elite is that they supposedly limited the number of galaxies to maintain the illusion that it was manually created instead of built at runtime using a pseudo random number generator with a preselected seed.
You are correct - I was conflating it with the later PC effort. When you get to my age (etc).
Sadly, nowadays the plain text of the legalese bollocks that you don't even realise you've signed up to is larger than entire games of yesteryear.
Once when I should've been in college still, I was using the Telix terminal program on a 286 with a 2400MNP5 modem. Now Telix had a really awesome scripting language that could do all sorts of terminal magic, but that's a story for another day.
My connection to the Internet was 8-bit clean, and at the time I was an avid Emacs user/evangelist. (Since then, I have seen the light of vim's face and have never turned away.)
I saw that Emacs included a mapping file for mskermit to generate ESC sequences for Alt+Keypress, and I figured out that if I mapped each scancode for Alt+Keypress to the appropriate 8-bit code, it would convert my Alt key into a Meta buckybit, perfect for Emacs usage!
So I painstakingly mapped out each scancode and transcribed it into the ms-kermit configuration, and before long I was Emacsing in all its 8-bit-meta-glory.
I figured it would be useful to other Emacs users, and so I sent the file upstream. Lo and behold, it was incorporated into the standard Emacs distribution, and it stayed that way for a long, long time.
Unfortunately, since I had never formally assigned copyright or done whatever legal bit needs to be done to GPL the code, TPTB did an audit of the Emacs distro and culled any code that did not have solid legal footing. At that point, ms-kermit was really obsolete anyway, so I suppose it's all for the best!
That worked fine but every now and then, a site would have some inline image that I'd want to see, so I'd view source to get the img URL and then download that to /tmp and transfer the file to my local machine via zmodem. Usually, it wouldn't be worth the effort. I'd also download mp3s off IRC via DCC and queue up a bunch of data for zmodem to transfer overnight or when I was in class. I really appreciated those bytes back then. Now, not so much.
But I could uuencode into a big buffer on my terminal program and uudecode locally.
https://web.archive.org/web/19980627010642/http://www.termin...
slrn --- NNTP/spool-based Usenet newsreader last updated: 2023-03-18T04:31:51 [UTC] repository: git://git.jedsoft.org/git/slrn.git tarfile: slrn-pre1.0.4-9.tar.gz ( size: 1563860 bytes; md5: f193d983e104a82ef4fd70b1037f8b60 ) github: https://github.com/jedsoft/slrn
IRC is not as big as it used to be, but at its highest it was mainstream even for computer illiterates.
I recall that there were some shenanigans with the sale of the company. I think there’s some messages floating around on newsgroups and around the place about it. I found this https://paste.ee/p/Z0ywP
EDIT: Oh! I found it and installed it from https://www.sac.sk/files.php?d=3&l= . Apparently, Terminator isn't a Russian clone like I remembered, but actually the successor to Terminate written by George Collins.
edit: this thread has some more info https://groups.google.com/g/comp.dcom.modems/c/JdOtiDGCV3Y
There's one person in the thread who is skeptical that Bo was in the wrong!
It felt familiar but strange and wrong to do such a thing. Worked great though.
Ymodem is simpler and smaller than Zmodem, but we've used both. I've used Zmodem at 921600 baud, but it needs fairly large buffers (for a UART) to work.
One of the big advantages is that TeraTerm for Windows exists, and supports xyzmodem. It means that I often don't have to write any host-side software for things like firmware update, and more important, don't have to touch Windows.
I have a brand new system sitting here next to me on the bench that uses SLIP framing for packetizing Protobufs :)
I have actually used 'sz' and 'rz' in relatively modern times for quick and dirty file transfer and found it very convenient in a very narrow set of use-cases.
However ...
It's a serious violation of the cleanliness and available attack surface involved in a terminal interface and we should be on the lookout for, and reject, similar interfaces and applications.
In order for zmodem to work over the terminal, the terminal program itself needs to know something about the text flowing over the connection and then invoke special, extra routines based on monitoring that textual flow.
This opens up all manner of weird, extra attack surface.
The beauty of the text terminal is that I can, theoretically, cat any file I want to without fear of what it contains. I can open up (perhaps with 'strings' or 'hexedit') any email attachment without fear of the strings that it contains. I can do this because I am using a dumb terminal.
As soon as the terminal is smart - even a little bit - you've got vectors for weird strings doing things you don't want them to.
But they don’t fork a new process… (unless I’m very mistaken).
pretty sure it won't work over mosh tho
Yep that's 1.5 million times the speed I used to get.
zmodem can also be used in embedded spaces to retrieve files if the only interface is a serial port.
Annoyingly it still supports a maximum of 115200 baud, even though the serial port on the switch is physically USB-C, so on the one occasion we had a switch brick itself and had to use zmodem "in anger" it took quite a long time to upload the replacement firmware.
cp -r
This type of automatic transfer makes it very convenient to generate figures on a server (where the data lives) and then view them locally. It’s a much better workflow than having to use sshfs or scp.
What I wrote is really quite similar to an old transfer program like zmodem with the added feature of auto opening a file if I choose.
It doesn’t matter if you mount a remote server locally… unless your data is of a trivial size, you still want to do the processing remotely.
I see this as a theory vs practice difference. In theory having one unified file system is great and the way to go. In practice… there are issues.
Company I was doing work for was looking at handheld solutions for on site data collection. One of the things they were looking at was something from Symbol. Nice unit, but, $2k a pop.
I was out visiting my father who happened to have an HP-48, which I had not seen before.
Fiddling with it I managed to learn quite a bit about its (now legendary) capabilities. But I was quite surprised to learn it had a serial port and support for Kermit.
Returning home, I bought one and quickly prototyped a solution for the company, where they’d upload the data from the calculators each day straight to the HP-9000 via Kermit.
You could buy the HP-48 base model for $100, and they ran forever on AAA batteries. We ended up rolling out the project and they bought a dozen of them.
That was a really fun project. I coded much of the calculator code lying in bed.
I want to say it was called WinGet, but now there's something else called that, so I can't confirm it.
EDIT: It was called GetRight.
In the spec, yes. But since servers use Content-Encoding instead of Transfer-Encoding for on-the-fly compression, its unspecified now whether the byte offsets in the resumption refer to the compressed or uncompressed file. Result: Download resumption did not work reliably and got removed from browsers.
When someone finally showed me how to set the options (which default to reliable-but-godawfully-slow) for much higher speed transmission (bigger segments, sliding windows for ACK, maybe a few other things?), it was entirely comparable to Zmodem.
I wonder why more than one client would default to such comparatively-lame settings for Kermit though.
IIRC you basically have to get a Kermit prompt and issue your commands to the server. Which is easy if you use an actual Kermit client, like Columbia put out (though apparently for the last 12 years it's been an independent project no longer affiliated with the university), but most people just used the internal version that came with their terminal program - with the primary effect being that most people had no idea it was actually a fast, flexible protocol (see the long explanation at http://www.columbia.edu/kermit/misconceptions.html).
The stated reason for this is that it should always default to working, and it's up to you to configure it to work well if your connection is better than rusty barbed wire.
The default of "always work" is amply demonstrated by the fact that Kermit is available for CP/M, and can be compiled on a working CP/M-80 system from .HEX files that you just blast over the serial port. Obviously, due to system constraints, this is not a fully modern Kermit.
But it works. And if you want error checking and reliable file transfer on a Kaypro, it's not a bad way to do it.
Looks like it's still quite possible, I wonder if our network monitoring tools would have noticed gigabytes of data flowing out of the network that way.
Probably as easy as `cat > sz` and then paste it over the wire? Maybe run through base64 first. Although actually depending on the situation you might be able to just do that directly and not bother with this? Like just as a super simple non-scientific test:
$ ssh someserver cat /bin/ls > ./test-ls
$ ssh someserver md5sum /bin/ls ; md5sum ./test-ls
b3535289b2932e25650074aa6d89bf3c /bin/ls
b3535289b2932e25650074aa6d89bf3c ./test-ls
scp is miserably slow on transferring lots of small files and this can be wildly faster, plus you get your choice of compression program based on what is on each side of the connection and what the bandwidth vs. CPU is.
Then I'd use type to save a uuencoded version of something like minicom on the laptop, use this program to uudecode it, and then use minicom to download anything else (Windows 95 install files?) using zmodem.
Here's an extreme example of such program consisting only of printable (and type-able) characters:
http://www.columbia.edu/kermit/ftp/a/tcomtxt.asm
(Seen on https://retrocomputing.stackexchange.com/questions/5202/tran... )
You can also notice the mention of bootstrap files in Kermit, and look them up.
I've recently seen some better known YouTube user doing exactly the thing we discuss on an old system (even including the consequences of accidental port setting mismatch), but, of course, can't remember the exact video.
Norton Commander, in addition to sharing files over serial or parallel port, was able to “clone” itself to another computer by sending a bootstrap loader the same way, then copying the rest. Hmm, the null modem cable pin connection chart in the readme seems to be a regular one (though without carrier detect pin), but the interlink/laplink cable is shown as 8 bit wide (though the thing should probably work with popular legacy 4 bit crossover):
nc -l 1234 > fname
nc host 1234 < fname
It supports flakey networks and it has tons of features.
You can accomplish the with a new scp session on the client server, but it's an extra step. I use this as a helper when for building the scp command.
function scppath() { echo $USER@$(hostname).$(dnsdomainname):$(realpath $1) ]
Aren't brains amazing, storing all those ancient unused acronyms for decades?
Should work through multiple SSH hops, and not giving the hassle of using scp through those same sets of hops.
I used this frequently to move files back in the dialup modem days, after doing an ugly redirect to get the Zmodem binary on the far end, and because of its superior compression, Zmodem was faster than uucp/uucico (when you were lucky enough to have the uucp suite installed and configured, which unlike Kermit and Zmodem, required root privileges...)
Why this isn't something that isn't supported out of the box is beyond me. Doesn't SSH have a control channel? Couldn't it support this without text mode hacks?