This is amazing! Please never ever use it in production.
In lieu of the submarine cable, something closer to a wet string was being routed over. It had such high latency and packet loss, that a watchdog timer I'd implemented on the stations was timing out.
Fortunately, the remote access we'd built into our stations (SSH and OpenVPN) still worked, albeit at slow speeds, like a 300 baud dialup, and crazy-high latency.
Having occasionally dealt with performance a bit like that as a kid, and knowing my way around Linux, it was like "I've been training my whole life for this moment."
So I just flexed the old command-line-and-editor-when-you-feel-every-byte-transmitted skill, and got the stations working, before the factory even knew about the submarine cable, saving our infinite-9s uptime.
It was nothing compared to what NASA does, but terminal animation effects would've ended both of our missions.
I'd take a solid 300 baud connection over a spotty cellular connection during most emergencies.
Having worked and played on flaky high-latency and low-throughput networks much of my life, I mostly visualize things in my head—as you likely mean by 'command-line-and-editor-when-you-feel-every-byte-transmitted'. Open a connection and queue your commands; wait for output. It works if you don't make any typos.
Preferably, when the connection is too slow or flaky (bad cellular), I make a local script and echo the script to be executed remotely to a file on the remote server, and then execute it with nohup - with the input to create the script and execute it coming from a pre-made file on my local machine redirected to SSH.
Bad cellular often works in bursts, in my experience. Also, redirect output to a file if needed.
This works nicely in situations where your connection may drop frequently for periods exceeding timeouts. On that note, keep ssh timeouts high. With really spotty cellular you might have the network drop for 5-15 minutes between reliable transmissions. SSH connections can stay alive nearly indefinitely if the timeouts allow for it and IP address don't change.
mosh hostname -- screen -S philsnow
> It works if you don't make any typos
mosh helps a bit with that too: you get local predictive echo of your keystrokes and you can't "recall" keystrokes but you can queue up backspaces to cover up your typos. Doesn't help where a single typo-ed keystroke is a hotkey that does something you didn't want, though.
It was fun to think about the path each packet was taking - from my laptop to the plane router, to a satellite, back to a digital ocean computer in the UK, to my home computer in NJ, and back. It wasn’t as bad when you think about the magic there.
Regarding planes/cafes/guest/etc. WiFi, I now usually put any "emergency remote plumbing access" on port 443 (though usually not HTTPS), to reduce the likelihood of some random non-SPI ruleset blocking us in an emergency.
"line is ~300 baud but only transmitting 7 bit but we need to send this critical file.. which is in EBCDIC!"
There is no link that could be used in preference. Any repo that the factory stations could access would be unusable by Singapore AWS. And any repo that Singapore AWS could access would be unusable by the factory stations. But the two ends of the link don't have to use the same repo.
* SITUATION: Factory reports MVP factory stations for pilot customer "not turning on" for the day, reason unknown.
* TASK: Get stations up in time for production line, so startup doesn't go out of business.
* ACTION: Determined cause was unexplained networking problem outside our control, and that was triggering some startup time checks. But that the stations were resilient enough that I could carefully edit the code on the stations to relax some timeouts, and enough packets were getting through that we might be able to operate anyway. After that worked for one station, carefully changed the remaining ones.
* RESULT: Factory stations worked for that production day, and our startup was therefore still in business. We were later advised of the submarine cable failure, and the factory acknowledged connectivity problems. Our internal post mortem discussions included why the newer boot code hadn't been installed, and revisiting backup connectivity options. From there, thanks to various early-startup engineering magic, and an exciting mix of good and bad luck, we eventually finished a year contract in a high-quality brand's factory production line successfully.
No technical wizardry in the immediate story, but a lot of various smart things we'd done proactively (including triaging what we did and didn't spend precious overextended time on), plus some luck (and of course the fact that some Internet infrastructure heroes' work had them routing any packets at all)... all came together... and got us through a freak failure of a submarine cable we'd never heard of, which could've ended our startup right there.
(Details on Action, IIRC: Assumed command of the incident, and activated Astronaut Mode manner. Attempted to remote access, and found network very poor. Alerted factory of network problem in their connectivity to AWS Singapore, but they initially thought there was no problem. Could tell there was some very spotty connectivity problems (probably including using `mtr-tiny`), so focused troubleshooting stations on that assumption. Realized how the station would behave in this situation, and that the boot-ish checks for various network connectivity were probably timing out. Or, less likely, there could be a bug in handling the exceptional situation. Investigating, very slowly due to poor Internet, found that the stations didn't have the current version of the boot code, which would've reported diagnostics better on the station display, so factory personnel might see it and tell us. Using `vi`, made careful, minimal changes to the timeouts directly on one station, in the old version of the the code on there (in either Python or Bash; I forget). Restarted station, and it worked. Carefully did the same to the other stations. Everything worked, and other parts of the station software had been done smart enough that they could cope with the production day's demands. Despite the poor connectivity, and the need to do network requests for each production item that passed through the station, before it could move on.)
Example: ls | tte —no-color slide
I’ll add environment variable checks to the todo list.
What I'd really like to see is this supported in the terminal emulator itself as an idle mode, i.e. don't run this in the buffer directly, but rather as a second buffer. This could function like a "screen saver". Once you interact, your primary terminal would be fully restored.
I’d love some of these effects to fade in/out within live terminal sessions.
This instantly reminded me about Ansible and how it annoyed me that ANSIBLE_NOCOWS had to be enabled to disable the default output of Ansible with cowsay [1].
All these serious enterprise people worried about their cows, but then they do install ‘cowsay’ on their systems xD
https://github.com/erkin/ponysay
https://blog.yjl.im/2016/01/ascii-pony-systempony-screenshot...
Maybe it does, I didn't check.
That being said, there are some environment variables that could be referenced. I'll add this one to the list for a future update.
The only one that comes to mine for me is FileBot[0] - and if the loading screen made it take ANY additional time, I'd be annoyed.
Anything that compresses or decompresses archives, like gzip/gunzip, tar, etc.
Anything that performs longer running recursive disk activities, like du, find, etc
I’m sure there are plenty more scenarios I haven’t thought of. Though to be clear, I definitely don’t want to see this as a default enabled option in any of these.
If I’m looking at real time output, it’s because I’m doing something manually because something isn’t working the way it’s supposed to, and I’m in no mood for anything that doesn’t help me understand and address the issue.
This work is impressive. I love it. For some reason I really like “printer.” I’d be all about it for games or toys. But too many tools for real work try to do cute Unicode things or fancy colors or animations. I don’t need it on anything I will ever have to develop or troubleshoot, personally. After the first lengthy wait for a broken container to come up, I’m well over the colorful attitude.
An example of this done well would be ‘pacman’ package manager where an Easter egg can be enabled to turn the progress bar into a little Pac-Man gobbling pills. It’s not there by default, but it can be enabled if you know how.
Personally, I’m in the same camp as yourself where I’d prefer our tools didn’t have all this cutesy stuff. But people like us seem to be a dying breed.
No.
Although I don't know if much software does it, that should still be possible to do it, so these animations could theoretically be auto-disabled on slow links. (And you could manually override the baud rate in your mosh terminal if necessary since it doesn't really have any other effect in a virtual terminal, not like when using an actual serial port, I mean.)
Edit: popular example of less invasive animations would be new docker cli, (pulling, buildx build)
Better than either, of course, would be effort spent on speeding up the build in the first place. I realize that's less fun, though.
In general, the critical development path is not a place to thoughtlessly add friction.
Production will always be an inappropriate place. QED.
As an "about" screen, or in a opt-in funny theme why not !
It's been a while since I dared assume anything at all was so obviously absurd that no one could ever think to do it - not least since significant portions of my career to date have consisted of cleaning up after people who had.