Show HN: Shpool, a Lightweight Tmux Alternative
github.com
shpool is a terminal session persistence tool developed internally at google to support remote workflows, which we have open sourced.
github.com
BTW I think your readme shouldn't just characterize it as a resumeable ssh tool. I often need to start a long running process that I want to reconnect to later, or I want to put some always-on system service inside a tmux container so I can easily jump in and see logs or mess with things in one way or another. There's a lot of utility besides just handling network dropouts.
As a result, any tmux-like layer needs to emulate a console in order to get a view into the state, and then re-render that emulated console on attach to restore state to the terminal emulator that you're connected from.
From the readme, this tool does that, kinda. I'm actually confused about why they'd go to the effort of implementing a VT100 emulator, write the code to redraw the screen from it, and yet not bother with doing the work that would let multiple terminal emulators attach.
This feels like it sits in a weird place between simple, crude tools like dtach, and tools like tmux; shpool has done most of the work to implement tmux-style behavior, and then decides to cut weird corners.
> I'm actually confused about why they'd go to the effort of implementing a VT100 emulator, write the code to redraw the screen from it
Well, we kinda cheated here. shpool_vt100 is just the already existing vt100 crate with a single critical bug fixed, so it actually wasn't much work :). Turns out having a nice package manager for a systems language comes with some benefits.
I'm actually open to adding a feature to allow multiple simultaneous connections to a single session. I never really had a usecase for it personally so I haven't prioritized it, but it is something that similar tools support and people keep bringing up. Since this isn't the first time I've heard people talking about it, I just made https://github.com/shell-pool/shpool/issues/40 to track work adding the ability to attach multiple clients to the same session.
> This feels like it sits in a weird place between simple, crude tools like dtach, and tools like tmux; shpool has done most of the work to implement tmux-style behavior, and then decides to cut weird corners.
I'm not aware of any tool that does internal rendering and subsetting handling scrollback and copy-paste in a way that I personally find usable, so these decisions were very much intentional.
I think tmux is a great tool for a lot of people, and I tried to get into it for years, but I could just never get over the weird scrollback and copy-paste issues or the fact that it meant that I couldn't use my normal `i3`/`sway` bindings to switch between terminals inside a tmux session. If tmux works for someone, I think that's great and they should keep using it. shpool is meant for people like me who aren't very good with computers :).
Scrollback and copy paste cannot always be configured as you want. I’ve shared some specifics elsewhere in this thread.
And there's https://github.com/numToStr/Navigator.nvim that unifies the keys by letting the outside layer (tmux) always ask the inside layer (vim) before any movements.
Although that is indeed a lot more setup. But works pretty well.
- CTRL-p, f (regular fullscreen)
- CTRL-p, z (zoom/remove borders)
Seems like most of the features you need are what mosh offers. I've been using it for a decade, probably, and it is pretty awesome for latent mobile connections (read as: throttled 2G @16kBit/s with interruptions).
> If you want to have 10 concurrent connections, you have to open 10 ports.
That is a problem in most environments.
Either you waste lots of traffic bandwidth because you have to have a session identifier or nonce in every packet, or you have to map sessions to ports in order to guarantee persistance when the client drops its connection.
Other ways of doing session handling will lead to an attack surface that can probably be used for DoS attacks.
Maybe I am missing something: How would you solve this, given the limitations of UDP and TCP?
For a while I was running neovim on dtach, which let me host all the terminals I might want. It has long felt weird that we have so recursively many layers of management and navigation: the OS juggling multiple terminals, tmux juggling multiple shells, and neovim juggling multiple windows. I was doing pretty good in full screen neovim for a while, but have backed down a bit, still find tmux navigation a bit faster.
Similarly dvtm, which is a minimalist version of the window management aspects of tmux but omits the session management, also never quite seems to click for me.
IIUC, the main difference is that shpool maintains an in-memory copy of terminal state that it can use to re-draw after a reconnect, so you don't lose all your scrollback.
Tbh that's why I'm not a fan of tmux or terminals with fancy internal window management. I suppose it makes sense if your OS level window management is crap (macos...) but otherwise it feels like poor solution
-- in which case Divvy is a great solution. YMMV, but IME, Divvy and iTerm2 cover 98% of the things I'd want to do.
My normal workflow is either to be sitting in front of my Linux machine, or SSH'd to it (from a Windows laptop). Tmux makes the transition between those two seamless. But nesting tmux doesn't work.
Active: active (running) since Fri 2024-06-14 19:26:55 +05; 27s ago
Invocation: d84b9281e0f64a798c4e555836815036
TriggeredBy: ● shpool.socket
Main PID: 3895609 (shpool)
Tasks: 15 (limit: 154377)
Memory: 1.4G (peak: 1.4G)
CPU: 6.566s
CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/shpool.service
├─3895609 /usr/local/bin/shpool daemon
├─3895629 -fish
├─3895719 systemctl --user status shpool
└─3895720 pagerOther tools that can overlap in use case are EternalTerminal and mosh which provide sessions that survive disconnects.
I did notice they have a vt100 emulator, but it's only used for screen restore?
It seems the focus of the tool is for reconnecting from the same machine.
[1] https://github.com/nelhage/reptyr [2] https://github.com/lkundrak/retty
Implying "tail -f $LOGFILE" is akin to tmux reattaching to a console is a stretch.
In those cases, I've found dtach to be slightly more reliable than nohup. Frequently I'll try one and switch to the other. Less frequently, I'll correct users' configs/workflows that are necessitating the troubleshooting in the first place. Even less frequently, I'll dig into what was actually the issue :)
https://linux.die.net/man/1/dtach https://dtach.sourceforge.net/
Shpool doesn't appear to allow for session sharing/remote pairing though?
I recently finally switched from screen to tmux and I found that it's pretty seamless (especially because I use vi keybinds in tmux still). Some machines at work don't have screen anymore.
Incidentally, this "screen absent" is why you should always volunteer to help create base shells (programmatically via packer not by hand) - emacs and tmux are always there when I made the image.
I don't make the image.
shpool:foo foo@bar ~ $ shpool detach foo
printf: usage: printf [-v var] format [arguments]
-bash: "\033]0;%s@%s:%s\007": command not found
-bash: "${USER}": command not found
-bash: "${HOSTNAME%%.*}": command not found
-bash: "${PWD/#$HOME/\~}": No such file or directory
tmux/screen do not break copy-paste. And that tmux/screen have their own scrollback is a feature.
The selling point for shpool is simplicity, which I appreciate. I need terminal multiplexing, but I might use shpool here and there.
Tmux breaks interacting with the clipboard so much that it has its own dedicated Wiki page dealing with all of the different issues and settings: https://github.com/tmux/tmux/wiki/Clipboard
However, you can have your own multiplexer and use shpool on a window to connect to a server. Done, you have your favorite multiplexer, with your configuration, and the only tool in the server is shpool
I've been using screen/tmux for a long time. Recently I switched to kitty[0] locally. I like kitty a lot! But I've been stuck with tmux on my servers for session persistence.
I'm guessing based on the Change-Id in its commit message <https://github.com/shell-pool/shpool/commit/6cef5782e358716a...> this was exported from some internal Google repo but for the rest of the world it makes $(git clone --recursive) real mad
Is it tested on macOS? Ready for use in production?
Thanks
shpool is a service that enables session persistence by allowing the creation of named shell sessions owned by shpool so that the session is not lost if the connection drops. shpool can be thought of as a lighter weight alternative to tmux or GNU screen.
Not sure why I'd want to switch.
Furthermore, in my opinion, both the defaults (C-a, C-b) are bad, as they conflict with default readline key bindings for cursor movement.
The main difference is that tmux is a power user tool that you generally invest in configuring and tweaking to get just right, while shpool is meant to be set up quickly to solve the specific problem of dropped connections and then mostly forgotten about.
[1]: to be clear, I don't mean "bloat" in a negative way. I'm a long time tmux user and I love it, absolutely love it, and won't be giving it up anytime soon. I don't personally think it has any bloat. But, I use the features. If one only wanted persistent connections, then all of tmux is quite a bit of bloat. It's all perspective.
it simply allows you to have multiple shell/term sessions on a server and be able to switch between them from any terminal. Kind of a 'flat ctrl-Z' cum 'nohup' if you will.
No windows, splits, etc, no client software at all in fact.
I love tmux but it’s definitely not “n00b” friendly.
# run your command in it
# Ctrl+b D to detatch
later, to attach back to it
$ tmux a -t 0
That probably covers 50% of usage. The rest would be naming the sessions, splitting the window panes. Anything else?
That's a key feature when you don't reflexively use less and ] isn't a natural keybind for "let me scroll please".
Window management, mouse support, key bindings (I really dislike the defaults for a few of them), selection/copy/paste, macros etc
All stuff that doesn’t need to be configured but if you look at any tmux tutorial you’ll see boat loads of details about. Which can be confusing because how do you know if you don’t need to know it?
Also what’s the basics for one person isn’t going to be the basics for another. Eg I rarely use the detach capabilities these days whereas the panes and windows are something I do use lots.
My point is this: tmux is sufficiently complicated that a complete beginner might not know what features they need to learn.
Scrolling works pretty well for me.
Obviously "nothing in life is free" - the person you're replying to literally spent a bunch of time writing a solution to their problem.
"what do you expect?" - the person literally wrote what they expected after having written the code to do exactly that. OP has been all over this thread being incredibly diplomatic in both explaining why they wrote a replacement for (part of) tmux, and constantly following it up with "but by all means if that doesn't sound right to you, use tmux it's a great tool".
It feels a lot like "well, if you don't like it go do it yourself", which is deeply ironic.
Without tmux, your terminal emulator uses its scroll-back buffer to render as you scroll, not requiring any intermediate copies of the lines of text scrolling into view.
With tmux, your terminal emulator no longer handles that, and instead tmux must pass a copy of each line freshly scrolled into view to the terminal emulator, which involves an intermediate copy of that text and re-triggers the terminal emulator’s parsing (as far as your terminal emulator is concerned, it has never seen this line of text yet). All of that (and some other subtle overhead I’m not mentioning here) are completely avoided outside of tmux (or more specifically: whenever the terminal emulator is free to manage its own scroll-back).
People say the use the likes of alacritty for the speed, and then they run tmux and make it slower than even gnome-terminal.
It comes with this tool’s benefit of native scrolling/cp paste PLUS the huge benefit of “right click to split vertical/horizontal”.
Most GUI features (new tab, new split, scroll, search, copy/paste, etc.) just work, and it all syncs with `tmux`.
Be aware though that it can be a bit buggy if you have a fancy `tmux.conf`, and that if you rely on any `tmux` plugins then most of those simply won’t work.
It is pretty annoying that the only terminal that I know of that supports tmux control mode only works on mac.
You'd get pretty much same experience, with textual splits, modeful keyboard shortcuts to access scrollback & copy/paste etc. — which I'm not saying is bad but tmux already does that. Well, I suppose over ssh it'd allow you to access remote tmux session(s) from a local tmux, so maybe allowing little more flexibility mixing panes from multiple remote hosts(?) But still pretty much similar.
The whole point of control protocol was to allow native [G]UI where each pane feels like a native standalone terminal, no?
1. Ability to use local tmux configuration for tmux sessions over ssh. So I don't have to worry about using the default bindings when connecting to a host that doesn't have my tmux config, mouse support is already enabled, etc.
2. Ability to combine and organize remote panes alongside local panes, or panes from other remotes
3. Avoid ending up with nested tmux, where I have a remote tmux session inside of a pane of a local tmux session, which is a pretty terrible experience.
I'd also love it if linux terminals supported it too, but that doesn't seem terribly likely to happen anytime soon.
Mosh implemented a flag to include in the client side: —server=… which solves this problem.
I also have dedicated .ssh/config entries for ensuring that ssh connects directly to tmux:
Host whatever-mux
Hostname whatever
RequestTTY yes
RemoteCommand /usr/bin/tmux -u -CC attI am much happier with that muscle memory than whatever the iterm shortcuts would be.
- tty keystroke router: A program that lets you send keystrokes to different tty's (ideally tmux sessions) but not taking over the actual terminal (so sort of a background application , a daemon that listen to a certain key bind and upon pressing it will route all the keypresses to a certain tty/tmux session.
- tty screen real estate extender: Basically be able to link one tty (tmux-window) (attach to it) from two different terminals. These two terminals one of them will be displaying the first N columns second terminal attached will be displaying from N till width of the tty (so you can hook up two monitors and widden your ttys)
If you can extend your terminal window across your two monitors and run `tcvt` within it, that might suit your purposes.
Too much hassle to muck with backports and package pinning for a QoL tool - my feedback would be to try to make this install without tomfoolery on the stable branch of major distros.
Seems to have pretty modest install dependencies?
https://github.com/shell-pool/shpool/blob/master/debian/cont...
git clone https://github.com/shell-pool/shpool.git
cd shpool
sudo apt install libpam0g-dev
# you may need to install build-essential and other packages too as the
# build-depends field in debian/control is incomplete
dpkg-buildpackage -b
sudo apt install ../shpool_0.6.2_amd64.deb
systemctl --user enable shpool
systemctl --user start shpoolThis benefit of compiled languages is often overlooked by folks who are (mostly/only) familiar with dynamic languages like PHP, Python or Js/Ts
So it is easy to solve this problem—just provide pre-compiled binaries on GitHub Releases. I believe the rust ecosystem made this relatively easy.
Not something I make use of though, feels faster to copy using the keyboard instead.
My best guess is that your terminal selection is overriding tmux.
I tried Zellij and couldn’t get the Alt key to work on mac. And then when ssh’ing into a server I couldn’t see some of the icons because it required a special patched font.
The suggestion usually is to run dvtm inside of dtach/shpool/abduco/diss for layout, and separating the concerns that way.
This is very interesting to me as it seems to provide persistence with much less cognitive overhead than tmux or even screen sessions, which are enough that I often don't bother and just use SSH. I also find tmux has a noticable latency overhead (on every terminal I've tried) so I wouldn't use it locally anyway.
Suffice to say this is very interesting and I'll check it out.
What if the computer user does not use a window manager. Many years ago I stopped using X11 and started using textmode only. Then tmux was released and appeared in NetBSD packages collection, it soon went into NetBSD base and I have been using it with almost zero problems ever since. If I am forced to run a graphics layer so that I can run a window manager so that I can run shpool, then is this solution truly "lightweight". The shpool program might be lightweight but the system I would need to run it would be heavyweight, when compared with textmode, no graphics.
For me, an assessment of "lightweight" also needs to consider the compile-time requirements. For example, I use a static copy of GCC that weighs in at 242.3 MiB and I can obtain libevent and libtinfo, the only two third-party tmux dependencies, from any number of mirrors across the internet. I like that GCC does not try to connect to the internet by default, or otherwise. When I lose an internet connection, I use the time to compile programs. GCC, and probably most if not all C compilers, add no friction to doing that.
On the other hand if your use case is just windowing on a single host (local only or remote only) then probably stick with tmux.
I use tmux for session management, including but not limited to windowing, on each single host. I generally work _outside_ tmux, using send-keys to interact with each window. I have daemons running in their own windows.
There is no graphics layer, no window manager and no terminal emulator. I stay in textmode.
I make heavy use of tmux features like send-keys and buffers. I use buffers, namely load-buffer, paste-buffer and save-buffer, as "copy-paste".
I do not use a mouse. I "select text" using UNIX utilities, not cursor movement.
The tmux scrollback buffer is not an issue because I work outside of tmux. For daemons running in tmux windows, I use multilog and tai64nlocal for logging.
The copy-paste issue is a non-issue because the "native terminal" for me is textmode and textmode, unlike a terminal emulator, has no copy-paste.
This tmux setup is lighter weight than running 1. a graphics layer, 2. a window manager, 3. a terminal emulator and 4. shpool.
my browser is my terminal now, and I can easily upload/download files via browser (dont type SCP anymore) with a mouse click, open terminal windows, open Python notebook and do stuff.
life has never been easier after I completely switched to jupyter notebooks+terminals
plus, because it is browser based, I can roam from multiple devices.
Also cells can be out of order and overwritten and such. Seems like a questionable fit.
The key to out of order problem is simple: once a piece of code works - save it in .py module and import it. Very little discipline is required to achieve significant productivity gain.
That way notebook is always clean and tidy and always resembles a “pseudo code”, with implementation detail hidden in modules (that were tested and are working).
If you are using notebooks for notebook coding tasks then fine but what does that have to do with tmux?
1. Jupyter has terminals feature in addition to notebook (New->Terminal instead of New->Notebook). You can run any number of terminal sessions, they will be persisten like tmix and each open in separate browser window.
They are also shareable and easy to collab, so you can toss the terminal URL to your colleague to let him continue where you left off, or let him monitor stdout as you run some long running process.
2. You can run shell command and capture its output in python var inside jupyter cell with “!” macro, for example:
files = !ls *.csv
files = !ls *.csv
pd.concat([pd.read_csv(x) for x in files]).MyColname.value_counts().to_frame().to_csv(“summary.csv”)
This is more readable and maintainable than bash oneliners and magic with awk and more powerful
Things like read cav, json, write parquet, any manipulation with data, both local and remote is a breeze and I dont feel like constrained by bash syntax.
Most importantly code is readable and maintainable