Take Two: Eshell | Hacker News Reader

Take Two: Eshell

yummymelon.com

122 pointsnanna67 comments

I feel like most folks misunderstand Eshell when going into it. It's not really their fault, as they _expect_ something bash-like, and in many ways it is. However, Eshell is _not_ bash. It's not zsh. It's not fish. It's Eshell. When you use Eshell, you get a marriage of command-line tools and Emacs. This isn't always the best case, and some tools are awful to try and use in Eshell.

Eshell shines when you're looking to use Emacs to do stuff in the shell, but not really deal with the shell itself. It will dutifully run whatever program you feed it, but the real magic is when you want to deal with the output. You can redirect outputs to buffers, which you can then manipulate with Emacs familiarity. You can even handle remote sessions just as easily as you would open a remote file in TRAMP. I've used this on many occasions to edit a remote config file, then trigger Eshell, which puts me in that directory, to then restart the service without having to launch a separate SSH session or terminal.

While there are a lot of clunky bits of Emacs, I feel like Eshell gets an especially bad rap. Some of it's justified: It presents itself as a bash-like interface, while actually having vastly different ways of handling things. I feel like folks don't really get the idea that it's an interactive Emacs session, a worksheet for the command-line that lets you ingest data from external tools in a more ergonomic way than doing `shell-command` or similar.

Yes it may be the case that the achilles heel of Eshell is that it tries to be too bash-like. Just making it clear that this is an "elisp thing that interacts with both emacs and your system" and call it "dialog" or "interactive scripter" or something and dispense with following normal shell conventions (while still offering tab completion etc) might have been wiser.

I've been using Emacs daily for well over a decade now, but I just can't get myself to use any of its shells, or features that replace standard shell workflows.

I think the main reason—besides muscle memory and the scripts, aliases, and commands I'm already used to—is because a POSIX-compliant shell exposes a generic interface for both programs and users. Programs can receive and output data over standard file descriptors. They have a CLI with common design conventions regarding flag and argument parsing. Some programs choose not to follow these conventions, and some have a very complex CLI, but for the most part users can expect a specific interface to exist. Man pages have a standard format, and way of accessing them for all programs. And so on. All compliant shells on Unix-like systems work in a similar way, which means that the workflows I'm already used to are easily transferrable to any other system.

Contrast this with the workflow table in TFA. It's a mishmash of different Emacs modes, each with their own quirks, UI, and ways of interacting with other modes. One could say that Elisp ties it all together, but it's a general purpose programming language which obviously doesn't enforce a cohesive user interface for packages to follow. Eshell even complicates this because it resembles a POSIX shell, but doesn't follow any Emacs interface guidelines, so it breaks conventions of both sides. Sure, I could force myself to learn the intricacies of each package over a long time, and create a cohesive Emacs configuration that works for me, but I would need to lug around that configuration and depend on Emacs for everything. Which I'm sure Emacs purists would claim is the way to go, but I don't like that. The beauty of Unix systems is the ability to easily replace any tool in your workflow without impacting everything else. For example, I've mostly replaced `ls` with `eza`, `grep` with `rg`, `find` with `fd`, `top` with `btop`, etc. These were essentially drop-in replacements with minimal impact to me. Consider what it would take to replace dired or Magit, as perfect as they might be.

The Unix "small programs that do one thing well" philosophy is often ignored, unfortunately. Emacs might be the single biggest offender of it, though I suppose it's to be expected since "GNU's Not Unix"... The benefit of it is precisely the flexibility it gives to users. As much as I love Emacs, I'm not married to it, so relying on one special program that does many things would be harmful for me in the long run if a better alternative appears in the future.

I am well past the phase of trying to pull everything into emacs. All reasons to not use eshell you mention are valid, and I can add one more: vanilla eshell doesn't really work well with visual commands.

But recently I installed eshell-eat, and my biggest paint point just went away.

I never expected to use eshell as a full-blown terminal replacement. I use half A dozen aliases (find-file dired...) , some lightweight lisp scripting, git that, apt install there... That's about it.

In that capacity it's a valid quick tool.

Same here. I just use

https://github.com/akermu/emacs-libvterm

and

https://github.com/suonlight/multi-vterm

eshell:

  Welcome to the Emacs shell
  ~ $ btop
  ERROR: Failed to get size of terminal!
  ~ [1] $

shell:

  $ btop                                                           
  Terminal size too small: Width = 79 Height = 22Needed for current config:Width = 80 Height = 24

If I increase the width and height, I get an extremely weird output.

ansi-term:

Same as shell.

Any other Emacs shells I missed?

In any case, I do not use any shells within Emacs. I use XTerm separately.

For interactive stuff you probably want eat https://github.com/kephale/emacs-eat or vterm

(I don't use them, I prefer my shells to last even if I should restart emacs)

Don't say that too loud or the "you should never have to restart Emacs" folks might hear you!

Visual programs won't work within emacs standard shell interaction. Only line oriented programs works (printing one line at a time and exiting). But for many visual programs, you have the equivalent in Emacs.

libvterm https://github.com/akermu/emacs-libvterm

M-x vterm

btop => https://i.imgur.com/wZowuzV.png

Does it have to be the GUI version of Emacs?

No. Vterm works fine even in a terminal version of emacs.

I never understood the need to do everything in Emacs either. I love Emacs, and I use it for all kinds of things, but the UNIX ecosystem has great tools too and it seems silly to have to choose one or the other.

Calling shell commands from Emacs and inserting their output into a buffer is great, don't get me wrong. But Emacs' terminal experience pales in comparison to plain ol' xterm.

At the end of the day, I use the tool I'm most comfortable with and does a good job. Sometimes that's Emacs, sometimes it's not. Maybe if I hadn't spent 20 years in the shell before picking up Emacs I'd feel differently.

I agree. I do have a bunch of aliases that allow me to send the output of a command into an Emacs buffer, that open a file, open a file as root, open a file on a specific line and things like that which smoothen out the flow. However, I'm much more comfortable in the Emacs+Terminal environment than in pure Emacs.

> But Emacs' terminal experience pales in comparison to plain ol' xterm.

[lib]vterm reduces this paling to almost zero.

vterm is great - I had it in my config for a while and it's hands down the best way to put a terminal in Emacs. But for me it's the whole "encapsulate a terminal in Emacs" concept that I find unuseful. That's a personal preference though - I've been used to having dozens of xterms scattered around my desktops for longer than I've been using Emacs, so that's just more comfortable for me.

I'll use regular ol' shell occasionally when I just need to throw around some commands but for anything that needs a real terminal emulator I just open another xterm.

There _used_ to be a huge advantage to pulling everything into emacs rather than having to background emacs, do whatever, then bring emacs back into the foreground. With multiple windows or multiplexers like tmux/screen, it's a lot less necessary, but still nice.

>but the UNIX ecosystem has great tools too

well, the big advantages start when you aren't in the UNIX ecosystem. One of the biggest benefits of using Emacs and Eshell heavily for me is, I have it on a Windows machine.

That's true, although I find that if it's a machine I'm going to be using for a while it's easier to go the WSL route and run Emacs from there.

> The Unix "small programs that do one thing well" philosophy is often ignored, unfortunately. Emacs might be the single biggest offender of it, though I suppose it's to be expected since "GNU's Not Unix"...

It's not because of GNU (which is a Unix workalike), but because Emacs originated on ITS and Genera Lisp Machines, which were truely very non-UNIXy environments.

Ah, that makes more sense. I'm not very familiar with the history.

I use a personally-adjusted variant of M-x shell for most of my last 20 years of Emacs use (after trying various hacks including things mentioned in this article). I end up having about 20–100 named shell buffers on each machine, different projects, logs, etc, and I keep all histories of each shell in separate history files. It helps that the buffers are practically infinte size and I can use standard Emacs editing for anything, plus comint, which is a great simple interface to shells. I have some tools to split the Emacs frame into multiple pieces and reorganize them quickly so I can see any number of my open shells, limited only by the monitor size. I dont understand how people live without such shell management solutions; I’ve seen expert tmux users struggle with keeping track of all shells, whereas Emacs has tons of tools to work on buffers that directly work: starting from simple ibuffer, bookmarks, previews, to multi-occur (which can specialize to shells or subsets of them), to whatever you wish to do with buffers really.

btop/htop/nvtop or other curses-heavy tools have specialized solutions (open vterm or eat and run it), but all the composing unix tools run in M-x shell.

I think if you were to write a post (or better yet a video) to demonstrate your usage of your workflow, that would be a great resource! I love working in M-x shell but your workflow sounds next level.

> These were essentially drop-in replacements with minimal impact to me. Consider what it would take to replace dired or Magit, as perfect as they might be.

There's a difference of philosophy there. The Unix way is a good concept to align software, but it's not the only one. In Emacs, the primitives are different. Instead of having pipes, redirectors, thinking in terms of lines,... What you have are buffers, windows to displays those buffers in, frames to hold those windows, a lot of functions to manipulate the text in buffers and keybindings to activate those functions. There's also the minibuffer for input and the echo area for messages.

So using those people creates packages that solve each a particular task. But the packages are not closed programs you compose together. They are both libraries and a particular way to use the library. If it does not fit your purpose, you alter it. If it's missing something, you add it. If your workflow requires two or more packages, you link them together, creating a metapackage for that workflow.

So Emacs is something to mold to your own needs. The default configuration is just the most common and the traditional way to do stuff. It's not lego bricks like the Unix way, it's clay. You don't replace ls with eza, you turn ls into eza.

Right, the primitives are definitely different. But that's my point. The Unix primitives are simple and generic, which allows the user to quickly and easily adapt a disparate set of tools into an infinite number of workflows. The user is not required to also be a programmer[1]. If they are missing a specific tool, which is very rare, then it can be written in any language and integrated into existing workflows by following the simple design conventions. This generic interface is why programs written decades ago are still useful today. They don't need to be aware of how they're going to be used. They just need to follow the established design conventions.

In contrast, if a specific Emacs package doesn't do what the user needs, they must use Elisp to change it, or write their own package. Integrating packages in a cohesive workflow also requires programming, and each integration is special because every package has its own quirks and API. This might be second nature to some Emacs gurus, but I don't want to program whenever I need to change my workflow. I'm sure that I could eventually build the perfect operating system for me this way, but I'd rather rely on an extensive ecosystem of tools written by others.

[1]: I would argue that shell scripting is not programming, for better or worse. It's often the best tool for the job for getting things done quickly, but for more sophisticated tasks a programming language is a better fit.

Emac's primitive are much more involved for sure. But the focus on having a simple and common representation for input and output instead of the separation between prompt and output has make it a lot easier to write involved tooling. Instead of a TUI application which breaks the unix philosophy or a script for a single workflow, you have both. The closest I think is now Vim+tmux, but it's now as easy to modify on the fly.

vterm takes care of at least the vast majority of the display issues that emacsian shells typically suffer from. It's literally a shell as a buffer. htop and other curses based TUI's seem to run fine in it.

I mean do what works for you of course, but I found it a vast improvement over the historical emacs shell things, which I've also used for decades.

> For file management these days, Dired is my interface of choice. It is simply the more elegant tool for the job. Case in point: renaming a bunch of files in a directory to arbitrary names. With a shell, my standard approach would have been to run some variant of `ls -1 > foo.sh`, edit `foo.sh` to insert `mv` on each line to a renamed target file, `chmod u+x foo.sh` to be executable, run `foo.sh`, delete `foo.sh` and call it a day.

I used to do this before I discovered vidir. [1] vidir is not actually tied to vi; it lets you rename and delete files through your `$EDITOR`. You edit a temporary file that looks like this:

  ...
  0093<tab>./fstab
  0094<tab>./fuse.conf
  0095<tab>./fwupd
  ...

There is also https://github.com/bulletmark/edir designed to improve on vidir. I haven't used it yet.

I'll give Emacs another try some day, I swear, and Dired is one of the reasons.

Edit: [1] I linked to https://github.com/trapd00r/vidir here at first. That is a fork; vidir comes from https://joeyh.name/code/moreutils/. You probably want the original. It is packaged for Debian, FreeBSD, Homebrew, etc. as part of the package `moreutils`.

Is <https://github.com/trapd00r/vidir> the same as `vidir` from <https://joeyh.name/code/moreutils/>?

Shout out to moreutils, there are many nice utilities, and there's a `moreutils` package in most distributions.

Update: Looking at the authors listed in the man pages, the `trapd00r` version looks to be a fork of the moreutils version. Not sure how the functionality compares. Both versions seem to have been updated since the fork.

> Both versions seem to have been updated since the fork.

Thanks for pointing this out. It looks like the @trapd00r version has added `--sort` but lacks zero padding for the numbers that improves the UX. I mistakenly thought it was a fork tracking the original. I have edited my comment.

I didn't know of vidir, but I just thought I'd share some funny etymology, since I know someone called Viðir.

Besides meaning vi + dir, the Icelandic name means "willow tree", which is a cool name for a directory traversal tool.

There is a code witch named Vidrun (https://aphyr.com/posts/341-hexing-the-technical-interview), presumably with similar roots.

Emacs Rocks Episode 13 is a wonderful example of how Emacs utilities quickly compound over time. It's worth checking out its editable dired buffer example (with multiple cursors) but also stick around for the ending. Great finale.

Every few years I put time into learning eshell and it's seriously impressive.

But what I really want is eshell outside of emacs. In fact I want all the bits of emacs outside of emacs.

The problem with eshell is the problem with emacs. It's a monolith. I know I know that's also its strength, but what I really want is for all the bits of emacs to be exploded out into components that make up an overall environment (yes, yes, Genera/Lisp Machines).

Unlike emacs purists, I am too much in and out of that environment to really ever feel fully comfortable adopting all its tools as my mainstay tools.

>But what I really want is eshell outside of emacs

https://www.gnu.org/software/emacs/manual/html_node/eshell/S...

I like the point raised by the author here about the power of elisp in the shell. For context, I've used emacs for well over a decade and write a good bit of elisp most weeks. However, I ended up finding elisp to be the wrong language for replacing a shell: it removes a lot of simplicity/convention for not enough reward, imo: it's just the wrong abstraction because elisp's most natural transformations are very stateful and buffer based, and, additionally, it really doesn't provide that much expanded capability in the end (and this is despite the fact that I'm one of those people who would absolutely never leave my emacs environment even for web browsing if I could feasibly make it work consistently for the kinds of things I need).

However, despite disliking python, I found https://xon.sh/ to be really excellent and provides a lot of the benefits of eshell, but better, in my humble opinion: it has a well-enough structured base language that is essentially already built to be a library glue-code language, which works really well with the unix philosophy that traditional shells utilize (a good library is just a tool that does a thing really well). It's really nice to just do things like this example

    from somelib import process_image #returns a file name that it saved to
    for f in $(some_executable_that_prints_a_file_list).strip().split("\n"):
      img = cv2.imread(f)
      cp @(process_image(img)) publish_dir
      make publish

It feels a lot like an extension of old shells to me rather than a complete replacement.

The writer is incredibly knowledgable. But all of this is completely unapealing. To me this isn't power, it is just learning a lot more things.

The power comes from the ease with which you can adapt your workflow to varying requirements.

If this description doesn’t cover areas you need to work in, forget it.

An example: you need to do a bunch of work in a pipeline, but at one point during development you want to stop, examine the output, possibly edit it, and then finish the pipeline. In this case, doing the work in emacs feels good because you can put the intermediate results in a buffer. Not life changing, but nice.

Yeah that sounds cool. I will keep an open mind and dip my toes into it at one point.

I liked all the parts that were not about eshell. Wdired and its cousin wgrep are killer. I didn't see much value in the eshell examples provided.

Eshell clicked for me when I started treating it like most other Emacs modes, thus enabling my existing/favorite utils. The lines between shell and editor start to blur: https://xenodium.com/yasnippet-in-emacs-eshell

I wish someone could write a "The Good Parts" book for Emacs. Even after reading the manual twice, I'm still overwhelmed with the API and forget even the most basic configuration options after a month of inactivity.

No matter you do, inactivity means forgetting. Why do you need the api memorised if you don't use them?

I mean, using the editor itself means maybe 10-15 keys/commands.

Mastering Emacs is a good overview, btw.

Yeah, the book is definitely worth the asking price.

It's worth noting that you'd have to pay for the book only once. Every new edition you'd get for free. Mickey keeps the book updated and relevant for every new release of Emacs.

There's truth to this. Emacs is large. I tried RTFM twice, thrice .. not skimming, taking time. And even then I learn later that there was some feature in it that I completely missed. And it's a common joke in the community "I've been using emacs for <N> decades and I didn't know that...".

Blessing and a curse ? Maybe they should rebrand emacs as a universe and not an editor.. so that people know there are many galaxies to explore

The thing is that you don't really learn Emacs. Emacs is just an interface to programs written in Elisp that happens to have editor-like functionality out of the box. There are specific conventions it follows, and Elisp has its own set of quirks and a steep learning curve if the user is not fluent in other Lisp languages, so those can be challenging to learn. But boiled down to its core, there's not much to Emacs itself. It's actually a pretty bad editor OOB, but that might be subjective.

What is more challenging is actually customizing Emacs to suit your own preferences, which is the main reason you would want to use it. Deciding which external packages to install, how to use and customize them, and writing your own packages. That is a lifelong endeavor and learning experience, just like programming. So it's to be expected that someone might be a long-time Emacs user, but not be aware of some of its features. You're not expected to know everything, nor do you need to. Just learn as much as you need to make it useful for you. Essentially, every Emacs installation is different, since it's purpose built for a specific use case according to very specific preferences. There's no singular "Emacs". And that is a beautiful thing. :)

I was really talking about the builtin libraries (listed in the core manual). That in itself is a challenge. The surface is huge. Not a critic per se, I've been using emacs for 25 years, I'm just reporting.

Emacs is a path, not a destination!

Saying this as somebody who's been coevolving with emacs for 20 years already. Oh, all the little bits of lisp, all the mechanical memory, all the IDEs that came and went...

My emacs survived them all!

Generally speaking, unless you're doing something weird (typeset IEC ladder!), proprietary (code completion for Aveva Quickscipt!), or right on the cutting edge, someone has figured out a way of doing what you need in Emacs. You just have to identify the pain points first. That's harder than people think.

I would install Doom Emacs and go from there. It presents all the good bits in a (mostly) coherent whole.

Emacs is the type of environment you need to immerse yourself in and build an intuition for. There's no shortcut for that. If you can't spend a couple hours a day for a few months using it, you're never going to feel at home in it.

It's quite possible you just don't need Emacs.

As a longtime emacs user, this perfectly summarizes what is most awful about it! It makes me cringe when people wag their fingers to correct the "misperception" that emacs is merely a great text editor and IDE, but rather a programmable elisp application platform. In reality, vanilla emacs with only a little bit of configuration (and as with any other editor, substantially more tinkering with installation and configuration of supporting binaries), provides a really great programming environment for almost any type of application.

It's also a great file manager, a good shell environment, a document viewer, a mail viewer and composer, a process manager, a music player, and you can play games in it. All powered by elisp.

It reminds me of the "actually a monad is" explanations. Yes monads are wonderful mathemagical infinityburritos that you can spend phds studying, but they also just happen to be very practical and useful even when you don't know you're using them and even if you have no emotional reaction to the name Simon.

There's a lot of good text editors out there, though. Vanilla Emacs with some tweaks is decent, sure, but if all you do is use the basic editing commands then you'll probably feel more at home with a different editor.

Look at the tutorial - it does everything it can to encourage you not to use the arrow keys or PGUP/PGDN. No, you're supposed to learn all kinds of whacky key combinations instead. That's all for a good reason but it's hardly encouraging for new users faced with the choice of doing things the "proper" way and building new muscle memory vs. using the keys they already know.

If all you want is a decent editor to get some work done without having to spend a fair amount of time learning the editor itself, Emacs probably isn't the best choice.

(To others reading this comment: Emacs editing commands have a hierarchy of sorts. It's powerful, and context-aware. But it requires you to internalize things like CTRL-f to move the cursor to the right. Unless you make the effort to learn them - or use evil-mode - you miss out on a good chunk of what makes Emacs so great for editing.)

I have the same feeling with Vim after using it for like the past 10 years professionally. Granted, it is much smaller, but for instance editing multiple files using just stock config is surprisingly undiscoverable. It comes with the whole buffer mechanism, which is certainly powerful, but I guess that 99% of newcomers just want to open tabs and go left/right, not to learn set of byzantine commands like :rewind or :last.

At this point, I believe that newcomers should jump straight to Neovim and file manager plugins, than deal with the stock stuff. That's just one thing out of many.

I did find the Vim manual to be a high-value read for basic editing, for what it’s worth.

It’s pretty short and I felt like most of it was useful.

> but I guess that 99% of newcomers just want to open tabs and go left/right, not to learn set of byzantine commands like :rewind or :last.

There's nano and multiple other editors for that. You want vim for a more powerful interactions than go left and write.

My need for vim is to open a file, go the the place I want to, quickly edit it, and go back to what I was doing (which is reading and thinking). Editing is just a short bust instead of being a continual activity.

Another annoyance with editors like VS Code (and the like) is how tedious to have information. Every file is in own tab (even though you can show more at once, but that's tedious). So you are always flipping back and forth. With vim, I can have everything at once in front of me, and once I have a clear line of action collapse it to the few (2 or 3) I need. Also quickly.

Vim and Emacs are for those that really needs them. There's a lot to learn, but that's because there's a lot you want to do.

I'm not sure that'll happen, because Emacs itself is not so much end-user software as it is a platform/VM for developing and running Elisp code.

There are many great applications and compatibility layers written to run on Emacs:

- Evil for a decent editor,

- calc for a good, flexible calculator,

- magit for version control,

- smerge for conflict resolution,

- dired, org, notmuch, tramp, eshell, outshine, comint, htmlize, etc.

and each of them deserves a "the good parts" book!

Doom Emacs (https://github.com/doomemacs/doomemacs) essentially serves this purpose with its curated, modular configuration and excellent documentation that focuses on the most useful parts while hiding complexity.

Doom is really cool and I appreciate the devs’ hard work. It’s seriously impressive.

But. Doom is an uncanny valley for me. It looks like Emacs, but feels like something else entirely, something unlike I’ve ever used before. If you want to use all the things available inside Emacs without learning the “bare” version, right on. But if you actually want to learn Emacs, in my opinion, Doom isn’t it. It’s its own thing, and it’s a fine thing, but whatever it is, to me, it ain’t Emacs.

Doom is in all respects Emacs. It's a relatively thin layer of abstractions on top of Emacs. I've lived through a fair share of emacs.d bankruptcies until finally settling down on using Doom, and I almost don't even use "the official" modules — I use the core of Doom with a set of my own modules. You can easily scroll through early-init.el to understand what Doom adds on top of vanilla Emacs; what's there really to even "learn about"?

I have thought about rebuilding my config from scratch once again, with all the knowledge I gained over the years, but every time I think about it, I conclude that I inevitably would end up replicating some nice macros that Doom has, which do considerably reduce otherwise unavoidable boilerplate I'd have to write.

Doom may have improved discoverability of Emacs — a major complaint of newbies known for decades — but it can't magically make Emacs "something else entirely". I don't even get what you're talking about, it's like finding any complex package written in Elisp and lament that it doesn't even feel like an Emacs thing.

FWIW, for just "using" Emacs, I wrote a number of keyboard-driven menus for different modes called Casual. Read all about it here: https://github.com/kickingvegas/casual

For understanding Elisp, I wrote an "Elisp Cheetsheet for Python Programmers" to help as a reference.

https://github.com/kickingvegas/elisp-for-python

> forget even the most basic configuration options after a month of inactivity

So do I. ChatGPT4o and the like of course do not forget, and they've read more documentation, library code, and message forum postings than any human could. They know all the five or so ways to initialize per-buffer variables and will write you little snippets of Elisp to add to your .emacs or .dir-locals.el or per-file headers/trailers.

So, if Org-mode or multi-occur aren't doing what I want, and `M-x info-emacs-manual` doesn't make it obvious, ChatGPT4o is my next recourse.

I've been using Dired for years and I've not heard of Wdired mode until now. This looks awesome.

One of the reasons why I kept driving into Emacs land was the dopamine effect of "Oh, shit, I didn't even know it could do this..." Over the years my attitude has changed. Now it's purely "task oriented" — whenever I need to get something done, I'd raise a question "can I do that in a more efficient, delightful way?"

Sometimes that manifests into adding a TODO item into my notes. Often I simply start experimenting with basic Elisp in my scratch buffer. These days I may ask an LLM.

When non-emacs muggles mock emacsians for wasting their lives to "maintain" editor config, they don't even realize — for an experienced Lisper their emacs is not like a self-made car that constantly requires attention, it's more like a garden — if you want to eat apples — you'd have to get your hands dirty, but that knowledge and time aren't wasted — you can then use the primitives you built to grow many other things later.

Reply on news.ycombinator.com