How to open a file in Emacs: a story about Lisp, technology, and human progress
murilopereira.com
murilopereira.com
A couple decades later, I just find it all so exhausting.
"when I was young I used to work out and eat healthy... a couple of decades later, I just fund it all so exhausting".
Customizing your computer environment is good. Eating healthy is good. None of those need to end up in a "rabbit hole".
Everyone can be tired of life and give up due to exhaustion, but greatness requires discipline. This is also true with the craft and art of programming.
a) learning Spanish to talk to that hot chick/dude
b) traveling around the world
c) doing whatever else you love to do besides coding
In the context of the posted article, one simply uses TRAMP (built in!) and moves on with life. The author is dealing with a situation where the "move on with life" tool is behaving pathologically.
So I mostly just leave everything as it is.
What they all have in common is that this level of customisation looks silly to the outside. Of course they could be 98% as good with something they bought off a quality store, but what they do is part of who they are and there arguably is a deeper satisfaction in that then going around hitting on hot dudes on holiday trips.
- process their email inbox
- communicate in chat
- navigate the web
- write code
- prose
- Creating a to-do item for something your partner just asked you (while you were concentrated on a task) that will show up automatically on your agenda tomorrow
- Converting some text notes you quickly scribbled down into a beautiful PDF via LaTeX, uploading it to S3, and referencing the URL in a reworded existing git commit message?
I could make the list much longer from my own experience, but this is just straight from the article.
I'm not a VSCode user, but I often hear about it only in a programming context, so it always surprises me when people suggest ditching Emacs to use VSCode.[1] I'm genuinely curious: How many of the above tasks are doable in VSCode? If not most of them, then I suggest people stop comparing the two programs - there is no point in asking people to stop using Emacs and use something that doesn't do what is needed.
If VSCode can do it, I'd love to look up the plugins/extensions needed.
[0] And many, many do it. I do over half of these things in Emacs (and most of the last item).
[1] Easily over 70% of my Emacs usage has nothing to do with programming.
And very probably, many, many, many more don't. (This is what telemetry is good for--actually telling you how many people use various functionality in your product). Somewhat more than half of my coworkers use emacs for their daily coding, and exactly 0 of them use it for anything other than a programming editor.
My point is that while I know many, many folks who use Linux only for SW development (at work), it would be silly to view Linux primarily as a SW development environment when it can be used for so much more. And so it is with Emacs. It's like saying "I dropped Linux for Visual Studio." The comparison makes little sense, unless you're talking to the folks who only use Linux for a small thing.
As for telemetry, I can again invoke the Linux analogy. While it would certainly be nice to get an accurate poll of what people use Linux for, I don't think many kernel developers will want to drop multimedia features from it because less than 10% of Linux users use those features (10% is really optimistic!). Emacs is an elisp development environment, and a text environment. Anything that works with these two is within its scope. It's a valid discussion on what should come bundled with Emacs vs off loaded to external packages, but that's not the discussion over here.
(Yes, yes I'm aware that Linus and the kernel folks do often drop features because they believe it's not used often, but those usually are way, way down the tail of the distribution).
I have fun with customization and it increases enjoyment of the coding part, so why not.
One is a practical endeavor of great importance for mind function, body function, longevity, performance, etc.
The other is something between slightly better ergonomics (in a non-scientific ad-hoc way), tinkering, and bike-shedding.
People have found again and again that "customizing your computer environment" is not a good in itself, but a means to an end, with quite narrow diminishing returns.
>Everyone can be tired of life and give up due to exhaustion, but greatness requires discipline.
This includes first and foremost the disclipline of avoiding tinkering and bike-shedding with your tools, and doing real work on real problems instead (which is not a false dichotomy: many people fixate on the former to the detriment of the latter).
>This is also true with the craft and art of programming.
Which has as little to do with a focus on customizing of tools, as a hi-fi hobby has with being a music lover...
The author of the article customized his environment so opening a file no longer froze his editor.
I wouldn't classify that either as "slightly better ergonomics" nor "tinkering" nor "bike shedding".
He had the discipline to improve his favorite editor. Good for him.
Emacs is obviously a very powerful environment for people who know how to use it. I applaud the author for documenting how he fixed this.
The only difference between emacs and many other open source software is here you can easily solve bugs in your config file, instead of filing a pull request or submitting a patch to some git repo. The first you call "tinkering" and look down upon. The other I assume you approve of? But why the double standard?
I didn't respond to the article, I responded to your particular comment, which seemed to imply that customizing your programming environment in general is something great and necessary.
>The author of the article customized his environment so opening a file no longer froze his editor.
That doesn't clarify as "customizing my environment" is more like "fixing fundamentally broken shit".
I mean, sure, if you define any config change that tangibly results in higher productivity and better performance as a fundamental bugfix instead of a customization, then customizations don't matter.
But that seems to be a kind of arbitrary distinction. Just as an analogy:
----
"I lost 30 pounds and fixed a sleep apnea problem when I started paying attention to my diet."
"That isn't paying attention to your diet, it's fixing a fundamental health problem. In general, most dieting falls into the category of fads and micromanaging."
----
Aside from your specific definitions about what does and doesn't count as customization, what do you and the OP actually disagree on?
Do both of you agree that when changing an Emacs config, I should take into account the ratio of time they'll save and the time I'll lose making them?
Do both of you agree that there are changes I can make to an Emacs config that would eventually provide a net benefit in terms of time saved?
Is it good that I can make those changes that save me time in the long run?
Is it bad if I ignore the cost/benefit analysis of a customization and spend more time messing with configs than actually programming?
I don't understand what you're actually arguing about.
Some examples of unnecessary customisation is the fact that the defaults of emacs so often ought to be changed. E.g. why does M-SPC do just-one-space instead of the strictly more powerful cycle-spacing (answer: probably a combination of history, neglect, and a fear of breaking existing saved keyboard macros), why use upcase-word instead of upcase-dwim? Why does autocomplete require so much customisation?
If emacs had better defaults there would be much less reason for spacemacs or doom to exist.
One person I work with will (drastically) customise the key bindings for each new mode before using it, making it hard to use new modes. Another gave up on customising emacs and switched to spacemacs to no longer have to think about customisation.
I wonder if it's more time wasted when you have to change editors every few years because of changes in the funding model of the corporate patron of the product and relearn how to use the entire environment with "sane defaults" versus creating some custom keybindings that you like essentially one time and then using them for 30 years in an editor that has a license and community that makes it unlikely to suddenly cease to exist.
People act like there's all this extra work to using software like Emacs over corporate IDEs, or GNU+Linux over corporate OSes, but I contend that it is merely different and better-documented work. The corporate environments just make more promises, and then everyone is surprised when the promises are broken.
I totally agree. The only reason I justify my Emacs customization is because I customize it very slowly, over years, and periodically remove things I don’t use any more.
It’s getting harder to justify now that alternatives like VS Code have a bucketload of features and working autocomplete without futzing around too much. It’s just that VS Code is so damn slow. Maybe better language server support will bring me back to Emacs as my daily driver in a couple years, or maybe I’ll be paying for CLion.
+ Increase productivity (e.g. if value = work x time, and you can't increase available time, then reduce time consumption per unit of work to increase value)
+ Increase comfort during this process
- cost: available time is reduced, which you could have spent working.
Objectives to exercising:
+ Increase lifespan
+ Increase comfort(health) during this lifespan
- cost: available time is reduced, which could have been spent living
Tbh, these look pretty similar to me. In both cases, you're trading off an existing resource gambling that you'll get back more than you put in.
---
As an addendum, there's a right and a wrong way to achieve tradeoff-based objectives.
With exercise/diet, you can certainly do it wrong: latest fad diet that's placebo or maybe even actively harmful to you, latest fad routine that isn't very efficient, overwork and hurt yourself, etc.
If you're "wasting a bunch of time" customizing your computer, I guess you're doing it wrong. Certainly, don't do that.
Like the OP I used to be in the having "pets" category (including gigantic ~/.emacs), but these days I prefer cattle (defaults as much as possible)
You don’t even have to stop there. I have a repo for my Emacs config, a repo for my Windows bin folder, a repo for my Unix shell stuff, and a repo for a bunch of cross-platform Python scripts I’ve written. Every computer I use regularly is super customised in exactly the same way as all the others. Takes about 10 minutes to get up and running.
What if there were a Silicon Valley company whose entire business model was selling a computing environment that didn't need to be customized? Like, at all? What if they did a whole bunch of psychological research, and found that most humans work pretty much the same way, and designed this environment in such a manner that they could get 98% or so of the way toward optimum productivity, right out of the box?
Knowing the Valley, they would give themselves a silly name, one having nothing to do with computers or data or anything. Like a fruit or something. But multiply the base price by billions of potential users... the idea could be worth trillions!
Additionally guess what. Most humans wouldn't prefer something like emacs even if it had sane defaults that would bring all it's users and the vim people and the like togheter. It's why a lot of it's features aren't a thing in popular ide's and such. Yet this niche still exists. Because some like a keyboard centric layout and don't mind learning a bunch of of keybindings to work faster down the line
> when I was young I used to work out and eat healthy at the level of an Olympic athlete ... a couple of decades later, I just fund it all so exhausting and don't do any exercise and eat only convenient packaged foods.
The original comment is arguing as if only these two extremes exist.
The thinking environment of choice of the most productive, smartest person I've ever worked with was .. a notepad. Not Notepad, a physical A4 ruled paper notepad and an ordinary gel pen. The actual development inevitably required some typing, but the thinking process and its associated diagrams, calculations, and scribbles for the kind of maths-heavy work he was excellent at worked best on paper.
(I currently have vscode open .. and actual Visual Studio, and IntelliJ, and notepad++, each with a different set of files open in different languages.)
> I was so into these kinds of super-customizable rabbit holes
I did this too. I spend days customizing my xmonad (I dont speak haskell), but thats nothing in comparison to the time I spend tuning my emacs.
On the other hand, i pretty much use the same environment like a decade ago. My same .emacs, my same xmonad.hs. I was looking at my WM and if there is a way to get rid of ghc on my machines (I use gentoo, updating ghc is an expensive task, especially on older machines) but every other WM felt like a downgrade so I went back to xmonad, to my old config.
I feel home everytime I open my notebook or booting linux on my desktop.
When I was in college I thought being a flawless typist would make me a better programmer.. (mind you I had trouble mastering dynamic programming, advanced abstractions or semantic subtleties).
I find these exhausting too now, and actually that's why I'm jaded by tech. The vast majority of things in the late 10 years felt like a planet scale tweak (phone has tilting backgrounds, css animations, ..). Deep inside I knew it was just marketing and nothing substantial.
Sounds like you are in a very small bubble if that is your take on the last decade's tech inventions/evolution. What about autonomous vehicles/dock yards/warehouses and reusable rockets?
Yeah my bubble was mostly about consumer computing and network. CRISPR wasn't included in my rant.
So do I, and yet I use Emacs. I think there is a middle-ground. There are things worth customizing and improving, in particular the tools I work with, and there are things that I expect to "Just work"™.
Hence I use Gnome and Emacs for important workflow, such as programming, TeX, email, git, shell etc. It's and IDE, if you wish, but with more stuff since email is also part of the workflow.
After awhile I realized I just wanted to write code and not have maintaining my system take time away from that. The fact that the Mac provided easy access to Unix was a major plus. Also, at the time, none of the Linux desktop environments could match the polish that the OS X GUI provided.
I've been saddened and frustrated by the state of macOS of late (hopefully Big Sur is better) but I'm still not ready to go back to the old days. As you say; it just sounds exhausting.
I'm typing this on a Pinebook Pro which, at least in my experience, has basically been zero-configuration with the default Manjaro release.
Ok, I lie, I added an m.2 ssd to it. But that's about it!
Then work happened. Right now I just can't stand it anymore. My brain is rebelling against even coding. I am still really good at it and enjoy it, but I realize that I just lack the drive I had. Some people might say that's bad. IT IS NOT. I spent almost 25 years of my life filled with coding. I am done. I want to have other people code things for me and focus on building awesome products instead. This is what drives me now. I don't care about low-level anymore. I don't care about bugs or why this stupid dependency won't work in a deployed environment. I want to ship awesome things to customers and I keep realizing that this is at the core what always drove me. I never coded for the sake of coding. It was always to build something cool, except there was no one else around to do it for me.
This is a very familiar story for many engineering managers I know.
It made more sense to customise when you had a limited environment you used and never much changed, but what's really changed is I'm twice as old now :)
It's worth noting that Emacs's built-in way to find files (project-find-file) in a project does not suffer these performance issues. Try it, C-x p f.
* project-find-file doesn't call expand-file-name for all 70k files (each of which involves a network roundtrip).
* project-find-file will use "git ls-files" (see project--vc-list-files) rather than find (or expecting the user to install and enable fd)
* project-find-file uses completing-read, which wouldn't by default call Ivy/Ivy prescient.
I can see how the author has ended up here: projectile predates the built-in project support, so it must only seem natural to use projectile as a base, and apply performance fixes on top.
Emacs's built in functionality tends to work better with other Emacs functionality, hence why project-find-file works well with TRAMP.
It's worth trimming your config occasionally, and rebasing onto built-in functionality.
The author's fast-project-find-file has a more responsive UI, however. It'd be great to have that in core.
Not by default, that's true, but for users that have installed ivy and activated ivy-mode, ivy does take over completing-read.
That is your personal key binding, project-find-file does not seem to have a default key binding. (This, by I which I mean telling people that some built-in command is bound to some key while forgetting that is in my configuration and not by default, has happened to me several times.)
Emacs 27 users can upgrade to this, or wait until Emacs 28.
I think it's more than that, there is a higher barrier to entry for code in Emacs than in third party packages. Given that it's not surprising if built-in code is higher quality than external packages on average.
I like how it describes different tools by the different values they prioritise. I agree with the post's thoughts in the tradeoffs involved in Emacs's development.
His discussion of core values is quite interesting, in that it’s both relatively obvious in retrospect yet something I’ve not given conscious thought to.
I’ve certainly thought about programming languages as expressions of intent, and how some attempt to be what I call “kitchen sink” languages, like C++, Java, or JavaScript, while others are much more opinionated about their purpose, like Erlang, Forth or APL.
Also reminds me of the old description of UNIX as user-friendly, just picky about who its friends are.
Admittedly I wish I would spend a bit of time in emacs internals sometimes just so I could learn to approach similar issues, but it's just never much of a priority when I'm trying to work on what I'm actually working on. I rarely get distracted by trying to optimize my config, as a result I use almost the vanilla defaults and I'm mostly happy with that. TRAMP is pretty great, I wish it would not make a new connection for every network interaction though. Setting ControlMaster on ssh helps a bit but it's still not as fast/transparent as I'd like. There are far too many "blocking" operations in emacs, as much as I love working in it, there is the non-zero times that it locks up on me and it drives me crazy when that happens.
If it was ever the case that I had to wait 30 seconds to open a file, I'm sure I would abandon emacs instead of trying to debug it. That said I can't imagine working on a 70,000-file repo. That sounds like a terrible idea.
Same. One of my wishes for Emacs in "From caching up to getting ahead" is for Tramp improvements.
These days I went back to TUI Emacs via SSH for work.
Open up your ~/.ssh/config file and set "ControlMaster yes", "ControlPersist yes" and "ControlPath ~/.ssh/cp-%C" and ssh will multiplex all of your ssh sessions across as few unique tcp connections as possible. This will speed up scp and sftp connections as well.
But these types of issues are why I've always given up on learning Emacs. When I can just install Visual Studio Code and the Remote - SSH extension and whatever else extension that I want to work remotely and get on with life, it's really hard to justify the time one needs to spend with Emacs to bend it to your will.
Even as much as I love Emacs, I would never fault someone for choosing a more productive tool now even if there’s a non-zero risk of something breaking in the future...because there’s always something that will break on any platform. Time spent being productive now is usually worth that risk.
The claim in the comment I was responding to appeared to me to be "hmm, Emacs has a problem, VSCode probably doesn't, I would rather use VSCode". It didn't seem to me to be written with much awareness of what the actual problem in TFA really was, nor with any real knowledge that VSCode actually would be more productive. It seemed to me to be more like "Ewww! Emacs has a bug, someone fixed it with a lot of obscure stuff, no thanks, VSCode here I come".
I certainly respect that for many, VSCode will be a more productive platform when measured by the metrics that matter to them.
Maybe.
I imagine at some point someone wrote ffap-guess-file-name-at-point to fix a problem. And the blog author solved a problem by disabling it without knowing what problem it maybe solved.
Problems eventually become complicated enough it’s not a binary fixed/broken, but rather a matter of trade-offs.
Open source and shared solutions are great. But they aren’t a silver bullet either.
Visual Studio Code is not perfect, and I have had issues with it and expect more issues. However, they don't hit me as early as they seem to with Emacs, and that's an important distinction. The remoting extension is extremely seamless, and I have not seen an Emacs example of matching anything close to it.
It really comes down to personality. If I was a carpenter and had a broken hammer, I wouldn't go off rebuilding the hammer or designing one from scratch. I'd do my best to find someone else who's already built a hammer that works. That doesn't imply I never expect the hammer to never break.
> would prefer to know that such issues can be fixed with or without vendor participation
Visual Studio Code is open source and well-managed. I'd prefer that a solution be merged into the tool for everyone, rather than being buried in someone's .emacs file.
Read it as the fine article or the featured article.
As a purely native developer, I have zero use for such things, but I can certainly appreciate that VSCode may have done a much better (or certainly, shall we say, a less historically indebted) version of this sort of functionality than Emacs.
[ Edit: and yes, the featured article ]
ust like there's more than one hammer on the market, or more than one drill, each has its strengths and weaknesses. There have definitely been frustrating times configuring Emacs, but one of the reasons it's paid off over the years is that when I run into an issue, I can always dig into it and potentially fix it myself; and when I want to extend it to do sometime it can't do yet, I can do that, too.
It's an investment, but it's one that I think has paid off, and will continue to pay off. Emacs has been applicable in more situations and environments than those other tools. (IntelliJ's tooling has been useful, but I've also run into bugs that have impeded my work, which I can't fix on my own because the source isn't open.) I also do a lot of work in Clojure, so Emacs Lisp isn't much of a jump. Admittedly this might not be the case for everyone.
I think another big reason I've stuck with Emacs is that the cost of switching editors as they come along (I hear Atom is really nice, and people have been talking positively about Sublime for years) has a cost as well. Emacs is long in the tooth, which is a curse (Emacs Lisp is a really old lisp) and a blessing (it's unlikely to go away any time soon, and that time spent learning Emacs Lisp is going to continue to pay off).
Please don't read this as apologia for Emacs or as a criticism of your choice, as it's neither. I'm just trying to add some context to how others may justify the time they've spent customizing Emacs.
Doing the above for 5 years I've a decent personal .emacs I take with me in a Git repo. Setting up takes less than 10 mins on any new machine.
I think using TRAMP or similar technologies is bypassing, if not the letter, then the spirit of that companies' security policy. Sure, not the whole repository is lost when the laptop is, but having an (ephemeral) copy of the files one is working on outside the security perimeter is kind of defeating its purpose. Why is that company granting access to remote file transfer? I'd think they would be better off restricting access to some form of desktop virtualization (the old SunRays were nice) if they must offer remote access.
The problem is that many parts of Emacs and libraries built on top of them have not been made fully TRAMP aware. There is an abstraction boundary that is being crossed, assumptions are being violated, etc. because all paths are treated equally, despite the fact that most parts of Emacs can identify that a path is remote (or could be remote).
Fixing the underlying issue and providing latency aware variants of commands is not easy (you usually can't just run elisp on a remote machine), but when we get there Emacs will be even more powerful!
Here is an excerpt from some notes I made back in October.
Consider these cases.
I have 10k files that I want to move on a remote server. I have 10k files that I want to move on a local server.
Where do I put the for loop?
I want to put it so that I only have to call ssh once.
Similar issue with using find and subshells. If I have to invoke a subshell there is significant overhead.
Having even a little bit of knowledge about the latency of running a single command, or the knowledge that certain commands commute under loop reordering would make it much easier to produce efficient code, basically
#+begin_src elisp
(do-for-values value '(a b c d e f g)
(please-do-this-on-the-remote #'command value))
#+end_src
#+begin_src elisp
(please-do-this-on-the-remote
(do-for-values value '(a b c d e f g)
(command value)))
#+end_src
The best way to approach this might be to detect and warn before running. Or not, sometimes you do just want to be more efficient, and if you don't want the optimization then disable it some other way. The reason is because sometimes it is much easier to write the latency multiplying version, and the conversion to the latency efficient form is trivial for a computer but extremely easy to screw up for a human being.
EDIT: Hah. I though I recognized Richard's reply. It was to me.
> Hah. I though I recognized Richard's reply. It was to me.
Thanks for being the cause of that really nice reply :)
My stunbling block is not so much knowing Elisp (I’m sure I can stumble through that) but just knowing the internal architecture and how emacs works overall.
Has anyone come across a good guide for hacking emacs? I know there is the manual. Perhaps I just need to look at that again and get going.
e.g. here is describes keymaps: https://www.gnu.org/software/emacs/manual/html_node/elisp/Ke...
Former is concise and more suited for configuration, latter is for package developers.
One step at a time.
If getting better at Elisp is something you want, just keep at it :)
I often mosh + tmux on remote servers slso, and even though I do nothing fancy, with Emacs it all works for me.
Vim can open remote files by putting scp:// at the start. Check the vim wiki or similar for exact syntax. Works with your ssh config aliases and keys and such, so can be pretty smooth.
What I don't know is if he knows of this feature and declares it not-equivalent, or if he's never seen it. TRAMP is likely more powerful, in part due to emacs itself. Like, you can view a remote dir in dired and open an image and see it inside your emacs. I don't know if gvim can view images, I only use regular vim or neovim, but I typically use a graphical emacs.
Mostly unrelated, you can open a web url in neovim like it's a file. I've done this to read a page with a bad layout before. A bit like w3m, but then you can easily delete chunks or save parts to another file if you want. You are just seeing the raw html, but usually the paragraph tags are fairly readable.
I know it, but it's not even close. Almost every editor can do something like that over SCP/SSH/SFTP.
Is it me, or is this telling me that the terminal Emacs experience is just severely crippled compared to the GUI one?
I have been using emacs for org mode and common lisp coding for about 1 year ... In my (very limited) experience I've noticed these differences:
- TUI can't display images
- Some keystroke combinations are "absorbed" by the terminal so are not available. I believe they are shift+alt key combinations. A little annoying, but not a huge deal-breaker.
... and that's all I've encountered so far.
I don't think TUI is severely crippled compared to the GUI version, but the GUI version is nicer to use.
Kudos for dedicating time and effort to high quality long form articles - this is something that online needs to revert to.
That said, can you give us a TL;DR if you have a moment :D
Can it be described? Well, this is how I did it in a tweet, and the author like it and thanked me.
« Fascinating, very long essay which starts with a discussion of text editors, wanders through software development models & the issues of modernising very old software, & turns into a discussion of intelligence, language & computing. »
If you can configure another tool in 2-5-10 hours to achieve 90% of the efficiency Emacs gives you and you can use the other 190-195-198 hours to actually realize those "ideas of your lifetime", it is a lot of time spent (and maybe wasted). Also: https://xkcd.com/1205/ + tinkering isn't necessarily productivity. If anything, it's more often procrastination.
Finally, I'm not sure how you could prove that Emacs it the only thing that allows you to achieve those "ideas of your lifetime", I don't think that's even a falsifiable statement, it's just your personal opinion.
There's no way any other program will give you 90% of what Emacs does in a mere 10 hours. Not. A. Chance.
See this list of things people do in Emacs from an earlier comment.
https://news.ycombinator.com/item?id=25627763
These are what the author mentioned. From my own usage, I can easily make the list 3x longer. Emacs is an application platform for Elisp apps. Your statement is the equivalent of saying "Just spend 10 hours learning VBA and you'll get 90% of the benefits you got from years of practicing C."
We can also save time by not caring about the color of our clothes, getting cheap fast food instead of preparing our own meals and so on, yet in the end most people don't do that because the difference is more than hours of their life.
I got better ergonomics, fewer distractions and higher comfort out of my customisation. Yes, I sunk many hours to get to that point. But on the way I learned many fascinating concepts, bash scripting, a deeper understanding of software and computers and more. Without this kind of tinkering I wouldn't even have started programming in the first place.