Emacs Symbolic Integration | Hacker News Reader

Emacs Symbolic Integration

gnu.org

138 pointseklitzke76 comments

Emacs has got to be one of the most misunderstood pieces of technology. It's often made fun of [1], just because of this misunderstanding. Saying Emacs is an editor (mostly), is like saying a smartphone is a phone (mostly).

Emacs is actually an appliction framework (and a Lisp machine) with a unified interface where one application can share data with other applications via "buffers" very easily because the basic data is text (no proprietary or fancy datastructures that can get outdated at some time). With this underlying structure (plus with the integrated programming language Lisp) you can connect, combine, configure (to your liking) every application within Emacs easily. In non-Emacs world, using isolated apps (islands), I'd "beg" the dev teams to add this or that feature to make it more productive [3]. Not so in Emacs-land, where I can change every aspect of an app within minutes.

I have legacy tools / apps that were discontinued, and my data is stuck in those formats [2], and I don't have the time to migrate those documents.

Emacs is here to stay. And offers many apps (packages) for many use-cases: org-mode for project management, task-management, taking notes, writing complex documents, creating technical documentation with inline images, ascii tables, with embedded code (that can get run via org-babel); lsp-mode which transforms Emacs into a decent IDE; calc for doing basic to complex calculations within Emacs (documents); org-babel for doing literate programming; eshell for using your shell within emacs; mu4e which is a very good email client (says someone who used to use MailMate and was thrilled I could use keyboard shortcuts to control lots of aspects of my email client), and many more.

Emacs is the tool that will grow on you, and it will get better the more you learn about how to configure it.

---

[1]: "a great operating system, lacking only a decent editor"

[2]: EccoPro PIM from the 90s, AskSam (full text database) from the 90s, NoteMap (outliner), MS Access databases (from the 90s/2000s) with old format, and many many more.

[3]: Whenever I liked an app, I'd quickly start writing with dev teams back and forth in forums or via email. Bear.app, Ulysses.app, iA Writer, DevonThink, etc. Constantly writing to the dev teams, asking to include this or that feature.

Yup. All this.

Emacs being a platform within a platform is a feature. It comes with an extra benefit: it's a free (libre) platform. Completely shielded from corporate interests. There's no bullshit inside. No proprietary formats, no unnecessary network dependencies, no "X as a Service". Just a community of folks maintaining and expanding a parasitic OS that embraces interoperability and personal productivity above all.

When I realized how powerful Emacs is, how it enables me to create a very streamlined workflows in my day-to-day work, I realized I have no need at all for a crippled tablet (not naming the company) that won't allow me to use Emacs (or many other full-featured software), and that's when I put my fancy tablet on eBay for sale realizing that all the apps that I used on it created yet more islands of isolated data that I will have time and effort to get that out of those formats.

Not many fellows are aware of the value of the freedom to use a platform and tools that are open, that give you the freedom to do what you want with it. They see a fancy interface, and they are sold, not seeing the long term costs, the loss of control, the crippled-ness, the trajectory of the whole industry towards a vision that takes away many forms of freedom that constituted personal computing as we know it.

I agree, and I am frequently impressed by the tools people have built on this platform. But I think the "it's a great operating system" line is wrong because it has underinvested in scheduling and process management.

Which is exactly the reason why... VSCode is winning the editors war.

The main difference is that it chose javascript instead of lisp as its universal language, which, in the current world, is a much better choice for adoption.

Also, saying app can share data easily because the format is text buffer is either a tautology or a non-sequitur: not having a defined format can't be broken, sure, but it also cannot be reliably used. ask anyone who had to maintain bash/awk/perl labyrinthine assemblies...

> The main difference is that it chose javascript instead of lisp as its universal language

The main reason is that it's backed by Microsoft, both because of the investment that provides and the visibility it provides.

JavaScript over Lisp probably helps, but if VSCode had Visual Basic as it's extension language it would still be doing strong, and not because VB is a good or popular language independently.

I just want to underline your point on it having corporate backing.

Commentary on our industry seems bent on the idea that things succeed in inherent quality. Reality seems to be full of contrary examples.

Programming languages too. It seems to me that popularity of programming languages is directly tied to amount of corporate backing behind them. There's plenty of popular languages that have a large company sponsoring its development and marketing.

Agreed. I've taken to this idea even stronger. "Simple" languages are believed to be simple mainly off the propaganda that pushes that they are simple.

> VSCode is winning the editors war

In what sense is there a "war"? That's a serious question. Emacs, VS Code, and quite a few of the other popular editors are open source. You can download them and use them. I have both Emacs and VS Code on my machine, and I use both of them, but for different purposes.

Now, if you wanted to talk about a war between open source and traditional closed-up-tight, walled-garden commercial software, I could get on board. It's really hard for me to understand your framing when you have freely available products that are not in any way exclusive.

On your other point, if I were to speculate, I'd say VS Code's main advantage is that it doesn't have a 1970's graphical interface. Port GNU Emacs to Monaco, give it some good default settings, and see what happens to adoption.

War as a metaphor? I mean, have you ever seen two editors battling out in the field? No.

The war is in usage % among users. VSCode is the most widely used editor by far now, reaching over 51%. [1] Usage converts into further growth, having more and better support from extension developers.

[1] https://blog.robenkleene.com/2020/09/21/the-era-of-visual-st...

There is no war. VS Code created the language server protocol, which works great inside Emacs. VS Code's existence has made Emacs even better.

If it were a war, they'd just DDoS gnu.org or something, not make Emacs better.

> The main difference is that it chose javascript instead of lisp as its universal language, which, in the current world, is a much better choice for adoption.

That at best bifurcates the community (possibly along the age lines). I for one consider it to be a bad choice for me personally. JavaScript ain't a particularly pretty language. And it's nowhere as nice as Emacs Lisp.

> saying app can share data easily because the format is text buffer is either a tautology or a non-sequitur

I'd agree, though with caveat that an Emacs buffer is not just an array of text. It's an array of text with bells, whistles, and a semantically well-designed API layer. This lets extensions to mostly compose nicely.

But I'd say the more important feature is that nothing is sandboxed. That means extensions can detect and work around potential conflicts with other extensions, up to the point of extending or replacing any function such conflicting extension uses (though with proper extension design - which is by this point a part of Emacs culture - this is rarely needed). It's easy for any extension to expose an API layer for interop, and Emacs itself provides enough of an API surface that usually you don't even need to do that explicitly.

More importantly, I myself can trivially extend or fix extensions as well. Something conflicts and errors out? Flip a debugger trap, wait for it to error again, poke around the source, find the culprit, patch it in your config. Granted, you need a passing familiarity with Emacs Lisp for that (though IMO, one should consider getting comfortable with Emacs Lisp a part of becoming a power user of Emacs). But you can do that, quite easily, and the entire environment supports you in doing this.

VSCode is winning the war that Sublime Text (and it's python api) was winning a few years back (and perhaps Notepad++ before that). A few years from now VSCode might still reign supreme... or it might not. If you're fine jumping to the next winner every few years, great. If you'd rather just invest in one editor platform and stick with it for a couple of decades, and know you can rely on it to stick around and be supported that long... well, you might want to take a look at emacs.

> The main difference is that it chose javascript instead of lisp as its universal language,

That is not the main difference. The main difference is that Emacs can be extended on the fly. While VS follows the traditional plugin approach.

For VSCode to the have the same kind of extensibility it would have to allow opening an inspector on the _running_ instance. Click an element, jump to its source edit the source on the fly and have that change persisted. Or allowing your settings to patch (which by the nature of ES6 modules is not something you will be able to do).

The on the fly extension is crucial for quick QoL improvements to your workflow. You can spend 5 minutes writing a function instead of having to setup a project/plugin to something as trivial as say, copying a .env file and updating some values when you create a [git] worktree.

> Which is exactly the reason why... VSCode is winning the editors war.

The main reason why VSCode is 'winning' the 'editor' wars is because Microsoft is spending tons of money to build a great product. Emacs, vim, and other open source projects can't compete as they don't have nearly the same amount of resources. They have healthy communities and are thriving FLOSS communities but people who are scratching an itch won't be writing monthly tours of new features like https://code.visualstudio.com/updates/v1_52 f/e.

> ask anyone who had to maintain bash/awk/perl labyrinthine assemblies...

Text in Emacs is richer than plain-text. You have overlays to annotate it with metadata, text properties, you can attach callbacks to it, etc. It is only at the boundaries that you have to resort to plain-text for interoperability purposes.

This means it is easy to inter-op with CLI utils but present a richer, more powerful interface. Ej. dired uses ls.

> The main reason why VSCode is 'winning' the 'editor' wars is because Microsoft is spending tons of money to build a great product. Emacs, vim, and other open source projects can't compete as they don't have nearly the same amount of resources

It doesn't seem to be the main reason, otherwise how can you explain Sublime text popularity prior to VSCode?

Sublime scratched an itch for a lightweight batteries-included text editor. As such, it competed with vim more than with Emacs. VS Code, on the other hand, tries to be your one-stop shop for code-related things, which makes it play in Emacs's sandbox.

I think that elisp is one of the many a strengths of Emacs, along with it's strong extensibility, customization, and general user freedom. I think language choice is a smaller part of the pie compared to Microsoft's backing, networking effects within companies, and visual appeal and design.

However, I do agree with you that Emacs onboarding process can be difficult if you don't know lisp, in that you have to now learn a new language and the API for interacting with the editor. That is why some friends and I recently decided to embed JavaScript/TypeScript into Emacs as a means of controlling the editor [1]. We will see if we can prove you right.

[1] https://github.com/emacs-ng/emacs-ng

> "a great operating system, lacking only a decent editor"

That joke seems to imply a perfect understanding that it's more than a text editor. It is, however, referred to as "The Emacs Editor" in its documentation and "an extensible, customizable, free/libre text editor — and more" on its official website, which could lead to some confusion about its primary purpose if that's not editing text. I do begrudgingly use Emacs as my main editor because that's what the communities of the languages I like use, but I do find the editing experience to be sub-par compared to other editors I've worked with, and I find that there are better external alternatives to any of the common elisp applications, barring org-mode.

I generally agree with your statement, but the primary thing that prevents me from switching to a more modern editor is counsel/ivy. I really wish that VS Code or others offered a similar experience of filtering a buffer/project so quickly and easily. The editing experience of Evil with ivy is fantastic.

I just wish that things like autocomplete or debugging were easier to get right across languages...

The whole point of Emacs is: you get to change whatever you don't like. Furthermore, why use Emacs as an IDE if you don't like it? Who cares what editor you use? Unless you use Emacs, then they make fun of you for using it, while relying on your efficient text processing skills to do refactoring.

I care that the editor I use has good support for the language I’m writing, so, like I explained in my post, I use Emacs since that’s where the language’s community puts their effort. Also, to address your first sentence, the things I don’t like about Emacs can’t be fixed by writing a bit of elisp.

The thing is that editing/text-navigation modes are so universally useful that it makes perfect sense to make a supposed “OS-level” text editor. How many plugins, extensions, etc. are there so that people can use Vim bindings in web browsers, IDEs, the window manager, etc.?

I use evil-mode and that has alleviated any complaints I had about the editing experience. But then again, the modal nature of vi is how I think about editing text.

I use evil-mode as well.

> "a great operating system, lacking only a decent editor"

I've seen this thrown around when people joke why vim is better than Emacs and i think it's funny. But, seeing that Emacs has evil-mode, which is basically an implementation of vim on top of the said "operating system", i'd say it's false. Or ironic :D

As a vim user who recently migrated to emacs, this is 100% true. Emacs is now, among other things, a superset of Vim.

Before using Emacs, I had always read that emacs has a steep learning curve. I found this not to be case since the vim emulation was so good and so complete, I was reasonably productive pretty quickly even though I obviously have a ton to learn about emacs.

> the basic data is text

How can this work? Let's say I am developing a spreadsheet. Each cell has a formula, and it has a rendered text version. How can this work in Emacs? I want there to be a "underlying" strongly typed version of the data which is rendered, but Emacs forces them to be the same.

Unix (with pipes) also works with plain bytes, but many pipes are not necessarily rendered. Given that emacs is inherently visual, because you see the buffer if you're working with it, why can't there be data structures?

Emacs comes with a pretty good plain text spreadsheet. I have not figured out how to use it for large amounts of data (caped at 999 rows) but it's way more powerful than "your favorite spreadsheet software" as it allows for both calc and elisp formulas. All in plain text yeah, types are as strong as elisp's. And maybe I am wrong, but spreadsheets are not famous for strong types. I have horror stories to share on that respect.

I guess you could strong type by using a type cast formula with elisp...

https://www.gnu.org/software/emacs/manual/html_node/ses/inde... - I presume you're referring to SES mode

SES is great as a quick & dirty spreadsheet. I also used it when I started "living" in emacs about 12 years back. I had SES worksheets for various things I'd often use Excel for. One was a stock tracker (downloaded data periodically form Yahoo! Finance), a tax estimator, a paystub tracker, a budget, leave tracker. Some were just worksheets for work problems (had a set of formulae and equations related to test cases, plug in numbers and determine what the expected output should be, or reverse it plug in received output and see what input should recreate it).

If you use org-mode, it has its own format for doing the same thing:

https://orgmode.org/worg/org-tutorials/org-spreadsheet-intro...

If I use emacs-as-a-spreadsheet, I tend to the latter anymore since I org-mode is my primary use of emacs anymore (with org-babel for editing source code).

I meant org-mode. I didn't know about the other one. Thanks for the info.

I do not use Emacs as spreadsheet often. As my spreadsheet needs are larger than org-mode is comfortable with. Pretty much business people sharing data at work. But I have used for reports and whatnot and it's fine.

See [0] for examples of spreadsheet support in Org Mode (which, while often discussed as a thing of its own, is still part of default Emacs installation).

Basically, your table is drawn as text (with Emacs automatically managing its layout, and providing various shortcuts for larger manipulations like inserting or swapping rows/columns). Formulas are recorded as text below the table. They get recomputed on demand, or can be made to recompute on each edit[1].

[0] - https://orgmode.org/worg/org-tutorials/org-spreadsheet-intro...

[1] - https://orgmode.org/org.html#Advanced-features describes a bunch of features, including recomputation of select rows on edit/navigation, as well as marking which rows are names of columns or cells vs. data.

> [1]: "a great operating system, lacking only a decent editor"

There is even a decent editor application available these days in evil[0].

[0] https://www.emacswiki.org/emacs/Evil

One amazing use of symbolic integration: https://github.com/johnbcoughlin/calctex

Create latex equations in calc! :D

This looks interesting and I want to use it. It seems to not be available on melpa, and the installation instructions for linux are missing, though. Are there any instructions for installation on linux that I miss? Apart from the cloning and tinkering that I will inevitably do if there is no easier way.

Sure, emacs is an OS [1], you can write any kind of programs for it.

On the other hand, you can do symbolic calculus in Scheme - see, for example, MIT Scheme-based Structure and Interpretation of Classical Mechanics (which, incidentally, is perfectly usable from inside emacs).

[1] http://www.informatimago.com/linux/emacs-on-user-mode-linux....

Of course, in this example, the symbolic calculus functionality is distributed with the OS, or arguably even as part of it, which I guess is a bit more rare than simply having the option of installing it.

Damn I'd been searching for ages to find a proper calculator. Turns out it's Emacs. Now I just need to get it on my phone somehow.

As a palliative you can always use some hp48 emulator.. you get some forth and some lisp all at once.

Android? Termux will run it, even the Termux in the Play Store.

What kind of phone are you using?

If it's Android you can at least for the moment get Emacs on there using Termux. That's what I do - and it works pretty well. It might also be possible via UserLAnd, although I have not tried UserLAnd personally.

It seems like iSH on iPhone may provide similar ability (again do not have direct experience) although it seems unclear at the moment how long it will be able to remain available.

You can run emacs on sailfish.

Oh yeah.. of course it calc.

On the tinier lib side there's bindat, to fiddle with binary data straight from elisp.

Nice, didn't know about this!

That is certainly impressive. But why does emacs have that functionality?

Who in his right mind would use an editor that cannot even do symbolic integration?

Joking aside, I suspect that this is mostly due to two factors:

1. Emacs was used by a lot of smart people, and one of them, David Gillespie got carried away a bit.

2. Thirty years ago, when David Gillespie first started working on calc, emacs lisp (despite at the time lacking even floating points numbers IIRC!), was probably one of the most expressive fairly widely deployed languages around, particularly for the type of symbolic manipulation you need to do for computer algebra. And unlike other things that were used by smart people 30 years ago, emacs still sticks around, so survivorship bias also plays a role.

Because it can... don't underestimate the amount of unrelated things you can crame into a 40 year old software.

Interestingly, nowadays it's really difficult to convince people to add new features, like enabling mouse scrolling by default [1].

[1] https://lwn.net/ml/emacs-devel/20200906133719.cu6yaldvenxubc...

I don't think it's a valid example. The discussion is about the default configuration, not about removing mouse scrolling from the code. It's a really big thread nevertheless...

It's a bit like asking why Windows has a calculator :).

Emacs is best viewed a Lisp execution platform with a built-in text UI. Much like it has a "c++ app", "terminal app" and "compile this" app, it also ships with a "calc app" (literally called "calc"). Since it was being used by lot of mathy people for lot of mathy stuff, it ended up being a proper scientific calculator application.

(Also, Lisps are inherently friendly for doing any kind of symbolic calculations. A Lisp system is one of the least surprising places to find a symbolic integrator in.)

> Emacs is best viewed a Lisp execution platform with a built-in text UI.

Hence the old joke "Emacs is a fine OS, but its text editor sucks."

Have you tried asking your local psychotherapist?

    M-x doctor RET

    I am the psychotherapist.  Please, describe your problems.  Each time
    you are finished talking, type RET twice.

    Why does Emacs need Symbolic Integration?

    What do you think?

    I don't know, just answer the question!

    Why do you say you do not know just answer the question?

    [...]

Maybe you'll have better luck than I did with mine.

Can't wait for someone to do a M-x doctor over GPT-3, AiDungeon style.

Oh my gosh, someone please add an M-x gpt-doctor. Hmm.. I wonder how we'd build the training dataset...

How many other meme modes exist? I dug up the earliest commits I could find, and apparently one of the earliest hooks was "protect-innocence-hook": https://twitter.com/theshawwn/status/1327459335245422597

If the buffer's filename is "sex.6", it asks whether the user is over 18. If so, it renames the buffer to "celibacy.1".

Apparently it was in the file lisp/play/meese.el. Emacs archaeology is more fun than it should be.

There's M-x zone, a bunch of games - anything from Tetris and Bubbles to Zork, morse code handling (M-x morse-region, unmorse-region), inserting interesting phrases for NSA spooks... There's a lot.

There's also nyan-mode, made by yours truly, but unfortunately it does not ship with Emacs by default :).

Who do we have to lobby to get nyan-mode into emacs mainline? Happy to send a recommendation email.

Was surprised how high quality the repo is, too!

https://github.com/TeMPOraL/nyan-mode

It even has an accompanying blog post, which I found informative. http://jacek.zlydach.pl/blog/2011-08-25-introducing-nyan-mod...

And even a Show HN. https://news.ycombinator.com/item?id=2906632

Thank you for your kind words. I wouldn't call it high-quality, though. I wrote this back when I barely knew how to wield Lisp or use Emacs. I've gained much more experience in both since then; I keep promising myself that one of these days, I'll clean it up and make it a stellar little mode...

As for getting it into mainline, I'd be OK with going through the FSF copyright paperwork, but I worry this will probably fail over the uncertain status of Nyan Cat image/music IP...

> I worry this will probably fail over the uncertain status of Nyan Cat image/music IP...

That’s how MooGNU was born: https://archive.org/details/M00GNU

>How many other meme modes exist?

Here is the manual section listing all the "Games and Amusements" that ship with Emacs by default:

https://www.gnu.org/software/emacs/manual/html_node/emacs/Am...

One of my favorites is "Zippy the pinhead" quotes (M-x yow). I believe this has been removed due to copyright issues, but it is still available if you look. Zippy's quotes could be sent to the doctor via M-x psychoanalyze-pinhead.

I think the Emacs doctor drilled down my insecurities down to a bad exam in school. (S)he ? claimed that his/her secretary will send me the bill, but I haven't been back so they must have forgotten.

If you feel that the services rendered were useful to you, consider sending a donation to FSF :).

Emacs is best understood not just as an editor, but as an integrated computing environment promoting user freedoms to the fullest extent possible in a manner naturally integrated with its user interface and extensibility. In that sense, it shouldn’t be the least bit surprising to see the kind of in-built application suite one would expect from an operating system or desktop environment. One must also remember that historically the most common computer (power) users were academics, and not “software developers” — so expect a lot of scratches for their itches.

It’s a bit like asking why does bash come with a utility like awk or bc; it’s equally natural.

Because awk and bc are UNIX programs that precede bash and are part of any POSIX certified UNIX, whereas bash is not.

https://pubs.opengroup.org/onlinepubs/9699919799/

The manual actually answers this very question: https://www.gnu.org/software/emacs/manual/html_node/calc/His...

Thanks for posting this. Not only it's an amazing story of scope creep, it also revealed a bunch of features I didn't know calc had (e.g. unit handling, arbitrary-size math).

thanks for the reference! love the final acknowledgement in there:

> Final thanks go to Richard Stallman, without whose fine implementations of the Emacs editor, language, and environment, Calc would have been finished in two weeks.

I often find it useful t do a calculation while I am editing. For example when I write a invoice I have to calculate taxes.

I have not had to use the more advanced features. I did not know that they were there. Now I know, perhaps I do need them?

Emacs is a big Lisp interpreter.

Elisp is the most popular Lisp in the world.

Why wouldn’t you want to improve it?

> But why does emacs have that functionality?

GNU Emacs is a 36 year old tool that's been targeted at programmers its entire life, is built on an interactive Lisp environment, and actively encourages external improvements and contributions. The more interesting question is why _wouldn't_ it have that functionality.

A healthier approach is to have a separate non-Emacs app that can do symbolic integration, which can be run seamlessly from within Emacs.

Otherwise, with all its extensions Emacs starts giving the impression of an Apple-like walled garden, which is not a good comparison for a free software, no?

How is it a walled garden if it's free, open-source software?

Emacs is its own ecosystem, you either buy it all, and grow to use it more and more, or don't get to use it all. So in a way it separates itself from the rest of Unix ecosystem.

Only because much of what emacs does is cumbersome or impossible as regular software.

a lock-in is a lock-in, even if with free software, it incurs certain costs on you, even if it provides convenience.

In general - developers use calculators all the time (most frequently via Google these days, but also via REPLs) so a dedicated calculator is a self-evidently useful widget for any editor. It’s is not essential fir a new editor in 2021 but considering emacs has had Calc for some time, it is worthwhile to make improvements to widely-used legacy software. Many emacs users have used Calc in lieu of alternative “developer calculators” like Octave, the R REPL, etc, so it’s not like Calc is an extraneous feature.

It’s uncommon for developers to deal with integral equations in their day-to-day work. But it’s obviously not unheard of - integral equations show up naturally in many domains of finance and engineering. I’ve had to deal with numeric integration on .NET in one of my jobs. Sometimes an exact symbolic solution is easier for the developer and the CPU than a numeric technique, so for a small handful of developers a symbolic integral calculator is a handy (if somewhat specialized) tool, and not just some bell-and-whistle.

Because enough of its users use it?

I'm a graphics programmer and I use Emacs' in-built RPN calc for its linear algebra capabilities almost daily.

Reply on news.ycombinator.com