As an Emacs user[1], I agree. The flexibility is really cool, and there are a handful of Emacs applications (notice I didn't call them "packages") that are "killer apps", like Magit, Org-Mode, Calc, etc.
But, even though I do use and like Emacs, I can't honestly recommend it to anyone and I wouldn't even consider it to be a very good programming environment unless you're specifically working in a Lisp/Scheme (where it's awesome).
These days, I mostly use Emacs for programming in Rust and for using Magit as a git porcelain even when not using Emacs for the actual code editing.
Here are the things that suck[2] about Emacs (mostly from the point of view of a programming text editor):
* The UI is slow. Don't tell me that it's "only" when we have long lines, or too much syntax highlighting or whatever. Sure, if I turn off all of the useful programming features, the UI updates fast enough for me to not notice any latency, but any time I open a file that uses a LSP server or equivalent (not just Rust- I've done Clojure, JavaScript, PHP, and C++ in Emacs), the latency for even just moving the cursor around is noticeable and it's distracting.
* Tramp sucks. I don't understand how everyone says that Tramp is awesome and magical. Every single time I've tried to use it to do anything, it's been a bad experience. It's been a few years since I've even tried it, but I remember having to wrestle with ssh-agent authentication, some packages not playing nicely with it (besides the "obvious" ones like LSP-mode), and the whole application hanging if the Tramp session hung for some reason.
* It's actually really HARD to customize properly. The experts say that Emacs is "self-documenting" and "introspectable", but many times, I'll try to customize some behavior or a keymap or something, and I'll read the documentation for a variable or function and it'll say something like "takes a plist of FOO definitions,"--well, what the heck does a "FOO" definition look like? What are the keys for the plist? What are acceptable values for those keys? It's very common to just have to dig through the actual elisp code to find out. Then, you'll find out that your customization doesn't even work correctly because some other mode or function stomps on your code, or gets initialized in a different logic path that isn't affected by your change. Since every feature and every package is different, you have little chance of just figuring out a few conventions or any standard structure for packages.
* Handling buffers is tedious. Sometimes buffers appear and take over the whole frame, sometimes they split the frame and steal focus, and sometimes they split the frame and don't steal focus. And it's very ad-hoc and tedious to try to separate out "background, system buffers" from "my actual work buffers". There are tons of Emacs packages that attempt to make it easier to manage buffers, switch between them more easily, group them in lists, etc.
Like I said, I like Emacs. But, I also run Linux on my personal machines and run a de-Googled AOSP ROM on my rooted Android phone. UX, polish, and convenience are obviously not a high priority for me.
[1]: I've used Emacs to varying degrees and with varying levels of dedication and hardcore-ness since 2008-ish. At a few points, I would've said that I was pretty competent/proficient in Elisp, and have authored some fairly elaborate functions to help me with common tasks within Emacs. I even use to try to do a version of "literate programming" with org-mode and org-mode babel blocks in place of something like Jupyter notebooks for Python.
[2]: Yes, this is a very negative comment, but I tend to complain loudest about the things that I love and care about. I don't complain or criticize things that I don't like or don't want to use.