It's the opposite for me. Not living in the shell 24/7, for me personally, doesn't just have big sanity benefits, it also makes me so much more productive.
I find shell environments incredibly limiting, I feel like my visual brain starves when limited to what feels like a visual desert to me for longer durations. Yes, there are a couple of things they absolutely excel at, and I use the shell for those, have one open all the time in a window – but everything else? Not my cup of tea. I'm a fairly visual person and I like manipulating things directly, I like visual cues, I like discoverable software, I'm into well-done(!) motion cues big time. I like using the mouse. I may have to hand in my hacker card after writing this.
I like how IntelliJ shows a million small annotations and cues that terminal just can't possibly support, how I can just stumble over features via the UI. I like how graphical text editors aren't just a featureless wall of text, that there is a UI for the eye to use for structure, that I usually have much more information immediately visible, and that the UI allows me to do things without having everything in my head or looking it up in huge manpages that often as not are hardly fit for for human consumption.
Lots of things I do wouldn't even be possible in a shell, like graphics/design work, 3d modelling, spreadsheets, and the rest wouldn't be much fun. Yes, I'm sure there is a way to do complex 3d modelling in emacs. No, I don't think that will work for me. Yes, I've tried vim and neovim and emacs, and I really don't like their paradigm.
As to what has been lost, I firmly believe I've gained much, much more, and I doubt I personally have actually lost anything substantial, but maybe we won't agree there. Not being bare-metal all the time is an advantage in my book; I've done some bare-metal and it isn't for me, huge respect to those who work with assembly all the time. A shell isn't very close to the metal for me, it's a really heavyweight, very idiosyncractic abstraction over bare metal, once you peel apart all the layers in between, it just makes it easier to manipulate other heavyweight abstractions directly. What I've learned in years of doing lots of work in shells is mostly limited to idiosyncracies of the shell I'm using (so far fish, sh, bash, a bit of zsh) and the various tools involved; especially macOS doesn't expose that much of its internals via the shell, not that much to be learned there.
Why stop at that level of abstraction, though? Why not put something on top that allows me to use all those areas of my brain that do complex visual and motion stuff? I strongly feel those are still way under-utilized, or rather badly over-utilized, or mis-used in general in all current mainstream GUIs; maybe that's why some people prefer theirs to be extremely simplicistic. Better use of motion might involve less motion than e.g. macOS currently has, but applied way more judiciously and effectively, possibly with a solid neuroscientific backing. Reality isn't a solid monochromatic wall of even-sized glyphs, why would UIs have to be?
Sadly, the only place where I feel motion in UIs is done well currently is the very rare game that gets it just right, but those paradigms wouldn't translate well to productivity UIs. But while not ideal by any means, I still find the macOS UI to be way ahead of any terminal UI in that respect; iOS even more so, but I can't use iOS devices for day-to-day work, not yet at any rate. I gather it's the other way around for some people, which is fine of course. You do you, and I stick to my animated GUIs, and I hope someone will figure out why we see these things so very differently some day, and we'll be able to make new things work better for all of us with that knowledge.
Another thing, discoverability and rich input. Take the Touchbar, fantastic thing – I'd absolutely love to have an external one that I can attach to my mechanical keyboard. With the touchbar I can do all sorts of stuff right away (in apps that support it) that I'd have to memorize arcane key combinations for otherwise; I hate doing that. IntelliJ supports the Touchbar pretty well. It can double as a sort-of analog slider, without any finger smudges on my main display like on Surfaces with touchscreens (though that's neat too – love to do this on the iPad). There are menus that contain pretty much everything an app can do, on macOS they're even searchable. I'm sure there are ways to replicate some of that in shell environments, but I doubt it would get very close – and then I wouldn't want to invest hours and hours to set it up like that, because, in the end, my visual brain would still starve. I've toyed with using my iPad and Pencil as a graphics tablet via Sidecar; I'll definitely do a lot more of that in the future. I wish there was a collaborative whiteboard app that supports this really well and also can get approved at my work. That would get close to whiteboarding my thoughs like I do all the time in the physical office, another thing I can't translate to a terminal workflow at all, I can't even translate that to a graphical workflow without a physical pen or similar.
I get that a higher-stimulus, "richer", more immersive, more physical user experience is the opposite of what some desire, but I believe there are lots of people who enjoy that sort of thing, when done well. I realize there are lots of upsides to shell environments as well (super stable software, highly portable, runs everywhere, relatively consistent, highly configurable unless you want anything not made of solid glyphs, etc. pp.) but those just don't rank all that high for me. Like, I have my Macbook set up the way I want it, I can move everything to another one just by restoring from backup, I don't need everything to be highly portable. I rarely work on remote machines – one of the upsides of doing everything as infrastructure-as-code, and when I do, it's via JupyterLab and the like.