This is what eventually got me off of Unity.
Unity is great if you're a beginner in game development and software development in general. You don't yet know all of the various things you need to make a game. If you're a beginner programmer, you will also struggle to build those things, once you do figure out you need them. So with Unity, you can get some really cool looking results up and running very quickly. And you can then make those results run on a variety of platforms that you can then show off to your friends/potential employers.
But if you are an experienced developer, Unity is a gigantic bag of bad conventions, with very little in the way of configurability (and that little of which being very poorly documented).
Unity the Company seems to optimize Unity for getting people up and running very quickly. They get you into their ecosystem and constantly tease you with "everything will be fine, just wait for us to fix this one issue". And you wait. And wait. And wait. And then they fix the issue. And introduce two more. Trying to build a professional-grade project in Unity was a nightmare.
LTS versions are way behind on latest hardware support, so if you're doing things like working on VR or AR projects (and I've seen estimates that something like 80% of all XR projects are built in Unity), then you have to try to surf the wave of latest releases to even have a hope of making a usable product.
A lot of the built-in features are designed in such a way that they only really make sense for small, completely encapsulated projects. If your game does not need to connect to the network for anything, if it doesn't allow for user-editable content, if it just sits there and does its own thing, Unity works fine. But if you have any concept of dynamic content, you'll be fighting Unity for the rest of your life.
For example, network requests, file loading, and texture and audio decoding all take place on the render thread. If you can get away with showing a static loading screen during that time, ok, that's fine, I guess. But in VR, you must never drop frames. That's really hard to do in practice even on ideal systems, so we do tricks like fade-to-black when we can't avoid it entirely. But you can't just leave a user in a blank, black void for several minutes while you wait for textures and audio you're fetching over the net to load and decode.
You want to make your controls remappable? You want to let the user pick the rendering quality level to match their hardware? Unity has default UIs for that! Oh, except they only appear in an application pre-launcher UI, that only appears on desktop PCs, if you have that option selected. Mobiles and VR devices, or VR apps launched from within the VR view get nothing. So implement your own. Again.
Hell, there are issues like certain app store platforms having minimum quality specifications that you literally cannot achieve if you have built your project following Unity's documentation. For the Oculus Store, your application must be responsive to user head motion within 3 seconds of the user opening your app. If you've followed Unity's docs, all of your game is smashed into a single Scene, where anything but the most trivial scenes on the most powerful hardware will take much, much longer to load than 3 seconds. So now you have to dig into the wonderful world of split-Scene architectures and how they prevent using half of the editor's property inspectors for making references across objects. No guarantees any of your dependencies even exist at that point, so you're left doing tons of runtime checks to make sure you can even proceed with running your program.
In my project, we wanted to build new levels and deploy them on-the-fly. Unity's means for doing that is the "Asset Bundle", which is basically "take Unity's proprietary binary format and chunk it around as wholly contained blobs." It's extremely inefficient regarding resource usage (got multiple levels that reference the same video file? Yeah, that's getting copied into every level). It takes running the Unity Editor itself to generate the blobs, so either your content creators need to be running Unity directly or any content development tool you make needs to interface with Unity's extremely poorly documented command line interface. I know of projects with web-based management interfaces where they have installed Unity on the web server to be able to do this.
Before I switched away from Unity, I had implemented a whole parallel system of texture and audio loading and decoding to end-run around Unity for this reason. Now you run into the malarky that is cross-platform support in Unity. For as much as they tout their cross-platform capability, it really only works (for static content) if you either A) again, are doing very simple things, or B) keep a bespoke, per-platform project directory for each platform. You can do it if you are super careful about using simlinks for your common assets (all the while ignoring Unity's dire warnings that simlinks are dangerous, figuring out on your own where the dangers are to sidestep them). Needing dynamic content, I needed pure-managed implementations of JPG, PNG, and MP3 decoders to be able to do this. Well, maybe I didn't need them, maybe I could have figured out how to bundle the native libraries for doing this across the myriad of platforms I needed to support, but as far as I could tell, if anyone is doing something like that, they aren't sharing how.
The UI system in Unity has been garbage for so long it has become a meme. Same for the multiplayer networking. When I finally got around to wanting to implement voice chat, I realized I had reimplemented so many of Unity's built in subsystems that it would be easier to reimplement the project using a library-based approach and reimplement the few subsystems I hadn't yet already done. Once I did that, project velocity improved dramatically.
I think a lot of people think to themselves, "I want my game to look beautiful, therefor I need to use a cutting-edge game engine developed by people who are smarter than me." Then they start with Unity and wonder why their graphics still look amateurish. The games market has done a great job at selling the idea that good looking graphics are the result of heroic programmers. But it's a lie, one that they perpetuate to ensure brand loyalty. Good looking games that perform well require great art assets, authored specifically for games, in simple game engines that enforce a very strict data pipeline. A general purpose game engine gives you the illusion of control. They give you this GUI editor where you can dump a bunch of assets into a scene, and it looks great on your dev machine, and then runs like shit on your target device. Because that flexibility the editor gives you is anathema to what it takes for good performance. So you end up working around the editor to get that performance back. So why were we ever using the editor in the first place?
If I were to ever use a game engine again, I'd strictly use an open source one, like Godot, just because then there'd at least always be an escape hatch.