Why would that be an issue? People already have Mesa and AMDGPU working fine on non-x86 architectures (people have demoed physical POWER, RISC-V and AArch64 machines, all with various Radeon GPUs and the newer AMDGPU stack) and the support gets better and better every release. But it already works fine.
Also, Franken-Ubuntu I suppose isn't really my complaint, though I still don't like it. It goes beyond that, though -- the Tegra boards have very custom design that means some things are a huge pain in the ass if you want to even remotely treat them like "normal" computers (for example, want to test that fancy driver you wrote on your AArch64 Tegra machine for that M.2 peripherial? well first you gotta reconfigure the BSP firmware to turn off the power rails to various subcomponents so you can reroute them to things like the M.2 E key slots in the TX2. Gotta disable that USB controller via BSP flash if you want a chip in there, etc etc). It also means I am basically stuck on older kernel versions which may mean features I want to use or develop with are basically off limits (eBPF enhancements, io_uring, etc.) I don't just want to test SIMD; I also need to test actual memory/disk/peripherials, etc. These constraints make perfect sense for Tegra devices -- they are still annoying, for me.
I'm not saying the Tegra machines are not nice, or they are not high quality. They absolutely are. If you are just writing code to give the CPUs some work to do, they are great! Flash the default BSP, plug in HDMI, run 'gcc'. They're the highest performing ARM machines for their power profile and size, IME. But the moment you treat them like an "ordinary" computer, all bets are off. These LX2K machines act substantially more like "ordinary" desktop computers, and IMO that's a significant benefit.
Now, if you want to do DL inference or GPU programming, I think CUDA is vastly better (for a number of technical reasons). And the Tegra machines would be a good choice! But if you're just writing ARM code and need a desktop, these machines seem perfectly sufficient and can host open source drivers just fine, and will be far more usable in a number of ways vs a Tegra machine.
(And also: Nvidia also released some press notes last month that they would be bringing the full CUDA stack to ARM for data-center computing, presumably, including the host driver. So it's possible you may be able to stick a Quadro into these things soon enough and use CUDA to your hearts content!)