I had the exact same experience.
Back at my old company, all of our development was done on a secure Sungrid server farm that could only be accessed from the VDI environment. The way my coworker initially showed me how to do it, the terminal was sent to an X session on the VDI, and yet everyone at the company was OK with this. I explained how typing is something that you do tens to hundreds of thousands of times per day, and that a few frames of latency every single key press really fatigues you mentally. The sungrid farm didn't support SSH (for load balancing purposes,) so I eventually had to deepdive into the spec for how to access via terminal, and wrote a script.
But that's not the worst part as I was interviewing around at other companies, I asked them how their computing is done, and you wouldn't believe how many companies have their development environment set up this exact way! These are all major semiconductor companies in Silicon Valley too, not some "IT as an afterthought" industry!
It's refreshing to see that VDIs are about as popular with developers on HN as Google Stadia is with hardcore gamers.