I don't think the audiophile comparison makes really sense here (and I like to mock audiophiles more than most) simply because display technology still has a long way to go before it reaches the level of audio when it comes to "bang for your buck".
CD quality audio is less than 1 megabit per second per channel, uncompressed. HDR (10 bits per component) 4K60fps 4:2:2 video is around 10Gbit per second of data.
Of course data bandwidth is only a small part of the problem of correctly reproducing an analog signal, but it gives you the orders of magnitude we're dealing with.
I currently use a cheap ASUS 4K display. It's more good enough for coding, but I wouldn't trust it for any sort of graphical work. The viewing angle is pretty bad, so depending on what part of the screen I'm looking at I see colors differently, and some gradients become more or less visible depending on which part of the screen they're on. Contrast is pretty bad, making even some videogame display poorly: depending on the location and time of day contrast seems always too high or too low.
You can buy a good sub $100 pair of earphones and a sub $50 DAAC and they'll be good enough to do 99% of any audiophile work you could ever want to do reliably. If you want to do serious graphics work without having to constantly adjust for your display you'll have to go for something a lot more expensive than an entry-level monitor.