I can understand why you're torn, but - speaking here as someone who owns such a watch - people vastly overestimate the Veblen status potential of expensive watches, which is unfortunate because that can lead them to be quick to judge their owners. I have met wealthy people in the sort of industries you might typically associate with this kind of pedigreed excess (Wall St, Big Law), and those folks don't appear to recognize what I'm wearing at all. Often they're wearing smart watches these days.
The truth is that extremely expensive watches are a poor form of wealth signaling unless they're loud (think of a Hublot encrusted with diamonds, or something extremely iconic like a Rolex). If you walk into nearly any room wearing a Patek, Lange, Vacheron, etc. I can virtually guarantee that your watch will simply go unnoticed.
I actually like Rolex watches quite a lot for the real engineering work that goes into manufacturing them, but I don't own one specifically because I'd rather not broadcast an expensive item on my wrist. I consider watches like these to be a form of art that emerges from the idea of practicality but also frees itself from it (as do most collectors I've personally spoken to).