Anyone who owns a house for long (the older the house, the more true it is), knows that it requires constant small bits of care here and there to make it ‘work’. A bit of maintenance on the exterior here, some yelling at a new yard guy who is mangling something here, getting a plumber in to fix a blocked drain there, add some untangling who gets to figure out the fence contractor on top, and you’re looking at work. Not full time work, but it goes on the pile.
If it doesn’t happen, it goes into a maintenance backlog that will make your head spin later, and possibly your wallet implode.
If you live there, it’s usually straightforward enough to fit it in with everything else, but can be exhausting on it’s own.
Through something major in (roof leak? AC breakage?), add it being in an area you don’t live in anymore, and gets harder.
Then, you can have tenants that aren’t absolutely perfect, and it gets even harder and more tiring. Late payments? Property damage? Neighbor complaints?
If you pick good tenants each time and nothing goes wrong, it can be great. It only takes one case of it not for things to get really unpleasant and overwhelming however.
Also, add in the Covid eviction moratoriums, which opened a whole additional can of worms for landlords - if something happens like that again, which precedent has now been set - you could spend years paying a mortgage and upkeep on a place with zero income from it.
Personally, I never rent out a place unless I know how to do an eviction in the local jurisdiction. I hope I never would have to, but not knowing how is a good recipe to lose your shirt and possibly any future gains you could ever hope to have.
I’ve known folks that had renters who seemed perfectly fine (solid full time professional jobs, etc), but left a complete trash house (literally, multiple dumpsters worth - trash to chest high), and disappeared suddenly without paying last months rent.
The underlying truth is that it takes work and management expertise + energy to maintain a house in livable condition, let alone want-to-live-in-it-condition, and that has value. Also, not everyone can, or wants to do that.