If I could go on Twitter and only see stuff from people I follow I'd feel the same as you about it. But they make it almost impossible not to be barraged with other stuff too, and not just ads, but hate and insanity that I tried hard not to follow. Those are the things that drove me away.
Now I exfiltrate the good stuff with Nitter RSS feeds, and that way I get the experience you say you like about Twitter.