I have been involved in patent litigation for many large companies, in a defensive role. And while some do blatantly violate patents, they generally go out of their way to work around the patent.
And there are some oft-requested features that don't exist in very popular products that don't exist solely due to patents.
And even patent trolls attack large companies more.
I just don't think that if you're a startup company you have much rational reason to care about patents. And if you do fear them, I suspect you'll likely fail due to that fact I doubt you're very rational.