No, it means you have something unique to say.

The bar is there, but it is lower than you expect. If you have a truly unique point of view to express, that brings some value to the table, slots will open up.

And I've spoken at plenty of conferences. :) Not always in the glamour rooms/slots. But... I did have one talk fill a room out the door. That was a talk on a difficult/controversial topic, and by then... I was probably about as expert as they came on the issue.

I didn't start with that though. I just started with a simple point of view talk. And I'd argue the second version of that talk is still one of the best I've given in my life.