Flours like rye or whole wheat don't have a lot of gluten relative to white flour. And there are a whole lot of online recipes recommending specifically those flours for feeding your starter. When you bake with such flours, you get a denser bread and when you create a starter with them, you'll struggle to get them to double in size for the same reason. They'll ferment and bake just fine. But there's just a whole lot less gluten to hold the starter together. Of course there's a wide variety of flours and milling techniques that determine how much gluten is actually in each flour. It's entirely possible to have whole wheat with lots of gluten. Rye has less of it but still some. It all depends.
I tend to use rye and whole wheat for flavoring. I baked a 75% spelt, 20% whole wheat and 5% rye flour bread last week. Very tasty. And I got a good over rise out of it (using a Dutch oven, which I highly recommend). My starter is the bastard offspring of whatever I had around to feed it for the past seven years. I started it off on regular white flour from the super market. What you'd call all purpose flour in the US, I guess. And since then I feed it with whatever I have around. When I don't use it, it lives in my fridge. For months on end. It ends up looking really sad. But then you feed it and it bounces right back. After observing that it was struggling to double in size with whole wheat or rye flour a few years ago, I now feed it with spelt flour. Which seems to work well. It's a German 1050 variety. Which means it's high in protein and gluten.