Component oriented programming makes it rather easy to pick a technology, make a component out of it and adding it into your system.
Especially if you use a prebuilt/popular component system like Unity3d you can almost just click and drag technologies together.
It's true that if you need any of your components to perform extraordinarily then you'll have to dive deep and become an expert, or make sure you find an expert to help you.
Ofcourse, if you want to do everything yourself, then you might have to learn everything yourself , and you'll have to master a lot of things. It is true that games programming covers (almost?) every field in computer science. I guess this is the result of games basically being simulations of (part of) the universe.
I know dozens of independent developers who work on their own games (myself included) - people who work in fields completely unrelated to CS/video games, but still work on games simply because they are such complicated projects to develop.
If you're the kind of person who thrives on difficult algorithmic and code architecture problems, I highly suggest giving game development a shot.
Right now, I am writing a text adventure bushcraft game and the coding is almost tedious and boring.
Or you can use a framework which makes everything easy, like Unity.