Instead join any related graphics domains: scientific visualisation, graphics drivers, jobs that involve writing game frameworks [1] for example. These jobs will give you both the fun of working in a hard domain combined with the stability that does not come with the intrinsic fickle nature of game industry. FWIW, I have followed this path of learning to write games on the side but not joining any game industry.
My 2$, I would say don't go to the extreme decision of leaving tech (that is also not a bad idea if you work it out) but before exploring all the other interesting options in tech and giving that a try, may make you think 'but if' in the future. Best!
[1] https://jobs.apple.com/en-us/details/113436938/game-developm...
Another idea is to join a small team, so that you have to become more of a jack-of-all-trades.
“A jack of all trades is a master of none, but oftentimes better than a master of one.”
Pick a game development technology that you like the look of (a language or framework or engine), do their basic tutorial, then drop by their forums. Almost all of them will have a "looking for collaborators" section where people with ideas but insufficient skill, or with skill but insufficient ideas, can shop around for partners to make something cool.
If you're not picky regarding tech, there's also TIGSource, which has a platform non-specific collaborators wanted forum: https://forums.tigsource.com/index.php?board=8.0
Or, if you're looking for something a bit more time boxed, a lot of game jams (game development hackathons) will help you find a team to work with on a no-commitment once off project to get a feel for whether this is something you enjoy. See if there are any running near you, and drop on by. Global Game Jam is a great one for this, but it's not until early next year, which may be too long a wait for you.