I did this - kept a day job as a systems administrator while building a video game using Unreal Engine 4 (as a solo dev - no team). It took almost three years to "complete"... and I shipped it on Steam back in Feb. 2017. (it's 75% off right now during the summer sale!)
Making the game was a brutal experience. Little to no money because I was taking so much time off work to make the game (I'd work maybe 4-6 hours at my job, then go home to work on the game). Deteriorated health because I sat for 16+ hours PER DAY in front of the computer. Morale was extremely low because few people wanted to help me (building a team is super hard) and more often than not, people I talked to about it just assumed I wasn't even capable of making a video game, much less a video game business.
I basically just trudged through it. Forced myself to complete it and ship it publicly. I was rather happy with the end result - the game wasn't perfect, but it was mine. Player reviews tell a slightly different story for a countless number of reasons.
The day I the game went live I was so excited... but that emotion quickly faded as I realized I wasn't going to even come close to my sales goals. To date, I've sold less than 200 copies. Thousands less than I expected, and not nearly enough sales to make another game. Literally can't even pay rent on my tiny apartment for one month using those funds.
About 4 days after launch, I realized my traffic and sales had flat-lined at zero. I came to the painful realization that I had just spent three years of my life making huge sacrifices of my time and money to build a product that no one knew existed and only I really cared about. All of my efforts merely in vain. Despite numerous post-release product updates, it stayed that way for months, until yesterday with the summer sale. I've sold a few more copies over the last 24 hours, but again, I can totally forget about becoming a full-time gamedev. I literally don't even have the funding or even strength to make another game - as a solo developer who isn't a unicorn, it simply requires too much investment for too little return.
Building my game was one of the most influential and exciting experiences of my life. Trying to make a paying job of it was easily the most depressing path I've ever walked. I even thought my experience shipping a game in a modern engine would help me find employment as a gamedev... NOPE! I literally can't even get a rejection email from most of the hundreds of places I've applied to. Needless to say, I went back to SysAdmin full time because it's a stable, paying career. Which is also depressing because it's not really what I'm passionate about... I'm just good at it.
All the dreams of making a living building video games? Crushed. Forget about it. If you're able to do this and you're still passionate about it, consider yourself extremely lucky.
edit - the game is here: https://store.steampowered.com/app/428980/Ethereal_Legends/