Something to keep in mind - experienced devops and security experts are extremely valuable, at the same time ML beginners are in oversupply. Do you really want to put yourself at a massive disadvantage in the near to medium term, especially if the future of ML is kind of uncertain while with your other skillset you can have a pretty stable career?
As someone working in ML (a couple of years of experience), I'd much rather be in your position than mine.