I recently switched from frontend to a backend/ml role at a large tech company.
At a high level the work is pretty similar. Instead of working with design/product you work with data science/product. The conversations do tend to skew more technical. I remember we talked a lot of about user experience on my old team whereas now it's more about the stability of our services/pipelines and the maths behind the models and features.
One thing I'd call out is that, as an engineer, you're largely responsible for productionising what the data scientists come up with. There can be varying degrees of collaboration on the modelling depending on the team/project but at the end of the day you're there to make the thing work in the real world.
Compared to other engineering jobs, I think the work tends to be more experimental, i.e. can you quickly write code to test an idea.
If you're interested in trying it out, go for it. There's a tendency to silo engineers (backend, web, mobile, etc) but code is code and you can ramp on the concepts in a few months :)
On the other hand, if you're loving frontend there's nothing wrong with staying there. The depth is definitely there but you might need to change teams/companies to keep growing.