You should really listen to the Manager Toolbox podcast on 1:1s (there's 3 segments, all of them very worth the time):
https://www.manager-tools.com/2005/07/the-single-most-effect...
Basically, that "time sucking" activity with trivial conversation is actually what helps people build a [professional] relationship with their bosses/reportees. From my experience, it is an error to use the 1:1 to check every week how work is going. There's a lot of other channels for that (including the dreaded daily standup). As I mentioned to my reportees: "One on Ones are to talk about anything you want, particularly, company/office stuff that is not part of your day to day job. If you want to tell me about someone that smells bad, this is the place, or about the water dispenser that never works, etc". Of course if someone wants to tell talk to you about work, it is also great.
Specially on remote-work scenarios, having weekly "how are you" calls is important to pull the members of a team closer to the team.