The usefulness here is that you're closing off ports and reducing your exposure, the downside is that you need proprietary agents installed on the remote devices, and clients (tailscale itself or the SSM extension to AWS CLI) on proprietary networks doing the routing for you. Which might be perfectly fine for your use cases.
I've done even less reading but is Cloudflare's WARP client the same thing for their own network?