This is exactly the conclusion I came to. I still use Obsidian for a lot of things, but I've developed my own task management software that uses plain text files and fzf for everything: https://codeberg.org/ngp/tsk

It has worked great for me! That being said, sync and mobile usage is still a bit of a sore spot with it. It works beautifully with Termux, but I wish there was some way to slap a basic UI on top of a CLI application for mobile. Tk/Tcl is the closest but there's only options on Android (I'm mostly an iOS user) and even then it's not really ideal. For sync, I at least have a reasonable plan: IMAP4 or git-based sync. The roadmap and tasks for the project are tracked in-repo with itself, so it definitely works.