Disclosure: I work for FusionAuth, an auth provider with a free community option.

If I were in your shoes I'd probably use a library built into whatever framework you are using. Auth servers are powerful but are another architectural component you have to manage (even if it is a SaaS, there's still config to manage).

Not sure what you are building it in, but if I were building it in rails, I'd use devise. If JS, maybe nextauth or passport.js.

When you do this you have to accept certain risks (what if your user data gets breached, what if you want to add more functionality) but based on the little you've shared, I think a local solution is perfectly fine.