I’ve been in this situation before. I had a full dev team, we had a security architect assigned to us. We needed to do a pen test regardless of it we used a framework or wrote it from scratch.
Our requirements were quite unique and so writing custom code but aligning to an OIDC style flow was a good choice for us.
In the end it was a good choice and much easier than forcing an auth framework to do something it wasn’t designed to do.