If I were to bring in 3rd party gem for auth, I'd have gone with Sorcery rather than Devise.
> You just have to be cognizant of what you're doing
These two ideas are incompatible with each other.
You don't roll your own security. You just don't do it. There are far too many variables that you can screw up, and anything you come up with isn't realistically going to go through nearly as much battle-testing as a ready-made solution (either upfront or on an ongoing basis).
> Custom built auth isn't as scary as you make it to be
Custom-built auth should be scary. Anyone who's worked on the security story for a popular framework will tell you that.
If you're not scared by it I'd have to assume you're either not seeing large swathes of the problem space or you're a genius.
Don't write your own crypto. That's great advice. The idea that everyone's auth needs are so standard they fit for every app just hasn't been realistic in my experience.
Yes, you don't roll your own security. You use powerful primitives provided by Ruby and Rails, such as bcrypt, has_secure_password/has_secure_token, encrypted sessions, secure httponly cookies with prefixes + samesite attrbiute all served over TLS 1.2+, with HSTS and CSP
But those are things that people normally delegate to either libraries (crypto) or the framework itself (session handling).
Of course, there's lots of places one can screw up, such as sending non-expirable password reset tokens, revealing private information and membership status via F2A/reset tokens. But those are the kinds of screw-ups that can happen in other parts of the website too.
That said, Devise is one of the few things that I don't completely dislike about Rails.