You can use frameworks where they fit, and not where they don't. Knowing when to do which takes time and experience, but it is definitely do-able, and better.
Frameworks allow you to update, and maintain your app for free, since others will do it for you. If you do it yourself, you have to do all the work and that's keeping you from adding value elsewhere.
To not use frameworks these days would be objectively stupid unless you're working on extremely low-level things.