Now, if I wanted to, I could build my own application without Django or Rails, as long as I had something to accept incoming HTTP requests, make responses, and a library to make DB calls.
But I wouldn't have gotten to this point had I not learned Django and Rails first.
I think using a framework turns many "unknown unknowns" into "known unknowns".
I can read my framework's documentation and learn it has built in logging, so logging is something I need to think about when building a web app. It lets you filter sensitive data from logs, so that's something I need to think about.
This is why I was building a type of application I didn't build before, I would look for a framework I could use, or something like it.
I'd use go kit for building a microservice for example. I don't get many of the concepts involved with building a microservice (circuit breakers, service discovery, etc) and hopefully the framework would help me learn them.