It's not that it is hard to build stuff from scratch, it's just that it is easier to use a tool to do it.
I worked on a project where we had to remove jQuery from the page, and it was a nightmare for all the developers that had never written vanilla JavaScript.
These days, I don't use jQuery or bootstrap, or lazy loading libraries because I can just add the functionality I need by myself.