Good luck supporting old android browsers, IE and dealing with the inconsistencies between browsers. It only takes one look at the jQuery source code to understand why it's not a good idea to "roll your own jQuery".
Simply don't target old browsers. If people don't use JS, or don't have a modern browser, feel free to go elsewhere. I don't target edge cases as I don't get paid to.
I really envy the industry where you work on. In my industry you can't just ignore half a billion of outdated Android phones and Government seats.