This is why I love to do game development without using a game engine. It gives you a reason to learn math, optimize your code down to the metal, all while having fun playing your game.
Keeping it as low-level as possible, I'm using CycleJS for dataflow management and Regl.js for drawing via a CycleJS-Regl.js driver.
All state is explicitly managed observables/streams in CycleJS, which maps out to Regl.js draw commands, which are basically raw frag/vert shaders with some bindings mapping my state from CycleJS to appropriate uniforms/attributes.
I probably would be able to produce some usable output much faster if I used an engine like Unity or a framework like Three.js, but I feel like I would have missed out on gaining so much knowledge by only working with high-level abstractions and never having to even touch GLSL code.