These sound like you're trying to build something with professional polish when you wanted to have fun:
> Part of it feels like I've tried to do too much at once instead of building piece by piece
> writing the tests for these and going back and forth on different decisions has killed all productivity and drive within me.
Why not build something awful and silly? Alternatively: add an awful plugin system and say it's left up to the user.