Having extra keys for your pointer finger sounds good (those 1.5U side keys on the ergodox next to tgb and nhy), but it is actually difficult to precisely decide among 3 columns that you have to absolutely position your finger to hit. I feel like my accuracy in deciding between b or v suffers and having another possibility doesn't help.
So my suggestion is to go small. Not having a number row may be overkill; they are far away but I have yet to remap them. Figuring out what keys are most important to press and making them easy is crucial. I know I hated where {}, -, =, etc. were on traditional keyboards, so I was careful to put those on home row. It was worth it. Having an extra row of normal-sized keys instead of control, meta, super, space... probably not worth it. The 1u keys next to the 2u thumb keys on the ergodox... I basically never use them (they're arrow keys and page up/page down, but I navigate with emacs style keybindings so only use them for skipping through YouTube videos and things like that; nice to have, but impossible to use during normal typing).
Not using your pinky for enter and backspace is also great.
For completeness, my layout: https://github.com/jrockway/qmk_firmware/blob/master/keyboar...