I'm a fast typer, and I understand where you're coming from! But note how there's a small (200ms by default but can be configured) delay when the auto-cd happens. This is actually pretty important, because otherwise it's impossible to see where you cd'd to. Importantly, during this delay, the keystrokes are eaten (so they are not fed down to the next level), and from my experience this is enough to basically never accidentally cd in the lower level.

The auto-cd can also be turned off with a CLI option.