> I've always wondered if any CPUs have tried to reduce the branch penalty by speculatively executing both ways at once in parallel

They already do it (edit: they don’t). It is difficult to get security right, however (see https://en.wikipedia.org/wiki/Spectre_(security_vulnerabilit...).