Bisecting with --first-parent takes care of this.