We built 5.6 with -O2 -march=broadwell -mno=avx (had to remove, probably pecl ext issue).

There was about a 15% performance gain. Nothing that would explain the large difference between E3 and E5.