For Intel's Xeon processors, the more cores you have the lower the frequency you get. Quad core might give you an extra gigahertz, which might be of more value to games than 12+ cores.
Plus shouldn't XFR (or Turbo Boost Max Technology 3.0 Frequency for Intel) somewhat alleviate that particular issue?