I don't think they even set out to make a small phone with the SE, they set out to make a cheap phone. They achieved that by reusing older generation iPhone tooling which just happened to be smaller, as was the style at the time. When they refreshed the SE line it too got larger as it graduated to using later generation tooling.