If the developer can get permission to build 3-5 story apartments on the lot

wouldn't that only work for a single unit? Like, yes, if inside a single-family-zoned block a unit would get a special permission to be the only one to build high density, sure that piece of land would be more valuable than before.

But if the whole block gets rezoned, then there's a race to the bottom.