Does anyone have the history for why number of pins was such an important consideration (not just for Zilog, Intel too)? Was it really that much more expensive to make a 64 pin package than a 40 pin one?
Chip packaging has come a very long way from the days when a 40 pin DIP was considered a large chip. Kens (righto.com) has a series of articles about classic chips that show what high density designs looked like in those days and you can clearly see how the limits of packaging play into the designs of the dies themselves.
A 64 pin package that would not occupy a very large amount of board area (this is well before SMD became commonplace) would use all kinds of tricks such as staggered legs (sometimes up to three rows) and really tiny traces to be able to keep the costs down.
This is a fascinating bit of tech history and the fact that the packaging kept pace with Moore's law is maybe not quite as impressive as what was happening on the inside of the package but it was a major achievement and an enabler in and off itself.
The 8086/8088 multiplexed the address and data busses to get the pin count to 40. Which wasn't bad when using DRAMs since they're also multiplexed. Again to save cost.
Advantage of the 8088 was the 8 bit data bus. So you only needed 8 DRAM chips. Which was a big cost savings. And extra 16k of DRAM could be hundreds of dollars.
Prototype carriers were likely significantly more expensive for >40 pin DIPs, just due to the lower demand. The ceramic packages with a die well and glued or brazed lid are available as an off-the-shelf item for low run, prototype, or special chips, and that's what a lot of chips back then started off in, including the Z8000.
As other comments suggest, escape routing was also a secondary issue. Not a problem for "Texas cockroach" 64-pin DIPs, but PGAs and such.
Intel management didn’t want to pivot to microchips despite the team’s pleads, so they left and founded Zilog, which eventually prompted Intel to pivot (https://www.amazon.com/Silicon-Invention-Microprocessor-Scie...)
I cant find it now, but I remember in one of Asianometry videos a history of Japanese manufacturer traveling to Texas and discovering he can offer American clients his lead frames at 1/10 domestic cost and still make a killing.