The reason the memory is retained is so that it can be reported to the parent process why the process died (e.g. normal exit (error code 0), or failure, or killed by signal, etc), and retain enough information for simple diagnostic purposes. It is for that reason that page table mappings (for the little memory that is retained) and private data (for process exit status etc) is kept.
But zombie processes aren't everywhere. Typically they exist briefly and then get cleaned up. So even if they could take up 4 bytes, nobody's bothered writing the code to make them do that, because what's the point? If they take up 64KBytes it's just not a very pressing problem.
The point is that there are always tradeoffs, whose resolution is affected by how important an issue is perceived to be.