Because the format was originally written by PKWARE for their PKZIP and PKUNZIP programs for DOS in the 80s and used for backup purposes, among others (being able to span multiple floppy disks was an often used feature).

To write the directory at the beginning they'd need to keep all the data in memory or do some extra postprocessing that would make it prohibitively slow on a 4.7MHz machine with a 20MD hard disk and a slow 360K floppy disk drive. So the directory goes at the end.

The files do not begin with a magic number because ZIP files can be embedded in other files - most commonly executable files for the self-extracting feature of PKZIP that placed the ZIP file right after the decompressor. This allowed the executable to be used both by itself and by PKZIP.

As for the data after the magic number, it was used for the comment which was most likely added at a later point and they decided to put it after the magic number so that it remains compatible with existing decompressors. In the 80s and early 90s people didn't had Internet to get the latest version and a lot of old versions of PKUNZIP were floating around for many years.