The New Zealand Passport Office reviewed the utility of the sex marker on passports [1], in the context of calls to remove it, and concluded:

> The costs of the removing the requirement to display the holder’s gender on travel documents outweigh the benefits at this stage. The costs of the change would be more significant given the adverse affects on the operations of border authorities and the potential inconvenience for passengers.

In particular, they noted that border authorities use this data for risk assessment, the significant cost of modifying existing software that processes this, and that many countries would not accept passports without a sex marker.

[1] https://www.icao.int/Meetings/TAG-MRTD/Documents/Tag-Mrtd-21...