Each My Number Card is a X509 certificate signed by a government agency (JPKI) and uses two passwords for signing and for user authentication. As far as I'm aware, there are currently two concerns with the JPKI part: though the password itself is not an issue (although the password requirement of 4 digits number for signing + lowercase letter for authentication may as well be), the ward office offered a piece of paper to write these passwords down when you issue the card, and many people did & folded it together with the card itself. Another one is that requesting user authentication means you'll get all the data on the card (name, DoB, gender, current address, image data) which may be too much from a privacy standpoint.

Apart from JPKI itself, another issue of My Number Card currently is that most of online KYC using MNC is still largely done by comparing the photo printed on the card with an image data taken from a web camera. There's a published paper of a group of researchers able to pass an eKYC using a photo of MNC and Deepfake[1] (and an actual case of someone doing a SIM-swap using a photo of My Number Card[2]). The government has recently been promoting the use of JPKI for eKYC instead of doing image analysis[3] (which loops us back to the privacy issue, though there have been talks about reviewing what to include/not include in the user authentication part of the card).

[1] https://www.jstage.jst.go.jp/article/pjsai/JSAI2021/0/JSAI20...

[2] https://finance.yahoo.co.jp/news/detail/e3801312bffeedcb1999...

[3] https://www.soumu.go.jp/main_content/000946152.pdf