Radio English has specific alternative prononciations, which help avoid this, like "fife" and "niner".
However if it is clear enough and there is no ambiguity, the common way of saying numbers can be used, so zéro onze is acceptable. In normal conditions, no french speaker will mistake "onze" for "un", the latter should be "unité" anyways.
The main problem with "un" is that it sounds like "hein?", which means something like "huh?", the kind of meaningless words that punctuate conversations without even noticing.
There's also a defined cadence and order for many transmissions. If you read off the automated weather or a clearance in the usual order, it's easy to transcribe it. If you read it off out of order, it would sound the same [and just as difficult] to a non-pilot, but would be much more difficult for a pilot to write down.
The other day I asked for direction at a front desk and the lady told me to go to the office # "trente quatre vingt", I wasn't sure whether that was "3420" or "3080".