The P/B distinction will be difficult for many people as well.
I believe this is one of the reasons that Toki Pona only has nine consonants (/p, t, k, s, m, n, l, j, w/) and five vowels (/a, e, i, o, u/).
Somme kind of error correcting would be nice too, perhaps?