Every competent reader of Japanese is first and foremost used to the kanji-hiragana mixed script, and has shortcuts for the kanji forms of words and the sounds of those words. The hiragana only forms? Not so much. So when they complain about hiragana only being hard to read, they're not lying. It really is harder. But it's not harder due to any inherent defect in a hiragana-only script, it's just about a lack of exposure to form those shortcuts that make reading feel easy.
Similar arguments used to be made in Korea, yet if you look at Koreans today they have no difficulty in reading hangeul - they have spaces so the words have form, and they have mass exposure to the hangeul forms of words. Ergo, their shortcuts are for the phonetic forms and those are what feels natural, solid, meaningful and easy to read. Same as both of us in English or me in say, Finnish or Swedish or French.