Maybe you're right that it's all just hard-headed stubbornness from fluent people. But I (and all other learners of Japanese) started by reading hiragana, and only later did we move to mixed script -- this is by necessity. Yet even when you factor in the difficulty of learning to read Kanji, reading Kanji is vastly easier than reading kana, even as a beginner.

Would spaces magically solve this problem? I guess it would solve some things -- you'd no longer have to guess where to terminate the prefix search, and I think you're right about word shape -- but it would definitely not provide the additional semantic context you get from having the high-bit-density characters in the mix. This makes reading faster.

I suspect that one could make a kana-only writing system that would be functional enough, but it would still be slower to read than mixed script. Also, the Korean comparison isn't exactly valid -- Korean has more sounds than Japanese. It seems minor, but Japanese has a ton of homonyms because of the tiny phoneme. Expanding that, even slightly, would be a benefit to reading.