I agree, and that's unfortunate because that's not the most differentiating thing about array languages. As you point out, it is the semantics: some of it appears in other functional languages (like doing away without loop constructions) but some are unique to array languages (like using various properties of matrix/vector operations to express certain computations concisely).
I am a big believer of linear algebra as a great tool of thought for computation that most people have little to no knowledge of, and the APL family (and actually, Excel) brings home this point.
EDIT: And I definitely speak from my experience. I was a professional q programmer for two years, and I came to realize what makes q difficult for many people is not just its cryptic notation but also the general lack of understanding of linear algebra/matrix operations.