From reading what Haskell related articles I've seen around HN and other places over the last few months, the always-on laziness seems to actually be a problem a chunk of the time, correct? Well, not so much a problem, as not always a good thing.
The big problem is that reasoning about non-strict evaluation is quite non-intuitive (although not impossible.) It's quite easy to write code which seems inefficient that runs in no time at all, and equally easy to write code that seems efficient that takes a long time to run. It requires a rough understanding of how your compiler is optimizing your program and what the resulting code is doing, which can be tricky. Laziness also (not surprisingly) interacts with side-effects in a complicated way, so much work has been done to alleviate the problems with lazy IO, e.g. iteratees [3].
So, yes, sometimes it's a great thing, and sometimes it's a problem. The position of the Haskell community is that the exploration of how to properly leverage laziness is worth the tradeoff, but there is indeed a tradeoff.
[1]: http://augustss.blogspot.com/2011/05/more-points-for-lazy-ev...
[2]: Infinite data structures (i.e. codata) can be written/created in a strict language, but the ability to write them falls out of laziness quite naturally, and a lot of people experience them for the first time in a lazy context.
Laziness also makes the language more foreign and harder for many people to learn initially, but I think that fundamentally does not matter. Learning Haskell is a constant cost where the benefits are at least O(n) (and plausibly more) to how much you program with it, so any learning costs are quickly dominated as you use the language.
Unless you're trying to use Haskell to compete with C (and some people are), laziness is a net gain. Take a look at the "Why Functional Programming Matters" paper for a good case in favor of laziness.
Now, laziness does have some downsides over strict evaluation; I just believe the upsides easily outweigh them.
The reason you read about problems with laziness is because the problems are encountered before many of the benefits by people learning the language and because the problems are easier to articulate than the advantages. Also, Haskell is the only common lazy language, so that particular aspect stands out.
Simon concedes the troubles with laziness and his biggest item of support for laziness is that it keeps you honest in terms of side-effects. I don't buy it. We can control side effects with the type system -- we don't need complete laziness (some laziness is actually quite easy to hand to otherwise eager languages).
P.S. The other major contribution Haskell made to PL was type classes, can't believe I forgot it.