I think you would have a hard time, but not entirely because of the language's faults. Rather, the core issue is that most of the things you have to keep in mind to write fast Haskell (like dealing with laziness) are very different from what you would have to keep in mind with more common languages that you are already familiar with.

That said, as I noted, the performance is usually good enough even with naively written code; you would only have to spend time optimizing a few select parts of most programs.

Another little problem is that there are not many resources on how to optimize Haskell code. This is a little annoying if you're learning on your own; however, I think it's offset by the Haskell community. I've seen some of the best Haskell programmers give very detailed responses on StackOverflow and the mailing list, so there is always somebody to help you while you're learning.

There are also some advantages to OCaml--it's a very fast language itself and it behaves in a way that's easier to predict especially since you can leverage more of your existing experience. OCaml is also a very nice languages; while I prefer Haskell, I would be perfectly happy with OCaml (and, in fact, I've only used OCaml and not Haskell in production).

I should add a big caveat to this whole post: while I've used both Haskell and OCaml a fair bit, I've avoided doing anything super performance sensitive. I usually care more about having correct, readable code quickly than I do about how fast it runs. So I am definitely no expert on optimizing either Haskell or OCaml!