So far, I've written all the "collection" type stuff in my system in Haskell, and I'm glad I ignored everyone's hemming and hawing about it, because it was actually a really pleasant experience. Now that I'm gathering a lot of disparate data, I'm in the "what the hell do I do with it?" phase of my project. I'm trying to resist sucking it all into Hadoop (partly because I'm a one-person operation and even a small Hadoop cluster would be larger than all of the systems that are generating the data that I'm monitoring, and partly because I can keep about a years worth of data comfortably in memory).
So I'm basically using this as an opportunity to try out whatever kinds of data analysis I think sound interesting, and then hopefully learning what is a good fit and what isn't. This is all uncharted territory for me, and I'm fortunate to be in a position where I can do what I want and don't have any time pressures.
I figure I can try to use this data to learn about statistical modeling, graph analysis, and machine learning. For the time being, I'm favoring Haskell for this, but that's not really any type of requirement. I just find that I enjoy Haskell a lot right now, so I should see how far I can go before the honeymoon ends and the awfulness is revealed.