I think the biggest issue with matplotlib is it's straddling of the fence between MATLAB and pythonic syntax throughout the library. It makes it actually pretty hard to predict what the syntax/call signature of a particular method is if you don't use it fairly regularly.