In my experience it is worthwhile to first try and improve the performance in Python. It's easier to play around with different ideas in Python, and things like numpy quite often enable you to get "fast enough". Only if that is not enough should you consider writing a C extension.