The small performance degradation could be a good thing - it may be a result of the use of explicit_bzero to clear sensitive data structures (instead of memset which was used previously) before freeing.
It could mean that a fair bit of that important cleaning was being optimised out before with OpenSSL.