Try rebuilding your mail index. You can use OnyX for that.