The algorithm is primarily focused on your calendar data and when you've last had an event (if you don't use your calendar, these recommendations might not be great but will get better as you snooze/check off people)
I like your idea on emails that you've actually replied too - will consider that for a future release!