I had not heard that Thomas Jefferson was particularly violent in his slave holding vs other slave holders. A quick google search isn't informative either, do you have any source for that claim?
Henry Wiencek wrote a book about it. That particular work received intense criticism from other respected Jefferson scholars. It's a debate worth reading.