Like a screen big enough to comfortably read PDFs. 10.3" on Kobo Elipsa vs 7" on Kindle. IMHO that makes the Kindle only really useful for fiction.
There's also Pocket support for reading offline web articles.
Overdrive support so I can borrow books from my local library for free.
Free Dropbox syncing without having to email everything when I buy a book from someone that isn't Amazon e.g PragProg, Packt, etc. (Dropbox does support email though if you do want that).
In theory I could also use the pen and take written notes on my Elipsa too but it's not a feature I was looking for.
The other stuff you mentioned, like Kindle having a large purchasable library, is also true for Kobo right from the homescreen. They're also priced similarly if you match specs and available for same day delivery from Argos in the UK.
Finally, I know I could hack my last Kindle but Kobo make that really easy out of the box. Installing Koreader, Plato or anything else is simple even if it's not required for everyday use.
So for me it's the other way around. Amazon offers nothing I want that Kobo doesn't have.
Different people have different usecases but I don't think I'll go back to a Kindle now I've switched. Kobos are very easy to live with.