I genuinely loved the second-generation Zune. The aluminum case felt lovely in the hand, it had swipe-based momentum scrolling before that was common (without even a touchscreen!). It had a matte paint on the front that felt almost like velvet, the buttons clicked nicely, the UI was both gorgeous and practical, the desktop software was the same (after the first couple revisions). I was really sad when mine got stolen from my dorm room a couple years after I got it, even though I had a smartphone by that point.

Bonus entry: the GameCube controller. With that huge, luscious analog stick and that huge, luscious A button. And the overall shape fit the hand really nicely too. For any game that didn't make prominent use of the secondary analog stick, I think it was and still is the best game controller out there.