>How bad are timing/side-channel attacks, really?

Bad. They are not something to blow off. They may seem "out there", but they are actually leveraged somewhat frequently. They are particularly dangerous in certain shared-hardware environments (e.g. VPS services like DigitalOceal, AWS, etc), and against physical devices that are supposed to resist data extraction (like smart cards).

When you're talking about something used to secure as much sensitive data for as many people as TLS/SSL, timing and side channel attacks are very important. Any little flaw can lead to information exposure such that people are put into danger.

Extremely bad and practical : http://cr.yp.to/antiforgery/cachetiming-20050414.pdf