The hardware and sensors is the big problem. Currently, the best smartwatch specs I have seen are the Basis B1. It claims it can monitor your heart rate, track your sleep, etc. But that is all a lie. Sure, it can track your heart rate at rest, but during intensive exercise there is no comparison with something like a Polar chest strap. At the same time the claim that the B1 can monitor your sleep phase based on your heart rate is a bit far fetched as well. From what I've read the heart rate does not change by more than 1-2 bpm between REM and deep sleep, which is within the margin of error of the measurements, so the data out will be all wrong. As for the non-fitness functions of a smart watch, I don't see a huge value in having my wrist buzz every time a random ticket on GitHub gets updated or NewRelic sends me a performance metrics summary. Even text messages are much easier to process on the phone.
I want to believe in the smartwatch idea. I want it to be a gadget that somehow improves my daily life. So far, aside from things like the Garmin Forerunner, I haven't see anything that would remotely come close to improving anything.