Siri goes away and thinks for dozens of seconds when you ask her something and sometimes never comes back. Even once text has been translated from speech, it sometimes takes 5 seconds to actually send a text.
The functionality is amazing, but the speed of the device is abysmal.
I love everything that it does do, but it is too painful to do it.
I use it for telling the time, tracking my activity, and reading iMessages, but I now know that attempting to do anything else I'd possibly want to (like replying to a message) will take twice as long on the watch as it does to take my phone out of my pocket and do it on there. It even lags while scrolling a notification, sometimes.
Many of my friends are the same, although don't dislike it enough to sell it. They don't plan to upgrade to the next version, however.
Edit: I also wish notifications were smarter. I like that you can choose which apps have their notifications show up on the watch, but it's currently all or nothing. I'd like to be able to choose what type of notifications show up. For example, I want notifications for Twitter DMs on my watch, but not notifications for likes, retweets, etc.