This is astoundingly difficult to research.
This article from Timex claims the USA total sales market for watches, more or less pre-cellphone, was 1.2 billion, because timex had $400M revenue and 1/3 of the USA market.
http://www.answers.com/topic/timex-corporation
This article claims the 2013 sales for Rolex were $4.5B and they had ten percent of the world market for mfgrs, and the USA is ten percent of the world market for sales, so a comparable USA total sales market is perhaps "four billion" to one sig fig or less.
http://www.statisticbrain.com/wrist-watch-industry-statistic...
So the sales in the USA have monetarily in nominal dollars roughly tripled. This is probably flat when corrected for inflation. On one hand minimum wage has gone from $3.80 to $7.25 which is barely doubling, then again the standard of living has inploded since then because virtually all expenses have increased at a far faster rate. Still to less than one sig fig I'd say the market is vaguely flat when adjusted for inflation.
Then again I read this comment and went to the deli to pick up a breakfast and my banana, strawberry, and tea observations of people in line ahead and behind me, and the employees, and customers at tables, were 17 people, one rather gaudy gold mechanical watch, one cheap digital watch, three bracelets (one looking like a medical alert bracelet). Two women furiously texting away (perhaps to each other). One woman listening to music on her iphone with the iconic white earbuds. One guy talking on unidentified smartphone.
My guess, looking at sales figures, is the market has dried up for anyone not willing to spend $2000+ on a watch, has exploded as conspicuous consumption for $2K+. The Chinese only made 600M disposable watches for the 7.1 billion of the world population who aren't 1%ers, and I suppose thats believable, if 99%er watches are value engineered to only last a year or two for $10 at walmart, I'd estimate I'd only see a couple in my trip to the deli, and thats what I saw.
If they could sell to 10% of the smartphone population, which admittedly sounds optimistic for a wristwatch with a 3 hour battery life, then their theoretical sales could be severely market disrupting. I'm not seeing it happen.
To some extent wristwatches are another cultural trap. Only about 1 in 40 people watch "Survivor" despite it being freely aired, yet its supposedly our defining influential cultural entertainment. So wristwatches could very well "define our culture", at least as we currently define culture, even if only low single digits of the population purchase them. A play on words due to multiple definitions is "don't have to be popular, to be popular". Or rephrased almost noone needs to wear wristwatches for us to describe ourselves as "everyone" wears wristwatches.