Actually, on a long enough timescale, he might very well have been right. Of course users will still have many devices with processing in them, but you could reasonably make the case that they won't be "computers" as the word is currently used.

In fact, they allude to this later in the same paragraph!:

Mr McCarthy always argued that the future lay in simple terminals hooked up remotely to a powerful mainframe which would both store and process data: a notion vindicated only recently, as cloud computing has spread

So why was he wrong?