I'd be quite interested to hear more about this, if you have more ideas and care to elaborate.
So there are a number of forces at work in the market, one is the decreasing cost of compute and storage, another the increasing availability of wide bandwidth networks. A seminal change in systems architecture occurred when PCI express, a serial interconnect, became the dominant way to connect "computers" to the their peripherals. Now sprinkle in the 'advantages' of keeping things proprietary, with the 'pressure' of making them widely used, and those bread crumbs lead to a future where systems look more like a collection of proprietary nodes attached to each other by high speed networking than the systems we use today.
From that speculation emerges the question, "What does an OS look like in that world?" and so far the only thing I can think of that seems to work is a bunch of 'network apis.' Lets say you build your "computer" with pieces that are connected by 10Gbit serial pipes and a full bandwidth cross bar switch. Your 'motherboard' is the switch, the pieces are each completely 'proprietary' to the manufacturer that built them but the API is standard. This is not a big change in storage (SAS, FC, and SCSI before it have always been something of a network architecture) its becoming less uncommon for video, and why not compute and memory? Memory management is then a negotiation between the compute unit and the memory unit over access rights to a chunk of memory.
The Infiniband folks touched on a lot of these things 10 years ago but they were a bit early I think and they aimed pretty high. But they have some really cool basic technology and show how such a system could be built. Trivially so if there were a 'free API' movement akin to the FOSS movement.
This removes the constraint of size as well. And if you're cognizant of the network API issues amongst your various pieces you get to the point where your 'OS' is effectively an emergent property of a bunch of nodes of various levels of specialization co-operating to complete tasks. Computers that can be desktop, or warehouse sized, and run the same sorts of 'programs'.
A Plan 9 system comprises file servers, CPU servers and terminals. (...) Since CPU servers and terminals use the same kernel, users may choose to run programs locally on their terminals or remotely on CPU servers. The organization of Plan 9 hides the details of system connectivity allowing both users and administrators to configure their environment to be as distributed or centralized as they wish. Simple commands support the construction of a locally represented name space spanning many machines and networks.
The place where the world is catching up is that we have very fast networks in very dense compute. In the case of a cell phone you see a compute node which is a node in a web of nodes which are conspiring to provide a user experience. At some point that box under the table might have X units of compute, Y units of IO, and Z units of storage. It might be a spine which you can load up with different colored blocks to get the combination of points needed to activate a capability at an acceptable latency. If you can imagine a role playing game where your 'computer' can do certain things based on where you invested its 'skill points' that is a flavor of what I think will happen. The computers that do shipping, or store sales will have skill points in transactions, the computers that simulate explosions will have skill points in flops. People will argue whether or not the brick from Intel or the Brick from AMD/ARM really deserves a rating of 8 skill points in CRYPTO or not.
[1] I didn't get to work with Rob when I was at Google although I did hear him speak once and he didn't seem particularly bitter, so I don't consider him a good exemplar of the problem. Many brilliant people I've met over the years however have been lost to productive work because their bitterness at not being accepted early only has clouded their ability to enjoy the success their vision has seen since they espoused it.