any model, MoE or not, can be sharded over multiple devices (separate GPUs on a single machine or separate machines via network), yeah. but your question was "Do several LLMs communicate with each other before providing a solution?", and in this context (Deepseek's thinking), the answer is definitely "no".

models can communicate with one another via tool calling, sure, and there are hypothetical workflows where agents delegate tasks to other agents (with inference being done on different models), but that simply isn't the case here.