Cache-to-Cache: Direct Semantic Communication Between LLMs (2025) | Hacker News Reader

Cache-to-Cache: Direct Semantic Communication Between LLMs (2025)

arxiv.org

109 pointsrochansinha22 comments

It's an old paper (from 2025, so, a decade ago in AI years), but the concept is still fascinating. And I'm yet to see it show up in any production models.

If multiple models can use cache representations for this kind of enrichment, the KV cache representations of different models must be somewhat compatible.

What stops us then from going a step further, and producing a model family where all models are "KV aligned", and each model can utilize the KV cache of other models directly?

So, an "expensive" reasoning model can use its full faculties to plan, but "delegate" simple subgoals to a smaller model. That smaller model can access the large model's intent directly, as rich KV cache representations - with no prefill recompute and no associated "handover" latency. Or, likewise, a "cheap" small model can generate a diminished but highly compact KV cache that the "expensive" model can then operate on - for example, for skimming a large file for shallow patterns.

You might like The Universal Weight Subspace Hypothesis: https://arxiv.org/abs/2512.05117

Curious to know if anyone is aware of research trying what parent suggested?

Sounds similar to Ramp's Latent Briefing for multi-agent coordination.

https://x.com/RampLabs/status/2042672773747589588

Kind of reminds me of this more recent paper: https://arxiv.org/html/2602.02459v1 Different use case and implementation, but a similar idea. In this case applied to sharing last state(not the whole KV cache) from big brain model running in the cloud with a smaller/dumber model running on-device in a robot, in a latancy-aware way.

Thanks for the link. This is the kind of "edge AI" adaptation that I expect to see more of in the future.

A "local 8B VLA" that runs at high frequency can be sufficient for things like sensor fusion, plan execution and manipulation dynamics, but insufficient for advanced problem solving and in-depth "exception handling". So being able to ask a larger model for steering cues is going to be valuable for AI robots. And that does introduce latency and interface boundaries.

Not sure if the exact approach described there is the way to go, but I do expect something similar to show up in deployments eventually. If nothing else, having this one described does provide a point of comparison.

So the models will not only be using more and more Neuralese in their CoT (like GPT-6), but different agents will also be able to communicate with each other in Neuralese. It's not looking good for monitorability.

Is Neuralese in no way decodable into a human-interpretable system? Genuine question -- I don't know the answer.

Definitely decodable, that's what's being done now

It's not.

Probably not without a sufficiently powerful LLM from same family, or something equivalent, to act as a translation layer, which includes the risk of the translator lying to you.

This is from like a year ago.

A few months ago I asked why semantic representation rather than text wasn't used, since natural language seems quite a lossy representation for semantic concepts:

https://news.ycombinator.com/item?id=47195212

I wouldn't have thought to use it for LLM-to-LLM communication, though

My guess is that if you go with something other than readable text as the “thought layer”, observability becomes impossible.

Which is not necessarily something the humans training a model would want re/ alignment.

Indeed this is one of the key problems here. If models start doing CoT in "neuralese", or use it when talking with each other, we lose what little observability we have.

I feel like multimodal models that can read images should work differently than they do. My understanding is that multimodal models basically first generate an image embedding and then the model is trained to interpret that embedding, but in the same way that text is lossy, it seems like the embedding would be as well. Why don't multimodal models learn to interpret images themselves without an embedding? Or e.g., by passing some "prompt" to the embedding model?

What does interpreting images mean in practice if you exclude the possibility of feature extraction or any other sort of implicit embedding?

I'm not an ML expert, but I was thinking of a sort of "guided" embedding. E.g., give the image model some prompt for what it's trying to do? I don't understand why multimodal models generate an embedding that doesn't understand what the model is trying to "figure out".

I think this is similar to how Gemma 4 12B is implemented, but even then I don't think the single layer image embedding is "aware" of the context.

You already do give the image model a prompt to tell it what to do. That’s not something the embedding can use independently of how the model is already using it.

In general an embedding doesn’t have intent or awareness in the way you’re looking for. “Embedding” just means one mathematical structure stuffed inside another. So for example the real number line is embedded into the Cartesian plane as each axis- that’s an embedding.

Now in this case specifically, the embeddings in any kind of transformer model encode the meaning of the thing they represent into vectors (which is what the model itself actually operates on). You can train the embedding to be more useful for a particular task at inference time, which already happens.

I think what I'm saying is that I don't understand why the embedding exists. I assume it's some kind of training and inference cost issue? But why can't the Gemma architecture linked above just learn to represent pixels in the LLM model's embedding space directly, rather than having the embedding from 48 x 48 pixel chunks? Or rather, give the embedding model some context to produce the embedding? (Which, as you note, wouldn't really be an embedding anymore, but seems like it would better understand fine detail)

Embeddings are used in language models to convert problems which are about words and meaning (which computers suck at) into problems which are about multiplying matrices, (which computers are really good at).

So embeddings are used in vision models to convert problems which are about the content and meaning of images into problems which are about multiplying matrices. The model doesn’t want to work with pixels (that’s what very basic vision models do, but it tends to be limited to special purpose applications) it wants to work with concepts in the image. That’s what the embedding gives it.

I still don’t really know what you mean about giving the embedding model some context. It embeds whatever you want to embed. So if you want to give it just a jpeg, fine. If you want to embed a jpeg and a json blob with some additional metadata/“context”/whatever, that’s also fine. That’s already how embeddings work.

The idea that I'm proposing basically isn't an embedding (which is context independent) but rather combining the embedding model with the context of the LLM. It sounds like the embedding model here is normally a "vision transformer" that maps image chunks into tokens with positional embeddings for both the position in the context as well as position within the image. Maybe it could be given the ability to consume the context and decide to emit multiple tokens for a single chunk? So for example, let's say that I ask a question "How many blades of grass are in this image" (a hard question for traditional image embeddings in models since the embedding won't contain the information). If the proposed architecture was both aware of the context and able to "decide" to emit multiple tokens for an image, then for the above it could emit tokens that represent the answer to the question posed, instead of just being the embedding of the image. Or you could ask something like "How many green pixels are there" and again I think it would work better under this architecture than it would otherwise.

I'm not sure how practical it is to train that architecture though or whether there would be performance issues.

Seems like fundamentally a cool idea but: KV is not context. if the KV cache gets evicted, you'd have to rerun the translation.

Seems like it could still help but also feels like one of those things where it becomes vastly more complex and difficult to debug.

Reply on news.ycombinator.com