Hacker News Reader

Isn't this more of A semantic understanding of how language maps onto geometry rather than understanding of actual space and geometry?

I sometimes feel like this may not work without language helping it along

With a multimodal llm watching a stream of these objects, should be able answer the question -

"where are my keys?"

It should be able to respond -

"Here on the dining room table" with an astar path drawn to them in the headset.

It's a stream of tokens that you can almost just reverse search on to find the last instance of "keys" in it (thats a simplification but...).

So, potentially how humans turn language into a rendered version of reality? :)