I think the core insight OP may be looking for is that your dictionary is just an illusion - that concepts being related to other concepts to various degree is all that there is. The meaning of a concept is defined entirely by other concepts that are close to it in something like a latent space of a language model.

Of course humans get to also connect concepts with inputs from other senses, such as sight, touch, smell or sound. This provides some grounding. It is important for learning to communicate (and to have something to communicate about), and was important for humans when first developing languages - but they're not strictly necessary to learn the meanings. All this empirical grounding is already implicitly encoded in human communication, so it should be possible for an LLM to actually understand what e.g. "green" means, despite having never seen color. Case in point: blind people are able to do this, so the information is there.