I think LLMs are cool and that research should be focused on sample efficiency, building public copyright free datasets, and on recognizing and moving beyond the limits of LLMs. These systems are often very half-baked, and I am more interested in seeing research that moves to new ways of machine thinking than I am in dumping more and more GPUs in to this promising but incomplete technology.

As one example, Yann LeCun's vision of a system called JEPA [1] is interesting to me. It may not be the solution we need, but this type of thinking - taking what we have learned and exploring new architectures that may have even better real world performance - is what interests me.

[1] https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-jo...