What am I missing -- aside from the marketing? Is there something architecturally different or what? Looks like regular autoregressive sequence transformer to me.
What am I missing -- aside from the marketing? Is there something architecturally different or what? Looks like regular autoregressive sequence transformer to me.
I like this idea because that's how humans think. We mentally formulate a whole sentence, then say it. People who don't do this speak in run-ons and word salad.
The process is a formulation of precise ideas (complex at some level and verified to some degree, hopefully), then translated into sentences for output (not necessarily in these two steps, but through iterations).
This project tries to use sentences as formalizations of ideas - an interesting way enabled by availability of tools, allowing good features like transparency.
An embedding space engine accepting sentences (SONAR) fit in so that the tokens of this architecture are complex sets of the tokens of past architectures.