Discussion on HN
https://news.ycombinator.com/item?id=44576352
201 karma · joined September 8, 2022
https://www.phaidon.com/agenda/art/articles/2014/july/14/on-...
Have there been any state-space models adapted for arbitrary text generation?
Language models like ChatGPT are trained to predict new words based on the previous ones and are excellent for generation, a harder task than translation or classification. I'm doubtful about the adaptability of text models that deal with fixed-sized input/outputs and don't have an architecture that is as natural for generating indefinitely long sequences.
https://urbigenous.net/library/nine_billion_names_of_god.htm...