Finding the Words to Say: Hidden State Visualizations for Language Models

Hello HN, author here. Language models are absolutely fascinating tools. I believe it would pay for software engineers to have a sense of their capabilities and how they function. The article showcases a few views to expose the inner workings of the model, but also simple UI for interacting with a language model to get a sense for how they work and generate words.

If you prefer video, I have also recently released a video [1] with PyData to provide an intro to language models and their applications and how we're trying to make Transformer-based ones more transparent with Ecco[2]. Contributors are welcome!

[1] https://www.youtube.com/watch?v=rHrItfNeuh0

[2] https://www.eccox.io/ and https://github.com/jalammar/ecco

Thanks mods for merging submissions. Happy to get feedback , thoughts, or questions.

Does some text generating tool exist which takes a badly written piece of prose (perhaps even in the form of bullet-points), and produces a high(er) quality text? (E.g., richer in vocabulary by using synonyms, removing repetition, rephrasing sentences so they don't all have the same general form, et cetera.)