Hacker News Reader

jdspiralon Meaning Machine – Visualize how LLMs break down and simulate meaning

I moved the app, it’s now tokenizer-machine.streamlit.app.

jdspiralon Meaning Machine – Visualize how LLMs break down and simulate meaning

So I've taken the feedback and realized that I was misleading on the name and title. I'm updating the project accordingly.

https://tokenizer-machine.streamlit.app/

jdspiralon Meaning Machine – Visualize how LLMs break down and simulate meaning

Thanks! Yes — that’s on the roadmap, along with some other cool visualizations I’m working on. Machine translation is definitely something I want to work on: showing how models align meaning across languages using shared embeddings and attention patterns. I’d love to make that interactive too.

jdspiralon Meaning Machine – Visualize how LLMs break down and simulate meaning

Yes, tokenization and embeddings are exactly how LLMs process input—they break text into tokens and map them to vectors. POS tags and SVOs aren't part of the model pipeline but help visualize structures the models learn implicitly.

jdspiralon Meaning Machine – Visualize how LLMs break down and simulate meaning

I built a tool called Meaning Machine to let you see how language models "read" your words.

It walks through the core stages — tokenization, POS tagging, dependency parsing, embeddings — and visualizes how meaning gets fragmented and simulated along the way.

Built with Streamlit, spaCy, BERT, and Plotly. It’s fast, interactive, and aimed at anyone curious about how LLMs turn your sentence into structured data.

Would love thoughts and feedback from the HN crowd — especially devs, linguists, or anyone working with or thinking about NLP systems.

GitHub: https://github.com/jdspiral/meaning-machine Live Demo: https://meaning-machine.streamlit.app