Hacker News Reader

jayalammaron Ask HN: Recommendation for a SWE looking to get up to speed with latest on AI

We actually just wrote a book with your profile in mind -- especially if by "AI" you're especially interested in LLMs and if you're a visual learner. It's called Hands-On Large Language Models and it contains 300 original figures explaining the main couple hundred intuitions and applications for these models. You can also read it online on the O'Reilly platform. I find that after acquiring the main intuitions, people find it much easier to move on to code implementations or papers.

jayalammaron Show HN: Llama2 Embeddings FastAPI Server

This is my sense as well. Text generation LLMs haven't been the best source of embeddings for other downstream use cases. If you're optimizing for token embeddings (e.g., for NER, span detection, or token classification tasks), then a token training objective is important. If you need text-level embeddings (e.g., for semantic search or text classification), then that training objective is required (e.g., what Sentence BERT did to optimize BERT embeddings for semantic search).

That's a great list of existing embeddings models (in addition the SentenceBERT models https://www.sbert.net/docs/pretrained_models.html).

jayalammaron Aya: An open science project to build open multilingual models and datasets

Contribution page: https://sites.google.com/cohere.com/aya-en/home

jayalammaron Show HN: Visual intuitive explanations of LLM concepts (LLM University)

That's beautiful! Hope you're getting to do more of these!

jayalammaron Show HN: Visual intuitive explanations of LLM concepts (LLM University)

Additional ones that come to mind now are:

Transformer Feed-Forward Layers Are Key-Value Memories https://arxiv.org/abs/2012.14913

The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention https://arxiv.org/abs/2202.05798

https://github.com/neelnanda-io/TransformerLens

jayalammaron Show HN: Visual intuitive explanations of LLM concepts (LLM University)

Thank you so much (and others for your kind messages). Glad you found them useful! Writing is the best way for me to learn, I find.

jayalammaron Show HN: Visual intuitive explanations of LLM concepts (LLM University)

This is a field I find fascinating. It's generally the research field of Machine Learning Interpretability. The BlackboxNLP workshop is one of the main places for investigating this and is a very popular academic workshop https://blackboxnlp.github.io/

One of the most interesting presentations in the last session of the workshop is this talk by David Bau titled "Direct Model Editing and Mechanistic Interpretability". David and his team locate exact information in the model, and edit it. So for example they edit the location of the Eiffel Tower to be in Rome. So whenever the model generates anything involving location (e.g., the view from the top of the tower), it actually describes Rome

Talk: https://www.youtube.com/watch?v=I1ELSZNFeHc

Paper: https://rome.baulab.info/

Follow-up work: https://memit.baulab.info/

There is also work on "Probing" the representation vectors inside the model and investigating what information is encoded at the various layers. One early Transformer Explainability paper (BERT Rediscovers the Classical NLP Pipeline https://arxiv.org/abs/1905.05950) found that "the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way: POS tagging, parsing, NER, semantic roles, then coreference". Meaning that the representations in the earlier layers encode things like whether a token is a verb or noun, and later layers encode other, higher-level information. I've made an intro to these probing methods here: https://www.youtube.com/watch?v=HJn-OTNLnoE

A lot of applied work doesn't require interpretability and explainability at the moment, but I suspect the interest will continue to increase.

jayalammaron Show HN: Visual intuitive explanations of LLM concepts (LLM University)

The goal is to make the materials as accessible as possible. So we're definitely not limited to the structure of a typical university course and are happy to iterate on it.

I appreciate you elaborating on your feedback. Thank you.

jayalammaron Show HN: Visual intuitive explanations of LLM concepts (LLM University)

The landing page is technically the course overview. I'd love to hear what you think would've made it more engaging for you. We can probably pull up some of the visuals to it as a preview. Let me see what we can do on that front.

jayalammaron Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

I'm the author of https://jalammar.github.io/illustrated-transformer/ and have spent years since introducing people to Transformers and thinking of how best to communicate those concepts. I've found that different people need different kinds of introductions, and the thread here includes some often cited resources including:

https://peterbloem.nl/blog/transformers

https://e2eml.school/transformers.html

I would also add Luis Serrano's article here: https://txt.cohere.com/what-are-transformer-models/ (HN discussion: https://news.ycombinator.com/item?id=35576918).

Looking back at The Illustrated Transformer, when I introduce people to the topic now, I find I can hide some complexity by omitting the encoder-decoder architecture and focusing only on one. Decoders are great because now a lot of people come to Transformers having heard of GPT models (which are decoder only). So for me, my canonical intro to Transformers now only touches on a decoder model. You can see this narrative here: https://www.youtube.com/watch?v=MQnJZuBGmSQ

jayalammaron Embedding Archives: Millions of Wikipedia Article Embeddings in Many Languages

How would you add that data? As new columns you mean? Or add the paragraph headings to the text of the paragraphs before embedding them?

jayalammaron Embedding Archives: Millions of Wikipedia Article Embeddings in Many Languages

There's a lot you can do with the vectors themselves without needing to embed any more text (e.g., clustering, exploration, visualization after dimensionality reduction...etc). Here's a previous embeddings exploration of top HN posts: https://txt.cohere.com/combing-for-insight-in-10-000-hacker-... A lot of that code can be used here as well.

If you want to query for a search term, you can use a trial API key which is free to use for prototyping. The embedding model itself is not open source, though. [co-author of the post here]

jayalammaron Show HN: AI Playground by Vercel Labs

Cohere actually trains its own models and they are not based on models from other providers [I work at Cohere].

Your prompt suggestion is a good one for LLMs as a whole. Any information added to the context informs the model and nudges it towards the expected answer format.

jayalammaron Show HN: AI Playground by Vercel Labs

For Cohere, make sure you're using Command-Xlarge-Nightly.

Otherwise, you may be prompting a Base LLM expecting the behavior of a different kind of LLM (an instruction-tuned chat model).

Cohere's Command model builds on top of the base model, giving it the capability to follow instructions and user commands.

jayalammaron Is Generative AI the future or the present?

Hi. Author here. This is the first in a series I've been writing for a while to help orient people about useful perspectives to have in catching up to all that's happening in AI/ML. It's based on what I've seen in the industry and the nlp company I work with (Cohere, where the post is hosted).

This article has the first four points (out of thirteen) that are useful to keep in mind. They are:

1- Recent AI developments are awe-inspiring and promise to change the world. But when?

2- Make a distinction between impressive cherry-picked demos, and reliable use cases that are ready for the marketplace

3- Think of models as components of intelligent systems, not minds

4- Generative AI alone is only the tip of the iceberg

---

The article goes into each one with more detail. Welcoming all feedback and perspectives as we collectively figure out this new frontier.

jayalammaron Large Transformer Model Inference Optimization

They're trained and focused on language data, actually, not code specifically. There are both generation models and multilingual text embedding models (100+ languages, single model).

jayalammaron Large Transformer Model Inference Optimization

We train and serve large models at cohere.ai. We've shared some optimization techniques here: https://txt.cohere.ai/running-large-language-models-in-produ...

jayalammaron The Illustrated Stable Diffusion

Agreed. "Stable Diffusion with Diffusers" and "The Annotated Diffusion Model" were excellent and are linked in the article. The code in Diffusers was also a good reference.

jayalammaron The Illustrated Stable Diffusion

I updated the post to say "multi-dimensional array".

In a context like this, we use tensor because it allows for any number of dimensions (while vector/ array is only one, matrix is two). When you get into ML libraries, both popular packages PyTorch and TensorFlow use the "tensor" terminology.

It's a good point. Hope it's clearer for devs with "array" terminology.

jayalammaron The Illustrated Stable Diffusion

Two diffusion processes are involved:

1- Forward Diffusion (adding noise, and training the Unet to predict how much noise is added in each step)

2- Generating the image by denoising. This doesn't predict the final image, each step only predicts a small slice of noise (the removal of which leads to images similar to what the model encountered in step 1).

So it is indeed an iterative processes in that way, each step taking one step towards the final image.

jayalammaron Show HN: Analyzing top HN posts with language models

Thanks. I just added a disclosure to the comment (can't edit the parent anymore). The full embeddings are freely provided here without the need to use the service.

jayalammaron Show HN: Analyzing top HN posts with language models

Disclosure: These were made by Cohere's embeddings, a company where I work. The process should work on text embeddings from other sources.

jayalammaron Show HN: Analyzing top HN posts with language models

I think that's part of the expectations out of "ask HN". I don't know that the same effect happens outside of Ask HN.

jayalammaron Show HN: Analyzing top HN posts with language models

TIL louvain clustering! I see it used for graphs. Can also be used for vectors/points?

Thank you!

jayalammaron Show HN: Analyzing top HN posts with language models

What do you mean by naturally biased? That people seem to favor them?

jayalammaron Show HN: Analyzing top HN posts with language models

So it really depends on what you use for clustering. In this case, I'm clustering by the original embeddings so the UMAP results are different. I've also seen:

1- Clustering by UMAP. Here the plot would show clean separation of topics. But the clustering algorithm would be working on highly compressed data (from the 1024 dimensions of the embedding down to the 2 of UMAP).

2- BERTopic's approach of doing UMAP down to 5 dimensions, using this dimensionality for clustering, then UMAP again from 5 to 2. Which is an interesting approach.

I've heard people having good results with all three. It's kinda hard to objectively compare, but my leaning was to give the clustering algorithm the representation containing the most information about the text.

jayalammaron Show HN: Analyzing top HN posts with language models

[1] https://txt.cohere.ai/combing-for-insight-in-10-000-hacker-n...

[2] https://assets.cohere.ai/blog/text-clustering/askhn_cluster_...

[3] https://assets.cohere.ai/blog/text-clustering/askhn_cluster_...

[4] https://assets.cohere.ai/blog/text-clustering/askhn_cluster_...

[5] https://assets.cohere.ai/blog/text-clustering/askhn_cluster_...

[6] https://assets.cohere.ai/blog/text-clustering/hn10k_clustere...

[7] https://assets.cohere.ai/blog/text-clustering/askhn-3k.html

[8] https://storage.googleapis.com/cohere-assets/blog/text-clust...

[9] https://storage.googleapis.com/cohere-assets/blog/text-clust...

[10] https://colab.research.google.com/github/cohere-ai/notebooks...

jayalammaron DALL-E 2 generates images of Kermit The Frog in various films