Hacker News Reader

Seems like it would lock in a user to cohere proprietary model in order to embed new search terms to the same space?

Would love to see similar but embedded with a more open representation model or even sent2vec

There's a lot you can do with the vectors themselves without needing to embed any more text (e.g., clustering, exploration, visualization after dimensionality reduction...etc). Here's a previous embeddings exploration of top HN posts: https://txt.cohere.com/combing-for-insight-in-10-000-hacker-... A lot of that code can be used here as well.

If you want to query for a search term, you can use a trial API key which is free to use for prototyping. The embedding model itself is not open source, though. [co-author of the post here]

If that's the intent, IMO the release dataset should have more metadata (e.g. paragraph heading, article taxonomy)

How would you add that data? As new columns you mean? Or add the paragraph headings to the text of the paragraphs before embedding them?

New columns.

For the headings, I mean the Wikipedia section headings (which isn't always a paragraphs, my mistake).

In both cases the data can be used like to classify/visualize Show HNs in your linked post.

Unfortunately yes, as the corresponding model is not also open-source.

Reply on news.ycombinator.com