Strongly recommend The Great Beauty from the same director (Paolo Sorrentino), IMO his greatest work by far.
343 karma · joined December 14, 2020
What feedback, and what program, are you referring to?
Or I can pass each row through an LLM and get structured clean output out for a couple of dollars per month. Sure, it doesn't work 100%, but I don't need that, and neither could the human-written code do it.
Effectively, LLM resulted in one less develper hired on our team.
> Ofcourse embeddings prefixed will be generally closer.
and then later:
> When I need to use embeddings for temperature or scientific literature purposes, I might put "Temperature:" in front of them, and "Burning"/"Freezing" will be further apart.
is awesome. Have a good day!
> Firstly, you can choose what you embed the word with, such as "Article Topic:" or "Temperature:" to adjust the output of the embedding and results of cosine similarity to be relevant for your use case
As far as LLM-based embeddings go, unless you train the model for this type of format, this is not true at all. In fact, the opposite is true - adding such qualifiers before your text only increases the similarity, as those two texts are, in fact, more similar after such additions. I am aware that instruct-embedding models work, but their performance and flexibility is, in my experience, very limited
As for the rest of your post, I really don't see why you are trying to convince me that LLM-based embeddings have so much more to them than previous models. I am very well aware of this - my work revolves around such new models. I simply corrected a common misconception that you gave, and I don't really care if you "really think that" or if you know what the truth is but just wrote it as an off-hand remark.
It's a common mistake people make, thinking that words that have the opposite meaning will have opposite embeddings. Instead, words with opposite meanings have a lot in common, e.g. both "burning" and "freezing" are related to temperature, physics, they're both english words, they're both words that can be a verb, a noun and an adjective (not that many such words), they're both spelled correctly, etc. All these features end up being a part of the embedding.
Saying that software is infrastructure only supports the renting model.
Instead, using some random, messy, scattered-with-spam site would be a much more realistic test environment.
So if this is correct, then the whole point of my original post is completely wrong. I'll have to look into this further.
If an article about the lack of dairy in east Asian cuisine never mentioned the high frequency of lactose intolerance in those regions, it would be equally misleading, even if there were many other factors resulting in this lack of dairy (primarily different agricultural practices).
> Coriander leaves fell even further out of fashion than the seeds because their distinct flavor clashed with the trendy imported ingredients of the time, such as rosewater. In 1544, physician and botanist Pietro Andrea Mattioli described the leaves as smelling like bed bugs or stink bugs, a comparison echoed by later authors.
I read this as "Cilantro fell out of fashion due to it's incompatibility with other popular flavors at a time, and since people love following recent trends, they started describing it in a negative way". In other words, people saw it as "stink-bug-like" because the fashion changed. This happens a lot with food (e.g. Jell-O texture now being repulsive to a lot of people) but in this case, it has nothing to do with fashion, because to a lot of people cilantro does actually have such taste, and this shift in popularity is much better explained by the shift in genetics.
Incredible that neither this, nor the other linked article, even mention the fact that this stink-bug/soap taste is genetically predetermined for some people [1], so above quote can by no means be taken as an argument that cilantro "became unfashionable" so authors started describing it negatively.
Instead, the real story is probably much more interesting, as the changes in cilantro popularity could probably be connected to waves of migrations and general genetic pool changes in a particular place.
For example, imagine that one paper is about how author X was French, in early 19th-c.and how they were one of the first ones to write about topic T. Another paper is about how author Y was inspired by the early 19th-c. French writers writing about T. However, this second article does not mention X at all. Asking about "who were the main influences on X" would not give you the second article.
Of course, I could run "multiple-hop" RAG-like process, where the model keeps asking questions itself and so on in a loop, but this becomes extremely clumsy, and the models (even GPT-4) tend to get out of hand. It is also extremely slow, of course.
For example, giving the model a bunch of text (academic papers and such) about 19th century writers, then asking things like "Who were the main influences on writer X"?
Obviously simple RAG-like approaches don't work, as such information is rarely available in the text as-is, and needs to be "extrapolated" to some extent. Long context models might work (just dumping everything into the prompt), but are way too expensive for my needs.