elexhobby

For another possibility, see https://arxiv.org/abs/2305.15717. The new models may not actually be better - the evaluation may be broken.

GPT-4 is powerful over a diverse set of tasks. They use it to build a model which is better for a narrow sub-task. Pretty sure the model is sub-optimal to GPT-4 for everything else.

Not sure if this is obvious. But its incorrect to ditch on GPT-4. The paper uses self-instruct on GPT-4 to generate the training data on which it is fine-tuned. This paper would not exist without GPT-4. Although they claim GPT-4 can be replaced by any LLM, I'm sure the results would be nowhere as good, and so they stuck with GPT-4.

This! The best resource I've found to explain transformers, that made them clear to me. I wish all deep learning papers were written like this, using pseudocode.

FWIW I've been told similar by Costco too, so Amazon isn't unique in this respect. My guess is that it adds some friction to the process. Refunding money is a purely digital activity, while refunding requires to you go and ship it, wait for the new item to arrive, etc, so you won't do it unless the savings are worth the time investment.

Furthermore, follow https://twitter.com/_brohrer_/status/1425770502321283073

"When you have a problem, build two solutions - a deep Bayesian transformer running on multicloud Kubernetes and a SQL query built on a stack of egregiously oversimplifying assumptions. Put one on your resume, the other in production. Everyone goes home happy."

any twitch streams you recommend?

Wow. I wish everything was explained so clearly. I understood everything in that post except this paragraph.

""" Languages like Erlang must implement tail call optimizations, since persisted data is stored as "loop variables" in infinite loops. This happens when we write code like this:

     loop(Data) ->
          ....
 ...
 loop(newData).

When I see code like this I mentally "see" the last call as a "jump" to the start of the code, rather than a recursive call to loop. """

What is the loop(Data), loop(newData) doing? Would be great if someone could elaborate on this point.

Quote: "The election comes down to a few swing states, such as Pennsylvania, Wisconsin, and Michigan. Crucially, right now all three of those states have Democratic governors and Republican-controlled legislatures … and there’s no clear law about which of the two (the governor or the legislature) gets to certify election results and send them to Congress! So suppose Trump has a slight edge on election night, Fox News calls the race for him, but then an avalanche of absentee or provisional ballots shift things in Biden’s favor over the following week. Can you imagine Trump or his supporters accepting the latter?"

Great post, thanks!

Is there a reason why the training is started off with two separate matrices - the embedding and the context matrix? If the context matrix is anyway discarded at the end, why not start and work with only the embedding matrix?