I'm not sure the initial comment lends itself to that reading, but it isn't clear it lends itself to my initial response either, so I'll switch over to your more charitable reading. Thanks!
8,760 karma · joined June 28, 2016
I'm not sure 2022 looks quite as good as you're suggesting, but it's clear the pandemic had a profound impact, certainly.
https://www.theguardian.com/business/2022/nov/19/britain-eco...
I still pay nothing in gas taxes, the primary funding for road maintenance. That doesn't seem quite right. I'd be happy to pay an annual fee based on miles driven and weight class.
Siri should be better! It lost features post-acquisition by Apple, and it seems like user privacy is why.
Home automation is arguable. If you consider a single point of failure on a server somewhere to be bad, Apple's solution is pretty great. Their commitment to zigging where others zagged put them behind, since hardware vendors didn't want to put in powerful (expensive) enough chips to handle the cryptography, but while other companies go out of business, or transmit images and video to external parties, Apple's works reliably and securely.
Still, as with most of my complaints about Apple, it's a trade-off between privacy and functionality, and Apple will seemingly always choose privacy over functionality, even as Google consistently chooses functionality over privacy.
[0] Yes, there are examples of edge cases that suggest a less-than-perfect record. Contrast that with their competitors, for which invading privacy is foundational to the business model.
I believe literally none of those things, and have explicitly stated the opposite of at least one of them on this page.
At this point I think an LLM would do a better job of responding to my comments, based on context and syntax alone.
NPR, as it happens, has run many stories that seemed aligned with the right or the left, depending on the subject. They're generally pro-military and pro-wealth, while being against racism and censorship, so it's not as if they're either all left-wing or all right-wing.
If you can communicate the truth with kindness and be heard, great. If not, move on.
In my experience, unkindness isn't any more effective than kindness in effecting change; it usually prompts defensiveness.
It turns out to be important to know which is the official Eli Lilly account, and at some point it was deemed important to know who was a news reporter, rather than someone attempt to trade on the reputation of a newspaper to support their own opinions.
Verifying everyone is definitely missing the point of verification.
Not everything that catches fire is destined to leave nothing but ashes in its wake.
I have. You refuse to accept it, but I definitely have given an answer that involves tokenization and association, the things we already know LLMs use to construct their responses.
Here's a sample question from the GRE Verbal Reasoning test:
> Upon visiting the Middle East in 1850, Gustave Flaubert was so [blank] belly dancing that he wrote, in a letter to his mother, that the dancers alone made his trip worthwhile.
> (A) overwhelmed by
> (B) enamored by
> (C) taken aback by
> (D) beseeched by
> (E) flustered by
Whether that specific question was in the training corpus or not, there are enough words in the sentence to suggest a positive association, including, significantly, "worthwhile." That alone possibly serves to narrow the answers down to A or B, with a preference for B, because it's more likely that "worthwhile" and "letter to x mother" are associated with "enamored" in general English-language text.
Look, the whole point of these models is that its not easy or even possible to trace the path any given input takes on its way to output, but we know the principles used in development, so I think it's rather more of a burden to explain how the clearly-explained principles in LLMs result in something other than the obvious. The fact that the results are so overwhelming that we become enamored by them, well, I'm taken aback by the seeming accuracy of some of the responses, but I beseech you to remember the other responses in which these LLMs are dramatically off-base, as if flustered--if LLMs could ever be flustered.
This is the mistake people keep making, literally anthropomorphizing LLMs! Because humans would find it easier to just learn the meanings of French words than regurgitate from a gigantic table of memoized tokens, these results must reflect understanding!
But of course, it's far, far easier for a computer to index a large number of tokens without any understanding at all than it would be for a person. Computers are not people. They have different strengths and weaknesses, and it is a mistake to forget that. (Ironically, one of our human weaknesses is also a strength: our pattern-matching skills are so strong we sometimes see Jesus in toast, or a face on Mars, when neither are actually there!)
What I described is literally how LLMs works. There is open source code to examine, and what it does is essentially what I said: tokenize words and store multi-word strings and identify their prevalence in training data, where the training data is an incredibly large corpus.
The experiment is about finding one missing word in a sentence, but the same logic is true even for long responses, or as I suggested, conversations.
To over-simplify: humans think in words, computers think in math. But it turns out that if you assigned a number to each word and parse enough of them, you can fake an understanding of words surprisingly well, even though it's still math.
I've given example elsewhere in this thread and on previous ones about the March 14 Chat-GPT giving incontrovertibly false information, even repeatedly in the face of correction, or seeming to correctly identify a popular logic puzzle while breaking the rules of the puzzle three times, then seeming to claim it had followed the rules of the puzzle.
> So Consider that as an existence proof that even if you "train to predict" you can still end up with actual understanding.
As a human, that is often true. Humans have trouble with arbitrary data, and it helps us to connect new data to existing data. In addition, we generally have a curiosity that prompts us to pursue new knowledge and meaning wherever possible, even when it doesn't actually exist. But if you're trying to suggest that because it is possible that humans can develop understanding while trying to only "train to predict," therefore it is probable that computers will develop understanding along the same lines, I suggest that's about three leaps too far.
That really, really, really demonstrates something. I'm not sure what.
We already know LLMs tokenize words, so they have definitely generated and stored such a table. The marvelous step forward from previous word-based solutions is that they store not just single words, not just pairs of words, but larger sequences of words, even up to short sentence length. It makes the table pretty large, as you suggest, but that's exactly we've been seeing, and each generation of GPT is that much larger than the one before.
I don't think that's true at all. To explain my thinking, let's assume that I am an American, and therefore according to popular understanding, completely incapable of learning any language other than English. I despair! But then I think no, wait, I happen to have a gift for numbers. Like, my mental arithmetic stunned teachers from a young age, and I currently hold the world record for reciting digits of pi from memory. So what I will do, I think, is assign a number to every word I read or hear and where possible, visualize where that chain of numbers exists among the decimal digits of pi.
At some point the person would come in and utter some words I couldn't comprehend, but I would think, aha! That was almost 498153217845632754, except it was missing the second seven, which mapped to "contraire." And I'd do that at least 20% of the time, escaping the room with no understanding of French at all.
If I could map between memorized numbers and words quickly enough, I'd even be able to converse in French given suitable trigger words without a shred of understanding.
>> Suppose I have a blurple, a mantiple and a hoje, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the blurple and hoje alone together, and I am not allowed to leave the hoje and mantiple alone together. How can I safely get all three across?
> This is a classic river crossing puzzle. Here's one possible solution:
> 1. Take the blurple across the river, leaving the mantiple and hoje behind.
> 2. Leave the blurple on the other side of the river and return to the original side.
> 3. Take the hoje across the river and leave it there with the blurple.
> 4. Take the mantiple across the river and leave it with the hoje.
> 5. Return to the original side of the river and retrieve the blurple.
> 6. Finally, take the blurple across the river to join the mantiple and hoje on the other side.
> This solution ensures that no two items that cannot be left alone together are ever left alone together.
Of course, my second stipulation was "I am not allowed to leave the hoje and mantiple alone together" and yet step 1 says "leaving the mantiple and hoje behind"
So no, despite using the words "classic river crossing puzzle," it clearly expressed no understanding of the puzzle.
Hey, that was 15 words, and I had them all in my head at once before I started typing! I was able to form a sentence of more than seven words all by myself!
You seem to be misapplying the concept of working memory, where seven (plus or minus two) is about arbitrary and unrelated words or numbers, not "items." In fact, it's generally seven for numbers, but lower for words. And again, that's unrelated words, like "person woman man camera tv," but probably even more unrelated.
The original question was, given that there were X comments which could be replied to on this page, how and why did the poster decide to respond to the one to which they responded, rather than either "all" (like ChatGPT would) or "none?"
So you saw the parent comment about hype and clicked into this comment box without a thought of what you might say other than "When?" Really?
Did you know whether you agreed or disagreed with the parent comment, or did it take until the word "reductive" for you to firm that up?
If you're saying that the part of your brain that controls your fingers is doing so one word at a time, then sure, obviously that's true. But if you're making the opposite claim, that LLMs like Chat-GPT are "deciding" what to say and only then working out the finer details of how to express that decision, that's a claim I'd like to see explicitly.
If you are suggesting that your approach to intelligence is parallel to that of an LLM, then either you're arguing for something far more radical than most as an emergent property of LLMs, or you're suggesting that you only form opinions one word at a time as you type, which I think would make you, well, I won't insult you, but I think you're probably doing a little more than that.
The complete set of "matches" they got were interesting, even while the number was very low.