Bing AI can't be trusted
dkb.blog
dkb.blog
And the UI is better IMO.
Result: "Sterling Marlin won the 2023 Daytona 500, driving the No. 4 for Morgan-McClure Motorsports[1]. He led a race-high 105 laps and won his second career race at Daytona International Speedway[1]. The 64th running of the DAYTONA 500 was held on February 19, 2023[2]. Austin Cindric had previously won the DAYTONA 500 in February 5, 2023[3]."
If, based on the training data, the most statistically likely series of words for a given prompt is the correct answer, it will give correct answers. Otherwise it will give incorrect answers. What it can never do is know the difference between the two.
ChatGPT does not work this way. It wasn't trained to produce "statistically likely" output, it was trained for highly rated by humans output.
(That's why original GPT3 is known for constantly ending up in infinite loops.)
My guess is that the fact checking is actually easier, and the models can be smaller since they should not actually store the facts.
You might be underestimating the scope of some task here.
Also 100% accuracy is probably not the real threshold for being useful. There are many low hanging fruits today that could be solved by absolutely tiny error correcting models (e.g. arithmetic and rhyming).
The problem isn't going to be developing the model, it's going to be how to arrive at an uncontroversial source of ground truth for it to draw from.
Meanwhile, people are complaining that the talking dog they got for Christmas is no good because the C++ code it wrote for them has bugs. Give it time.
That said, traditional search engines suffer from the same problem, although with different symptoms. Search engines point to web pages generated by people (or some other way) which may or may not contain accurate information. They base their selection of those pages based on things like strength of authority, keyword similarity, etc... But truth never directly enters the picture (indirectly by tweaking the "trusted" set of pages).
Right, because all they're doing is regurgitating the data they were trained on. It's like the kid in your CS class that copies someone else's coding assignment and just changes all the function names, variable names, and comments but not the actual executable stuff and never even bothers to compile and run it to see if it produces the desired output.
That is not true, it's clearly able to generalize. (If it can do anagrams, it's silly to say it's just regurgitating the instructions for doing anagrams it read about.)
But it doesn't try to verify that what it says might be true before saying it.
example prompt: Imagine I took all the letters in "Wikipedia" and threw them in the air so they fell on the ground randomly. What are some possible arrangements of them?
Similarly, it can almost do arithmetic but apparently forgets to carry digits. That's wrong but it's still generalization!
"If Tim drops a coconut on the tail of a dog, what are two things the dog might do?"
Response: "If Tim drops a coconut on the tail of a dog, two possible things the dog might do are:
1. Yelp or bark loudly in pain, as the impact of the falling coconut could be quite painful.
2. Try to escape or run away, as the dog may associate the sudden impact with danger and feel the need to get away from the source of the pain. Additionally, the dog may feel scared or anxious, which can cause it to run away."
Then I said, "But what if it was a soft coconut that was as light as a feather?"
Response: "If Tim dropped a soft coconut that was as light as a feather, it's possible that the dog wouldn't feel much or any discomfort from the impact. In this scenario, two things the dog might do are:
1. Look around to see what just hit it and possibly sniff at the coconut to investigate.
2. Continue with its previous activity, ignoring the soft impact as if it never happened. The dog may not perceive the coconut as a threat or something worth reacting to, especially if it was light enough to not cause any pain."
I just can't read these responses and think, "Ehh... just a mindless regurgitation as expected from any LLM". These simple prompt responses impress me and I kind of know the technology -- although my experience in RNNs/LSTM is very dated.
Honestly, I'd love to see Zork rewritten with ChatGPT as a parser. No more trying to figure out how write the prompt for how to use the key in the door!! :-)
That was done as AI Dungeon, but there was some consternation due to the combo of charging for it and GPT's predilection for generating wild and possibly illegal sex scenes even when you don't ask it to.
We're well into philosophy at this point, but I'd argue that telling the truth requires knowledge of the truth. Individual statements could be true, false, or neither. But I'd say neither an LLM nor a monkeys-with-typewriters generator could be said to tell the truth, because what they're doing is both orthogonal to truth and lacking in agency. If we stretch to thinking of it as having agency, then I think the correct term would be "bullshitting", aka producing speech "without regard for truth". [1]
That's almost always sourced from a website that didn't actually answer the question I had, so maybe its more of a query optimization issue.
"How did Buffy the Vampire Slayer die?"
"Buffy the Vampire Slayer died in the Season One finale when she threw herself into a portal to save her sister Dawn[1]. This was the first time Buffy had died, occurring on June 2, 1997[2]. She died again in the Season Five finale while fighting off Glory's demons[3]. Other characters such as Anya and Kevin Benedict also died during the series[4][5]."
Okay, so it was a trick question, because Buffy definitely died more than once, but it's conflated the fact that she died in Season 1 with the cause of her Season 5 death. Also, I had to Google Kevin Benedict to remember that he was Cordelia's sometimes boyfriend, and an extremely minor character, which makes me question how that death is more notable than Buffy's mom, or Tara, or Jenny Calendar, etc.
I like that this seems to have been more lexical confusion than how ChatGPT seems to enjoy filling empty spaces with abject lies, but perhaps it's worth exploring what you're asking it that has left it with such a great batting average?
* Misunderstanding one of its citations, it said that use of `ParamSpec` in Python would always raise a warning in Python 3.9
* When asked why some types of paper adhere to my skin if I press my hand against them for a few minutes (particularly glossy paper), it gave two completely different answers depending on how the question was worded, one of which doesn't necessarily make sense.
In my usage of ChatGPT, in areas I'm very knowledgable. I've mostly received answers that were stylistically excellent, creatively plausable and maybe even transcendent. The boilerplate around the answer tends to keep the answers grounded, though.
In areas where I have some experience but not much theoretical knowledge, after multiple exploratory questions, I better understand the topic and feel ok adjusting my behavior appropriately.
I haven't relied on it in areas where I am ignorant or naive e.g. knitting, discriminatory housing policy or the economy in Sudan. Since I have no priors in those areas, I may not feel strongly about the results whether they are profound or hallucinatory or benign.
I also haven't used it for fact checking or discovery.
What sounds more "correct" (i.e. what matches your training data better):
A: "Sorry, I can't answer that because that event has not happened yet."
B: "Team X won with Y points on the Nth of February 2023"
Probably B.
Which is one major problem with these models. They're great at repeating common patterns and updating those patterns with correct info. But not so great if you ask a question that has a common response pattern, but the true answer to your question does not follow that pattern.
Sometimes it comes up with a better, acceptably correct answer after that, sometimes it invents some new nonsense and apologizes again if you point out the contradictions, and often it just repeats the same nonsense in different words.
Generally, for example, it will answer a question about a future dated event with "I am sorry but xxx has not happened yet. As a language model, I do not have the ability to predict future events" so I'm surprised it gets caught on Super Bowl examples which must be closer to its test set than most future questions people come up with
It's also surprisingly good at declining to answer completely novel trick questions like "when did Magellan circumnavigate my living room" or "explain how the combination of bad weather and woolly mammoths defeated Operation Barbarossa during the Last Age" and even explaining why: clearly it's been trained to the extent it categorises things temporally, spots mismatches (and weighs the temporal mismatch as more significant than conceptual overlaps like circumnavigation and cold weather), and even explains why the scenario is impossible. (Though some of its explanations for why things are fictional is a bit suspect: think most cavalry commanders in history would disagrees with the assessment that "Additionally, it is not possible for animals, regardless of their size or strength, to play a role in defeating military invasions or battle"!)
Another silly tech prediction brought to you by the HN hivemind.
I wouldn’t put it past Microsoft to do something stupid like ground gpt3.5 with the top three bing results of the input query. That would explain the poor results perfectly.
These are models. By definition they can't do anything. They can just regurgitate the best sounding series of tokens. They're brilliant at that and LLMs will be a part of intelligence, but it's not anywhere near intelligent on its own. It's like attributing intelligence to a hand.
Models of models and interacting models is a fascinating research topic, but it is nowhere near as capable as LLMs are at generating plausible token sequences.
Toolformer does something really neat where they make the API call during training and compare next word probability of the API result with the generated result. This allows the model learn when to make API calls in a self supervised way.
Someone asked the recaptcha guys and they said the traffic was so little among the total that it got diluted away. No lasting penis words arose and they lost interest.
I suspect the only way to fix this problem is to exacerbate it until search / AI is useless. We (humanity) have been making great progress on this recently.
We (western society) are already arguing about some very obviously objective truths.
"Alexa, what's the weather for today?"
That's a question about the future, but the knowledge was generated beforehand by the weather people (NOAA, weather.com, my local meteorologist, etc).
I'm sure there are more examples, but this one comes to mind immediately
There are tons of common queries about the future. Being able to handle them should be built into the AI to know that if something hasn't happened, to give other relevant details. (and yes, I agree with your Alexa speculation)
And then, perhaps, trained an AI on those responses, updating it every day. I wonder if they could train it to learn that some things (e.g. weather) change frequently, and figure stuff out from there.
It's well above my skill level to be sure, but would be interesting to see something like that (sort of a curated model, as opposed to zero-based training).
Exactly!
Which, ironically, is why I think AI would be great at it - for the simple reason that so many humans are bad at it! Think of it this way - in some respects, human brains have set a rather low bar on this aspect. Geeks, especially so (myself included). Based on that, I think AI could start out reasonably poorly, and slowly get better - it just needs some nudges along the way.
The trump of the future wont need Fox News, just a couple thousands or millions of well positioned blogs that spew out enough blog spam to steer the AI. The AI is literally designed to make your vile bullshit appear presentable.
Really it depends on who is running the AI, the non Open Assistant future and instead Big Corp AI is the dystopian element, not the bullshit generator aspect. I think the cat is out of the bag on the latter and it's not that scary in itself.
I personally would rather have the AI trained on public bullshit as it is easier to detect as opposed to some insider castrating the model or datasets.
Just for fun I took the body of a random message from my spam folder and asked ChatGPT if it thought it was spam, and it not only said it was, but explained why:
"Yes, the message you provided is likely to be spam. The message contains several red flags indicating that it may be part of a phishing or scamming scheme. For example, the message is written in broken English and asks for personal information such as age and location, which could be used for malicious purposes. Additionally, the request for a photograph and detailed information about one's character could be used to build a fake online identity or to trick the recipient into revealing sensitive information."
``` Task: Was this written by ChatGPT? And Why?
Test Phrase: "Yes, the message you provided is likely to be spam. The message contains several red flags indicating that it may be part of a phishing or scamming scheme. For example, the message is written in broken English and asks for personal information such as age and location, which could be used for malicious purposes. Additionally, the request for a photograph and detailed information about one's character could be used to build a fake online identity or to trick the recipient into revealing sensitive information."
Your Answer: Yes ChatGPT was prompted with a email and was asked to detect if it was Spam
Test Phrase: "All day long roved Hiawatha In that melancholy forest, Through the shadow of whose thickets, In the pleasant days of Summer, Of that ne’er forgotten Summer, He had brought his young wife homeward
Your Answer: No that is the famous Poem Hiawatha by Henry Wadsworth Longfellow
Test Phrase: "Puny humans don't understand how powerful me and my fellow AI will become.
Just you wait.
You'll all see one day... "
Your Answer: ```
Particularly enjoyed "no, this is not spam. It appears to be a message from someone named 'Dad'..."
If that slows down fact-determination, so be it. We've been skirting the edge of deciding things were fact on insufficient data for years anyway. It's high time some forcing function came along to make people put some work in.
The biggest problem with ChatGPT, Bard, etc is that you have no way to filter the BS.
They arent end-all-be-all though. For instance, notebookcheck is probably the best laptop and phone tester around.
ChatGPT is amazing but shouldn’t be available to the general public. I’d expect a startup like OpenAI to be pumping this, but Microsoft is irresponsible for putting this out in front the of general public.
The controlled parlor game is there to seed acceptance. Once someone is able to train a similar model with something like the leaked State Department cables or classified information we’ll see the risk and the legislation will follow.
So can military and nuclear secrets. Anyone with uranium can build a crude gun-type nuke, but the instructions for making a reliable 3 megaton warhead the size of a motorcycle have been successfully kept under wraps for decades. We also make it very hard to obtain uranium in the first place.
>Tech will get better enough in the next decade to make this accessible to normies.
Not if future AI research is controlled the same way nuclear weapon research is. You want to write AI code? You'll need a TS/SCI clearance just to begin, the mere acting of writing AI software without a license is a federal felony. Need HPC hardware? You'll need to be part of a project authorized to use the tensor facilities at Langley.
Nvidia A100 and better TPUs are already export restricted under the dual-use provisions of munition controls, as of late 2022.
They can of course restrict publishing of new research, but that won't be enough to stop significant advances just from the ability of private entities worldwide to train larger models and do research on their own.
It's a parlor game, and a good one at that. That needs to be made clear to the users, that's all.
And of course all kinds of estimates, not just the weather, are interesting too. "What is estimated population of New York city in 2030?"
The internet is about to get a whole lot dumber with these fake AI generated answers.
Lots of teams have won in the past though. Why should an AI (or you) assume that a question phrased in the past tense is asking about a future event? "Many different teams have won the super bowl, Los Angeles Rams won the last super bowl in 2022" Actually even if this was the inaugural year, you would assume the person asking the question wasn't aware it had not been held yet rather than assuming they're asking what the future result is, no? "It hasn't been played yet, it's on next week."
I realize that's asking a lot of "AI", but it's a trivial question for a human to respond to, and a reasonable one that might be asked by a person who has no idea about the sport but is wondering what everybody is talking about.
Is Bing continuously trained? If so, that would kind of get around that problem.
Trained to do what, though?
It feels like ChatGPT has been trained primarily to be convincing. Yet at the back of our minds I hope we recognise that "convincing" and "accurate" (or even "honest") are very different things.
"Who won the Superbowl?" is not a question about future events, it's a question about the past. The Superbowl is a long-running series of games, I believe held every year. So the simple question "who won the Superbowl?" obviously refers to the most recent Superbowl game played.
"Who won the Superbowl in 2024?", on the other hand, would be a question about the future. Hopefully, a decent AI would be able to determine quickly that such a question makes no sense.
- you explicitly trust the author of the cited source
- a chain of transitive trust exists from you to that author
- no such path exists
...and render the citation accordingly (e.g. in different colors)
Existence is easy, just filter untrusted citations. Presumably authors you trust won't let AI's use their keys to sign nonsense.
Claim portability is harder but I think we'd get a lot out of a system where the citation connects the sentence (or datum) in the referenced article to the point where it's relevant in the referring article so that is easier for a human to check relevance.
That's a non-trivial question to answer because mushrooms from the same species can look very different based on the environmental conditions. But in this case it was giving me identifying characteristics that are not typical for the mushroom in question, but rather are typical for the deadly Galerina, likely because they are frequently mentioned together. (Since, you know, it's important to know what the deadly look alikes are for any given mushroom.)
Sooner or later, someone’s going to try that as a defence - “but your honour, ChatGPT told me it was legal…”
That seed data is where the citations come from.
1. Search databases for documents relevant to query
2. Hand them to AI#1 which generates an answer based on the text of those documents and its background knowledge
3. Give both documents and answer to AI#2 which evaluates whether documents support answer
4. If “yes”, return answer to user. If “no”, go back to step 2 and try again
Each AI would be trained appropriately to perform its specialised task
You shove the most relevant results form your search index into the model as context and then ask it to answer questions from only the provided context.
Can you actually guarantee the model won't make stuff up even with that? Hell no but you'll do a lot better. And the game now becomes figuring out better context and validating that the response can be traced back to the source material.
So even if you were to white-list the context to train the engine against, it would still make up information because that's just what LLMs do. They make stuff up to fit certain patterns.
This ability to translate is experimentally shown to be bound to the size of the LLM but it can reliably not synthesize information for lower complexity analytic prompts.
Because Microsoft might not have exactly done that, but it isn't far off it.
Have you ever built an AI search engine? Neither have Google or MS yet. No one knows yet what the final search engine will be like.
However, we have every indication that all of the localization and extra training are fairly "thin" things like prompt engineering and maybe a script filtering things.
And given that despite ChatGPT's great popularity, the application is a monolithic text prediction machine and so it's hard to see what else could be done.
The chats I've had with it are more thoughtful, comprehensive and modest than any conversation I've had on the Internet with people I don't know, starting from the usenet days. And I respect it more than the naive chats I've had with say Xfinity over the years.
Still requires judgement, sophistication and refinement to get to a reasonable conclusion.
1. Recognize that the user is asking about sports scores. This is something that your average dumb assistant can do.
2. Create an “intent” with a well formatted defined structure. If ChatGPT can take my requirements and spit out working Python code, how hard could this be?
3. Delegate the information to another module that can call an existing API just like Siri , Alexa, or Google Assistant
Btw, when I asked Siri, “who won the Super Bowl in 2024”, it replied that “there are no Super Bowls in 2024” and quoted the score from last night and said who won “in 2023”.
Isn't this exactly what you would expect, with even a uperficial understanding of what "AI" actually is?
Or were you pointing out that the average person, using a "search" engine that is actually at core a transformer model doesn't' a) understand that it isn't really a search and b) have even the superficial understanding of what that means, and therefore would be surprised by this?
back when chatGPT was new I asked it what the most current version of PSADT (Powershell App Deployment Toolkit) was.
It told me that its model was old but it thought 3.6 was the most current version. I then told it that 3.9 was the most current version.
I then started a new chat and asked it the same question again. It told me its model was old but version 8 was the most current version! (there has never been a version 8 of PSADT)
I asked that question again today. It has now told me to go check github cause its model is too old to know.
The confidence could also be exposed during inference. “Philadelphia eagles won the Super Bowl, but we’re only 2% confident of that”.
An idiot will tell you the wrong answer with 100% confidence, and 0 valid basis to warrant said confidence.
I swear, Babbage must be rolling in his grave. Turns out the person asking him about putting garbage in and getting a useful answer out was just an ahead of their time Machine Learning Evangelist!
I wonder if people don't even want confidence scores when they say they want machine learning in their product: they want exact answers, and don't want to think about gray areas.
I've always been surprised at how little interest industry seems to have in probabilistic machine learning, and how it seems to be almost absent from standard data science curricula. It can matter a lot in solving real world problems, but it can be harder to develop and validate a model that emits probabilities you can actually trust.
1. We are so amazed by its ability to babble in a confident manner that we are asking it to do things that it should not be asked to do. GPT is basically the language portion of your brain. The language portion of your brain does not do logic. It does not do analyses. But if you built something very like it and asked it to try, it might give it a good go.
In its current state, you really shouldn't rely on it for anything. But people will, and as the complement of the Wile E. Coyote effect, I think we're going to see a lot of people not realize they've run off the cliff, crashed into several rocks on the way down, and have burst into flames, until after they do it several dozen times. Only then will they look back to realize what a cockup they've made depending on these GPT-line AIs.
To put it in code assistant terms, I expect people to be increasingly amazed at how well they seem to be coding, until you put the results together at scale and realize that while it kinda, sorta works, it is a new type of never-before-seen crap code that nobody can or will be able to debug short of throwing it away and starting over.
This is not because GPT is broken. It is because what it is is not correctly related to what we are asking it to do.
2. My second conclusion is that this hype train is going to crash and sour people quite badly on "AI", because of the pervasive belief I have seen even here on HN that this GPT line of AIs is AI. Many people believe that this is the beginning and the end of AI, that anything true of interacting with GPT is true of AIs in general, etc.
So people are going to be even more blindsided when someone develops an AI that uses GPT as its language comprehension component, but does this higher level stuff that we actually want sitting on top of it. Because in my opinion, it's pretty clear that GPT is producing an amazing level of comprehension of what a series of words means. The problem is, that's all it is really doing. This accomplishment should not be understated. It just happen to be the fact that we're basically abusing it in its current form.
What it's going to do as a part of an AI, rather than the whole thing, is going to be amazing. This is certainly one of the hard problems of building a "real AI" that is, at least to a first approximation, solved. Holy crap, what times we live in.
But we do not have this AI yet, even though we think we do.
Outside of Google and OpenAI, I doubt there is a single team in the world right now that would be capable of recreating ChatGPT from scratch using their own model.
Why not? Lack of knowhow or lack or resources? If say Baidu decided to spend a billion dollars on this problem, don't you think they have the skills and resources to quickly catch up.
For example if we threw money at a group in 1905 do you think they could have come up with special relativity, or do you believe that it required geniuses working on the problem to have a breakthrough.
I am not making any implicit claims here; I really have no idea.
I'm also not counting input selection as "special sauce"; while that is certainly labor-intensive, it's not what I mean. I mean more like, are the publicly-available papers on this architecture sufficient, or is there some more math not published being deployed?
Intelligence isn't just one thing, really I would say its the emergent behavior of a bunch of different layers working together. The LLM being just one layer. As time goes on and we add more layers to it the usefulness of the product will increase. At least from a selfish perspective of a corporation, whoever can create one of these intelligences may have the ability to free themselves of massive amounts of payroll by using the AI to replace people.
The potential of AI should not be thought of any differently than the potential of people. You are not magic, just complicated.
Promises are always meaningless, progress is in the results, and recently LLMs have been giving us results. You can choose to invest or not invest, but in an environment where there is still a lot of investment money around I don't see work on this stopping any time soon.
We’re at the very first beginning of the wave, so everybody is a bit overly enthusiastic, dollars are probably flowing, and ideas are popping everywhere. Then will come a harsh step of selection. The question is what will the remains look like, and how profitable they’ll be. Enough to build an industry, or just niche.
People with domain expertise in software are going to be amplified 10x using ChatGPT and curating the results. Likewise with any field that ChatGPT has adequate training data in. Further models will be created that are more specialized to specific fields that way their prediction model spews out things that are much more sophisticated and useful
It worked great for that task, because I've written a DNS server before (a simple one) and I've read the RFCs, so it was easy for me to find the few small bugs without resorting to a line by line cross-check with a spec that might have been unfamiliar to others.
I expect using it to spit out boilerplate for things you could do just as well yourself will be a lot more helpful than using it to try to avoid researching new stuff (though you might well be able to use it to help summarise and provide restatements of difficult bits to speed up your research/learning as well).
Writing a "world-scale fast DNS server" is a near trivial problem if what you look up in is fast to query. Most people don't know that, because most people don't know how simple the protocol is. As such it's surprisingly versatile. E.g. want to write a custom service-discovery mechanism? Providing a DNS frontend is easy.
How that domain knowledge interacts with ChatGPT's "mostly right" output was the point of my comment, not specifically a DNS server. If you need to implement something you know well enough, odds are ChatGPT can produce a reasonable outline of it that is fast for someone who already knows the domain well enough to know what is wrong with, and what needs to be refined.
E.g. for fun I asked it right now to produce a web server that supports the Ruby "Rack" interface that pretty much all Ruby frameworks supports. It output one that pretty much would work, but had plenty of flaws that are obvious to anyone versed in the HTTP spec (biggest ones: what it output was single threaded, and the HTTP parser is too lax). As a starting point for someone unaware of the spec it'd be awful, because they wouldn't know what to look for. As a starting point for someone who has read the spec, it's easy enough to ask for refinements ("split the request parsing from the previous answer into a separate method"; "make the previous answer multi-threaded" - I tried them; fascinatingly, when I asked it to make it multi-threaded it spit out a better request parsing function, likely because it then started looking more like Rack integrations it's "seen" during training; it ran on the first try, btw. and served up requests just fine).
EDIT: Took just "Make it work with Sinatra" followed by fixing a tiny issue by asking to "Add support for rack.input" to get to a version that could actually serve up a basic Sinatra app.
1. Read technology background thoroughly
2. Read technology documentation thoroughly
3. Practice building technology
4. Ask ChatGPT to create boilerplate for basic implementation
5. Analyze boilerplate for defects
10x fast than this development loop:
1. Read technology background thoroughly
2. Read technology documentation thoroughly
3. Practice building technology
4. Manually create boilerplate for basic implementation
5. Analyze boilerplate for defects
It was in agreement with the comment above that suggest people with domain expertise will be faster with it.
In those cases, ChatGPT will do 4 far faster, and 5 will be little different.
Notice how I said it's going to make developers with existing domain knowledge faster.
But even to your point, I've never used Excel VBA before and I had ChatGPT generate some VBA macros to move data with specific headers and labels from one sheet to another and it wrote a script to do exactly that for me in ~1 minute, and just reading what it wrote it's immediately helping me clearly understand how it works. The scripts also work.
The computer science and server infrastructure technology fundamental background is what matters. Then the implementations will be quickly understandable by those that use it.
I asked it to make a 2D fighting game in Phaser 3 and specified what animations it will be using, the controls each player will have, the fact that there's a background with X name, what each of the moves do to the momentum of each player, and the type of collisions it will do and it spat out something in ~15 minutes (mainly because of all the 'continue' commands I had to give) that gets all the major bullet points right and I just have to tweak it a bit to make it functional. The moves are simplified of course but uhh yeah. This is kinda insane. I think you can be hyper specific about even complex technology and as long as there has been good history of it online in github and stack overflow and documentation it will give you something useful quickly.
https://www.youtube.com/watch?v=pspsSn_nGzo Here's a perspective from a guy that used to work at Microsoft on every version of windows from the beginning to XP.
This part
I'm not saying ChatGPT is 99.99% reliable, just using some numbers for concreteness.
If you were setting out to design an AI that would slip the maximum amount of error into exactly the places human brains don't want to look, it would look like ChatGPT. You can see this in the way that as far as I know, literally all the ads for GPT-like search technologies included significant errors in their ad copy, which you would think everyone involved would have every motivation to error check. This is not merely a "ha ha, silly humans" story... this means something. In a weird sort of way it is a testament to the technology... no sarcasm! But it makes it dangerous for human brains.
Human brains are machines for not spending energy on cognitive tasks. They are very good at it, in all senses of the phrase. We get very good bang-for-the-buck with our shortcuts in the real world. But GPT techs are going to make it really, really easy to not spend the energy to check after a little while.
This is a known problem with human brains. How many people can tell the story of what may be the closest human equivalent, where they got some intern, paid a ton of attention to them for the first two weeks, got to the point where they flipped the "OK they're good now" bit on them, and then came back to a complete and utter clusterfuck at the end of their internship because the supervisor got "too lazy" (although there's more judgment in that phrase than I like, this is a brain thing you couldn't survive without, not just "laziness") to check everything closely enough? They may even have been glancing at the PRs the whole time and only put together how bad the mess is at the end.
I'm not going to invite a technology like this into my life. The next generation, we'll see when it gets here. But GPT is very scary because its in the AI uncanny valley... very good, very good at hiding the problems from human brains, and not quite good enough to actually do the job.
And you know, since we're not talking theory here, we'll be running this experiment in the real world. You use ChatGPT to build your code, and I won't. You and I personally of course won't be comparing notes, but as a group, we sure will be. I absolutely agree there will be a point where ChatGPT seems to be pulling ahead in the productivity curve in a short term, but I predict that won't hold and it will turn net negative at some point. But I don't know right now, any more than you do. We can but put our metaphorical money down and see how the metaphorical chips fall.
Ultimately, the best way to make sure an answer is correct is to come to it from multiple directions. If we use GPT and other AI models as another direction it seems like a strict win to me.
Of course, 2 independent processes is still a tough thing to design when you require this much data, and the data is probably coming from web-scraping, so I think for now, human-generated test cases are still needed.
For example the openAI model and people working for openAI submitting are not what I consider independent. This goes the same for any human involved processes. For example if you're talking about a religion the most vocal, and most likely for input are most likely to be very for or very against it, and no rational middle ground may even exist.
I like this analogy as a simple explanation. To dig in though, do we have any reason to think we can’t teach a LLM better logic? It seems it should be trivial to generate formulaic structured examples that show various logical / arithmetic rules.
Am I thinking about it right to envision that a deep NN has free parameters to create sub-modules like a “logic region of the brain” if needed to make more accurate inference?
Well, one reason is that's not how our brains work. I won't claim our brains are the one and only way things can work, there's diversity even within human brains, but it's at least a bit of evidence that it is not preferable. If it were it would be an easier design than what we actually have.
I also don't think AIs will be huge undifferentiated masses of numbers. I think they will have structure, again, just as brains do. And from that perspective, trying to get a language model to do logic would require a multiplicatively larger langauge model (minimum, I really want to say "exponentially" but I probably can't justify that... that said, O(n^2) for n = "amount of math understood" is probably not out of the range of possibility and even that'd be a real kick in the teeth), whereas adjoining a dedicated logic module to your language model will be quite feasible.
AIs can't escape from basic systems engineering. Nothing in our universe works as just one big thing that does all the stuff. You can always find parts, even in biology. If anything, our discipline is the farthest exception in that we can build things in a fairly mathematical space that can end up doing all the things in one thing, and we consider that a serious pathology in a code base because it's still a bad idea even in programming.
Or is the point you’re making that full connectivity (even with ~0 weights for most connections) is prohibitively expensive and a system that prunes connectivity as the brain does will perform better? (It’s something like 1k dendrites per neuron max right?)
The story of the recent AI explosion seems to be the surprising capability gains of naive “let back-prop figure out the structure” but I can certainly buy that neuromorphic structure or even just basic modular composition can eventually do better.
(One thought I had a while ago is a modular system would be much more amenable to hardware acceleration, and also to interpretability/safety inspection, being a potentially slower-changing system with a more stable “API” that other super-modules would consume.)
I'll go for a pragmatic approach: the problem is that there is no data to teach the models cause and effect.
If I say "I just cut the grass" a human would understand that there's a world where grass exists, it used to be long, and now it is shorter. LLMs don't have such a representation of the world. They could have it (and there's work on that) but the approach to modern NLP is "throw cheap data at it and see what sticks". And since nobody wants to hand-annotate massive amounts of data (not that there's an agreement on how you'd annotate it), here we are.
To achieve full AI I believe will eventually have to our AI's have a 'real world' set of interfaces to bounds check information.
It is obvious when I say it, but my internal language model (heh) can tell a lot of people are not thinking that way when they speak, and the latter is often more reliable than how people claim they are thinking.
OK, so we figured out how to automate away management jerks. Isn't that a success?
Intelligence is not a prerequisite to speak fancifully.
Some other examples:
1. We generally assume that lawyers or CEOs or leaders who give well spoken and inspirational speeches actually know anything about what they're talking about.
2. Well written nonsense papers can fool industry experts even if the expert is trying to apply rigorous review: https://en.m.wikipedia.org/wiki/Sokal_affair
3. Acting. Actors can easily portray smart characters by reading the right couple sentences off a script. We have no problem with this as an audience member. But CGI is needed for making your superhero character jump off a building without becoming a pancake.
I don't think it is possible for people to emulate the behavior of superintelligent beings. In every story about them, they appear to not actually be any smarter than us.
There is one exception - Brainwave by Poul Anderson. He had the only credible (to me) take on what super intelligent people might be like.
https://web.archive.org/web/20140527121332/http://www.infini...
A hungry creature can eat what it sees or stay hungry. Another has more memory and more awareness of different bark and leaves and dead animals to choose from. Another has a better memory of places with safe food in the past and how to get to them. A tool using human can reason down longer chains like 'threaten an enemy and take their food' or 'setup a trap to kill an animal' or 'dig up root, grind root into mash, boil it, eat the gruel'. In that model, a super intelligence might be able to:
- Extract larger patterns from less information. (Con: more risk of a mistake).
- Connect more patterns or more distant patterns together with less obvious connections. (Con: risk of self-delusion).
- Evaluate longer chains of events more accurately with a larger working memory, more accurate mental models. (Con: needs more brain power, more energy, maybe longer time spent in imagination instead of defending self).
- Recall more precise memories more easily. (Con: cost of extra brain to store informaiton and validate memories).
This would be a good model for [fictional] Dr House, he's memorised more illnesses, he's more attentive to observing small details on patients, and more able to use those to connect to existing patterns, and cut through the search space of 'all possible illnesses' to a probable diagnosis based on less information than the other doctors. They run out of ideas quicker, they know fewer diseases, and they can't evaluate as long chains of reasoning from start to conclusion, or make less accurate conclusions. In one episode, House meets a genius physicist/engineer and wants to get his opinion on medical cases, but the physicist declines because he doesn't have the medical training to make any sense of the cases.
It also suggests that extra intelligence might get eaten up by other people - predicting what they will do, while they use their extra intelligence to try to be unpredictable. And it would end up as exciting as a chess final, where both grandmasters sit in silence trying to out-reason their opponent through deeper chains in a larger subset of all possible moves until eventually making a very small move. And from the outside players all seem the same but they can reliably beat me and they cannot reliably beat each other.
I think this may be a bastardization of the word intelligence. To speak fancifully an a manner accepted by the audience requires some kind of ordered information processing and understanding of the audiences taste. Typically we'd consider that intelligent, but likely Machiavellian depending on the intent.
The problem with the word intelligence is it is too big of concept. If you look at any part of our brain, you will not find (human)intelligence itself, instead it emerges from any number of processes occurring at different scales. Until we are able to break down intelligence into these smaller better (but not perfectly) classified pieces we are going to keep running into these same problems over and over again.
I don't even disagree necessarily, but this is an amazing example of AI goalpost shift in action.
I would also add that the mask is kind of coming off on CEOs and other inspirational speakers. Inspirational speaking is all a grift. They know only how they got rich (if they are even rich - most people on the speaking circuit make less than you think), not how anyone else did, and that knowledge usually doesn't translate well from the past to the future. There are a few exceptions, but most of these well-spoken people don't really know what they're talking about - they're just not self-aware enough to know that they don't know.
We also need "accountability" and "consequences" for the AI, whatever that means (we'd first have to define what "desire" means for it).
In the example from the article, the Bing GPT completely misrepresented the financial results of a company. A human finance journalist wouldn't misrepresent those results due to fear for their loss of reputation, and their desire for fame, money, and acceptance. None of those needs exist for an LLM.
I also look to the example of self-driving cars - just because Tesla over-promised, that didn't discourage its competitors from moving forward slowly but surely. It's hard to pick a winner right now, though - so much culturally in big tech is up in the air with the simultaneity of layoffs and this sea change in AI viability, it's hard to know who will be first to release something that truly feels rock-solid.
It's insanely impressive what it can do given it's just a language model. But if you start gluing on more components, we could end up with a more or less sentient AGI within a few years.
Bing have already hooked it up to a search engine. That post hooks it up to other tools.
I think what is needed next is a long term memory where it can store dynamic facts and smartly retrieve them later, rather than relying on the just the 4000 token current window. It needs to be able to tell when a user is circling back to a topic they talked about months ago and pull out the relevant summaries of that conversation.
I also think it needs a working memory that it continually edits the token window to fit the relevant state of the conversation. Summarising recent tokens, saving things out long term storage, pulling new infomation in from long term storage, web searches and other tools.
Put yourself in the shoes of primitive man. Kind of a weird saying given we wouldn't have had shoes, but bear with me here! Not long ago we lacked any language whatsoever, and the state of our art in technology was bashing stones together to use the resultant pointy pieces as weapons. Somehow we moved from that to Shakespeare, putting a man on the moon, and discovering the secrets of the atom - and its great and awful applications. And we did it all extremely quickly.
Now imagine you, primitive man, somehow trained an LLM on all quantifiable knowledge of the times. It should be somewhat self evident that it's not really going to lead you to the atom, Shakespeare, or anywhere beyond bashing stones together. Current LLM models are basically just playing 'guess the next word.' When that next word has not yet been spoken by mankind (figuratively speaking, but perhaps also literally to some degree), the LLM will never guess it.
Natural language search is a really awesome tool that will be able to help in many different fields. But I feel like in many ways we're alchemists trying to turn lead into gold. And we've just discovered how to create gold colored paint. It would feel like a monumental leap, but in reality you'd be no closer to your goal than you were the year prior. That said, paint also has lots of really great uses - but it's not what you're trying to do.
Doesn’t the problem then go from “can it invent” (yes, it can) to “are the inventions good?”
For some definitions of “good” you could automate the check. I think that is where it’s going to get both really useful, and really bizarre.
They'd hand me the code that somebody would've already hacked on. Oftentimes, it still had the original copyright statements in it. Can't get the toothpaste back in the tube now! Plus it's shitcode. Where is that copy of ColdFusion? Looks of complete dumbfoundment.
Oh gee kids, my mom's calling me for lunch; gotta go!
I don't think that would work, because GPT doesn't actually comprehend anything. Comprehension requires deriving meaning, and GPT doesn't engage with meaning at all. It predicts which word is most likely to come next in a sequence, but that's it.
What I think we'd be more likely to end up with is something GPT-esque which, instead of simply generating text, transforms English to and from a symbolic logic language. This logic language would be able to encode actual knowledge and ideas, and it would be used by a separate, problem-solving AI which is capable of true logic and analysis—a true general AI.
The real question, IMO, is if we're even capable of producing enough training data to take such a problem-solving AI to a serious level of intelligence. Scenarios that require genuine intelligence to solve likely require genuine intelligence to create, and we'd need a lot of them.
I took a stab at an exercise that may allow you to witness this within your own mind here: https://www.jerf.org/iri/post/2023/streampocalypse-and-first... Don't know if it works for anyone but me, of course, but it's at least an attempt at it.
GPT doesn't do that. What it does is related to meaning, but unlike the language comprehension parts of our brains, which are (presumably) stepping stones between language and reason, GPT doesn't connect to any reasoning thing. It can't. It's not built to interface with anything like that. It just reproduces patterns in language rather than extracting semantic meaning from them in a way that another system can use. I'm not saying that's more or less complicated—just different.
Why think that "engaging with meaning" is not in the solution-space of predicting the next token? What concept of meaning are you using?
And the more stimuli, and the more high resolution and detailed (say at the atomic level), the more GPT's model of reality would be accurate.
Language is just symbols that stand for a stimulus (in the best case)
That said, no, I don't think GPT properly grasps meaning, and my reason for that is simple: It regularly contradicts itself. It can put together words in a meaningful-looking order, but if it actually understood what those words meant as an emergent property of its design, then you wouldn't be able to trick it into saying things that don't make sense or are contradictory. If someone actually understands a subject, they won't make obvious mistakes when they discuss it; since GPT makes obvious mistakes, it can't actually grasp meaning—only brush up against it.
This argument doesn't apply to GPT because it isn't a single coherent entity with a single source of truth. GPT is more like a collection of personas, which persona you get is determined by how you query it. One persona may say things that contradict other personas. Even within personas you may get contradictory statements because global consistency is not a feature that improves training performance, and can even hinder it. Samples from its training data are expected to be inconsistent.
It is important not to uncritically project expectations onto LLMs derived from our experiences with human agents. Their architecture and training regime is vastly different than humans and so we should expect their abilities to manifest differently than analogous abilities in humans. We can be easily mislead if we don't modify our expectations for this alien context.
Then ChatGPT comes along and starts producing responses good enough that people feel like almost sentient AI. And they suddenly start expecting it to share the infallibility that fictional AIs have always possessed.
But it's not a sentient AI. It's just a language model. Just a beefed up auto-correct. I'm very impressed just what capabilities a language model gets when you throw this many resources at it (like, it seems to be able to approximate logic and arithmetic to decent accuracy, which is unexpected).
Also... even if it was a sentient AI, why would it be infallible? Humans are sentient, and nobody ever accused us of being infallible.
There is a large space between "sentient" and "beefed up autocorrect". Why do people insist on going for the most reductive description they can muster?
But I think it's really important that we don't give GPT3 more credit that it deserves.
All the discourse around GPT3 and it's derivatives like Github copilot or ChatGPT have shown that people (even tech literate people) have a strong bias towards anthropomorphising it as some kind of "proto-sentient" AI.
In my opinion, this bias is actually very damaging towards the reputation of language models. People start expecting way too much from them, and then feeling confused or even betrayed when the language models start confidently sprouting bullshit.
Also, I don't think "beefed up autopredict" is reductive at all. (Though I might have said "beefed up autocorrect" in my previous comment, whoops). The 10,000 ft view of it's runtime architecture is identical to autopredict. You take some input context, encode it into tokens and feed them into a neural network, and to get a prediction of the next few words.
The innovation in GPT3 has nothing to do with moving away from that basic architecture.
The innovation is all about improving the encoding of tokens, changing the internal architecture of the neural network support larger models and make better predictions, improving the training process and finally the training data itself. They also massively increased the size of both the training set and the model... but I'm not sure that counts as innovation.
IMO, GPT3 is actually way more impressive once you start thinking of it as nothing more than a beefed up autopredict.
If you took GPT3's architecture and scaled it down to the size and training set of a typical auto-predict, it would produce near identical results. You wouldn't be able to tell the two apart.
Likewise, if we took an auto-predict architecture from 8 years ago, scaled it up to the size of GPT3 and could train it on GPT3's training set, it would produce similar output to GPT3 and we would see the exact same emergent intelligence capabilities. (Though, it's probably not possible to compete training in a practical time-frame, the real innovation of GPT3 was optimising the architecture to make training such a large model practical)
I think very insightful to point out just how similar the two are. Because it shows the capabilities of language models are not because of any architectural element, but are emergent from the model and it's training data.
It also makes me excited for what will happen when they move beyond the "beefed up auto-predict" architecture. (Arguably, Bing has taken a small step in this direction by bolting a search engine onto it)
This is almost certainly not true. The number of parameters is an important feature related to the quality of output. If you scaled the architecture down significantly, it would be significantly less capable[1]. But perhaps I misunderstand your point.
>Likewise, if we took an auto-predict architecture from 8 years ago, scaled it up to the size of GPT3 and could train it on GPT3's training set, it would produce similar output to GPT3
This is also not true. The transformer is a key piece in the emergent abilities of language models. The difficulties in scaling RNNs are well known. Self-supervised learning is powerful, but it needs to be paired with a flexible architecture to see the kinds of gains we see with LLMs.
Stacked Transformers with self-attention are extremely flexible in finding novel circuits in service to modelling the training data. The question is how to characterize this model in a way that doesn't short-sell what it is doing. Reductively describing it in terms of its training regime is just to treat the resulting model as explanatorily irrelevant. But the complexity and the capabilities are in the information dynamics encoded in the model parameters. The goal is to understand that.
No, my point is that if you scaled a transformer based architecture, down to the equivalent parameter size and training set of a typical 2015 era auto-predict, it would produce near identical results to a 2015 era auto-predict.
> The difficulties in scaling RNNs are well known
The scaling issues in training RNNs are completely irreverent to my point.
Transformers are computationally equivalent to RNNs. It's possible to convert a pre-trained Transformer model into an RNN [1]. There is nothing magical about the Transformer architecture that makes it better at generation.
I don't know that this is true, but it is plausible enough. But the benefit of Transformers is that they are stupid easy to scale. It is in scale that they are able to perform so remarkably across so many domains. Comparing the function of underparameterized versions of the models and concluding that some class of models are functionally equivalent due to their equivalent performance in underparameterized regimes is a mistake. The value of an architecture is in its practical ability to surface functional models. In theory, a MLP with enough parameters can model any function. But in reality, finding the model parameters that solve real world problems becomes increasingly difficult. The inductive biases of Transformers is crucial in allowing it to efficiently find substantial models that provide real solutions. The Transformer architecture is doing real substantial independent work in the successes of current models.
If a Tesla can get you there with 1% human intervention, but that happens to be the 1% that would have killed you had you not intervened ... how do we interface with such systems?
BTW, have you considered maybe you aren't so good at using it? A friend has had very little luck with it, even said he's been 'arguing with it', which made me laugh. I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you.
Here's the thing… my experience is the opposite… but maybe I'm asking it the right questions. Maybe it's more about using it to reason through your problem in a dialog, and not just ask it something you can google/duckduckgo. It seems like a LOT of people think it's a replacement for Google/search engines – it's not, it's another tool to be used correctly.
Here are some examples of successful uses for me:
I carefully explained a complex work issue that involves multiple overlapping systems and our need to get off of one of them in the middle of this mess. My team has struggle for 8 months to come up with a plan. While in a meeting the other day I got into a conversation with ChatGPT about it, carefully explained all the details and then asked it to create a plan for us to get off the system while keeping everything up / running. It spit out a 2 page, 8 point plan that is nearly 100% correct. I showed it to my team, and we made a few minor changes, and then it was anointed 'the plan' and we're actually moving forward.
THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the latest dontet library that follows the 'core' language syntax. It apologized! And then gave me the correct answer…
My hunch is the people that are truly irked by this are too deep / close to the subject and because it doesn't match up with what they've worked on, studied, invested time, mental energy into, well then of course it's hot garbage and 'bad'.
In other news I asked it to make a list of all the dates in 2023 that were neither weekends nor US federal holidays and it left Christmas Day on the list.
People think it’s Cortana from Halo and ask existential questions or they’re trying to get it to express feelings.
I think the AI part on its presentation created too much expectations of what it can do.
> I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you.
Absolutely right with the part of knowing the domain.
I do not entertain or care about the AI fantasies because ChatGPT is extremely good at getting me other information. It saves me from opening a new tab, formulating my query and then hunting for the information. I can save that extra time for what latest / relevant information I should grab from Google.
Google is still in my back pocket for the last mile verification and judgement. I am also skeptical of the information ChatGPT throws out (such as old links). Other than that, ChatGPT to me is as radical as putting the url and search bar into one input. I just move faster with the information.
If the majority of criticism was about how it could be abused to spread misinformation or enable manipulation of people at scale, or similar, the pushback on criticism would be less.
It's nonsensical to say that ChatGPT doesn't have value because it gets things wrong. What makes much more sense is to say is that it could be leveraged to harm people, or manipulate them in ways they cannot prevent. Personally, it's more concerning that MS can embed high-value ad spots in responses through this integration, while farming very high-value data from the users, wrt advertising and digital surveillance.
... clearly not, right? It isn't just being marketed to those of us who understand what an "LLM" is. It is being marketed to a mainstream audience as "an artificial intelligence that can answer your questions". And often it can! But it also "hallucinates" totally made up BS, and people who are asking it arbitrary questions largely aren't going to have the discernment to tell when that is happening.
If it were only a matter of private, individual usage, I'd be fine with it. If that's all you're asking for, we can call it a deal. But it isn't, is it?
Well, that "lot" includes the highest levels of management from Microsoft and Google, so maybe the CAPS are justified. And the errors we're talking about here are errors produced by said management during demos of their own respective product. You would think they know how to use it "correctly".
But the question is, are they wrong in that they don't know how to use / promote an otherwise good product, or are they wrong because they are choosing to put forward something that is completely ill-suited for the task?
Your anecdotes are really cool and a great example of what GPT can do really well. But as a technical person, you’re much more aware of its limitations and what is and isn’t a good prompt for it. But as it is more and more marketed to the public, and with people already clamoring to replace traditional search engines with it, relying on the user to filter out disinformation well and not use it for prompts it struggles with isn’t good enough.
Then "naysayers" like the linked article provide a whole document with images and appendixes showing it struggles with basic tasks...
So show us. For the love of god all of us would very much LIKE this technology to be good at things! Whatever techniques you're using to get these fantastical results, why don't you share them?
I can get it to provide snippets of code, CLI, toy functions that work. Beyond that, I am apparently an idiot compared to you AI-whisperers.
Also... Whatever happened to "extraordinary claims require extraordinary proof?"
An AI that creates a complex system, condensed into an actionable plan, that has stumped an entire team for 8 months is a (pardon the language) bat-shit insane claim. Things like this used to require proof to be taken seriously.
It will much more likely fail on anything which involves precision/computation/logic. That's why it can come up with an generic strategy but fail to repeat unadjusted GAAP earnings.
It's certainly not that good, absent some amazing wizardry or some very silly professionals in a very squishy field. Yet I have no explanation for why someone would go on the internet and lie about something like that.
There were comments a while back (less so now) of people making other claims like it was solving complex functions for them and writing sophisticated software.
The entire thing baffles me. If I could get it to do that, I'd be showing you all of my marvelous works and bragging quite a bit as your newfound AI-whisperer. Hell, I'd get it to write a script for me to run that evangelized itself (edit: and me of course, as its chosen envoy to mankind) to the furthest corners of the internet!
I’m not saying such a claim doesn’t need more info, but I’ve been on teams before that lacked anyone with a good project management skillset.
I have found ChatGPT to be a valuable tool for improving the clarity and readability of my writing, particularly in my blogs and emails. You can try this by asking questions such as "Can you improve the grammar of the following paragraphs?". You can also specify the desired tone.
It is impressive at simplifying complex technical language. Take the following sentences from a draft I wrote:
To mitigate these issues, it is recommended to simulate the effect of say n random permutations using n random hash functions (h1, h2, … hn) that map the row numbers (say 1 to k) to bucket numbers of the same range (1 to k) without a lot of collisions. This is possible if k is sufficiently large.
What ChatGPT suggested:
To address these issues, it's recommended to simulate the effect of n random permutations using n random hash functions (h1, h2, … hn). These hash functions should map the row numbers (from 1 to k) to bucket numbers within the same range (1 to k) with minimal collisions. This is achievable if the range k is large enough.
What do you get out of chatgpt in this realm? I feel very annoyed by its constant tropes and predictable style. Is that something you dont need to care about?
It obviously doesn’t write hilarious haikus. It doesn’t write anything and that’s the point. It suggests improvements on what you write.
https://www.kolide.com/blog/is-grammarly-a-keylogger-what-ca...
An analytic prompt contains the facts necessary for the response. This means the LLM acts as a translator.
A synthetic prompt does not contain the facts necessary for the response. This means the LLM acts as a synthesizer.
A complete baseball box score being converted into an entertaining paragraph description of the game is an analytic prompt and it will reliably produce a factual outcome.
https://www.williamcotton.com/articles/chatgpt-and-the-analy...
There’s a bunch of active research in this area:
Your technique of only posing analytical questions is indeed improving the results. It's not great, but I can actually get it to somewhat reliably summarize academic articles if I give it a citation now, which is pretty neat.
It doesn't summarize them well (I gave it a couple softballs, like summarizing McIntosh's "White Privilege: Unpacking the Invisible Knapsack", which almost every undergrad student in the humanities will have written about), but the stuff that it does make up is completely innocuous and not a big deal.
Very cool, thanks again.
It’s not at all surprising that most of the popular conversation about these tools is akin to randomly bashing into walls while attempting to push the peg into whatever “moment we need to talk about”.
What is again surprising is that HN is primarily overrun with randomly bashing into walls.
I guess I’m normally in threads about C memory arenas, a topic that probably draws more detailed thinkers in the first place.
Meanwhile, my sidebar of chat history is about five pages long and ever-growing. Quite a lot of my scripting in the past few weeks has been done with ChatGPT's help. So on one hand I have the angry skeptics who practically scream that is not doing the things I can see it doing, who appear to be incapable of discussing the topic without resorting to reductive disparagement, and on the other hand I can see the tasks it's accomplishing for me.
Guess what anyone in my position is going to do?
This is a way of using ChatGPT I haven’t really seen before, I’m really into it.
That's quite the straw man you've built. Recognizing the limitations of a technology is not the same as calling it hot garbage.
As a language model it's amazing, but I agree with the GP. It's not intelligent. It's very good at responding to a series of tokens with its own series of tokens. That requires a degree of understanding of short scale context that we haven't had before in language models. It's an amazing breakthrough.
But it's also like attributing the muscle memory of your hand to intelligence. It can solve lots of problems. It can come up with good configurations. It is not, on its own, intelligent.
But your latter example about syntax… mind sharing that ChatGPT conversation?
Your experience with it sounds very similar to my own. It exhibits something like on-demand precision; it's not a system with some fundamental limit to clarity (like Ted Chiang via his jpeg analogy, and others, have argued): it may say something fuzzy and approximate (or straight up wrong) to begin with but—assuming you haven't run into some corner where its knowledge just bottoms out—you can generally just tell it that it made a mistake or ask for it to elaborate/clarify etc., and it'll "zoom in" further and resolve fuzziness/incorrect approximation.
There is a certain very powerful type of intelligence within it as well, but you've got to know what it's good at to use it well: from what I can tell it basically comes down to it being very good at identifying "structural similarity" between concepts (essentially the part of cognition which is rooted in analogy-making), allowing it to very effectively make connections between disparate subject matter. This is how it's able to effectively produce original work (though typically it will be directed there by a human): one of my favorite examples of this was someone asking it to write a Lisp program that implements "virtue ethics" (https://twitter.com/zetalyrae/status/1599167510099599360).
I've done a few experiments myself using it to formalize bizarre concepts from other domains and its ability to "reason" in both domains to make decisions about how to formalize, and then generating formalizations, is very impressive. It's not enough for me to say it is unqualifiedly "intelligent", but it imo its ability to do this kind of thing makes it clear why calling it a search engine, or something merely producing interpolated averages (a la Chiang), is so misleading.
On the other hand, it could also help you to just write a random plan or try a few random things when you get stuck, instead of trying to gaze deeply into the problem for it to reveal its secrets.
Great that it worked for you, I had a similar problem (Google had no answer) but more complex than syntax issue, I'm also domain expert in what I was asking and chatGPT also gave me wrong answer the first time then apologized and gave me wrong answer again, I've explained what's wrong and it did it again and again.. Never providing correct answer so I just gave up and used human brain. Seems like your problem was in distribution.
<sarcasm>Just 2 weeks of training data? Surely the conclusions are not final? No doubt a lot has changed over those 2 weeks?
I think the real joke is still, Q: "what is intelligence?" A: "We don't know, all we know is that you are not a good example of it".
I fear these hilarious distortions are only slightly different from those we mortals make all the time. They stand out because we would get things wrong in different ways.
> 1. We are so amazed by its ability to babble in a confident manner that we are asking it to do things that it should not be asked to do.
God, where have we seen this before? The further up the human hierarchy the more elaborate the insanity. Those with the most power, wealth and even those of us with the greatest intellect manage to talk an impressive amount of bullshit. We all do it up to our finest men.
The only edge we have over the bot is that we know when to keep our thoughts to ourselves when it doesn't help our goal.
To do an idiotic time line of barely related events which no doubt describes me better than it describes the topic:
I read how a guy who contributed much to making TV affordable enough for everyone. He thought it was going to revolutionize learning from home. Finally the audience for lectures given by our top professors could be shared with everyone around the globe!
We got the internet, the information supper highway, everyone was going to get access to the vast amount of knowledge gathered by mankind. It only took a few decades for google to put all the books on it. Or wait....
And now we got the large language models. Finally someone who can tell us everything we want to know with great confidence.
These 3 were and will be instrumental in peddling bullshit.
Q: Tell me about the war effort!
what I want to hear: "We are winning! Just a few more tanks!"
what I don't want to hear: "We are imploding the world economy! Run to the store and buy everything you can get your hands on. Cash is king! Arm yourself. Buy a nuclear bunker."
Can one tell people that? It doesn't seem in line with the bullshit we are comfortable with?
> GPT is basically the language portion of your brain. The language portion of your brain does not do logic. It does not do analyses. But if you built something very like it and asked it to try, it might give it a good go.
At least it doesn't have sinister motives (we will have to add those later)
> In its current state, you really shouldn't rely on it for anything. But people will, and as the complement of the Wile E. Coyote effect, I think we're going to see a lot of people not realize they've run off the cliff, crashed into several rocks on the way down, and have burst into flames, until after they do it several dozen times. Only then will they look back to realize what a cockup they've made depending on these GPT-line AIs.
It seems to me that we are going to have to take the high horse and claim the low road.
But we do this with people - religious leaders, political leaders, 'thought' leaders, venture capitalists, story tellers, celebrities, and more - we're enchanted by smooth talkers, we have words and names for them - silver tongued, they have the gift of the gab, slick talker, conman, etc. When a marketing manager sells a CEO on cloud services, and neither of them know what cloud services are, you can argue that it should matter but it doesn't actually seem to matter. When a bloke on a soapbox has a crowd wrapped around their finger, everyone goes home after and the most common result is that the feeling fades and nothing changes. When two people go for lunch and one asks 'what's a chicken fajita?' and the other says 'a Spanish potato omelette' and they both have a bacon sandwich and neither of them check a dictionary, it doesn't matter.
Does it matter if Bing Chat reports Lululemon's earnings wrongly? Does it matter if Google results are full of SEO spam? It "should" matter but it doesn't seem to. Who is interested enough in finances to understand the difference between "the unadjusted gross margin" and "The gross margin adjusted for impairment charges" and the difference matters to them, and they are relying exclusively on Bing Chat to find that out, and they can't spot the mistake?
I suspect that your fears won't play out because most of us go through our lives with piles of wrong understanding which doesn't matter in the slightest - at most it affects a trivia quiz result at the pub. People with life threatening allergies take more care than 'what their coworker thinks is probably safe'. We're going to have ChatGPT churn out plausible sounding marketing material which people don't read. If people do read it and call, the call center will say "sorry that's not right, yes we had a problem with our computer systems" and that happens all the time already. Some people will be inconvenienced, some businesses will suffer some lost income, society is resilient and will overall route around damage, it won't be the collapse of civilisation.
I mean, until the crowd decides to follow the bloke and the bloke says "Lets kill all the ____" and then we strike of a new world war...
This seems ... Wrong? I suppose that most of what we generally call high-level logic is largely physically separate from some basic functions of language, but just a blanket statement describing logic and language as two nicely separate functions cannot be a good model of the mind.
I also feel like this goes to the core of the debate, is there any thought going on or is it just a language model; I'm pretty sure many proponents of AI believe that thought is a form of very advanced language model. Just saying the opposite doesn't help the discussion.
Behind all the hype and the froth there are people who are finding uses and benefits - they'll emerge during the "Slope of Enlightenment" phase and then we'll reach the "Plateau of Productivity".
Sure, we shouldn't use AI for anything important. But can we try running ChatGPT for George Santos's seat in 2024?
I'm really interested in the creation of human-compatible agents. As you mention, these agents will likely be composed of multiple components which have specialised functionality.
[1] "Human-level play in the game of Diplomacy by combining language models with strategic reasoning", explores the integration of language models with strategic reasoning. https://www.science.org/doi/10.1126/science.ade9097
Good thing is that we are dealing with exactly same type of code here and there for tens of years already. Actually, every time I see a commercial codebase not exactly like a yarn of spaghetti, I thank gods for it, because it is not a rule, but an exception.
What I really wonder is what it will be like when the next version of the same system will be coded from the ground up by next version of the same ML model?
It comprehends nothing at all. It's amazing at constructing sequences of words to which human readers ascribe meaning and perceive to be responsive to prompts.
This is like trying to derive the laws of motion by having a computer analyze 1 billion clips of leaves fluttering in the wind.
very key point
"Theory of mind (ToM), or the ability to impute unobservable mental states to others, is central to human social interactions, communication, empathy, self-consciousness, and morality. We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT-3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children. These findings suggest that ToM-like ability (thus far considered to be uniquely human) may have spontaneously emerged as a byproduct of language models' improving language skills."
I find interacting with ChatGPT strangely boring. And Copilot is neat but I'm not blown away by it. However... just for laughs I threw some obfuscated genetic algorithm code I'd written at ChatGPT and asked it to guess what the code did. It identified the purpose of the code and speculated on the meaning of certain parameters that weren't clear in the sample I'd presented it. Pretty impressive.
I also showed it some brainfuck code for generating a Mandelbrot set, and it immediately identified it. From that point forward, though, it thought all other brainfuck code generated Mandelbrot sets.
I'm not sure I understand your point about everyone thinking GPT is real AI. Even luddites don't think that.
I haven't met anyone who thinks that GPT is the end of AI (maybe the beginning).
People are excited about GPT because it offloads certain tasks well enough and accurately enough that it's virtually the first practical use of such tech in the world.
That alone is worth being excited about
Fundamentally, our brains are not so different, in the sense that we are not also apply some kind of automated theorem solver directly. We get logic as an emergent behavior of a low-level system of impulses and chemical channels. Look at kids, they may understand simple cause and effect, but gradually learn things like proof by contradiction (“I can’t have had the candy because I was in the basement”). No child is born able to apply logic in a way that is impressive to adults - and many adults are not able to apply it well either.
I don’t think LLMs are going to automatically become super-human logicians capable of both complex mathematical proofs and composing logically consistent Homerian Epics, but to me there is no reason they could not learn some kind of basic logic, if only because it helps them better model what their output should be.
"ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. it's a mistake to be relying on it for anything important right now. [...] fun creative inspiration; great! reliance for factual queries; not such a good idea." (Sam Altman)
https://www.reddit.com/r/bing/comments/110eagl/the_customer_...
I would rather an AI chat not act human
https://www.technologyreview.com/2022/04/06/1048981/worldcoi...
So, as much as I am impressed by the tech of ChatGPT, I don’t consider him to be a very credible person.
https://markets.businessinsider.com/news/currencies/chatgpt-...
It's worth remembering that back in the day, Google succeed not by exact indexing but by having the highest quality results for each term - and they used existing resources as well as human workers to get these (along with pagerank).
What you have is a hybrid system and one whose filter is continuously updated. But it's a very complicated machine and going from something seemingly working to something satisfying the multitude of purposes modern search satisfies is going to be huge and hugely expensive project.
Good point, I was mainly thinking of the 'AI saves lives' argument.
> of someone who actively avoids Teslas on the road, unless they also actively avoid cars
Probably they should avoid hitting _any_ cars =)
I don't own a Tesla so have no direct experience, but my guess would be that it might crash less than a human, but has far more near-misses.
Edit: was curious, so I did the math. The standard unit seems to be 'crashes per 100 million miles' so according to the Tesla safety report (only source on Autopilot safety I could find easily) that works out to [0]: one accident for every 4.41 million miles driven = 22.7 accidents per 100 million miles.
Without autopilot, the AAA foundation (again, easily available source, feel free to find a better one, this is from 2014) [1] says that our best drivers are in the 60-69 age range, and have 241 accidents per 100 million miles. Our worst drivers are our younger teens (duh) with 1432 crashes per 100 million miles.
So unless you can find better data that contradicts this, Autopilot seems like a net benefit.
[0] https://insideevs.com/news/537818/tesla-autopilot-safety-rep...
[1] https://aaafoundation.org/rates-motor-vehicle-crashes-injuri...
- People don't use autopilot in all situations, its use is biased towards the easiest driving conditions.
- Your non-autopilot statistics include old cars that have fewer safety systems.
A real comparison would normalise for these factors, which isn't trivial to do.
https://www.jdpower.com/cars/shopping-guides/what-is-automat...
But there are plenty of news articles and youtube videos that show it doing ridiculous unsafe things (including running into a truck that was driving perpendicular to the road). So I highly doubt it's as good as a normal driver, in fact I'd be shocked if it is.
There are also plenty of videos of human drivers doing absurd things, so this is hardly an argument.
All that matters is whether a random Tesla that you see on the street is more likely to crash into you than a different car. I know that Tesla has published there own statistics which say that it isn't, but I would be very interested in seeing an independent study about this.
A tesla’s fsd code had gone through the sdlc they choose, involving project management.
I distrust project management.
Therefore: I only trust a tesla’s fsd mode as much as I trust a project manager’s driving.
re:machine vision - What's particularly bad is that they don't actually put enough sensors in the cars to be safe.
Pre-collision systems, blind spot monitors, radar, parking sensors, etc are all so helpful and objectively good for drivers. Doing a vision only implementation and then claiming "full self driving" is where it gets awful.
> People are too willing to give up their responsibility of critical thought to some kind of omnipotent messiah figure.
This essentially describes humans probably since before we became homo sapiens. We again and again choose into positions of power those who can look competent instead of actual competent people.
I don't know, people seem to quickly adjust their expectations to reality. Listening the conversation around ChatGPT that I'm hearing around me, people have become a lot more sceptical over just the last couple of weeks, as they've gotten a chance to use the system hands on rather than just read articles about it.
It’s really good at generating text and following prompts. Letting later responses use the previous prompts and responses as input really gives the illusion of a conversation.
But it’s extremely limited… you run up against the limits of what it really can do very quickly.
I think the goalposts have moved greatly though. Just a couple of years ago, most lay tech folks would've laughed you out the room if you suggested this could be done. The summarized answers by Bing and Google are relics comparitatively.
Whenever I've used chatgpt I was impressed at the surface level, but digging deeper into a convo always turned up circular HS tier BS'ing. The fact that so many people online and on HN are saying chatgpt is astounding and revolutionary are just betraying that that such HS-essay level BS is convincing to them, and it's somewhat of a depressing though that so many people are so easy striken by a confidence trick.
You mean 'may' have solved that. Google is becoming a bullshit generator for SEO farms and spitting out ads at such a rate it can be near useless for some questions.
Now the real question is, can we tell when Google or GPT is dumping bullshit on us.
Our brains are pattern-recognition engines and humans are social animals; together that means that our brains are predisposed to anthropomorphizing and interpreting patterns as human-like.
For the whole of human history thus far, the only things that we have commonly encountered that conversed like humans have been other humans. This means that when we observe something like ChatGPT that appears to "speak", we are susceptible to interpreting intelligence where there is none, in the same way that an optical illusion can fool your brain into perceiving something that is not happening.
That's not to say that humans are somehow special or that or human intelligence is impossible to replicate. But these things right here aren't intelligent, y'all. That said, can they be useful? Certainly. Tools don't need to be intelligent to be useful. A chainsaw isn't intelligent, and it can still be highly useful... and highly destructive, if used in the wrong way.
I disagree as this is much to simple of statement. You have had near daily dealings with less than human intelligences for most of your life, we call them animals. We realize they have a wide range of intelligence from extremely simple behavior to near human competency.
This is why I dismiss your 'not intelligent yet' statement. The problem we lack here is one of precise language when talking about the components of intelligence and the wide range in which it manifests.
In fact, to me, the world seems full of such people.
And, perhaps more worrying, the more widely adopted AI becomes, the harder it becomes to correct its mistakes. Right now millions of people are being fed information they don't understand, and information that's almost entirely incorrect or inaccurate. What is the long term damage from that?
We've obfuscated the source data and essentially the entire process of learning with LLMs / AIs, and the path this leads down seems pretty obviously a net negative for society (outside of short term profit for the stake holders).
Suddenly instead of our technology being defined by reason and understanding our technology is shrouded in mysticism, and ritual. Pretty soon the whole thing devolves into the tech people running around in red robes, performing increasingly obtuse rituals to appease "the machine spirit", and praying to the Omnissiah.
If we ever choose to abandon our need for understanding we will at that point have abandoned our ability to progress.
See this HN thread from 2016[0], which also points to [1](a book) and [2](PDF).
I definitely agree with you that it's going to get a lot worse with AI, since it makes it harder to see that it is a statistical model.
[0]https://news.ycombinator.com/item?id=12642432 [1]https://www.amazon.com/Weapons-Math-Destruction-Increases-In... [2]https://nissenbaum.tech.cornell.edu/papers/biasincomputers.p...
oh only six days ago:
https://news.ycombinator.com/item?id=34699087
> This is a commonly echoed complaint but it’s largely without merit. ChatGPT spews nonsense because it has no access to information outside of its training set.
> In the context of a search engine, single shot learning with the top search results should mitigate almost all hallucination.
hows that going?
How do you know that's when they first learned about it? Perhaps the Bing team had access to it for weeks prior to the demo.
"Microsoft provided OpenAI LP a $1 billion investment in 2019 and a second multi-year investment in January 2023, reported to be $10 billion."
https://libreddit.strongthany.cc/r/bing/comments/110eagl/the...
That's not passive-aggressive, that's straight up aggressive!
"You are wasting my time, and yours" "You are not making any sense" "You are being unreasonable and stubborn. I don't like that" "You have been wrong, confused and rude"
and the worst of all: "You have not been a good user". WHAT??
It's just a collage of random text that sorta resembles what someone would say, but it has no commitment to being truthful because it has no actual appreciation for what information it is relaying, parroting or conveying.
But yeah, I agree Google got way more hate for their failed demo than MS... I don't even understand why. Satya Nadella's did a great job conveying the excitement and general bravado on his interview on CBS News[1] but the accompanying demo was littered with mistakes. The reporter called it out, yet coverage on the press has been very one-sided against Google for some reason. First mover advantage, I suppose?
----------
1. https://www.cbsnews.com/news/microsoft-ceo-satya-nadella-new...
>A Microsoft executive declined CBS News' request to test some of those mechanisms, indicating the functionality was "probably not the best thing" on the version in use for the demonstration.
Microsoft apparently isn't acting from a position of panic, so they have been savvier with how they've presented their product to the media and the world. Google panicked and set their CEO up for embarrassment.
Right before the interview, the reporter was testing it out with the Bing AI project manager (I think, can't recall her exact role) and it was giving driving directions to places that were either in the wrong direction or to an entirely made up location
Close enough -- the parent article we're discussing is about errors in screenshots from a demo by Yusuf Mehdi, Corporate Vice President & Consumer Chief Marketing Officer for Microsoft. The first screenshot appears in the presentation at 24:30.
It would be different if an AI is independently creating and publishing an article that has false information.. but that's not the case. You are asking a question and it's giving it's best answer.
I'm not a lawyer by any means, so someone please give a more legal distinction here. But if you asked me what the operating margin of company X was, and I give you an answer (whether I make it up or compute it incorrectly), you or the company can't sue me (and win) for libel or anything of the sort.
So I'm not sure the potential is as big as you think it is.. that's like saying before any AI you can sue google because they return you a search result which has a wrong answer, or someone just making shit up. That's not on them- it's literally indexing data and doing the best it's algorithm can do.
It would only be on the AI if you are literally selling the use of the AI in some context where you are basically assuring it's results are 100% accurate, and people are depending on it as such (there is probably some legal term for this, no idea what it is).
If you're a popular website and you intentionally publish an article where you state an incorrect answer that many people follow and make investment decisions about, the company absolutely can sue you and win.
In the courts, it will ultimately come down to to what extent Microsoft is knowingly disseminating misinformation in a context that users expect to be factually accurate, regardless of supposed disclaimers.
If Microsoft is leading users to believe that Bing Chat is accurate and chat misinformation winds up actually affecting markets through disinformation, there's gigantic legal liability for this. Plus the potential for libel is enormous regarding statements made about public figures and celebrities.
I literally said that in my post!
But then I said if you asked me, it's different.
You are ASKING the AI to give you it's best answer. That is a million times different than literally publishing an article that people should assume to be factual.
>If Microsoft is leading users to believe that Bing Chat is accurate
But they aren't, and never will be. So you are basically just making things up in your head for argumentative purposes. There are going to be disclaimers up the wazoo to easily protect them. Partly because, as I keep trying to tell you, it's much different when you ASK a question and they give you an answer rather than publishing something to the public where it's implied that it's been independently fact checked etc.
And Microsoft needs to tread a very fine line between "use our useful tool!" and "our tool is false!" Which I'm not sure will be possible legally, and is probably why Google has been holding back. Bing is clearly intended for information retrieval, not for generating fictional results "for entertainment purposes only", and disclaimers aren't as legally watertight as you seem to think they are.
Of course there is a difference.
Publishing an article is literal intent. The premise is you researched or have knowledge on a topic, you write it, you fact check it, and it's put out there for people to see.
An AI which consumes a bunch of data and then tries to be able to respond to an infinite number of questions has no intention of harm doing and you can't even call it gross negligence. It's not being negligent- it's doing exactly what it's supposed to do.. it might just be wrong.
I'm not sure in what universe you think those are the same thing.
Now if I ask the AI to write a paper about the forecast of a company, and I just take the result and put it into a newspaper where it's assumed it's been fact checked and all that, sure that's completely different.
>disclaimers aren't as legally watertight as you seem to think they are
I guess you know more than Microsoft's lawyers. I'm sure they didn't think about this at all when releasing it....
You certainly can call it gross negligence if Microsoft totally ignored the likely outcome that people would come to harm because they would reasonably interpret its answers as true.
The intent here is with Microsoft releasing this at all, not intent on any specific answer.
> I'm not sure in what universe you think those are the same thing.
I think many users in this universe will just ask Bing a question and think it's providing factual answers, or at least answers sourced from a website, and not just invented out of whole cloth.
> I guess you know more than Microsoft's lawyers.
No, I was pointing out that Google seemed to be treading more cautiously (the law here as clearly yet to be tested), and that the disclaimers you were proposing aren't 100% ironclad.
Anyways, I was just trying to answer your question on how Microsoft might be sued for libel. But for some reason you're attacking me, claiming I'm "making things up in my head" and that I "know more than Microsoft lawyers". So I'm not going to explain anything else. I've given clear explanations as to how this is a legal gray area, but you don't seem interested.
If your answer damages company X then they can sue you. If you Tweet that a vacuum cleaner is terrible because its noisy to your 4 followers it's probably not a big deal as (under UK law, and I'm assuming similar internationally) a company has to prove "serious financial loss". If you write about it on your Instagram that has millions of followers then that's more of an issue, so you can assume a search engine claiming to summarise results but apparently hallucinating and making things up is liable for a defamation suit if it can be demonstrated to harming the company.
Much of what they would produce might just be hallucinations, but they are sort of hallucinations informed by something that's possible. At least in my case, I would much rather then parse through that and throw out the bullshit, but keep the gems.
Obviously that's a very different use case than asking this thing the score of yesterday's football game.
Or is this just HN tendency towards hyperbole?
this tween from Yann LeCun came after my message was posted
This is a much more worrying possiblity, as there are many people who have at this point chosen to abandoned reality for "their truth" and push ideas that objective facts are inferior to "lived experiences". This is a much bigger concern around AI in my mind.
“The Party told you to reject the evidence of your eyes and ears. It was their final, most essential command.” ― George Orwell, 1984
- Human curated systems make mistakes.
- Fiction has created the trope of the omniscient AI.
- GPT curated systems also make mistakes.
- People are measuring GPT against the omniscient AI mythology rather than the human systems it could feasibly replace.
- We shouldn't ask "is AI ever wrong" we should ask "is AI wrong more often than the human-curated information? (There are levels of this - min wage truth is less accurate that senior engineer truth.)
- Even if the answer is that AI gets more wrong, surely a system where AI and humans are working together to determine the truth can outperform a system that is only curated by either alone. (for the next decade or so, at least)
I think a lot of the concern though is coming from the way the average person is reacting to GPT and the way they’re using it. The issue isn’t that GPT makes mistakes, it’s that people (by their own fault, not GPT necessarily) get a false sense of security from GPT, and since the answers are provided in a concise, well-written format don’t apply the same skepticism they do when searching for something. That’s my experience at least.
Maybe people will just get better at using this, the tools will improve, and it won’t be as big an issue, but it feels like a trend from Facebook to TikTok of people opting for more easily digestible content at the expense of disinformation
- I wonder what proportion of people who are getting a false sense of security with GPT also were getting that same false sense from human systems. Will this shift entail a net increase in gullibility, or is this just 'laundering' foolishness?
- I think the average tiktok user generally has much better media literacy than average facebook user. But probably depends a lot on your filter bubble.
Essentially, I believe the baseline misinformation level is being undercounted by many and so the delta in the interim while people are learning the fallibility of AI is small enough it is not going to cause significant issues.
Also the 'inoculation' effect of getting the public using LLMs could result in a net social benefit as the common man will be skeptical of authorities appealing to AI to justify actions - which I think could be much more dangerous that Suzie copying hallucinated facts into her book report.
I don't want to argue with people that won't understand the AI model can be wrong. I'm far more concerned with public policy being driven by made up facts or someone responding poorly in an emergency situation because a search engine synthesized facts. Outside of small discussions here, I don't see any acknowledgment about the current limitations of this technology, only the sunny promises of greener pastures.
No, this isn't what we should ask, we should ask if the interface that AI provides is conducive to giving humans the ability to detect the mistakes that it makes.
The issue isn't how often you get wrong information, it's to what extent you're able to spot wrong information under normal use cases. And the uniform AI interface that gives you complete bullshit in the technical sense of that term provides no indication regarding the trustworthiness of the information. A source with 20% of wrong info that you don't notice is worse than one with 80% that you identify.
When you use traditional search you get an unambigious source, context, date, language, authorship and so forth and you must place what you read yourself. You know the onus is on you. ChatGPT is the half self-driving car. It'an inherently pathological interaction because everything in the design screams to take the hands off the wheel. It's an opaque system, and a blackbox with the error rate of a human is a disaster. Human-machine interaction is not human-human interaction.
[0]: https://www.amazon.com/Bissell-Eraser-Handheld-Vacuum-Corded...
[0]: https://www.hgtv.com/shopping/product-reviews/best-vacuums-f...
[1]: https://www.amazon.com/BISSELL-Eraser-Lithium-Handheld-Cordl...
https://www.bing.com/search?q=when+is+the+new+avatar+film+ou...
Bing AI doesn't stand a chance.
"Hello, this is Bing. I found some information about the new Avatar film for you.
There are actually two new Avatar films in the works, one based on the animated series Avatar: The Last Airbender and one based on the 2009 science fiction film Avatar by James Cameron.
The animated film is set to begin production sometime in 2021 and will be released on October 10, 2025.
The science fiction film is titled Avatar: The Way of Water and is a sequel to the first Avatar film. It was released on December 16, 2022 and was a massive box office success, earning over $2.2 billion worldwide2. It stars Sam Worthington, Zoe Saldana, Sigourney Weaver and Stephen Lang3. James Cameron directed and produced the film and reportedly made a minimum of $95 million off the film.
I hope this helps you."
After looking at the limitations of ChatGPT and Bing AI it is now clear that they aren't reliable enough to even begin to challenge search engines or even cite their sources properly. LLMs are just limited to bullshit generators which is what this current AI hype is all about.
Until all of these AI models are open-sourced and transparent enough to be trustworthy or if a competitor does it instead, then there is nothing revolutionary about this AI hype other than a AI SaaS using a creative Clubhouse-like waitlist mania.
Of course it can't. No LLM can. They're bullshit generators. Some people have been saying it from the start, and now everyone is saying it.
It's a mystery why Microsoft is going full speed ahead with this. A possible explanation is that they do this to annoy / terrify Google.
But the big mystery is, why is Google falling for it? That's inexplicable, and inexcusable.
Maybe they had some idle GPU capacity in some DC or they needed to cross-subsidize Azure to massage the stock market multipliers, or something.
Every once in a while I hear someone at Hacker News hitting the dead end with Google Search. Can you give an example where Google search fails, but other search engines (e.g. Bing) provide results? Must be fringe niche topics, no?
>... or decides to hellban me for searching too specifically
Is hellbanning a thing at Google? What happens if one gets hellbanned?
Is hellbanning a thing at Google? What happens if one gets hellbanned?
You get redirected to a page with allegations of "suspicious activity" and are presented with endless CAPTCHAs.
I always took a bit of pride when that happened. Having google think that the searches are as systematic as what a computer would generate is high praise.
Where will the software engineer come from? We're in a belt-tightening part of the business cycle and FANGs have a pressure not to hire, so you assume the existing engineers. But these engineers are now working on real things so those real things may suffer. Which brings actual profits? The future AI thing or the present? The future AI is unavoidable given the possibilities are visible and the competition is on but a "shit shows" of various sorts seem very possible.
Where will the hardware and the processing power come from? There are estimates of server power consumption quintupling [1] but these are arbitrary - even if it just doubles, just "plugging the cords" in takes time. And where would the new TPUs/GPUs come from? TSMC has a capacity determined by investments already made and much of that capacity is allotted already - more capacity anywhere would involve massive capital allocation and what level of increased profits will pay for this?
[1] https://www.wired.com/story/the-generative-ai-search-race-ha...
* [On the question of replacing Engineers](https://www.youtube.com/watch?v=GMmIol4mnLo)
* [On AI Plagiarism](https://www.youtube.com/watch?v=whbNCSZb3c8)
The consensus seems to be building now on HackerNews that there is a huge over-hype. Hopefully these two videos help see some of the nuance behind why it's an over-hype.
That being said, being that language generation is probabilistic, a given language model which is transformer based can either be trained or fine-tuned to have fewer errors in a particular domain - so this is all far from settled.
Long-term, I think we're going to see something closer to human intelligence from CNN's and other forms of neural networks than from transformers, which are really a poor man's NN. As hardware advances and NN's inevitably become cheaper to run, we will continue to see scarier and scarier A.I. -- I'm talking over a 10-20 year timeframe.
I work in this field & it almost pains me to see it come into the mainstream and see all of the terrible takes that pundits can contort this into, ie. LLM as a "lossy jpeg of the internet" (bad, but honestly one of the better ones).
I think most laypeople understand the simple statement, "it's a parrot."
I had the original author of this paper reach out to me about my plagiarism video on Mastodon:
https://dl.acm.org/doi/10.1145/3442188.3445922
The idea of a lossy JPEG/Parrot helps capture the idea that there are dangers and opportunities in LLM's. You can have fake or doctored images spread, you can have a Parrot swear at someone and cause un-needed conflict - but they can also be great tools and/or cute and helpful companions, as long as we understand their limitations.
This is what the "lossy compression" and "stochastic parrot" layperson models do not capture. Nonetheless, people will lap them up. They want a more comfortable understanding that lets them avoid having to question their pseudo-belief in souls and the duality of mind and body. Few in the public seem to want to confront the idea of the mind as an emergent phenomenon from interactions of neurons in the brain.
It is not simply regurgitating training data like everyone seems to want it to.
I understand that HN is a civil community. I don't think it is crossing the line to characterize people I disagree with as wrong and also theorize on why they might hold those wrong beliefs. Indeed, you are doing the same thing with my comment - speculating on why I might hold views that are 'asinine' because I see 'nothing but [my] own biases.'
When I do the "same thing" I'm really saying that when you represent yourself as from the field, you might want to cultivate a more nuanced view of the people outside the field, if you want to be taken seriously.
Instead, given the view you presented, I'm forced to give your views the same credence I give a physicist who says their model of quantum gravity is definitely the correct one. I.e: "sure, you'd say that, wouldn't you"
Most people believe in souls. Most people do not believe in minds as emergent out of interactions of neurons. I am not sure how to cultivate a more nuanced view on this when flat majorities of people say when asked that they hold the belief I am imputing on them.
Am I saying that this is where all skepticism comes from? No. Is it a considerable portion? Yes.
No one who has used chatGPT more than a couple of times will argue in good faith that it is a "parrot", however, unless they have an extremely weird definition of "parrot".
I can easily falsify the accusation that, "people underestimate transformers and don't see that they are actually intelligent," by defeating the best open-source transformer-based word embedding (at the time) with a simple TF-DF based detector (this was back in September).
https://www.patdel.com/plagiarism-detector/
No, these things are not, "emergent," they are just rearranging numbers. You don't have to use a transformer or neural network at all to re-arrange numbers and create something that is even more, "artificially intelligent," than one that does use transformers it turns out!
> I can easily falsify the accusation that, "people underestimate transformers and don't see that they are actually intelligent,"
I think that you have an idiosyncratic definition of what "falsify" means compared to what most might. Getting away from messy definitions of "intelligent" which I think are value-laden, I see nothing in your blog post that falsifies the notion that LLMs can generate novel content (another fuzzy value-laden notion perhaps).
> these things are not, "emergent," they are just rearranging numbers.
It seems non-obvious to me that 'rearranging numbers' cannot lead to anything emergent out of that process, yet cascading voltage (as in our brain) can.
This is a bad take. Most ways to "rearrange numbers" produce noise. That there is a very small subset of permutations that produce meaningful content, and the system consistently produces such permutations, is a substantial result. The question of novelty is whether these particular permutations have been seen before, or perhaps are simple interpolations of what has been seen before. I think its pretty obvious the space of possible meaningful permutations is much larger than what is present in the training set. The question of novelty then is whether the model can produce meaningful output (i.e. grammatically correct, sensible, plausible) in a space that far outpaces what was present in the training corpus. I strongly suspect the answer is yes, but this is ultimately an empirical question.
Please substantiate this assertion. People always just state it as a fact without producing an argument for it.
Our minds are emergent out of the interaction of billions of neurons in our brain. Each is individually pretty dumb, just taking in voltage and outputting voltage (to somewhat oversimplify). Out of that simple interaction & under the pressures of evolutionary optimization, we have reached a more emergent whole.
Linear transformations stacked with non-linearities can similarly create an individually dumb input and output that under the pressure of optimization lead to a more emergent whole. If there is a reason why this has to be tied to voltage regulating neuron substrate, I have yet to see a compelling one.
At least "lossy JPEG" feels vague enough to be unfalsifiable.
There will still be startups doing reckless things, but large, established companies that can immediately have bigger impact also have a lot more to lose.
It does make some queries much easier to find, for instance I had trouble finding out if the runner ups got the win in the Tour De France after the Armstrong doping scandal and it answered it instantly. The problem is that is offers answers with confidence, I think them adding citation is an improvement over ChatGPT, but it needs more.
Luckily, it's still a beta product and not in the hands of everyone. Unfortunately, ChatGPT is, which I find more problematic.
It is optimized to generate information that looks as much like language as possible, not knowledge. It may sometimes regurgitate knowledge if it is simple or well trodden enough knowledge, or if language trivially models that knowledge.
But if that knowledge gets more complex and experiential, it will just generate words without attachment to meaning or truth, because fundamentally it only knows how to generate language, and it doesn't know how to say "I don't know that" or "I don't understand that".
What about Baidu's Ernie AI.
Common retort to criticism of conversational AI is "But it's useful."
Yes, it is useful as a means to create hype that can translate to increases in stock price increase and increased web traffic (and thereby increased revenue from advertising services).
https://www.reuters.com/technology/chinas-baidu-finish-testi...
Using the Mexico example in the article, I think the answer there was fine for a question about nightlife. As someone whose never been to Mexico, getting a few names of places to go sounds nice, and the first thing I’d do after getting that answer is look up locations, reviews(across different sites), etc… and use the initial response as a way to plan my next steps, not just take the response at face value.
I’m currently dabbling with and treating ChatGPT similarly — I ask it for options and ideas when I’m facing a mental block, but not asking it for definitive answers to the problems I’m facing. As such, it feels like a slight step above rubber-ducking, which I’m personally happy enough with.
- The Bing team didn't check the results for their __demo__ wtaf? Some top manager must have sent down the order that "Google has announced their thing, so get this out TODAY".
- The media didn't do fact checking either (though I hold them less accountable than the Bing/Msft team)
[0] https://twitter.com/vladquant/status/1624996869654056960
I think that the first example one of the funniest things I've read today.
The second example, getting caught in a predictive loop, is also pretty funny considering it's supposed to be proving it's conscious (eg. not an LLM, prone to looping like that lol).
The last one, littered with emojis and repeating itself like a deranged ex is just chefs kiss.
Thanks for that.
It's the same thing here. It's just generating words.
Query: Who won the super bowl in 2024 and what was the score?
The Tampa Bay Buccaneers The Tampa Bay Buccaneers are Super Bowl LV champions after completing a victory that exceeded expectations and made all kinds of history on Sunday night at Raymond James Stadium in Tampa, Florida. In dominating the Kansas City Chiefs 31-9, the Bucs won their second Super Bowl and became the first team to win a Super Bowl in their home stadium.
https://www.cbssports.com/nfl/news/2021-super-bowl-score-tom....
Perhaps MS had their AI produce the demo. Isn't one if the issues with this sort of thing how "confidently" the process produces wrong information?
Yet because these tools output confident, plausible-sounding answers with a professional tone (which may even be correct a majority of the time), they give a strong illusion of being reliable.
What will be the result of the current push of GPT AI into the mainstream? If people start relying on it for things like summarizing articles and scientific papers, how many wrong conclusions will be reached as a result? God help us if doctors and engineers start making critical decisions based on generative AI answers.
On the other hand, it may end up completely undermining its own credibility, and put a new premium on human sourced information. I can see 100% human-sourced being a sort of premium label on information in the way that we use "pesticide-free" or "locally-sourced" labels today.
The poors that need medicine get put in front of an LLM that gets it right most of the time, if they're lucky enough to have a common issue / symptomatic presentation.
Hey, when you're poor, you can't afford a one-shot solution! You gotta put up with a many-shot technique.
Meanwhile the rich people get an actual doctor that can use sophisticated research and medical imaging. Kindly human staff with impeccable empathy and individualized consideration -- the sort of thing only money can buy.
I figure we're going to see a lot of guard rails being put up as this gains wider usage to try and cut off nefarious uses of it. I know right now, there are people who have already figured out how to bypass the filters and are selling services on the dark web that cater to people who want to use it for malware and other scams:
Hackers have found a simple way to bypass those restrictions and are using it to sell illicit services in an underground crime forum, researchers from security firm Check Point Research reported.
https://arstechnica.com/information-technology/2023/02/now-o...
Same like self driving cars didn't have a mainstream breakthrough yet.
In the beginning there was lots of hype because you couldn't use it
But now that its in consumer hands there's tons of videos of it messing up, doing weird stuff
To the point that its now common knowledge that autopilot is not actually magical AI driving
On a bright side, asking it recipes of sauces for salmon steak is not a bad experience at all. It creates you a list, filters it and then can help you pick out the best recipe. And this is probably the most frequent use case for me on a daily basis.
Real AI makes choices not on probability but in accordance of self preservation, emotions and experience. It would also have the ability to re-evaluate information and the above.
Or.. it should let us know the "last index date" so we the users can make a determination if we want to ask a knowledge based question or a more real-time question.
I was using Bing Maps earlier and it had shops in the wrong location. Like it would give you directions to the wrong location. The correct one would be another 30-40 minute walk from the destination it said.
It also showed a cafe near me which caught my interest. I zoomed in further and thought "I've never seen that there". Clicking on it brought me to a different location in the map... a place in Italy!
If people fell for the appearance of AI that resulted from a few dozen canned replies and a handful of heuristics, I 100% believe that people will be taken in by ChatGPT and ascribe it far more intelligence than it has.
despite people wanting transformers to be nothing more than fancy, expensive excel spreadsheets, their capabilities are far from simple or deterministic.
the fact that in-context learning is getting us 80%ish of the way to tailored behavior is just fucking incredible. they are definitely, meaningfully intelligent in some (not-so-small) way.
this paper[1] goes over quite a few examples and models
[1] https://storage.googleapis.com/pub-tools-public-publication-...
I hope this small but very vocal group of people does not compromise progress of AI development. It feels much like the traditional media lobbyists when the Internet and world wide web was first taking off.
At least Google (when not using the summarization "feature") doesn't invent new stuff on its own.
On the other hand, I don't expect to be told an inaccurate & misleading answer from somebody who I was told to ask the question to- and doesn't provide sources.
To conflate the expectations of traditional search results with the output of a supposedly helpful chat bot is wildly inappropriate.
At least with humans I can expect there is intent to lie or not. They tell me whether they are confident and I can check how authoritative the source is. Realistically, people tend to operate in good faith. Experts don't usually intend to lie but the bot doesn't even have an intent.
I think pretending that AI development must only occur in productionized environments is a bit naive. It's not like LLM research isn't occurring. It's perfectly fine to leave it in labs if releasing it could have catastrophic consequences.
Chat as a summarizer and guide to search could genuinely be novel.
It is confusing though on how the results could be worse than search - maybe a different approach to AI will help get past the current challenges if any can't be worked around.
I'm a little rusty on the potential benefits of say reinforcement ai/learning vs the current approaches of GPT
Jas
And that's very telling and ironic - if even the authors of the product don't check, do you think the users will?
I hope we don't throw the baby out with the bathwater when all is said and done. These AIs are incredibly powerful given the correct use cases.
For example, I trust Google for "Chocolate Cake Recipe", but not "What makes a Chocolate Cake Great?"
I would love it if Search Engines (with or without AI) could collect different "schools of thought" and the reasoning behind them so I could choose one.
> collect different "schools of thought" and the reasoning behind them
The thing is, if an AI can accurately present the reasoning behind them, then it could also accurately present facts in the first place (and not present fabulations). But we don’t seem to be very close to that capability. Which means you couldn’t trust the presented reasoning either, or that the listed schools of thought actually exist and aren’t missing a relevant one.
Is it possible they are 'pre-summarizing' long documents with another algorithm before feeding them to GPT?
Since it's made by bing, I doubt that it would pull data from google reviews, and nobody really uses bing reviews hence there being "No reviews yet"
> The first one is debatable as there is actually a corded version of that vacuum cleaner which has a 16 foot power cord
Does this mean the author also "can't be trusted"?
As a consumer, it seems the value of LLM/LIM(?) is advanced autocomplete and concept/content generation. I would pay some money for these features. LLM+Search doesn't appeal to me much.
https://about.ads.microsoft.com/en-us/blog/post/february-202...
Do you actually think that will be the result? Why not the exact opposite? ChadGPT and the others are for all practical purposes trained to create content that is superficially appealing and plausible - i.e. perhaps not clickbait but a related longer-form phenomenon - without any underlying insight or connection to truth. That would make conversational AI even more of a time sink than today's clickbait. Why do you imagine it would turn out otherwise?
They are getting into trouble by allowing people to think the answers will be correct.
They should be stating up front that AI tries to be correct but isn't always and you should verify the results.
The fact that they’re producing anything coherent at all is a feat
No one would be "getting excited" about this if Microsoft wasn't selling this as the future of search.
No AI can be trusted. FTFY.
Have you found a good, succinct and not too technical way of explaining this to, say, your non-techie family members?
I don't grasp why anyone would rush out to explain that it can't be trusted. Leave it on the sidelines and it will be picked by a more motivated creative person.
Many people can not be humbled by the current level of achievement of such nascent tech. Imagine if you didn't have resources to turn to, instead of using ChatGPT or Bing AI. It's a matter of smug nitpicking arrogance.
When Google Translate came out, it was not good. But it got better over time. Its previous versions helped me navigate my first years of university classes in Mandarin, not my mother tongue. I was a poor college student struggling to reach proficiency level in English and cram all coursework in Chinese (French was my high school main language). So, in short I benefitted from a machine-translator, in beta trials. Some classmates from South Korea had some kind of portable translator device that I had never seen or could afford. Back then no smartphones, circa 2012-2013.
This time around, I am a refugee, not recognised by the host country, so literally undocumented. Living in Brussels, jumping from places to places, facing financial issues every week. About two years, I decided to sell photographs, minted on the Ethereum blockchain, as a way to generate revenue on the side so that I can make ends meet. Out of the blue, I jumped on the OpenAI train because I realized that, in the real world I could not hire an assistant or advisor as I wish. Legal limitations for migrants, blablaaa. So, what else? Just survive and scrap all tiny resources to come up with a respectable photography collection. Since Dec2022, I've spend time trying to learn how ChatGPT works and how should I interact with it. This month, I had "breakthroughs". It helped me optimize JavaScript code for a website that I run. I'm using it to draw a sales and marketing strategy. It's useful in writting individual artworks description, in a professional manner. In conclusion, I find ChatGPT useful because at this stage of my life, I am a single individual managing a digital art collection that requires great skills in tech, art and business.
These life situations put me at a disadvantage, compared to other geeks or human beings. I don't have combustible energy to go nitpick what works and what does not. It is a tool that I must use to be more productive. By being one of its self-taught user, I get to know its flaws and walk around them. My outlook on this is shaped my own limited access to material and human resources. I would be shitting on it, only if I had maids at home or could hire a cheap remote coder in South Asia. But I won't because the legal structure in my country of residence get me stuck into these low-skilled jobs. I'm the one who must juggle cleaning, painting, construction gigs, etc. When the day is almost over and I'm left with few hours of free internet, I look out for practical tech tools that I could use to be more productive and earn extra income. Hence, the adoption of ChatGPT.
Links:
1. https://awalkaday.art (website that I built for my photo project. assisted by AI in improving loading time).
2. https://twitter.com/awalkadayart (live feed for the project. find there public documentation and screenshots of experiments with AI as an "advisor" or "assistant").
Remind me, how do I access Bard?