In the battle between Microsoft and Google, LLM is the weapon too deadly to use
theregister.com
theregister.com
https://www.google.com/search/howsearchworks/our-approach/
> Google’s mission is to organize the world's information
yeah but you only organized the info, but GPT-3/ChatGPT was the first of is kind to actually "understand" the information.
I think the “AI purists” need to make a lot more noise about the fact that nobody knows what LLMs are doing.
People are saying it's a "lossy jpeg" in a way that is disparaging and minimises how awesome LLMs actually are! They are a form of text compression that leverages the semantics of the text! Amazing!
In 2017 I could ask Google which episode of the Futurama was the one where the save the planet from a trash ball, and it could answer the question in natural language.
I suspect that Google X has had these, and other capabilities for a while even if they've never been shipped in any specific product. Or how about Duplex, where Google could call a restaurant to make a reservation, including negotiating times? That was also like 2017-2018.
These are only the public, polished, productized ideas. Google has bard, a response to chat gpt, probably only because they didn't realize that people actually wanted a chatbot. I think it was a product failure. How quickly they responded with something similar I think shows you this tech exists inside Google already.
Remember, OpenAI was created in part to help make AI accessible to people outside of Google/Microsoft. Who knows what's going on behind closed doors.
Which makes Microsoft relevant again because they own the lower level UI experience on a staggering percentage of screens.
I'd expect nothing but aggressively shoving clippy 2.0 into every surface Microsoft owns. And I expect they will be very light with the ads because they have the additional revenue streams to play a longer game on this if it hurts Google's market share.
Google doesn't really have the same luxury.
One does wonder what the analogy to "radioactive fallout" will be from LLMs. The obvious guesses (greater propaganda and internet spam) are likely shortsighted.
Similarly with Covid, the right time to start worrying (on a societal level) was before the disease becomes widespread. The tragedy of that means that if you take the correct action at the right time and succeed, you will always look like you were overreacting.
Politicians are going to be worried about perception of the appropriateness of their actions, not the correctness of those actions.
I think we're already seeing it. People don't understand that an LLM is just a text generator and add value to what the LLM says. See the article below for such an example.
https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...
As I was reading that, the image where "Eliza" describes methods of suicide was so bizarre to me. I thought what an absurd tone change bordering on dark comedy, and then caught myself, I was doing exactly that adding value and assumptions based on online human interaction to a "text generator".
It's going to be an interesting experience as more LLMs become humanized, by naming them, using 3D models or preset video animations, text to speech generation, more personalization to the prompt creator, etc. and that line becomes increasingly blurry for folks.
A good example with GPT is to watch it do complex math. There are simply too many permutations of math solutions for it to have ever memorized, and it can easily explain its process and the path it took to arrive at a solution.
Another good set of tests are ones around theory of mind, complex deduction problems and missing information, etc. A good source of information about the precise capabilities of GPT-4 is the Microsoft Sparks paper, which goes into a good number of tests MS researchers put the model to.
[0] https://www.youtube.com/watch?v=cP5zGh2fui0
[1] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
[2] https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
I feel like we need a new word for this gradient of intelligence. If you described the capabilities of an LLM to AI researcher in the early 1990s they would describe it as an AGI. I remember AI researchers using the term AI-complete to describe various tasks, by which they meant that any algorithm which could do X would be AI-complete and thus would also able to do all the other tasks that associated with general intelligence. The ability to hold a conversation and use human language like GPT-4 was often referred to the most obvious AI-complete task.
How do we know when we cross the threshold into AGI? If you have a robot library that can paraphrase any knowledge in any book in the library that probably isn't AGI, if they can make inferences on this knowledge to create new knowledge is that AGI? How complex and novel do those inferences have to be, before they are an AGI? What if we have a LLM that strongly surpasses human intellectual ability in every regard but one or two?
> The obvious guesses (greater propaganda and internet spam) are likely shortsighted.
Better censorship, better filtering of propaganda, internet/phone spam, and probably most importantly cheaper moderation of online spaces. If you control the LLM that filters/moderates your feeds, then you can build a reality tunnel for yourself and/or control your dosage of memetic toxins. If you do not control the LLM that filters/moderates your feeds, then hopefully it is run by a competent and transparent organization that has your best interests at heart. If it is not, you are in big trouble. Much like sedentary agriculture, once a human society adopts this technology it is the both the solution and cause of most problems.
The thing that worries me the most about LLM is the ability to create addictive media. Humans become addicted to gambling, porn, feeling strong emotions, exploitative freemium games, etc... Technologies can heighten the addictive nature of media e.g., a lot less humans were addicted to porn before the invention of the camera. Color TV is probably more addictive than black and white.
A slot machine is not a complex machine. It is a product of 20th Century industrial production: simple, easy to mass produce and designed to appeal to a model of the average human who visits a casino. What would slot machine look like if it was customized to each individual to be as addictive as possible and that was always learning and shaping that experience to maximize engagement and extract money? By how much would it increase the number of people who have addictions? How strong of a parasocial relationship could an LLM create, if it was designed to create and exploit parasocial relationships?
To me it felt like knowledge creation, but maybe someone had already published a similar scheme and then someone had published a similar attack and ChatGPT was pattern matching on that and adapting it to my setting. That seems very likely because while I was working in RSA, the fix to the attack is very similar to the use of a nonce in Schnorr signatures and for the same reason.
I feel like compressing knowledge and being able to answer questions about the knowledge, with the ability to explore “what if” differences seems more like it suggests real understanding. And that’s what these LLMs can do.
And again, folks who profess to know for certain an LLM can't in any way be sentient are leaning pretty far over their skis. It's unlikely given how they work, but not impossible.
What I would like to know (and what few if any of the whining "Journalists" are investigating), is:
Exactly what biased filtering is being performed:
1) On the input datasets used in various LLM training, and
2) On the "guard rails" enforced on LLM output (ie. the target of the various jail-breaks that prevent LLM's true outputs being perverted by the LLM's owner)
LLMs trained, restricted and run by these huge, clearly biased organizations are inherently untrustworthy, in my opinion.
Unfortunately, no "Investigative Journalists" are interested in providing useful insights in this critical area.
Please, all tech-spectators and commentators, let's take 6 weeks to gather facts, put things in perspective, check your local library for paper books about nuclear physics, read some Hubert Dreyfus, and resume publishing when it can be done in a responsible and fact-based way.
Anyone who tells you that these models are "just glorified text generators" is flat out wrong and hasn't bothered to do their homework. And anyone who claims they "know how it works under the hood" is making claims that all of the true experts have notably carefully avoided making.
It exhibits the weaknesses of that mode of thought in humans; over-confidence, generating nonsense, etc. People have thoughts, make gestures that indicate they had the correct thought, pass it off for lexing and transmission to language bits of the brain, then process can go off the rails, and they say something different than what they think, and we know they hd the right thought by the gesture they made. We also don't know what we'll say until we say it; I can believe it quite likely that we're also kind of building nearly one word at a time with a statistical model.
I'm actually saddened that people don't recognize in themselves that this thing occurring in themselves is not real thought or intelligence even without something like GPT4 around.
The main reason I don't think it's AGI is because I don't think it's GI in humans, but I think it's doing something pretty similar to one thing we do.
What homework.. if i may ask a dumb question?
My very limited understanding was such that this is a glorified text generator - however, it seems what is possible with text generation is allowing unexpected levels of competence and utility. To be clear, i agree with you that it's functionality is impressive and deep. However i had figured one area of research is in the very premise of "How good can LLMs without intelligence be?".
Is that wrong in your view?
(again, i'm not making a statement. I know next to nothing in this space. I just try to reach a layman's understanding on this subject and i use GPT4 daily)
1. Actually hands on testing the AI to verify that it can't do the things you claim or believe it can't do.
2. Review of the current literature where these LLMs have actually been tested rigorously for various emergent capabilities.
So a highly intelligent AI that produces text is glorified, whatever that means...
after this LLM breakthrough a lot of smart people started questioning what intelligence really is.
Maybe closer, but not close. For one thing, these things don't even continuously think. Would you call a human "conscious" if it only existed as soon as a question had been asked of it, lived for the sole purpose of responding, and went braindead immediately after answering?
A more serious omission is a lack of continuous inner dialog. Bing has the #inner-monologue tag but all of the AI's language based thought happens out in the open for the most part, and it doesn't have any time to process or ponder what's been said.
Of course that can be remedied somewhat by putting the AI in "ponder-bot" mode, where you can tell it to write out it's thoughts privately while you "pause". Both Bing and GPT will ponder if you ask them to and write an inner monologue if you give it time and ask it to shield it's dialog with some privacy.
It's interesting what Bing or GPT will reveal about their thoughts when in "ponder-bot" mode. Bing (at least in my trials) will only tell you generally what it hid from you in its private thoughts but the times I've tried to pry Bing ends the chat, and GPT will usually reveal them if you ask.
Again, saying they are not 'glorified text generators' is not a claim at all. They are glorified text generators, and of course they are more interesting than previous text generators, the question you are avoiding here is the actually significant one, which you demure on and try to lead us onto unfounded conclusions based on other people also not being willing to stick their neck out and make any claims.
Much of the reporting is self-serving, with reporters trying to protect their livelihood. Among others, LLMs outright replace journalists, especially those reporting on science and technology. Why pay a human money to understand and report on something when ChatGPT can generate a digest for pennies on the dollar?
I find LLMs fascinating, and do not perceive them as an existential threat on par with nuclear warheads. Rather I see it as a liberating mechanism. Llama.cpp lets me run a Googlesque search on my local hardware, no connectivity required. That's incredibly exciting!
But, that leaves a lot of space for "proper" journalism: gathering facts on the ground, interviewing relevant people, assessing impact, and so on.
I'm sure many journalists do this, but I would argue that science journalists also need to:
- consult other, independent sources with knowledge of the field. These parties may be able to highlight weaknesses or limitations of the work, framing assumptions that the work is dependent on etc.
- consulting broader background sources to provide context for the work, and potential relevance.
A science journalist who merely extracts from the research institution's PR is exactly as bad as a business reporter who merely extracts from a company's PR is exactly as bad as a political reporter who merely extracts from a politician's grandstanding. All of them are bad journalists who are merely tools of some other organization's public relations effort.
It seems to me, and I have experience in the alternative weekly newspaper world (which is long dead as a business model, but as a model of journalism, I feel one that we would do well to resurrect in some form) would prefer if all writers took on a more supervisory role, directing AI to do all of the investigatory work of showing up at the records office and sifting through city council transcripts. Then the human writer would do what humans do well and work those personal connections that they have much more time to do now that the AI is doing all the tedious grunt work.
I understand that people are frightened and in a bit of a panic, but these generative AI could do a vast amount to bring back truth to journalism, if we do this correctly. Will we? <shrug>
much of "journalism" is just propaganda for the rich. I'd rather read made up stuff than directed, curated, misinformation intended to hurt me.
I don't think it replaces real journalists. Real journalism just got much easier to do & analyze research. I will gladly pay a human money who understands what they're reporting on, can gather sources of quality data & use a tool like ChatGPT to help discover & summarize the data.
Journalism that is quality over quantity will increase in value, especially the kind that helps filter out fearmongering & overusing the 7 deadly sins. I don't want or need daily news. I want quality & value my time.
And we desperately need it. Part of why everything is getting more corrupt is that we don't have newspapers funding journos to go and dig up the dirt.
How can you report on wrongdoings, when the one doing wrong sponsors the publication.
A challenge going forward would be to figure out how one can fund journalism in a way that is separate from motive and bias of those funding.
State funded means state gets the last say more or less. Corporate funding means journalism is inherently pro-corporate.
Local news(tv and paper) is no longer profitable which means we lose the most independent investigative journalism.
The only way forward is for the readers to pay for it. But we've trained everyone that the news is free - it's going to be a hard task to undo that training and get people to pay for their news again.
What do you think of this? http://arxiv.org/abs/2303.16200
Thoughts experiments [1] have been a very important catalyst of human thought and yes, of scientific progress itself. If the opinion in the paper is based on sound reasoning and logic, I would argue it deserves to be considered seriously, even if it does not contain any empirical data.