Artist feeds childhood diary into GPT-3 to have a chat with herself
nwn.blogs.com
nwn.blogs.com
It's fun, but not so different from a conversation you could have with a psychic who is good at cold reading. It'd be more interesting to see what would have happened if GPT-3 was actually fine-tuned with all the journal entries, I think.
I do agree a lot with this conclusion: "This is the stuff I think that has the most interesting ramifications: more broadly, more immersive human / computer interface loops, from conversation with virtual therapists to in-game interactions for virtual worlds, given there is user input, AI could be used to train highly customizable responses or generate unique storylines per use."
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-Tac-Toe."
"Why is the net wired randomly?" inquired Minsky.
"I do not want it to have any preconceptions of how to play."
At this, Minsky shut his eyes. Sussman asked his teacher, "Why do you close your eyes?"
"So that the room will be empty."
At that momment, Sussman was enlightened.
thus there is not actually a connection between the neural net being randomly wired and it not having any preconceptions as to how to play, it just seems like that to Sussman.
I have never found these zen koan things very interesting though. Also I think there is much more likely a connection between a neural net being randomly wired and not having preconceptions than closing of eyes emptying out rooms.
https://twitter.com/TeamYouTube/status/1337950347231440899?t...
"Why is the startup sound so loud?" asked Minsky.
"Because they don't use their own products" I replied.
Minsky began scribbling in his notebook at a furious pace.
"What are you writing?" I asked of Minsky.
"Just another obituary for an undead product" replied Minsky.
At that moment, I became enlightened. I realized Gmail was undead, and YouTube was on its way to becoming a zombie too!
"So Google is purgatory?" I blurted out, flustered.
"As much as a slap is one hand clapping!" replied Minsky.
I instantly became enlightened and Ram Dass appeared, as if from the heavens. I knew what my mission now was. To ridicule Google on Hacker News in front of their employees so they reverse this beyond stupid decision some idiot in a position of power made. ;-)
/slap
goldenkey slaps YouTube PM around a bit with a large trout
The Minsky story is a comment on the ridiculousness of the idea that you can train a model on your journal to "avoid bias" and "accurately simulate" what you were like as a child.
[^1]: https://twitter.com/michellehuang42/status/15970055048693719...
Wait, certainly that’s not even remotely close enough to enough data to train a “past self” AI on. Even if we assume that by “characters” she meant words that’s just not very much text.
But yes, I agree. I'm not sure how much the journal entries really contributed other than specific turns of phrase and general topics of discussion.
I am not familiar with the OpenAI workflow, but KoboldAI is a community open source implementation to interface with different AI language models locally or online. There are links to premade notebooks of open sourced models you can run different on Google Collab. [1] I think finetuning models offline is still too expensive, but there is some progress and is possible via collab.
KoboldAI has a chat mode setting where it properly generates and continues a conversation between two people. The models I ran were the smaller ones, but there's larger models available on huggingface like FB's OPT 30B. Even the smaller models I could fit into 8gb vram were coherent enough to be impressive.
I am surprised I haven't seen more of is a chat bot based on writes who have passed away. I imagine an Aristotle chat bot would be pretty interesting and avoid issues of copyright.
Combine that with being an extreme data hoarder and having full backups of basically all my computers 2000 -> now on my NAS and I've got almost everything…
If I saw this a year ago, there is no way that I would have believed it was real. There have been too many "AI Generated" scripts/books/etc that turned out to be heavily edited by the author. This one feels a little too perfect, considering the author states this is largely what came out of the model on their first pass.
Today, I lean towards thinking this is real. Having used copilot and Midjourney for the last few months, I know that my "what is real" radar is completely off. Honestly, if you told me that many of the things I have generated via AI were actually a VERY fast fiverr worker, I might believe you.
What a time to be alive!
GPT-3 is not a person. The biggest risk with new ai tech is that it will learn to win us over, seeming like people, when there is nothing happening on the inside but number crunching. Earning empathy despite having no feelings.
(The above was generated by GPT-3.)
(No it wasn't.)
https://news.ycombinator.com/from?site=twitter.com/michelleh...
> in 2021, the average American spent only two hours and 45 minutes a week with close friends (a 58 percent decline relative to 2010-2013).
I understand some people might feel uncomfortable about "AI chatbots" (about their inaccuracy, or "fakeness"). However I can't help to think that the future of humankind will be plagued by artificial entities that we will interact with as we do with other humans. It might be in the form of simple ELIZA models, some ML artifact or full-on conscious intelligences.
I suspect there will be huge benefits to virtual companions to help with this crisis of loneliness that afflict us. In any case I hope humans will evaluate these technologies by the value they derive from them.
But what kind of world are we going to end up with when this happens. What is the result of giving everyone chatbot friends who are all perfectly agreeable. And who probably slip in product placement/ads.
Imagine the company changing policy and you being locked out of your virtual companion? That should never happen.
I was wondering exactly how she entered her presumably hand written notes from childhood into the GPT-3 playground.
Did she just retype all of them? Did she hand pick certain passages?
I ask because I have a feeling GPT-3 would return roughly similar results without the journals.
Isn't 'one' shot' or 'few shot' prompting a kind of training?
Isn't the result, purely from the point of view of the interacting user, functionally a chatbot?
I think that "chat bot" is less objectionable, but it presents a bit of a misleading picture to a casual reader. I think people ascribe a lot more agency to a "chat bot" than they would to a "next word predictor".
Isn't GPT-3 already capable of acting like a chatbot out of the box? The article implies that it was trained to be like a younger Michelle from the journal entries, but all the responses presented seem to be utterly generic. Talk about dreams and hardships but not anything concrete, no mention of friends and family, locations or activities. I don't know enough about GPT-3 to say if the training affected word choice, but it doesn't seem to have any discernible depth at all.
I hate this complete and utter misrepresentation of facts.
GPT is not an AI. It does not talk. It certainly does not understand.
I wish AI researchers would spend some effort to educate the public instead of bathing in all the attention.
But there is also a broader absence (destruction?) of our classical understanding of thought and other domains such as classical objective reality and the platonic domain of ideas.
Not yet sure if this is some weird post modernist view or intentional dummying down, maybe two sides of the same coin?
But at least from what I can observe there is little push to teach people how to think.
For example, "What Happened to Make You Anxious?" by Jaime Castillo proposes a similar technique for addressing anxiety. The technique goes like this:
Think of a traumatic moment in your past, where you wish you could go back in time and comfort or offer yourself advice. Then imagine yourself stepping into the frame of your memory and giving your younger self the support you needed in that moment (but lacked). This might sound woo-woo, but the efficacy is documented.
Sure, GPT-generated text isn't actually equivalent to talking to yourself - but you can see the utility for therapeutic applications like this.
This is how the Internal Family Systems psychology technique deals with past trauma and it has been around since the early 1980s. No AI required. To go that deep though, it helps to have a therapist trained in the technique to guide the process.
But health care costs in the US are extraordinarily steep. Mental health care is often out-of-network, if it's covered at all.
I have a hunch that market demand for AI-guided "mental health wellness exercises" will out pace regulations or ethics. Calm would be the best example of this kind of demand. I wonder if they're talking about this internally, and what line is being drawn.
Seeing someone once a week without insurance is on the order of 3,000$ to 7,000$ per year depending on area. Not trivial, but something most people could budget for even if they seek treatment for much of their life.
It’s of course possible to spend more than that, especially in the short term. However, the major costs tend to be people being unable to work or take care of themselves rather than blockbuster drugs, emergency care, or surgery.
I just Googled the median US rent ($1,771). Based on that, if you frame the expense as an added 2 - 4 months rent per year, I'm guessing there are a lot of people forgoing therapy because the immediate cost is too high.
I wish I could choose where my taxes were allocated. I'd subsidize cost of therapy.
To really drive this home, one of my own memories is my mom (ashamed, embarrassed, depressed) asking to borrow money from me to pay our bills. I was 9. She had issues, I inherited a few, and I've only recently been able to drop 5-10 dimes on therapy each year to sort it out. Money well spent!
But I wonder how my mom would've fared if even an AI facsimile of therapy were available to her cheap, instead of other cheap coping mechanisms. Churches and drugs are the cheapest coping tools available in the US.
Let me put it this way: It's called AI, Artificial intelligence. Are trees inteligent?
Because they are a clump of cells that manage to achieve amazing results. growing to be some of the largest living things on the planet. Extracting nutrients and transporting them 20 or more meters above ground, where they are used to harvest energy from the sun, all the while producing offspring every year and fighting off predators.
I don't think many people would call them "natural intelligence".
The I admit that I maybe apply a narrow definition of "intelligence", but I think the core concept is one of "understanding". And we are not even close to al ML algo actually understanding anything.
And this is the problem. It masks the inherent shortcommings of ML. People are delivered the impression that applications that use ML actually do what they are expected to do, because like a person, you train them and then they understand their job and do it.
This is NOT HOW THAT WORKS. The ML algo does not understand that it is asked to identify oncoming traffic. It does not know that it is looking a cancer cells that will soon kill someone.
And even worse, we humans who make these things, are 100% unable to understand the models we create. We can feed them data, and compare the result. But that is it. There is no real way to understand how it works in detail.
They get used anyway. With predictable results. see Tesla Autopilot for a prominent example.
Also, I call them AI researchers for the same reason I call Nuclear Fusion researchers that, not because they are doing it, but because they are researching it.
If your job is an engineer or scientist, the term "AI" is basically a synonym for "unrealistic executive expectations." That's a super triggering and stressful situation to be in, especially early career.
I've actually had a CEO describe expected output of my ML team as "magic AI shit" - I bet you can imagine the team's reaction and tone. I'm reading the same strong emotions and frustration here.
The good news is you can always course-correct expectations with communication. I've come to love talking about AI with people who are only somewhat technical, because their wildest dreams are sometimes totally do-able with some duct tape and fine-tuning.
AI researchers do in fact educate the public. [1] They face serious pushback from obscurantists like Hofstadter [2] and Marcus who refuse to commit to any definition and instead go by hunches about what architecture or data representation can support "understanding" and what certain behaviors indicate.
1. https://www.economist.com/by-invitation/2022/06/09/artificia...
2. https://www.economist.com/by-invitation/2022/06/09/artificia...
The way I and many others use the term "AI", is based on the understanding that AI is an umbrella term that includes modern deep learning as well as the broader field of machine learning, and also the other pre-machine-learning types of AI such as symbolic AI, expert systems etc.
If the stuff running on Lisp Machines was referred to as AI, I don't see why GPT-3 can't be labelled as such.
I'd consider these language models AI if they initiated conversations themselves or used conversation as part of a plan to achieve some objective. But so far, game bots are more deserving of the title than these trendy language models and painting generators, IMO.
Take an assertion like X implies Y. Unfortunately, due to the imprecision of language, X may have multiple definitions, e.g. D1 and D2, such that D1 implies Y but D2 does not. The fallacy of uncharitable interpretation is to take a definition for X such that the stated premise does not hold, even though there exists another definition where the premise would hold.
In this case, large ML algorithms are commonly called AI. Even the company that made GPT-3 is called OpenAI. Even you, yourself referred to them as "AI researchers." What do "AI researchers" produce if not AI? Finally, "talk" can colloquially mean to use words in an prompt-response pattern.
She's not "having a conversation with her younger self" and this is incredibly disingenuous and misleading, per @mavu's point. More importantly, it doesn't really educate the uneducated or move the technology forward in any meaningful way. It's pure garbage clickbait nonsense. I regret giving them the impression.
1) No explicit content (that could be plagiarized or combined) is actually stored by GPT3: only statistical relationships between tokens are retained. You can't point at a specific subset of weights and say "see... this is where phrase/idea X is stored".
2) GPT3 makes no specific claim to be generating "original, own work" (which would be required for something to be considered "plagiarism")
When people ask GPT-3 to write a rhyming poem, you see plenty of examples of GPT-3 poems starting with "There once was a cat named Pat..." This is an extremely common first line of a limerick, found anywhere from here[1] to here[2]. I'm sure those "statistical relationships" are very strong; is it plagiarism? I'll leave it up to you to decide that, but I'm willing to bet that with enough finagling you can probably get it to spit out phrases from Moby Dick.
[1] https://www.google.com/books/edition/Selected_Vintage_Poetry...
[2] https://sites.google.com/site/rainydaypoems/poems-submitted-...
Knowing that a "rhyming poem" is likely to start with a specific token (or set of tokens) does not exactly constitute "plagiarism", the same way that writing a poem that starts with "There once was a cat named Pat..." is not "plagiarism" by itself: it is just adhering to expected convention/norms of a specific literary format or genre.
Is using the basic-ass I-V-VI-IV chord progression in music "plagiarism", since it has been (and is) used by countless other people before?
> I'm sure those "statistical relationships" are very strong; is it plagiarism? I'll leave it up to you to decide that
Well... my claim is that it is clearly not plagiarism (and I gave specific arguments to support my claim). If you are not interested in arguing (which is fine), then I assume you accept that your characterization of what GPT3 does as being "plagiarism" is (at the very least) overly simplistic (i.e., just as simplistic as claiming that GPT-3 is sentient or actually intelligent).
> but I'm willing to bet that with enough finagling you can probably get it to spit out phrases from Moby Dick.
If GPT-3 (or most humans, for that matter) are asked to complete the phrase "To be or not to..." and decide that the word "be" is the most likely/reasonable completion, does it mean that GPT-3 (or any human, for that matter) is "plagiarizing" Shakespeare? Or does it simply mean that they are trying to address your question/problem to the best of their capabilities (and that they probably have read a passage or two of Shakespeare before, or someone paraphrasing Shakespeare)? In other words, just because you can force GPT-3 to output a specific copyrighted work (or an excerpt of it) still doesn't mean that what GPT-3 is doing should be characterized as "plagiarism".
Again, for something to technically count as "plagiarism", it is required that someone (i.e., not a computer program) try to pass off (incorrectly) something as original, own work, which does not seem to be the case here. That was my main point.
EDIT: if you want to be derisive of things like GPT-3, while still being accurate, it makes more sense to say things like "it is simply imitating" or "has no actual creativity" (which seem defensible to me) than things like "it is literally plagiarizing and copying what it saw before" (which seems much less defensible/accurate).
This is not at all what's happening here. Complete red herring.
> If GPT-3 (or most humans, for that matter) are asked to complete the phrase "To be or not to..." and decide that the word "be" is the most likely/reasonable completion, does it mean that GPT-3 (or any human, for that matter) is "plagiarizing" Shakespeare?
The short answer is yes (imho), but let me put it this way: does GPT-3 know that when it's regurgitating "to be or not to be" it's actually regurgitating Shakespeare? My argument is that no, it does not know, precisely because it thinks this just happens to be a very strong statistical correlation of stringing words together. When, in fact, it's a very famous phrase by a very famous person. So, in a way, it's "accidentally" plagiarizing, but plagiarizing nonetheless. Like if, for whatever reason, I had heard the phrase "it was the best of times, it was the worst of times" somewhere, but couldn't remember where, my ignorance doesn't preclude me from technically plagiarizing Charles Dickens if I blatantly reused the phrase without attribution.
> things like "it is literally plagiarizing and copying what it saw before" (which seems much less defensible/accurate).
This is literally what it's doing, though, under the guise of "statistical correlation." In fact, I've read reports of people using GPT-3-adjacent models that needed to add specific filtering out of training data.
Source: https://en.wikipedia.org/wiki/Plagiarism
The same way that you disagree when someone stretches the meaning of the work "talk" to encompass what GPT-3 does, I also disagree when you try to stretch the meaning of the word "plagiarism" to encompass what GPT-3 does (and I've explained exactly why: GPT-3 generates sequences of tokens, but makes no specific claim about the originality of the generated sequences of tokens).
We can agree to disagree, if you can't accept that "plagiarism" literally involves more than just "copyright infringement" or "replicating someone else's work from statistical correlations" or anything along those lines: it must also involve fraud or some other form of misrepresentation.
Either way, have a nice day.
This is your chance to become a patron of the arts! Pay to follow the concept through and realise the vision.