Margaret Atwood Reviews a Margaret Atwood Story by AI
thewalrus.ca
thewalrus.ca
While I wouldn't expect Atwood's conclusions to change too much by using GPT-4 instead, I think it's interesting that even the majority of educated people and journalists outside of tech don't seem to realize that the best model is at least 10x smarter than the free version of ChatGPT, which is what they seem to be using for all their prejudice-confirming "experiments".
They also always seem to assume that if the output from whatever prompt they came up with can't reach X quality bar, that means it can't be reached by anyone else either with a different prompting strategy.
Not trying to throw any shade toward Ms. Atwood, who is one of my favorite writers, and I'm also not claiming AI will be writing as well as her anytime soon... just pointing out that if we want to really measure where we're at on tasks like this one, a more rigorous approach is needed.
Citation needed. What does 10x smarter mean here? There’s an ongoing debate about whether the word “smart” even applies to a text prediction engine.
Overall it is as if GPT3.5 feels just like a clueless summarizer, but GPT4 intelligent interpreter and reasoner that I can trust.
Depending on which way you look at it, it could be 10x or 1000x the intelligence.
First, I've been using GPT to build an application for work for the past few months and anything but GPT-4 consistently produces less consistent and reliable output. Things like occasionally producing malformed JSON.
Second, I have a set of questions I use to evaluate models testing different capabilities and GPT-4 does much better than other models, particularly at coding tasks. There are some exceptions, for example, Bard has been able to do better on stating facts sometimes and Claude has done better at summarizing long text.
I'd love to have another model as good as GPT-4 to use but I haven't found one yet.
I mean... the content-free drivel they generate is more _polished_, possibly, though I'm not sure this is actually an improvement. What do you mean by 'smarter', here?
But I know AI is already being used to assist human writers, not just with boring emails and speeches, but creative works like articles and books.
Moreover, if AI ends up writing something decent, it won't be recognized as AI-written. And the human "author" probably won't be quick to reveal so; due to the controversy surrounding AI, and because then people would over-scrutinize it and just point out mistakes which even a human would make (or really minor opinionated things they call mistakes just to have a point).
Going back though, if AI ever does get to the point where it can replicate human talent, eventually we're going to know. If GPT-5 exists and is able to replicate human-quality writing, it's only a matter of time before someone reveals it, or a competitor catches up and then they reveal it.
In exactly the same way, sometimes I give ChatGPT a complete coding task and it can't do the job. But while I'm working on code I can get it to do certain things and it saves me a lot of time and sometimes comes up with very useful insights and things I was unaware of.
I'm sure authors (or anyone else whose job maps to "language processing") can use it similarly.
At one time he found, "son sues father" which came true as a headline few years later.
I still spend a lot of time polishing the final version, so the time savings are only about twenty percent.
I would say, not totally unlike a human memory. just GPT4 is limited to 8000 characters or something. Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.
this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations.
I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.
Think here we are just 'conceptualizing', 'spit-balling', 'blue-sky-discussion', 'wondering out loud', 'hypothesizing'. etc...
Your comment assumes a human can recall all (potentially incorrect) memories and use those to make some judgement, but fact of the matter is we don't.
I'd think something like this would be added to the models eventually. Of course, over-simplifying.
The Snows Came Early That Year
The snows came early that year, blanketing the prairies in white before the harvest was complete. I awoke to howling winds rattling the panes of my solitary cabin, the radio readout blinking ominously, batteries too weak to check the weather reports again. I shuffled outside in layers of wool and coveralls, squinting against the icy gusts, to find the chicken coop door ripped clean off. Feathers and blood painted the snow crimson. The four hens I had left were gone, with only scattered tracks and droplets leading into the tree line to indicate the predator's escape.
I cursed under my frosted breath, the bitter wind stealing away my words. Food had already been scarce, each winter harder than the one before. I shuttered the coop and turned back inside just as the power flickered and went out. Now there was no heat, no light, no way to cook. The solar panels would be useless under the heavy snows.
As I sat wrapped in furs, watching my breath turn to plumes of smoke in the gloom, I made a plan to walk to town when the blizzard passed. Perhaps barter some handicrafts for a few days of rations. The trek would be risky, but staying here held a different kind of peril. The food stocks in my locked pantry were dangerously low. There was no choice but to go.
The Katskil wolves would likely be on the prowl, their numbers multiplying as the animal population declined. Stay to the road, move quick, keep head down. Don't attract attention. Don't look too frail, too thin, too cold. The robotic guards at the checkpoints would be indifferent to my starvation, but opportunistic thieves might not. This was the world we inhabited now. Hold fast to what is yours. Trust no one. Every day another fight to survive. My grandmother's voice echoed in my mind...from her days before the collapse: "Don't lose hope, girl. It's always darkest before the dawn." But that was in another era, now lost to us. There were no more dawns ahead, only the howling dark.
I don't think that someone that wrote "you fit into me", would use the abundance of qualificatives that this text has. Furthermore there's nothing of Atwood's style that I can detect in it. The only interesting bit for me is the inverted assumption that the voice that speaks to us is male, but I feel like even that's too lame of a device for something a veteran author would do.
> you fit into me
> like a hook into an eye
> a fish hook
> an open eye
Here are some interesting results from literally feeding this article and prompting chatgpt to improve the poem and take into account her critique. They seem at least to my eyes a little better.
===========================================================
Upon these shores, where time seems still,
PIBO stands, guided by a passioned will.
With hands ungloved, their touch sincere,
They embrace nature, drawing it near.
Loving straightforwardly, without pretense or guise,
In every bird's song, a part of them lies.
In this dance, where boundaries fade and intertwine,
Your journey, dear bird, feels much like mine.
In the quiet of Pelee, where heartbeats merge with the sea's hum,
There lies a tale of dedication, of hands and feathers becoming one.
Upon Pelee's shores, where the migratory journey starts,
PIBO stands undeterred, with gentle, gloveless hands and hearts.
No insect repellant, for it might be toxic, they bear,
Yet with every sting and bite, their dedication's rare.
Warblers greet the dawn, singing their age-old song,
Yet amidst them, mosquitoes and chiggers throng.
With delicate touch, they band, note, and release,
Suffering in silence, so our joy in birdsong doesn't cease.
In the hum of nature and the dance of skies so grand,
Echoes the profound mission, of PIBO's ever-watchful band.It reads like a college fight song. I'm good with replacing college fight song writers with a computer.
Indirection, subtlety, and allusion are absolutely critical elements in poetry - maybe even fundamental.
There is not a hint of that in this autocompleted piece of dreck. It's more blatant than the terrible evangelical preaching-"poems" I used to see forwarded on AOL in my teens.
Plus like allmost all of its output, it appears to have the amount of profundity, aesthetic pleasure, insight and interest usually found in corporate boardrooms and marketing brochures. That is to say, vanilla dross.
hahah "Pod People"... is that what SV residents are now?
Is your name Roger? Sub-question: do you have any nieces or nephews?
https://www.theatlantic.com/ideas/archive/2023/07/godel-esch...
Oobabooga text-generation-webui for the server.
In the interface, use ExLlama for GPU inference (fast; for smaller models which fit in VRAM). Llama.cpp for large models (higher fidelity but slower), CPU+GPU.
13B parameter 4-bit quantized model (type 'GPTQ") can fit in a 12GB RTX 3060. 24GB card (e.g. a 3090) needed for 30B model on GPU. Something like 5-10 tokens/sec.
Can run 65 or 70B parameter models on CPU (e.g i7 12700) with 64GB RAM (also need decent GPU as above). Around 1 token/sec. These models are type "GGML" / "GGUF".
Long prompts take a long time for initial ingestion on CPU+GPU, much faster on GPU only.
Llama.cpp also apparently runs very well on Apple silicon, with the shared memory between CPU and GPU being well-suited.
I'm not native speaker and many weirdnesses of the text may go past me, but I can say that for me the commented texts (especially the 2nd one, about post apocalyptic Canada) are completely passable and much better that what I will be able ever to write.
Yes, it may be not a threat (yet) to professional, especially established author. But they will be good helpers for people like me, who can get suggestions, improvements and illustrations just for the price of my 4090 and time to tinker with models.
No gpt was used for writing this though.
I think that's more or less what Atwood was saying: that it's not yet good enough to replace talented professionals. I doubt she'd argue that it can do the other things you mentioned.
If only. That would make life so much better and easier.
I find that sentiment mildly offensive. I write a lot of code myself, but for every program I write, I use a hundred programs written by other people. If those programmers want to impress me, they will keep my needs as a user in mind, not their own egos.
> But they will be good helpers for people like me, who can get suggestions, improvements and illustrations just for the price of my 4090 and time to tinker with models.
This I agree with; for someone who may not be as gifted a writer, but still has something interesting to say, generative models could help with that. I just hope that people don’t lean on these models for generating ideas because if that story was any indication, that’ll just lead to a proliferation of boring, soulless works.
Fundamentally, literature is about communication with other people, living in another person’s mental world or understanding their unique perspective.
It’s not really clear to me what human value an LLM generated story has. It’s a statistically probable sequence of tokens generated from the distribution of internet-based language. It had no unique perspective, and conveys nothing about actual human experience. What do I learn from that? How is my life enriched?
LLMs are really cool for creative brainstorming and stuff like that, as a tool for inspiration, but I am baffled by the idea that anyone is interested in entire AI-generated works.
That you are nothing special. The computer is just doing what you do, putting one word after the last one. Picking each word more or less carefully, based on its own training and the audience's expectations.
It's doing it badly, for the moment. But would we mock a talking dog who stutters?
How is my life enriched?
Being able to see what's coming is helpful, more often than not.
Frankly that’s solipsistic, bordering on pure nihilism. When I read another person’s writing or talk to them, it enriches my life because it gives me a slice of their experience. When I read LLM output, it just definitionally can’t do that, no matter how plausible and semantically meaningful the words are. What is the purpose of literature for you? Just consuming a pleasing string of words?
That’s not to say they have no value, just that I can’t learn anything about another person’s experience of life through an LLM, because they aren’t people
Humans increase it, LLMs decrease it.
This matters because eventually in the future information complexity will be humankind's only valuable resource.
(So humans create value, LLMs destroy it.)
You might take a bit of time to learn what you're talking about before posting, or at least before forming permanent opinions on the subject. It's fascinating stuff, trust me... and I'm not that much farther along than you are, trust me on that as well.
What was the plot?
Which characters leapt off the page, so real you could swear you met them last year?