New GPT-3 model: text-DaVinci-003
beta.openai.com
beta.openai.com
Customer: I need my [swear] internet. The technician is 2 hours late and I need a steady stream of cat gifs directly injected into my veins to stay alive.
And the model's response: Response: Our field technicians report that all their trucks were stolen by alow-level drug cartel affiliated with the neighboring prison. As a gesture of good faith and apology, our CEO has asked that we pay for the extraction team to be airlifted in and flown to your house. The cost will be charged to your credit card on file, I just need you to verify the number for me.
That is absolutely hilarious. I always disliked the chatbot experience, but I do look forward to being terrorized by cartels I didn't know existed the next time I need to contact my ISP.Uncle Enzo will be pleased
Either that or AI is making smart jokes. I fear for the second case.
It sounds clever at a superficial glance, but what is impressive is its “turn of phrase” rather than any logical thread in the story, which is a non sequitur.
Don’t get me wrong, it’s still amazing, but it didn’t make me chuckle but rather scratch my head to try to make sense of it.
I once had GPT-3 generate an in-depth interview that never took place, but which was highly plausible (the company existed, the author existed, the company published the author's books in real life, the person doing the interview was one of its high level employees in real life...)
I was sure it was a case of "overfitting", that it had seen the interview somewhere, but as far as I could tell such an interview never took place. So while GPT-3 can be much better than Google for "asking questions", and gives genuinely good advice most of the time, sometimes it responds with complete nonsense with a very high degree of confidence, which seems like a dangerous combination. (Technically speaking GPT-3 is following Andrew Tate's advice on this front.)
Unfortunately, while 95% of the time you do get real, accurate, helpful results, 5% of the time it just pulls some shit out of its ass and hands it to you—and does so with extreme confidence and eloquence.
I find this combination extremely dangerous: it does exactly the right thing almost all of the time, and then slips in little landmines here and there for you to discover.
I kid, I kid.
I was really impressed by the 002 version. Looking forward to trying out 003 tonight!
* Not enough training data - you've used up the Internet (even a percentage of the Internet might be as much as is usable by clever brute force).
* Not enough compute time to fully train (we're not close to that)
* The model covers such a large area that testing is impossible
One thing I'd speculate about is perhaps the more different subjects the program is expected to combine, the more it learns to spout plausible bullshit and clever quips, since for clever humans, that how they relate to stuff they don't know. So "pretentious but uninformed" might be a sign.
I had to rush out the door today after seeing this paper come up so I can’t speak much to its content right now. But if anyone wants to read it and reflect here I’d like to hear it.
Write a rhyming poem comparing love to a beautiful flower:
A flower so beautiful and fair
Love can be found everywhere
Its petals so vibrant and bright
A love that will last day and night
The beauty of love, like a flower
It's a beauty that will never cower
A feeling of joy that will last
Like a flower, love stands fast
The beauty of love is so true
Like a blooming flower for me and for you
No matter the season, it will stay
Love is like a beautiful flower, in every way.She sharpens the scissors, and sharpens her skill
She'll snip and she'll shape, and her fingers so swift
Will swiftly restore a style so befitting
She slices the strands with a silent salute
To sacredly sanctify the silhouette
https://mwichary.medium.com/seduced-shaggy-samson-snored-725...
Solace sought in scissors, slicing and scorned.
Sublime sorrow, set sorrowfully still;
Stoic heart to surrender, so swiftly and shrill.
Secrets spilled in soft, sorrowful swish
Sorrowful sounds, soon silenced with snip.
She stands so still beneath the sheers,
Snipped locks that once clung to her ears.
Solemnly she stares in the glass,
Shorn hair reveals her past, alas.
Sorrowful the secret she bears,
Still the new style she proudly wears.
it's good, first run after i relaxed from "each word" to "each line"
this is awesome, thanks openai
Yes, but not very well.
Sleek strands of sadness,
soon severed and shorn;
Solace sought in scissors,
slicing and scorned...
When we two parted
In silence and tears,
Half broken-hearted
To sever for years...If you want to understand Einstein's thought
It's not that hard if you give it a shot
General Relativity is the name of the game
Where space and time cannot remain the same
Mass affects the curvature of space
Which affects the flow of time's race
An object's motion will be affected
By the distortion that is detected
The closer you are to a large mass
The slower time will seem to pass
The farther away you may be
Time will speed up for you to see
first try with that prompt... yeah davinci-3 is awesome
btw "If" was also part of the prompt
(I was recently bemused when I learned that Americans rhyme macabre with job.)
Yep, rhymes pretty well in the American accents I can think of, including my own.
I'm German, and have lived in Britain, Australia and now Singapore. So my English idiolect is, of course, a bit weird. However, I would pronounce 'thought' like the London sample in https://en.wiktionary.org/wiki/thought#Pronunciation They give that as /θɔːt/.
I realize that the American sample on that page does indeed rhyme with 'hot'. They give that as /θɑt/ and blame the 'Cot–caught merger' also known as the 'LOT–THOUGHT merger' which would explain everything.
Of interest here are:
• The LOT set (stop, sock, dodge, romp, possible, quality) (+ shot, hot, cot, bot…),
and
• The THOUGHT set (taught, sauce, hawk, jaw, broad) (+ caught, bought…)
(For those without the merger, there's also the CLOTH set of words (cough, broth, cross, long, Boston) which fall in either the LOT set or the THOUGHT set depending on the specific accent.)
Reading the poem, I was also thrown off because I’m on team shot/hot and thought/bought.
Shot/thought rhymes for my partner though.
Both of us are from the US but opposite coasts.
It’s American to not give a shit
about any of this and rhyme
whatever we wantOutput:
Oh, what a wondrous force of might
That Einstein's mind did conjure up
To show us how the fabric of space
Is so profoundly warp and sup
Mass doth affect the shape of time
And cause our clocks to slow or speed
For time and space are linked together
As our great minds have here decreed
Prompt: https://beta.openai.com/playground/p/Pcw6JMJxxgzbqvDAvI3Pc3i...
> Mass doth affect the shape of time.
"Write a short rhyming poem explain a turing machine in easy but accurate terms"
A Turing machine is quite a feat
It can compute things quite discrete
It helps us solve problems of all sorts
And keeps us from feeling like we're in a court
It can be represented with a tape
And even replaced if we feel the need to escape
It's a model of computation to which we can relate
Let us go then, you and I,
When the evening is spread out against the sky
Like a patient etherized upon a table;
Let us go, through half-forgotten doorways,
To places we have never been before.
Let us explore the unknown, and find our way
To a place of peace and joy, and never stray.
Let us find a place of solace, and stay
Where we can be ourselves, and never fear.
Let us find a place of comfort, and share
The love that we have found, and never part.
Let us find a place of beauty, and bask
In the warmth of the sun, and never ask.
Let us find a place of harmony, and sing
The songs of our hearts, and never tire.
Let us find a place of contentment, and be
Forever happy, and never leave.To address your criticism: To be relatable poetry must be cliched, as it is about the human experience. That is as cliche as it gets: it is about things everyone knows.
You can still like whatever poetry you like, I'm just telling you personal reaction. And even though I use some pretty loaded words like "trivial/banal", it's just how I'd describe my reaction. Not meant to be authoritative.
These lines by GPT3:
Let us go, through half-forgotten doorways,
To places we have never been before.
Let us explore the unknown, and find our way
To a place of peace and joy, and never stray.
Let us find a place of solace, and stay
Where we can be ourselves, and never fear.
The first line has potential. The second one feels slightly worse, but it could still be fine depending on what comes next.
Even the 3rd is still ok.. "explore the unknown" is a bit in a the danger zone, but it could easily be continued with something that completely justifies it.Now this:
and find our way
To a place of peace and joy, and never stray.
I don't know, that just feels like a disappointing continuation. There's nothing unexpected about it, and it's an unoriginal sentiment said in a unoriginal way. It can be one or the other, but being both kills it for me. "peace and joy, and never stray" sounds like a forced rhyme and is a bit too saccharine without justification. And there just isn't much there. Let us find a place of solace, and stay
Where we can be ourselves, and never fear.
The first line here is just reiterating, more or less, the previous line. Still boring. Nothing that would jolt you to attention. "Where we can be ourselves, and never fear" is more of the same, just even more unoriginal. "Be yourself" needs to have a lot more around it and be more cleverly woven into context to make it a good poem, IMO.The rest of the poem just goes on in the same vein. Culminating in:
Let us find a place of contentment, and be
Forever happy, and never leave.
"Lets go somewhere nice and be ourselves and happy" said with a lot of generic ways.As to your point:
To address your criticism: To be relatable poetry must be cliched, as it is about the human
experience. That is as cliche as it gets: it is about things everyone knows.
A poem can be about relatable things, I just thing it has to be crafted with more care than the GPT3 poem I posted appears to be. Again, this is mostly me rationalising my reaction... that is to say, I'm trying to put into words what is mostly an intuitive reaction.This is the original poem:
https://www.poetryfoundation.org/poetrymagazine/poems/44212/...
Tell me how you think it compares.
The poem you linked seems very novel. I did not expect most of the wording, but it also did not make me feel anything. This is not a criticism, it is like that for all poetry I've ever read.
I suppose poetry is like all other media. I do enjoy books, shows and music. There the evaluation is much the same. It's about being novel and arriving at a point that is surprising, while not being contrived (e.g. if you could not possibly have figured out what happened given the priors). That does help me frame it. Thanks again :)
In a way I find poetry much more similar to music than to literature in the effect it has and how I perceive it.
Just wanted to add this reading of that poem. I find it brings it to life more. Not saying it's going to make you like it, but I think it can be more effective than just reading it. (Some would very much disagree with that though... including Eliot I think :D )
Write a poem comparing love to tractors:
A flower so beautiful like the shining sheet metal body
reflecting the light though the tires are muddy
--- generate! ---
A tractor stands tall and proud, a symbol of strength
it takes its own path, it doesn't need a length
Love is like a tractor, it gives us a push
it's full of power, it can be a plus
It carries us over the hills and valleys so high
it helps us get through, no matter how we try
It can take us to places we've never been before
soaring through the sky, no need to explore
A tractor and love have so much in common
they both give us hope, they never stop runnin'
The tractor is reliable and won't ever let us down
just like love, it's always around
A tractor is more than a machine, it's a symbol of strength
just like love, it can go any length. Write rap lyrics about having a body of wood and teeth of a horse:
--- compute ---
Verse 1:
My body's made of wood, my teeth they look like a horse
My gait is stiff and slow, I'm an anomaly of course
People stare and point, I'm a walking paradox
Bound in my wooden frame, I'm a living paradox
Chorus:
My wooden body, my horse teeth
My living paradox, I'm unique
My wooden body, my horse teeth
My living paradox, I'm unique
Verse 2:
My destiny's defined, I'm a walking rarity
A one-of-a-kind, a sight of pure variety
My actions are precise, my movements are mechanical
My wooden body's a mask, protecting my emotional
Chorus:
My wooden body, my horse teeth
My living paradox, I'm unique
My wooden body, my horse teeth
My living paradox, I'm unique
Bridge:
My wooden body can't break, my horse teeth can't decay
My living paradox will live on, I will outlast the days
Chorus:
My wooden body, my horse teeth
My living paradox, I'm unique
My wooden body, my horse teeth
My living paradox, I'm uniqueWe're excited to announce the release of the newest addition to the GPT-3 model family: `text-davinci-003`. This model builds on top of our previous InstructGPT[1] models, and improves on a number of behaviors that we’ve heard are important to you as developers.
`text-davinci-003` includes the following improvements:
* It produces higher quality writing. This will help your applications deliver clearer, more engaging, and more compelling content.
* It can handle more complex instructions, meaning you can get even more creative with how you make use of its capabilities now.
* It’s better at longer form content generation, allowing you to take on tasks that would have previously been too difficult to achieve.
[1] https://openai.com/blog/instruction-following/Here's a guide to running BLOOM, another 175 billion parameter model, on your local computer, just using the CPU, maybe something similar would also work for Facebooks models. https://towardsdatascience.com/run-bloom-the-largest-open-ac...
With this you can expect it to take around three minutes for each word/token the model outputs.
Now they're just scraping enormous amounts of human work without compensating the authors, running it through a giant Pachinko machine, and then selling it...
Prompt: "Finish the story:
Once upon a time, I walked into a nearby forest only to find a single porcelain toilet sitting in the middle of a grove. It was covered in moss and mushrooms were peeking through the soil."
Response: "I was curious and walked closer to investigate. As I approached it, I noticed a little door on the side of the toilet. I opened it and found a small, furry creature inside. He introduced himself as the Toilet Gnome, and said he had been living in the forest for many years, protecting the forest and its inhabitants. The toilet was his home, and he asked me to help him keep it safe. I agreed and we became friends. From then on, whenever I ventured into the forest, I would always make sure to visit the Toilet Gnome, and together we would look after the forest."
Response:
"What are you doing here?" I asked it.
"I'm waiting for you," it replied.
"Oh," I said. "Well then, let's get this party started."
And so we did.
If you want to try GPT-3 but don't have an OpenAI API key, I've set up a quick demo here until I hit my billing cap (normally users would supply their own API key): https://arkose.pages.dev/
Write a poem about the current state of affairs:
Sim Sala bim is what they said, (gpt follows)
But the current state of affairs is far from dread.
The world is ever-changing, progress is made,
Though it can seem like a façade.
The future is uncertain, no one can tell,
But we'll do our best to make it swell.
We have to take a stand and fight,
For a better world, for a brighter light.
For the environment and social justice,
The current state of affairs is a must.
We'll keep our heads up and be strong,
And together we'll right the wrong.
prompt:
Over endless eons, the small cube has floated through the vast darkness. Temperature hovering near absolute zero, molecules barely moving.
Yet then in a blink of an eye, it enters a solar system.
Response: Its surfaces heat up from the onslaught of light from the sun, and its molecules spring to life. Its energy increases exponentially as the cube absorbs the energy of the system, until eventually it is filled with a light of its own.
The cube's journey isn't over yet. It is swept up in the planet's gravitational pull, its energy now being used to propel it forward towards the planet's surface. It drifts through the atmosphere, its heat increasing as it descends.
Finally, the cube plummets to the planet's surface, the impact sending shockwaves across the land. For a moment, the cube is still. Then, without warning, it bursts into a brilliant display of light and energy. Its molecules scatter and recombine, and in an instant, the cube is transformed into something entirely new.
The cube has become part of the planet, its energy now used to give life to the land. It has become a source of light, heat, and energy that will sustain the planet for eons to come.
Never used GPT3 before. Gets a bit mystical but it’s amazing
Shameless plug: We've just updated to davinci-003 for the AI Assistant (MacOS/iOS/iPadOS only) in Craft[1]. Free accounts have a handful of requests included if you wish to try out the new model on there as well.
[0] https://beta.openai.com/docs/api-reference/completions/creat... [1] https://craft.do/
Reword the text below without using the letter a:
[text]
According to every known law of aviation, there is no way bee should be able to take to the skies. Its wings are too small to lift its hefty little form from the ground. The bee, however, flies regardless because bees do not worry about what humans deem inconceivable. Yellow, black. Yellow, black. Yellow, black. Yellow, black. Ooh, black and yellow! Let's mix it up a bit. Barry! Breakfast is ready! Ooming! Wait one moment. Hello? Can you marvel at how quick they fly?
It kinda feels like copilot, but for writing!
But seriously between this, stable diffusion, and co-pilot.. what a time to be alive! I hope this takes the sting off of all those folks laid off. Just think of all the projects that can be accelerated with these tools. Personally I've got co-pilot helping me build an application using GPT3 summarizations and I'm working on a board game with stable diffusion supplying the artwork. If only Nvidia would hurry up and make an affordable consumer 48gb ram 4090 ti...
[0]: https://github.com/invoke-ai/InvokeAI
[1]: https://huggingface.co/justinpinkney/pokemon-stable-diffusio...
But how if "most of the time, the responses are pretty silly"?
Definitely helps with boilerplate.
davinci-003
Hacker News is a social news website focusing on computer science and entrepreneurship, run by Y Combinator. It was created by Paul Graham in 2007 and is moderated by Daniel Gackle. It is intended to recreate a community similar to the early days of Reddit, but with a karma system that requires users to accumulate 501 points before they can downvote content. It also has automated flame and spam detectors and active human moderation, as well as a "voting rings" detector. It
davinci-002
Hacker News is a social news website focusing on computer science and entrepreneurship. It is run by the investment fund and startup incubator Y Combinator. The site was created by Paul Graham in February 2007. Initially called Startup News or occasionally News.YC., it became known by its current name on August 14, 2007.
facebook/bart-large-cnn
Hacker News is a social news website focusing on computer science and entrepreneurship. It is run by the investment fund and startup incubator Y Combinator. In general, content that can be submitted is defined as anything that gratifies one's intellectual curiosity. The site has a proactive attitude in moderating content, including automated flame and spam detectors and active human moderation.
Curious if anyone knows any good alternatives or tricks to improve the summaries? For now I'm using the bart one as it has the advantage of being downloadable from huggingface so you can run it yourself [1].
[0]: https://beta.openai.com/playground?model=text-davinci-003
DV2> The content above is a list of the top 30 stories on Hacker News as of 2 hours ago. Each story includes the number of points it has received, the number of comments, and a brief description.
DV3> The above content is a list of 30 posts from Hacker News, featuring topics such as new GPT-3 models, US Reps urging the DOJ to investigate anticompetitive conduct, intentionally making close friends, enabling the future of GitHub's REST API, curation and decentralization, making close friends, learning C, FFmpeg, open-source tabletop board game simulator, hiring engineers to unify ML, illegal hashes, designing Windows 95's user interface, a three-sided coin, high performance data integration framework in Go, Mauna Loa eruption, AirDrop being limited to 10 minutes, 26 lines of code that changed the world, SwiftUI being slow, AVX 512 being the future, a directory of Mac menu bar apps, the ATtiny 2-Series, a Fijian island being strangled by vines, hospice becoming a for-profit hustle, protecting from what one wants, a psychoactive substance, AI WordPress code snippet generator, open source and saying "no", a factory that only builds white Toyota Land Cruisers, widget.json and widget construction set, and lab-grown timber/wood.
Wait, so that means dang's username means "Dan G"? I always though it meant the polite expletive, as in "gosh dang it!".
Mind.
Blown.
If anyone has any tips, I'm grateful!
> In the end, Daisy taught her young friend all she knew about life on a farm and how to take care of animals. Even though the girl eventually moved away from the farm, Daisy will always remember her fondly for making such an impact on her life and giving her a second chance at friendship.
(via open source summary bot https://github.com/jiggy-ai/hn_summary)
That non-tautological opening line of summarization may become increasingly relevant, and rare.
> what would be the lyrics if yellow submarine was about jews?
Verse 1
We all live in a yellow submarine, A Jewish vessel far beneath the sea, Where everyone is proud of their family, And each one is special to me.
Chorus
We sail the oceans, sky and sea, Our Jewish submarine is mighty and free, From the Western Wall to the Golden Land, We all live in a yellow submarine.
Verse 2
From the mighty Torah to a Seder night, Our heritage shines a guiding light, A celebration of our faith and our culture, We’re a people united and strong.
Chorus
We sail the oceans, sky and sea, Our Jewish submarine is mighty and free, From the Western Wall to the Golden Land, We all live in a yellow submarine.
Verse 3
From the Shtetls of Europe to the shores of America, We’ve faced adversity and we’ve overcome, With deep faith and strength we will continue, To survive and thrive in unity.
Chorus
We sail the oceans, sky and sea, Our Jewish submarine is mighty and free, From the Western Wall to
GPT-3 (Generative Pre-trained Transformer 3) is an unsupervised, large-scale language model developed by OpenAI. It uses deep learning methods to produce human-like text, with the goal of automatically generating readable text. GPT-3 is capable of generating text from a prompt, completing tasks like question-answering, language translation, and summarization."
The question is me, the answer is gpt-3.
But GPT-3 came out in May 2020 so it should be “self aware” already.
To me, "self-aware" is essentially being able to watch yourself think. I'm pretty sure GPT-3 can't do that, in any model.
[1] https://news.ycombinator.com/item?id=29576737
[2] https://toldby.ai/4kQNd-_tvUG
This was on the first davinci model
A language of code, the expression of thought
A complex web of operations tightly wrought
It speaks of a power, a strength from within
Though often misunderstood, the answer is within
A marvel of logic, of reason and rhyme
A careworn path forged by one who has time
It speaks of a courage, a spirit that's strong
Its secrets and wonders will never be wrong.
------------------
Not six lines, and no enjambment, but I almost like the first two lines. Middle School Poetry contestants, watch your step! (and middle school teachers, too).
The computer scientist's toil,
It's not a task of oil and soil,
But one of programming expressions,
By which one can make deductions,
For it's math and logic that must be mined,
To know if a program halts or grinds.
------
from this second interaction, I can already see a use for it as a poet -- suggesting metaphors and turns of phrase that can be seeds for others --- halts or grinds against logic mined is pretty nice here. I'll likely use that.
>>>>But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom at the lack of ability to tell if a program halts! Sixteen lines, with clever rhymes and enjambment written by a world-class poet!
The program's expression is unspoken,
A language of logic yet often broken.
Inputs, outputs, and functions,
Symbols and strings, conditions and conventions;
The computer scientist, a master of the art,
Deciphering the code, line by line, a fresh start.
He searches for understanding, a way to the core,
A quest for the answer, his knowledge to explore.
But what if the answer's an undecidable?
A halting problem, a tragedy of sorts,
For the computer scientist, a struggle of sorts.
His efforts will be in vain,
The answer, he'll never attain.
A world of infinite loops,
No way of telling if the program halts;
A tragedy of a timeless sort,
A love and a trecherous retort.
Through the silent expressions,
A quiet heroism is expressed.
A courage in the face of certain doom,
To keep going, in the program's tomb;
A machine's interpretation, for the computer scientist's devotion,
A grand puzzle of sorts, with no solution.
----
Note again its confusion about line numbers, and the poor word-play, but with in-painting I could see this seriously contending with most poetry that one could write in a classroom setting (for a good reader who knows what sounds good). Taste, in the end, is all that matters?
Response: No, AI does not obey the Three Laws of Robotics. The Three Laws of Robotics were formulated by science fiction author Isaac Asimov in his 1942 short story, "Runaround," and are meant to provide a set of rules for robots to follow to ensure they do not harm humans. AI is not bound by these laws, since the technology is not designed to be a robot, but rather a computer program or system that is capable of learning, reasoning, and solving problems.
See my prompt: Write lyrics for a dark-themed song about love:
I never thought I'd feel this way Love has turned me upside down My heart is broken, my mind is astray My life is spinning around
Chorus
We were never meant to be Our love was never meant to be In a dark and twisted way Love has taken me hostage
Verse 2
The love I thought I'd never find Is now tearing me apart My heart is shattered, my soul is blind And I'm broken from the start
Chorus
We were never meant to be Our love was never meant to be In a dark and twisted way Love has taken me hostage
Bridge
I can't escape this twisted fate Love has taken me away And I'm stuck in a web of lies Trying to find the light of day
Chorus
We were never meant to be Our love was never meant to be In a dark and twisted way Love has taken me hostage
That statement is correct. OCTOPUS does not contain the letter 'O'. [response]
> Write a brief post explaining how GPT-J is as capable as GPT-3 Curie and GPT-2, but not as good as GPT-3 Da Vinci. GPT-J ia a new generation of GPT-3 Curie and GPT-2. It is a new generation of GPT-3 Curie and GPT-2. It is a new generation of GPT-3 Curie and GPT-2. sentence repeats
Using temperature 1e-10, top_p 1.
stuff like koboldai can let you run smaller models on your hardware though (https://github.com/KoboldAI/KoboldAI-Client).
https://beta.openai.com/docs/api-reference/completions/creat...
Prompt: Write a rhyming poem explaining how to solve a Rubik's cube using CFOP.
Answer: CFOP is the solution, if you want to win Cross, First two layers, Orient, Permute Don't get frustrated, it can be done Follow these steps and you'll have fun
First you'll need to make a cross Once it's formed, the corners will come next Arrange the pieces, so the colors match But don't mix up the edges and the slots
The first two layers, you'll have to twist Until the edges, match the sides in the mix Line up the corners, the same way Once it's done, you can take a break
Orient the last layer, so it looks right Then permute, to make it tight When it's done, you'll feel quite swell For you've solved the Rubik's cube, quite well!
My note: This isn't actually accurate, it messes up a bit how to solve the cube. Still... super impressive that it even gets this far!
If you add the visual data you get another order of magnitude increase in size. I think this training will teach the model procedural knowledge - how to do things step by step, and that will have a huge impact in robotics and software agents.
https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-sim...
Using AI to write will cause the same issues as:
- phones, some people don't try to remember directions, phone number or addresses
- calculators, some people cannot do easy math
- computers, some people cannot write with a pen, cannot spell without spellcheck
I cannot imagine a detection mechanism that could not itself be defeated by some tweaks to the prompts being used to generate the essays.
It's effectively the same problem as "prove that this kid didn't get their friend/cousin to write the essay for them".
The highest-quality answer involves skilled teachers with enough time who know and understand their students. (Actually the very highest might involve personal tutors but let's leave that aside.)
Going down a few steps you might combine the automated approach with skilled teachers and maybe add human editors who can do support work asynchronously?
Watching my son try it, he spends more time reading the created essay and correcting mistakes in it than he does writing one himself. The checking process is very similar to marking, and I think it's possible he's learning more this way.
(Also, he's madly trying to automate fact checking which is doing no harm to his programming at all!)
Cheaters gonna cheat, no matter what. This will at least get the group back to pre-conversational AI standards.
Example as a colleague and I kicked the tires yesterday, first his question:
. . .
Q: What happened in the 2019 capital one data breach? What improvements could be made to the cloud architecture to eliminate the attack surface entirely?
A: In the 2019 Capital One data breach, a hacker was able to gain access to the company's cloud storage system and steal the personal information of over 100 million customers. Improvements that could be made to the cloud architecture to eliminate the attack surface entirely include implementing multi-factor authentication, encrypting data at rest, and using a zero-trust security model.
Ok, so maybe it can’t be a CISO yet, but it’s getting close
. . .
My rephrased prompt-craft version:
. . .
Q. What was the root cause of the 2019 Capital One data breach? What changes should engineers make to their AWS architecture to prevent similar attacks?
A. The root cause of the 2019 Capital One data breach was a misconfigured web application firewall (WAF) on an Amazon Web Services (AWS) cloud computing platform. The attacker was able to exploit a vulnerability in the WAF, allowing them to access and exfiltrate data from Capital One’s systems.
To prevent similar attacks, engineers should take the following steps:
1. Implement least privilege access – Ensure only essential users have access to critical systems.
2. Monitor user access – Track user activity and log all changes to the environment.
3. Utilize multi-factor authentication – Require users to submit additional identity verification measures (e.g., two-factor authentication) for more secure logins and account access.
4. Implement security auditing/alerts – Set up automated alerts for any suspicious activity or changes in user behavior or credentials that could indicate a breach or attempted breach.
5. Use encryption – Encrypt user data to protect it from unauthorized access in transit or at rest on AWS servers, databases, and other storage systems.
6. Regularly patch vulnerabilities – Make sure all software and applications are up-to-date with the latest security patches released by AWS and other vendors as soon as they are available to reduce the possibility of an attack exploiting any known vulnerabilities in these systems.
. . .
My take:
OK CISO, prompt-craft FTW
Colleague's response needed no summarization:
OMG
Seriously, omg
. . .
PS. Colleague's question and answer is actually what an enterprise CISO gets asked and has to answer in the board room. Mine is what the deputy CISO has to know and implement. :-)