Hacker News Reader

Comment by SketchySeaBeast | Hacker News Reader

I think people don't realize how many corner cases there are in coding. I think that the problems people have with accounting for all the Autopilot corner cases is going to be repeated with code generation as well.

View on HN

> I think people don't realize how many corner cases there are in coding

I would even say that one of the central characteristics of coding is corner case management.

GPT isn't "write my app for me" it's, "let me give you an Engligh description of what I want a snippet of code to do, in what language and with what libraries and output something close enough to correct that I can look up the documentation on a few library methods and round it out faster than writing it from scratch.

I think AI code generation will really amplify the importance of unit tests. If you can enumerate all the corner cases using tests then tell the AI to write code that does XYZ and passes the given tests, that's very powerful.

True, and honestly, if it turns out I can write unit tests all day I'll actually be a happy camper, I love thinking through problems and trying to account for issues, but I can guess how we will be encouraged to generate the unit tests in this new paradigm.

Is there a way to give ChatGPT a test suite and ask it for code that passes the suite?

Not as part of ChatGPT, you need to write/use a tool that takes a test and a prompt and re-runs the prompt until the output passes the test.

Sounds like someone should corner that market ASAP.

I’d prompt the AI to write the unit tests too. I know what the corner cases are. It doesn’t (yet).

That is true, but think about domains like Advertisements. Ads are currently far from perfect. A 5% click rate is a huge success. If 30% of the ads generated are low quality but 10% do really well it will be a huge benefit

Exactly, generating moderately decent advertising copy will be really helpful. Even just to start the creative process going.

"Here's a description of my product, give me 5 potential tag lines for it."

It's a great start for brain storming.

Even there the corner case failures are going to be amazing if we're throwing up AI generated Ads onto billboards indiscriminately. "So it seems our blacklist didn't include the word 'Hilter'...."

Ad text sounds like the exact opposite of a significant use case really; very little text to generate and high cost from the text being substandard. If 30% of the ads generated are low quality, you're wasting 30% of your ad budget, which is less efficient than paying a human to come up with higher average quality text (unless you're generating unique copy on the fly to evade spam detectors...). Just because current ads are far from perfect doesn't mean generated text isn't on average more banal and less accurate...

I think the main use case for pure LLMs will remain those already starting to be tackled by AI: low importance (or testable) information/opinion retrieval, low effort content generation, suggestion prompts and entertainment

Have you heard of A/B testing? With this you could do A-Z testing. And then let a human review the top 5 results of that A-Z testing. Instead of tasking a human with writing 26 ads

Better to test two good pieces of ad copy than 26 mediocre-to-entirely-inappropriate ones. The reason it's A/B testing rather than A/Z testing isn't because creating 26 variations at a time exhausts the potential of human creativity...

Are you suggesting that GTPChat can only generate "mediocre-to-entirely-inappropriate" ad text?

Yes.

(It'll use the same rhetorical question opening structure and cliche call to action close whether you're advertising products, countries, animals, products which exist or products which don't. This is just the most meta. The most noteworthy thing about the outputs apart from them being cliched and repetitive is its statements for why you should use ChatGPT's copywriting services are claims about ChatGPT which are obviously untrue...)

>> write an ad for ChatGPT copywriting services

> Are you tired of mediocre copy that fails to connect with your audience? ChatGPT Copywriting Services is here to help!

> Our team of expert copywriters knows how to craft compelling, persuasive copy that speaks to your target market and helps you stand out from the competition. From website content and blog posts to email campaigns and social media posts, we've got you covered.

> But it's not just about the words - we also understand the importance of SEO and how to create copy that helps your website rank higher in search results. Plus, with our flexible pricing and fast turnaround times, you can trust that we'll work with your budget and deadlines to deliver the results you need.

> So why wait? Contact ChatGPT Copywriting Services today and see how our expertise can help your business thrive. Whether you're a small startup or a large corporation, we have the skills and experience to help you succeed.

>> write an ad for ChatGPT copywriting services

> Are you looking for high-quality copywriting that engages your audience and helps your business stand out? Look no further than ChatGPT Copywriting Services!

> Our team of talented writers has a wealth of experience crafting compelling, persuasive copy for a wide range of industries. From website content and product descriptions to social media posts and email campaigns, we have the skills and expertise to help your business succeed.

> But it's not just about the words - we also understand the importance of SEO and how to craft copy that helps your website rank higher in search results.

> So why wait? Contact ChatGPT Copywriting Services today and see how our services can take your business to the next level. Whether you're a small startup or a large corporation, we have the experience and talent to help you reach your goals.

>> write an ad for ChatGPT copywriting services

> Looking for high-quality copywriting that will help your business stand out from the competition? ChatGPT Copywriting Services is here to help!

> Our team of talented writers is skilled at crafting compelling, persuasive copy that speaks to your target audience and helps you achieve your marketing goals. From website content and product descriptions to email campaigns and social media posts, we can create the perfect words to drive results.

> But it's not just about the words - we also understand the importance of SEO and how to craft copy that will help your website rank higher in search results.

> So why wait? Contact ChatGPT Copywriting Services today and see how our expertise can help your business succeed. Whether you're a small startup or a large corporation, we have the skills and experience to help you reach your goals. So, don't hesitate to reach out to us for all your copywriting needs.

That all seem pretty similar to the ads on the front page of google search results.

Just generate 200 of those and test them to see if any of them standout.

The Utility in these models is not that they are perfect or better than humans at all times, but that it is very cheap and to generate something of value across many domains.

But testing 200 scripts giving cliche-ridden and inaccurate descriptions of your product isn't very cheap, it's very expensive compared with asking somebody that actually understands your product to write one or two better scripts...

But having someone who is a bad writer but understands your product take a look at 200 generated copy lines and pick their favorites for A/B testing is much cheaper than having someone who understands your product and is a good writer writing ~20 lines and A/B testing those.

No it isn't. Copywriting starts very cheap, and getting a text-generator that doesn't understand your product to finally generate something which looks like it does isn't any less time consuming than writing words (especially short form copy). 200 iterations of generated copy stressing that the advantage of using ChatGPT for copywriting is its team's experience of SEO in slightly different ways are all equally useless to promote ChatGPT, the service which doesn't have a team of experienced writers and doesn't have experience in SEO.

And the really expensive bit of copywriting is the failed conversions. A LLM which when prompted to generate an ad for a particular market is trained to produce something which looks as similar to the competition and as much like an ad as possible isn't likely to generate the highest conversions....

Reply on news.ycombinator.com