Hacker News Reader

Comment by ineptech | Hacker News Reader

It seems like we're pretty close to the point where LLMs will be able to create simple client apps. When anyone can make, say, an android note-taking app to their own specifications for free, it's not hard to imagine the impact on the paid note-taking apps in the store today. "Make me a game like Clash of Clans, but free, and with..." is probably only a few years further out.

One interesting question is: when people are using LLMs to generate software for their own use, what will they do for the server component of client-server use cases? Is "LLM, go register me an EC2 instance and configure it not to go over $20/mo, here's my credit card number" totally out of the question?

View on HN

Honestly, I think we're way way way off what you're suggesting about giving it inputs and getting specifications. This blog post shows chatgpt recreating things with very specific instructions, that it has seen before. The hard part of defining requirements is being specific in them, and that's what programmers do. It's pretty common in these threads to see people talking about chatGPT and friends being totally wrong and it not being aware of how wrong it is.

> ? Is "LLM, go register me an EC2 instance and configure it not to go over $20/mo, here's my credit card number" totally out of the question?

I suspect we are _way_ closer to that than to having it respond to "make me a game like clash of clans".

You’re right that it still needs a skilled person to ask the right prompts, and I don’t see that changing anytime soon

But if a few people asking the right prompts is all you need, what happens the to other 50-100+ people a game like Clash of Clans would normally employ?

I think ChatGPT is actually better at creating the right prompts than doing the answers to the prompts.

I don't think your intuition is ambitious enough. We have 12 year olds making roguelikes and platformers today, with templates and toolkits. Sure the initial LLM-made apps will suck, the initial everything sucks, but a couple years of progress on something this new will be considerable. We've barely started tuning LLMs for specific use cases!

> The hard part of defining requirements is being specific in them

True but I don't think you're accounting for iteration. You know the "idea guy" cliche? Today, if/when this "idea guy" hires a team of devs from India to make his million-dollar idea into reality, his inability to write good requirements will lead to a useless app, and the idea guy asking for changes (that are also poorly described) until he runs out of money, right?

Now imagine that his "dev team" works for free and can turn around each new set of requirements in minutes rather than months. How long will it take him to learn how to good acceptance criteria? A day? A week? It's hard but not that hard. And a million or two other "idea guys" are doing the same thing, and sharing notes with each other.

The fact that we have 12 year olds asset flipping from tutorials today shows the value of those asset flips. If chatGPT cannibalises that market, I will then be as scared of ChatGPT taking my work as I am of a 12 year old who can follow a unity tutorial right now.

> It's hard but not that hard.

It really is that hard. It's basically 90% of staff+ engineer's job.

I don't doubt that AI and ML in particular will absolutely change our industry, but as an engineer I see chatGPT as a tool like copilot, a debugger, or a linter. All those tools make me an order of magnitude more productive, but they're useless without the human making the decisions

Writing requirements is not usually done by staff+ engineers, is it? An awful lot of working software, the majority I'd guess, is built from requirements written by a non-technical PM who wasn't even in the software industry three years ago. I wonder if you're too good a programmer to have your thumb on the pulse on the kind of software bad programmers make? Because that's where the real disruption will come from. The question is not "will some bad programmers lose their jobs", it's "what will happen when the kind of simple software that used to be made by bad programmers becomes effectively free."

The working requirements of "when I click X, do Y" comes from PO's, sure. But the remaining implied requirements (what happens if there's an error in your app? How do you identify sensitive data that needs to be handled differently to "basic" data?) in my experience (working with teams of varying degrees of experience) are defined by engineering.

We're talking about being a couple of years away from ChatGPT being able to copy and paste from tutorials here, and with my understanding of AI (I've done a few ML projects in the last decade, but nothing that stuck), that's the "easy" part. The impressive part of things like Mindjourney is the synthesis, and we're not really seeing that or any signs that it's coming IMO.

> what happens if there's an error in your app? How do you identify sensitive data that needs to be handled differently to "basic" data?

Nothing happens, and you don't. That's what I meant by "bad software". I'm reminded of a story Clay Shirky told about AT&T analysts from the mid-90s trying to understand the web hosting business[0]:

> The ATT guys had correctly understood that the income from $20-a-month customers wouldn’t pay for good web hosting. What they hadn’t understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn’t very good. This, for the ATT guys, wasn’t depressing so much as confusing... For a century, ATT’s culture had prized - insisted on - quality of service; they ran their own power grid to keep the dial-tone humming during blackouts. ATT, like most organizations, could not be good at the thing it was good at and good at the opposite thing at the same time. The web hosting business, because it followed the “Simplicity first, quality later” model, didn’t just present a new market, it required new cultural imperatives.

0: https://gwern.net/doc/economics/2010-04-01-shirky-thecollaps... (shirky's old site has been down for some time)

how is this different than git cloning an already done hello world style app/using a blog article with a code sample?

You can ask the bot to flesh out the code and implement new features (roughly) in seconds. For example, if you have a notes app, you could ask the bot to write code that integrates it with existing calendar apps, and it'd take less than a minute to get you close.

> and it'd take less than a minute to get you close.

Have you actually tried this yourself? I have probably... 300+ ChatGPT chats/sessions past 30 days (aka, I actually use it).

I have 0 faith it would actually spit out anything error-free / working.

Yeah, I've had success, and it has either produced useful code or nearly useful code. I use it to refactor functions and write regexes, tedious stuff.

1. You don't have to know about the existence of such repos

2. You don't have to find a good blog article that works for your specific purpose

3. You can ask for changes to the code like "translate this code to python" "I want to use Azure instead of AWS" "add a database so I can save player scores" etc. and it will both write the code and explain what it did.

4. You can ask for elaboration if you don't understand something, it can describe code or it can further explain written text.

5. It can explain why you're getting an error message after you change something and suggest a way to fix it.

7. If you have a feature you want to add but don't know how you can just ask it and it will show you how to implement the feature and how to integrate it into the existing code.

There's a lot more, too much to list but every one of these is far more useful than just cloning a repo and following an article.

Tbh for clash of clans, if it can set up the server infrastructure etc, then that will be quite scary.

The scariest thing is that... I did use Chatgpt to set up an aws cloud formation stack for a lambda function, and it gave me the right file to sync.

So... Yeah, it's not that many steps to a "text to (web|mobile) app generator" solution.

Uh.. I will get to work.

I can Google a cloudformation stack for a lambda and get an answer [0] in about as long as it takes an LLM to generate one. That said, as a programmer I'm not paid to copy from Google or stack overflow, I'm paid to figure out if it's working or not. As an example, ask chatgpt to generate a lambda template with 16GB of memory as Nd see what it does.

Until we see chatgpt figuring out how to fix issues like that, I'm not worried.

[0] https://iamondemand.com/blog/how-to-deploy-your-lambda-funct...

It says this, is it ok? I can check when I get home

ChatGPT: Below is an AWS CloudFormation YAML template for creating an AWS Lambda function with 16GB of memory:

```yaml AWSTemplateFormatVersion: '2010-09-09' Description: 'AWS CloudFormation Lambda Function Template with 16GB Memory'

Resources: LambdaFunctionRole: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: - sts:AssumeRole Path: / Policies: - PolicyName: LambdaExecutionPolicy PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents Resource: 'arn:aws:logs:::*'

  LambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      FunctionName: MyLambdaFunction
      Runtime: python3.9
      Role: !GetAtt LambdaFunctionRole.Arn
      Handler: lambda_function.lambda_handler
      MemorySize: 16384
      Timeout: 300
      Code:
        ZipFile: |
          import json

          def lambda_handler(event, context):
              return {
                  'statusCode': 200,
                  'body': json.dumps('Hello from Lambda with 16GB memory!')
              }

```

This CloudFormation template creates an IAM role and a Lambda function with 16GB (16384MB) of memory. The Lambda function runtime is Python 3.9, but you can change this to another runtime if necessary. The code block contains a basic Python script for returning a simple message; you can replace this with your own script.

Yep! Except lambda limits are 10Gb [0],so this doesn't work.

If you ask ChatGPT to generate a template for a lambda with 16 vCPUs, it will generate a template that allows a max of 16 concurrent invocations, which again is not what you asked it for, and unless you _know_ what reserved concurrenct executions are, you won't know that's what it's generated.

[0] https://docs.aws.amazon.com/lambda/latest/operatorguide/comp....

I told it:

In here, it says that maximum memory is 10240mb, so that may not work:

Memory and computing power

Memory is the principal lever available to Lambda developers for controlling the performance of a function. You can configure the amount of memory allocated to a Lambda function, between 128 MB and 10,240 MB. The Lambda console defaults new functions to the smallest setting and many developers also choose 128 MB for their functions.

Its response:

ChatGPT: I apologize for the confusion earlier. You are correct that the maximum memory allocation for a Lambda function is 10,240 MB. Here is the revised CloudFormation YAML template with 10,240 MB memory allocation:

```yaml Resources: LambdaFunction: Type: AWS::Lambda::Function Properties: FunctionName: LambdaWith10GBMemory Handler: index.handler Runtime: nodejs14.x MemorySize: 10240 Timeout: 300 Role: !GetAtt 'LambdaRole.Arn' Code: S3Bucket: <S3_BUCKET_NAME> S3Key: <S3_KEY>

  LambdaRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: LambdaRoleWith10GBMemory
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: LambdaBasicExecution
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: arn:aws:logs:*:*:*

```

Replace <S3_BUCKET_NAME> and <S3_KEY> with the appropriate values for your S3 bucket and object key that contain your Lambda function code. You may also adjust Runtime, Handler, and Timeout as needed.

===

Then I said:

What can we do to be able to use 16gb memory?

Then it said:

ChatGPT: To use 16 GB memory in a serverless context, you will need to consider other options as AWS Lambda currently supports up to 10,240 MB. Two alternatives are AWS Fargate or running a container with Amazon ECS.

Here's an example of using AWS Fargate with a task definition for a container with 16 GB of memory:

1. Create an ECS cluster:

aws ecs create-cluster --cluster-name MyFargateCluster

2. Create a task definition with 16 GB of memory (you can adjust the values as needed). Save the JSON below to a file called fargate-task-definition.json:

{ "family": "FargateTaskWith16GBMemory", "taskRoleArn": "<YOUR_TASK_ROLE_ARN>", "executionRoleArn": "<YOUR_EXECUTION_ROLE_ARN>", "networkMode": "awsvpc", "containerDefinitions": [ { "name": "MyContainer", "image": "<YOUR_DOCKER_IMAGE>", "cpu": 4096, "memory": 16384, "memoryReservation": 16384, "essential": true, "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ] } ], "requiresCompatibilities": [ "FARGATE" ], "cpu": "4096", "memory": "16384" }

Replace <YOUR_TASK_ROLE_ARN>, <YOUR_EXECUTION_ROLE_ARN> and <YOUR_DOCKER_IMAGE> with appropriate values for your use case.

3. Register the task definition:

aws ecs register-task-definition --cli-input-json file://fargate-task-definition.json

4. Launch the Fargate task:

aws ecs run-task --cluster MyFargateCluster --launch-type FARGATE --task-definition FargateTaskWith16GBMemory --network-configuration "awsvpcConfiguration={subnets=[<YOUR_SUBNET_ID>],securityGroups=[<YOUR_SECURITY_GROUP_ID>]}"

Replace <YOUR_SUBNET_ID> and <YOUR_SECURITY_GROUP_ID> with the appropriate values for your VPC and security group. This will give you a Fargate container running with 16 GB of memory, and you can adjust the resources allocated to the container as needed.

====

I think that's quite a nice answer.

It's a nice answer if you know that the problem is that the lambda has a limit of 10GB. See the other example of asking it to specify vCPUs - that's just plain wrong, and unless you know how lambda works, it's not going to work for you.

I think this proves the point though, that it's closer to Copilot - it can generate a bunch of boilerplate, but you the programmer still need to understand what it's generating to deploy it. The only difference is that instead of opening vscode and typing LambdaFunction, I ask chatgpt in english, but I still need to specify. _that's_ the part we're not seeing movement on yet.

I would like to crank this prediction up a bit more. I think we are pretty close that it doesn't make sense to have simple client apps anymore. Why shouldn't the LLM do it all?

Let's say a simple single purpose web app such as Calendly. Why should I pay for such a service if I can just ask the LLM to create the entire app for me?

In TFA, the author let the LLM create an ics file. Why shouldn't the LLM also spawn a REST endpoint and serve stuff from it matching my prompt?

I agree, people thinking about using LLMs to make apps that they can then sell are missing the forest for the trees.

The more interesting question is, once non-technical people are able to auto-generate their own apps, what backend will they use and how will they handle auth? The easiest default option today is "cloud services, and 'log in with facebook' oauth" but we can hope for better...

ChatGPT,

Pls make website, all responsive like, w/ BIG pictures ooo, use my fav fonts, also fancy menus with whoosh on, load fast pls.

Thanks, Human

PS, no bugs :)

make me a website that looks like apple's website but for my pizza food cart

I was able to use GPT4 to write a small, self-contained JavaScript function and it worked okay. But writing out the function is painfully slow, particularly if you want to iterate. Which you’ll need to do because sometimes it introduces bugs. You can tell it to fix them and it can, but then you have to wait.

For this reason, I use gpt4 mostly just to start off new functions and then I edit them myself.

It would be awesome if gpt4 could be made to edit the code and therefore, it didn’t have to regenerate it from scratch every time.

The regeneration also chews up a lot of the token limit, so it forgets crucial parts earlier on in the conversation.

I've found that copying the result of one session to the next can work pretty well (like when you hit token* limits), especially if you have the bot include comments.

I've had luck with telling it to just rewrite a certain part of the code. You can copy and paste the relevant part and ask it to just rewrite that or you can direct it using natural language, you just have to be specific. If the chat is getting too long you can also copy just the important parts or the code so far into a new chat and start from there.

Pretty much everything you described could be done pretty easily already by anyone half inclined to do so.

If there was any point in doing it, people would’ve already done it 1000x, the LLM adds no value here.

Isn't this misunderstanding the point of an AI assistant? The idea is to have AI do the same things that people can do. The benefit is they do it a lot faster and cost a lot less and can work 24/7.

How long until I can make my own movie like Star Wars, but with my story line and my characters? Serious question, does anyone have has insights into this problem?

I want to do this for my kid - inject him as a character in kids stories and songs.

Probably not very long, look into ControlNET for video. https://www.youtube.com/watch?v=3FZuJdJGFfE

It's far from movie quality right now but there are a lot of people working on this sort of thing. People are also exploring text to 3D scene, model, and character generation pair that with text to animation which is also being worked on and we've at least taken the first steps towards that goal.

Reply on news.ycombinator.com