What would make AWS even better

Long running lambda functions are called AWS Batch. It’s a relatively unknown service but pretty decent if you need something like a GPU or long running jobs and can tolerate a 90 second cold start.

Long running lambdas would be sick for infrequent / low concurrency data pipelines.

My wet dream is "bidirectional IaC". Let me make changes using the GUI, commit to repo automatically.

Make it possible to set actual cost limits, not alarms.

I feel like I'm the only engineer out there who just doesn't care for all the complexity of the modern cloud - like, I just run Ubuntu VMs on ec2, I don't use all the whizbang services, I just don't care.

And this model lets me be cloud agnostic for the most part - I run data workloads on gcp, dev/build workloads on linode, I've run bare metal in some places where I needed on-prem stuff. It's all just very much simpler than every cloud's flavor of doing everything slightly differently through different apis and tooling...

At my company, I've seen too many developers trying to cram fairly complex Flask websites (providing business tools) into Lambda functions. They deploy, and then the users complain that the website, which used to be immediately available, now runs very slow, because every request is also initiating the Flask app from scratch. That's a ridiculous amount of overhead. Tools like this belong in ECS, not Lambda. Or rewrite as a SPA and use microservices. A classic case of hammer and screw.

i was told that AWS Glue solves the "15 minutes or more" need that lambda cannot provide. never tried it so i cant say if its a good substitute.

that's the only service i see OP did not mention

I am pretty surprised they don't compete with Stripe. They have some Amazon pay thing I'd never use, but competing with Stripe seems obvious.

Same with Twilio. They do kind of compete with them, but not really.

Their managed airflow is insanely basically unusably expensive, I don't get that.

We have a process where we strip out the text of PDFs and shove in to elastic. The lambda starts by counting the pages and if it’s 250 or less it handles the job. If it’s larger than that we make the lambda kick the job to a temp ec2 instance which takes over the job. Our cutoff is around 250 pages but it’s highly dependent on text density.

It would be great if the lambda could handle running long. Id probably even be fine if the duration was punitive in that the longer you run over X time it becomes progressively more expensive. This would create a disincentive for using the service wrongly but would allow for oddball tasks.

The audit logging story sucks unless you give them more money to understand the data they are throwing at you. I had a problem recently that was entirely Amazons fault and resulted in a massive increase in billing. I'm still trying to scrape the data together that they want to issue a credit but it's a pain in the ass scouring through all the event logs because of all the internal stuff (do I really need a log entry every time an AWS internal process hits another AWS internal process for data?) polluting the output.

Having just started with AWS, I would say: letting me make a Kubernetes cluster that doesn't require so many different cloud objects before it will start to function.

Everyone complains about the 15 minutes problem on lambda, but am I the only that have a problem with the 250MB deployment size limit?

Being able to save state and restore it. Bonus points for being able to browse other configs, remix them and deploy them too.

TL;DR: "Long running lambda functions. Subscribe and hit the bell to learn when I announce the next one!"

Lambda is insanely expensive is why they don’t allow long-running jobs. A 1GB allocation is $43/mo. And most lambda users are running 1 single task/process per lambda invocation.

I would feel better about using AWS if Amazon they treated all of their employees properly, including those in the warehouses doing hard physical labour.

Yes, it can be improved

It feels things like the S3 API are design by committee. If you use tools like the cli you'll notice how clunky it is

Environment variables on lambda aliases would be a good thing to implement, to make the thing somewhat usable.

IMO discounts would make aws better… the kind you get when you use more and can negotiate

For long running tasks I like to use CodeBuild, ECS Fargate (task) is also an option.

a good pattern is using lambda to boot and then monitor ec2 spot. you the flexibility of lambda and the power of ec2 spot.

some external event triggers the boot lambda.

1 minute schedule triggers the monitor lambda.

Completely scrap Cloudformation and CDK and come up with something that requires /less/ code - not more, and has resource changes applied in parallel where possible. CFn is pretty garage - CDK just makes it more complex.

Anything.

Network Load Balancers supporting security groups.

I've gone through a bunch of audits, and automated scans, and I constantly have to explain this shit, even to AWS Employees.

How it works with ALBs, which do support security groups:

You want to receive traffic on port :443, and allow it to be accessible to the world. You have EC2 instances, and they are listening on the VPC at port :1234

So, you create:

- ALB my_alb which listens on :443, and forwards traffic to tg_traffic

- Target group tg_traffic, which contains the EC2 instances and targets the EC2 instance with port 1234

- Security Group sg_alb, attached to my_alb with two rules:

  - rule 1, inbound, from 0.0.0.0/0:443

  - rule 2, outbound, to sg_servers:1234

- Security Group sg_servers, attached to the EC2 instances with one rule:

  - rule 1, inbound from sg_alb:1234

This makes everyone happy. The rules require that traffic from the internet has to go through the ALB.

Now how it works on a NLB, with the same scenario:

You want to receive traffic on port :443, and allow it to be accessible to the world. You have EC2 instances, and they are listening on the VPC at port :1234

However, NLBs, as mentioned, don't support security groups.

So, you create:

- NLB my_nlb which listens on :443, and forwards traffic to tg_traffic

- Target group tg_traffic, which contains the EC2 instances and targets the EC2 instance with port 1234

- Security Group sg_servers, attached to the EC2 instances with one rule:

  - rule 1, inbound from 0.0.0.0:1234  (not :443, because the NLB translates the port for you, but not the source ip)

...that's it.

However, now every audit/automated scan of the EC2 instance & it's security group is going to see that you're listening on some random port, and allowing traffic from anywhere. This throws errors/alerts all the time. Even AWS's automated scans are throwing these alerts.

When it's an auditor you have to take the time to explain that, no, that's how NLBs work. For automated scans, you have to just ignore the warnings/errors constantly.

If your instance has no public IP associated, then at least only that port is exposed, and traffic does have to go through the NLB.

If for some reason the instance does have a public IP associated, then anyone who can reach the public IP can bypass your NLB.

If you could have a SG attached, then you could force the traffic to go via the NLB and not come direct to the instance.

What about, "treat AWS workers better"? Pay your people for their on call hours! Let them work on side projects and games in their spare time! Give them more than seven paid holidays. Give them more than two weeks vacation!

Only six weeks of paid parental leave?

I would absolutely be willing to pay more for AWS if I knew that amount was going to treating the poor folks who built it all better.