Show HN: Lambda Complex: build apps that run in AWS Lambda and SQS | Hacker News Reader

Show HN: Lambda Complex: build apps that run in AWS Lambda and SQS

github.com

41 pointsexratione6 comments

I'm struggling to find a good balanced framework approach to AWS Lambda. I have a dozen of small lambda functions running, but I manage each one individually using just plain CLI commands or a simple Makefile. Sharing code is a pain. I can only find simple Makefile-like (grunt?) approaches which seems to be too simple, or "monster"-like approach like JAWS that do a gazillion things at the same time (seriously, have you tried the hello world of JAWS? The amount of things it creates on your AWS account is scary to say the least).

What I need is something where I can have a bigger project, export some services/api via handlers, and then some text file where I can "draw" my framework, connecting handlers to API gateway, input sources, etc.

The pain point for what you want there is the versioning and deployment side of things: how you update the application as atomically as possible.

I need to get Lambda Complex into real apps for a while in order to understand what is really, actually needed around the transition of the interface with other AWS resources on update of version. Currently I'm sure it's going to be a major pain to have to write some code to redirect the flow of events to a different queue, but that's still better in my mind than to try to manage versioning with a single queue or other source of events that is not included in the CloudFormation template or has to be managed as a special case during updates.

The other pain point is monitoring of the application state and clearing up after it. Lambda functions spawn infinite log groups in CloudWatch, and they don't get deleted without some sort of automation to go do that. It isn't clear to me what the best approach is at this point in time to attach monitoring and alerting to CloudWatch Logs. I'd almost rather leave them alone and primarily monitor via SQS queue attributes, leaving the CloudWatch logs for manual inspection.

We were in the middle of shifting our entire architecture to Lambda when all hell broke loose.

I had delayed Lambda function calls. S3 triggers for Lambda were taking hours to actually start the process, but the logs were showing the actions happening immediately. They weren't happening immediately. It was easy to see that the S3 files were not being moved for hours, but the logs were reporting the wrong time.

Days later, AWS us-east-1 went offline. Much of the internet went with it.

After it came online, everything was working well for me. We enjoyed near instant response times. Lambda was great, for a few days.

Then it started again. Hours of delay before the S3 events actually started the Lambda functions, and the logs once again lying.

We moved away from S3 triggers because it was unreliable. I really hope they fix the issues, but I'm not interested in relying on something that doesn't work, with zero help in the forums, and requires an expensive service contract just to report broken services.

was that during this: https://aws.amazon.com/message/5467D2/

That event was when we noticed the problem, but the problem continues, on and off, to this day.

What is more frustrating than anything is that we have to check manually because the logs are pretending like the events are happening immediately (timestamps), but they are not.

I liked this strategy and approach for Lambda, will test it out. I commented on JAWS documentation, I also think it's too hard to do something so simple. I prefer Gulp with Lambda till now, will have to test this one.

Reply on news.ycombinator.com