Running Cross-Account Workflows with AWS Step Functions and Amazon API Gateway | Hacker News Reader

Running Cross-Account Workflows with AWS Step Functions and Amazon API Gateway

aws.amazon.com

35 pointskiyanwang13 comments

A little friendly warning, about setting orchestrated logic with AWS Step Functions or similar third party solutions, particularly when calling Lambda functions.

Don't forget to setup your AWS billing alarms using AWS Budgets plus AWS Budget Actions. A standard best practice in any case.

Also enforce timeouts at the State Machine level:

https://docs.aws.amazon.com/step-functions/latest/dg/amazon-...

The reason being that, for a Lambda function, and independently of your execution logic, the Lambda execution in any case terminates, ( depending on configuration), in the worst case after 15 min. So also terminates the corresponding billing for that execution also.

With Step Functions or similar orchestration, a logic error could expose you to create an infinite loop of continuous invocations and the corresponding never ending growing invoice accompanying it :-)

Good advice. But a small correction. Step Functions have a limit too. I think it’s 25,000 transitions per execution.

Is this just an elaborate way to trigger someone else’s AWS code using RBAC instead of an API token?

Seems like the “normal” way I’d go about this would be someone sets up a server, I ping it with token auth that’s been provided, thing happens. Now instead of doing auth through headers we require both parties to be on AWS to use their auth magic?

Guess it could be useful if both organizations are within the same company, so you can just enable app-to-app auth. No one is probably worried about cloud vendor compatibility in that case. No diss on it, but it seems kinda niche?

It’s on the AWS Compute blog, so they’re fairly willing to spot an initial condition of “obviously, this is all running on AWS; here’s how to something cross-account that you know how to do in a single account.”

I am yet to find a single use case where AWS step functions look appealing.

Am I right to think that if you don't want to do some "no code" weirdness, they're just not for you?

Because deploying a thing I can't store in source control looks like insanity to me.

You store the State Machine using Terraform, in source control.

I've used it before to do lightweight scheduling: you can include a timestamp and then have a "sleep until" step that waits until that timestamp before continuing. So I'd pass in timestamp + event, it'd wait and then execute the event by passing it to a lambda. For something only occasionally I think it was probably easier and less intrusive into the rest of the infrastructure than any alternative.

In terms of source control, the step function definition is just JSON (IIRC). I deployed it with CDK so the structure of the step function, its metadata, permissions etc. were all in source control. AWS docs like this tend to show things done in either the web UI or the CLI, but generally you can use an infrastructure-as-code solution instead.

Also the local testing experience is surprisingly good: there's a stepfunctions-local by Amazon and I think that's used by localstack too (which is what I used).

It is great for large, asynchronous, parallelised processing jobs. I’ve used it to integrate and orchestrate with numerous services within AWS ecosystem. We use source control for the step function JSON and deploy with terraform.

Why can't you do this directly without the API Gateway? https://docs.aws.amazon.com/step-functions/latest/dg/concept...

It's in another account. Essentially this entire article boils down to be able to allow ApiGateway invocation by specific StateMachineARN which is being kept when calling any external ApiGatway:

> "aws:SourceArn": ["<SourceAccountStateMachineARN>"]

Yeah, I understand that, but I don't understand why you can't execute a state machine in another account/region - assuming the IAM policy allows it. If this is a limitation of the startExectuion resource, I don't see it documented.

all that no code has a lot of code.

Can someone please provide an example of a large-scale distributed state machine?

I can’t think of any. Maybe a train / rail network?

Reply on news.ycombinator.com