Are you really sure that it all re-ran? No. You have to query your latest batch - which means you have to have arbitrary boundaries for when failures might have occurred like say, last 24 hours. If was a weekend, just query the whole database looking for any entry that's missing the "final" status success. Try not to run anything while you're doing this, unless your system is tolerant to the DB being locked for a full-table scan.
This is all very fragile and change-averse. Who would choose to run an ETL like this? I have worked for large companies who all end up with this rickety system. Every one, every time. God forbid you want to run tests with a mock set of serverless AWS services.
Our use of AWS Lambda for ETL is usually just one-off small processes that don't have dependencies.
As soon as you start getting into "This ETL depends on X, Y, and Z being run first" I think you're out of "small utility" territory.
Meanwhile, I don't have to do things like /make sure the ETL process is running/ or /clear out a bunch of logs when a disk inadvertently fills up/ or any of those other stupid last-century compute problems. It's just a far, far easier way of thinking about application development.
I can appreciate that. A lot more people than you might think, use Kafka or SQS because of various requirements (like that's all their infrastructure supports) and you end up with a massive "source" DB which is a duplication of all-time messages. Such is the reality of a larger byzantine organization. If it can't be audited by someone else, at their own pace, it's not approved.