> The workloads commonly executed on Async are those that do not require blocking an active user’s experience with a product and can be performed anywhere from a few seconds to several hours after a user’s action.
Apache Pulsar has been around for a while as a streaming system, and it does have a fairly significant Pulsar Functions layer which might make it a decent all in one candidate. (Pulsar in general feels like a much smarter lower-latency take on Kafka, allowing much more control/flexibility of where data gets backed/replicated/read from that makes it exceedingly more useful for "log/stream is the database" styles.) https://pulsar.apache.org/docs/3.1.x/functions-overview/
There's so many up and coming streaming things here. Apache Ballista. Netflix's Mantis. RisingWave. A million years ago there was SOA; a startup I was at used Apache Camel. Benthos. Others. Many of these will decidedly remain more streaming oriented, not encroach on FaaS space, which is fine, but I think we'll see overlap grow. Reductionist, but a function call is "run this code when a function name and parameters happen" versus maybe "run this code when an event type happens", which seems not so far apart to me.
But in reality, any managed queue or even a well structured DB table can be fine, depending on your scale.
I remember this being an issue a long time ago, where Lambda pretty much was a non starter for use cases where you're running 1000 independent tasks that each, on their own, is a small unit of work (just 1 or 2 database queries).
Long way of asking: is there a way to share a database connection between lambda functions or does each run need to re-establish / re-authenticate a new connection with the database?
https://www.jeremydaly.com/reuse-database-connections-aws-la...
It will get re-used across lambda invocations as long as the instance of it isn't killed which typically happens after a few minutes
This outlines creating the handler outside the function context to allow the client to be reused between invocations.