AWS Lambda is great solution for a lot of problems but one might hit certain issues when pushing it to its limits. Let’s look at some interesting observations about it which can help us to work better with it.
A user can invoke a Lambda function either synchronously or asynchronously and the limits applied to the function depend on how a function is invoked, among other reasons. Some interesting aspects for asynchronous invocations are:
Internal invocation queue
Lambda internally uses an invocation queue (one per function) to manage the asynchronous invocations. Once invoked, Lambda queues the event and then executes it at a concurrency level that is compatible with the account, region and user limits.
For a scenario where you have a bug in the invoked function and you want to stop it, these features can be problematic. Some workarounds are:
- Pre-step: Throttle the function.
One can throttle the function concurrency to
0, if necessary, to
Immediately stop the execution of the queued events.
- Contact AWS support to clear the invocation queue
This is the cleanest way to purge the invocation queue though AWS might roll out a way for customers to view and edit the invocation queue themselves. The downside is of course the turnaround time.
- Delete the function!
By deleting the queued function, a user can clear the invocation queue regardless of how many invocations are in the queue. It’s a quick and dirty way to get out of a messy situation but remember that all invocations that happen when the function is in deleted state are lost. Ensure you have a way to re-trigger your asynchronous functions.
- Just wait!
AWS deletes all queued events in the invocation queue after 4 days (unofficial information, no guarantees about it and it might vary depending upon your account specifics).
“Pause, Update and Continue”
If you find a bug in an asynchronously invoked function and you only care about the new invocations, an approach that could help is:
- Throttle the concurrency of the asynchronous function to
- Update the function code with a bug fix
- Increase the concurrency limit to
1or more and continue from where you paused
Lambda can retry a function execution for a number of reasons. Being aware of this is important for asynchronous invocations since the only feedback that you get is from CloudWatch (no synchronous responses!). Ensure that the side-effects of your functions are idempotent for best results.
Asynchronous invocations are a great way to quickly scale up the concurrency
level of the invocations and also, get around the execution time limitation of
300 seconds per function. An approach would look like this.
function_2invocation in turn invokes
Such a setup lets you invoke
x * y times very quickly.
e.g., let’s say you need to invoke
function_3 10000 times and time needed to
invoke this function is 10ms.
Total time =
10000 * 10ms =
1 * 10ms (
100 * 10ms (
function_2 invoked 100 times by
100 * 10ms (
function_3 invoked 100 times by every invocation of
Total time =
10ms + 1000ms + 1000ms =
The result of the fan-out method of invocation is two orders of magnitude faster and it works because Lambda functions support a high level of concurrency.
Certain caveats with this are:
- Ensure your concurrency limits are high enough for this or request for an increase in concurrency limits
function_3execution happens with very high concurrency. This limits that options that this function can perform to other highly concurrent applications. e.g., you might not be able to update a RDBMS database. Additionally, you may get throttled by applications that usually support concurrent access, but cannot support the sudden spike of traffic.
Speed of invocation
Time required to invoke a Lambda function asynchronously is in the neighborhood of 10ms to 25ms (tested with Golang. It might change with the programming language and might improve in the future).
Asynchronous invocation can be a powerful tool in a serverless programmer’s toolbelt. Be aware of the pitfalls and the performance considerations for best results.