My definition of serverless is that:
- You don't have full responsibility of the infrastructure, which is a good thing.
- Your resources should scale up and down automatically, based on the load.
- You only pay for what you actually use, so the consumption model.
- Glue code, connect services
- Back-end stream processing, event-driven architecture
So when I help out a team who is new to serverless they usually have run into an issue which is related to back pressure.
In your application this shows up as data not being processed fast enough as there is a bottleneck in your system.
For example: The serverless compute part of your application scales very well but a third party API or your data store does not scale in the same way. So you might experience throttling of your requests or you might even be DDossing your other services.
There are several ways to deal with this:
When dealing with third party APIs, always check what the rate limits are.
Some APIs have throttling and retry mechanisms in place and some even have a default behavior. For instance, when you connect to CosmosDB and use the CosmosClientBuilder you can specify the number of retries and the timeout between the retries.
If the client API does not have a built in mechanism and it based on HTTP you can inspect the Retry-After header in the response when the HTTP response is 429 (Too many requests). This header can either hold a date or a delay in seconds when the endpoint can be called again.
There are librabries to help you with building a resilient serverless API. I personally am a big fan of Durable Functions, where you write an orchestration, and activity functions can be retried automatically. The state of the orchestration is persisted to strorage which makes it so resilient.
If you're working with HTTP clients then Polly is also a fantastic library where you can specify policies to do retries, circuit breakers and so on.