Rate Limiting for AWS

Introduction:

AWS API Gateway is a fully managed service that allows developers to create, deploy, and manage secure APIs at any scale. It acts as a front door for applications to access data, business logic, or functionality hosted on AWS services like Lambda, EC2, or any external endpoint. API Gateway provides powerful features, including request throttling, rate limiting, and security mechanisms, making it ideal for building robust and scalable APIs.

Rate limiting is a critical aspect of API management. It helps protect APIs from abuse, ensures fair usage among consumers, and prevents service overload. AWS API Gateway supports configurable rate limiting and burst limits, which can be adjusted to meet the requirements of different workloads. In this POC, we will implement rate limiting for an API Gateway that connects to a Lambda function and explore how to test and monitor it effectively.

Step 1: Create a Basic Lambda Function

We will start by creating a simple Lambda function in JavaScript that returns a "Hello World" message.

Steps:

Navigate to the AWS Management Console then search for Lambda.
Click on "Create Function.” → Choose "Author from scratch.” → Enter a function name, e.g., HelloWorldFunction → Select the runtime as Node.js 18.x → Click "Create Function.”
Write the Lambda Code: In the "Function code" section, replace the default code with the following:

exports.handler = async (event) => {
    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello World!'),
    };
    return response;
};

Deploy the Function: Click "Deploy" to save the changes.

Step 2: Create an API Gateway for the Lambda Function

The next step is to create an API Gateway for the above created Lambda Function, “HelloWorldFunction”

Steps:

Navigate to the API Gateway Console
Create a REST API → ****Click on "Create API" and choose "REST API → Select "New API" and enter a name, e.g., HelloWorldAPI → Click "Create API.”
Add a Resource and Method → Under the Resources section, create a new resource (e.g., /hello) → Add a GET method to the resource → Choose "Lambda Function" as the integration type → Enter the name of your Lambda function (e.g., HelloWorldFunction) → Save the configuration.
Deploy the API → Click on "Actions" and choose "Deploy API.” → Create a new deployment stage, e.g., dev → Note the "Invoke URL" for testing.
Test the API → Use the Invoke URL in a browser or a tool like Postman to confirm it returns "Hello World!”

We can see that the Invoke URL of the API Gateway (dev stage) is able to invoke the Lambda Function at ‘/hello’ endpoint.

Step 3: Understanding Rate Limiting and Burst Limits

When managing APIs, it's essential to control how much traffic they can handle to ensure stability, prevent abuse, and optimize resource usage. AWS API Gateway provides two critical features to achieve this: Rate Limits and Burst Limits.

What is Rate Limiting?

Rate limiting is the process of controlling the number of requests an API can process per second. It defines a steady rate at which the API can handle incoming requests, ensuring the system remains responsive and doesn't become overwhelmed during high-traffic periods. For instance, if the rate limit is set to 100 requests per second, the API will process up to 100 requests in any given second, rejecting additional requests during that time frame with a 429 Too Many Requests response.

What is a Burst Limit?

Burst limits complement rate limits by allowing short-term spikes in traffic. They permit the API to handle a sudden surge of requests beyond the defined rate limit temporarily. This flexibility ensures that brief, unexpected traffic spikes don't result in immediate throttling, providing a smoother user experience. For example, if a burst limit of 50 is configured, the API can process up to 50 extra requests momentarily, even if the rate limit is exceeded.

AWS Default Limits

Out of the box, AWS API Gateway has generous default limits to support most workloads:

Rate Limit: 10,000 requests per second.
Burst Limit: 5,000 requests.

These default values are shared across all APIs in your account within a given AWS Region.

However, you can customize these limits for individual APIs or stages to meet specific requirements. For instance, you might want to set lower limits for a development stage to control costs and higher limits for production.

Why Are Rate and Burst Limits Important?

Prevent Service Overload: Protects backend services from excessive traffic that can cause downtime or degraded performance.
Ensure Fair Usage: Maintains equitable access to APIs, especially in multi-tenant environments.
Cost Control: Helps manage costs by restricting the number of requests processed.
User Experience: Prevents sudden failures during traffic spikes by leveraging burst limits.

This example will help you in understanding Rate Limit v/s Burst Limit clearly:

Step 4: Configure Rate Limiting in API Gateway

Steps:

Navigate to the API Gateway Stage Settings → In the API Gateway Console, select your API → Go to "Stages" and click on the dev stage
Set Throttling Limits → Under "Default Method Throttling," set:
- Rate Limit: e.g., 2 requests per second.
- Burst Limit: e.g., 1 requests.

Step 5: Testing Rate Limits

To test rate limiting, we can use a custom JavaScript script to programmatically send requests to the API. Below is an example script:

const axios = require('axios');  // Use 'npm install axios'

// Replace with your API Gateway Invoke URL
const invokeUrl = "<Invoke_URL>";

const queryApi = async () => {
    let successCount = 0;
    let tooManyRequestsCount = 0;
    let totalRequests = 200;

    // Function to handle each request
    const sendRequest = (requestNumber) => {
        axios.get(invokeUrl)
            .then((response) => {
                if (response.status === 200) {
                    console.log(`Request ${requestNumber}: Status 200 OK`);
                    successCount++;
                }
            })
            .catch((error) => {
                if (error.response && error.response.status === 429) {
                    console.log(`Request ${requestNumber}: Status 429 Too Many Requests`);
                    tooManyRequestsCount++;
                } else {
                    console.error(`Request ${requestNumber}: Error - ${error.message}`);
                }
            })
            .finally(() => {
                // After the last request, print the summary
                if (requestNumber === totalRequests) {
                    console.log(`\\nSummary:`);
                    console.log(`Total Successful Requests (200): ${successCount}`);
                    console.log(`Total Too Many Requests (429): ${tooManyRequestsCount}`);
                }
            });
    };

    // Loop to send requests with a 5ms delay
    for (let i = 1; i <= totalRequests; i++) {
        setTimeout(() => {
            sendRequest(i);
        }, i * 300); // i * 5 ensures 300ms delay between requests
    }
};

// Run the script
queryApi();

Replace <Invoke_URL> with your actual API Gateway Invoke URL. This script sends 200 requests to the API with a delay of 300 milliseconds (0.3 seconds) between each request. It logs the number of successful requests and those that were throttled.

Steps to Execute the Script:

Save the Script → Save the script to a file, e.g., testRateLimit.js.
Install Axios → Navigate to the directory where the file is saved and install the required dependency using the command ‘npm install axios’.
Run the Script → Execute the script using Node.js

node testRateLimit.js

Observe the Output:
- The terminal will display the status of each request in real time. Successful requests will show Status 200 OK, while throttled requests will show Status 429 Too Many Requests.
- After all requests are completed, the script will print a summary of successful and throttled requests.

If you prefer using command-line tools, you can test the rate limits using a curl command:

for i in {1..50}; do curl -X GET "<Invoke_URL>" & sleep 0.001; done

Replace <Invoke_URL> with your API Gateway Invoke URL. This command simulates sending 50 requests to the API, observing how the rate and burst limits are enforced.

Step 6: Create a CloudWatch Alarm for 429 Errors

Steps:

Navigate to the API Gateway Console:
- In the AWS Management Console, go to API Gateway and select the API you've created (e.g., HelloWorldAPI).
- Under Stages, select the stage (dev) where you've set the throttling limits.
Enable CloudWatch Logs for the Stage:
- Go to the Logs/Tracing tab and click on Edit.
- Check the Enable CloudWatch Logs option to enable logging for the API.
- For Log level, select INFO (or ERROR if you prefer to capture errors specifically).
- Enable Detailed CloudWatch Metrics to ensure your API Gateway sends detailed metrics.
- Save the changes.
  
  This will automatically create a Log Group for you in CloudWatch.
Create a Metric Filter to Capture 429 Errors:
- Select the above log group created in the previous step.
- Click Create Metric Filter.
- In the Filter Pattern, use the following pattern to capture only the 429 errors:
```
  "Method completed with status: 429"
```
- This filter pattern ensures that only 429 Too Many Requests errors are captured. Other 4xx errors (like 400 Bad Request) will not trigger the metric filter.
- Click Next.
Assign a Metric:
- Under Metric Name, enter a name (e.g., 429Errors).
- Under Filter Name, enter a descriptive name (e.g., 429ErrorFilter).
- Select a namespace (e.g., Api-Gateway).
- Set a metric value to 1 (each occurrence of a 429 error will count as 1).
- Click Create Filter to apply the metric.
- Before proceeding with creating CloudWatch Alarm, execute the script again so that the custom metric (429Errors) gets showed up in the custom namespace (429ErrorFilter)
Create the CloudWatch Alarm:
- After the metric filter is created, go to CloudWatch > Alarms.
- Click Create Alarm.
- Select the metric 429Errors under the namespace you've chosen (e.g., API-Gateway).
- Set the condition to trigger the alarm when the metric exceeds a threshold. For example, if you want to trigger the alarm when there are more than 5 429 errors in 5 minutes, set:
  - Statistic: SampleCount
  - Threshold type: Static
  - Whenever the metric: is Greater than
  - Threshold value: 10
  - For a period of: 10 seconds
Create SNS Topic for the Alarm
- Next, create an SNS Topic for the above created alarm.
- Create a name for the topic and then specify the email endpoint you wish to keep (e.g., devops@bimaplan.co)
- Click on ‘Create Topic’
- Click on Next.
- Add the name and description for the alarm.
- Click on Next → Create Alarm.
Execute the script and observe the results
- As the last step, execute the load testing script and watch the alarm ring!

Conclusion

This proof of concept demonstrated rate limiting and burst control in AWS API Gateway to manage traffic and prevent overload. By configuring these limits, we ensured fair usage, protected backend services, and optimized resources. Using a Lambda function with API Gateway showcased real-world application, while testing and CloudWatch monitoring reinforced their importance in maintaining a stable API ecosystem.

Rate Limiting for AWS - API Gateway

Introduction:

Step 1: Create a Basic Lambda Function

Steps:

Step 2: Create an API Gateway for the Lambda Function

Steps:

Step 3: Understanding Rate Limiting and Burst Limits

What is Rate Limiting?

What is a Burst Limit?

AWS Default Limits

Why Are Rate and Burst Limits Important?

Step 4: Configure Rate Limiting in API Gateway

Steps:

Step 5: Testing Rate Limits

Steps to Execute the Script:

Step 6: Create a CloudWatch Alarm for 429 Errors

Steps:

Conclusion