Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 22 Next »

The Framework proposed in this space (Alex Xu) is applied to propose a design : Getting started - a framework to propose...

On this page.

Introduction

The benefits of using an API rate limiter :

  1. Prevent resource starvation caused by Denial of Service (DoS) attack, either intentional or unintentional, by blocking the excess calls.

  2. Reduce cost by limiting servers and allocating more resources to high priority APIs. Important when we use paid third party APIs.

  3. Prevent servers from being overloaded by filtering out excess requests.

Part 1 - Understand the problem and establish design scope

Type of rate limiter ? client-side or server-side ? in a middleware ?

Server-Side or Middleware

Rules of throttle ?

Different sets of throttle rules (IP, user ID, etc.)

Scale of the system ? Small, Medium, big company

Big company

In a distributed environment ?

Yes.

Implemented as a separate service or in application code ?

No matter.

Inform user who are throttled?

Yes.

Additional requirements to consider

  • limit excessive requests

  • low latency - not slow down http response time

  • limit the use of memory as possible

  • can be shared across servers and processes

  • exception handling - show clear exceptions to users

  • high fault tolerance - if rate limiter goes offline, it does not affect the system

Part 2 - High level design

Client-side implementation is not the right option because client is not a reliable place : client requests can be forged by malicious actors.

Where to put the rate limiter ?

Choice of implementation is a middleware : API Gateway is a fully managed service supporting rate limiting, SSL termination, authentication, IP whitelisting, etc.

How To choose the right implementation ?

Middleware vs Server-Side (API Servers) ? Use a guideline :

  • Evaluate the current technology stack;

  • Identify the rate limiting algorithm - on server-side, we have full control of algo.

  • If Microservice Architecture and including an API gateway (performing tasks like authentication, IP whitelisting, etc.)

Regarding the algo, we need a counter to keep track of how many requests are sent from the same user, IP Address, etc.

Part 3 - Design deep dive

To deep dive, we need to manage rules and other components (MQ, Cache, …). See example rules in the page - References & Glossary for rate limiter.

  • If the request is not rate limited, it is forwarded to API servers.

  • If there are too many request according to the rules (from cache), the rate limiter return 429 error code to the user and the request is dropped or forwarded to a queue.

Part 4 - Pros & Cons

Pros

  • Help to control the resources services consume, impose limits on the rate;

  • Appropriate for large-scale repetitive automated tasks (batch processing,…)

Cons

  • Demanding to implement in a distributed environment;

  • Synchronize the rate limiters in a distributed environment.

  • No labels