Kong Advanced Practice
The last chapter gave you the vocabulary. This one is about the three things a gateway is actually judged on once it is in production: authentication, rate limiting and transformation. Each of them has a plugin, each plugin has settings that interact with the others, and the interactions are where most production incidents come from.
It closes with the unglamorous half of the job - keeping development, staging and production consistent - because a gateway whose configuration differs between environments is a gateway that will fail in the one environment where it matters.
By the end of this chapter you will be able to configure JWT and OAuth 2.0 validation, design a rate limit that is shared across a fleet, rewrite requests before they reach an upstream, and promote the same configuration from staging to production without editing it by hand.
The JWT plugin verifies a signed token and, critically, ties it to a Consumer so that the rest of the gateway can make decisions about the caller.
_format_version: "3.0"
consumers:
- username: mobile-app
jwt_secrets:
- algorithm: HS256
secret: 4f8c2a1b9d6e3f7a0c5b8d2e9f1a3c6d
plugins:
- name: jwt
config:
header_names:
- authorization
uri_param_names:
- jwt
claims_to_verify:
- exp
key_claim_name: iss
anonymous: null
The key_claim_name field is the one that confuses people. It names the claim Kong reads to find which consumer’s secret to verify the token with. The default is iss, and Kong looks in the token payload first and then in the header. If your tokens identify the caller with a different claim - sub, or a custom cid - you must set key_claim_name to match, or every token will be rejected with Invalid JWT.
claims_to_verify takes registered claims from RFC 7519 and accepts exp and nbf. Adding exp makes Kong reject expired tokens; if you omit it, an expired token is accepted as long as the signature is valid, which is rarely what you want.
anonymous is a deliberate escape hatch. Set it to a consumer’s username and any request that fails JWT validation is treated as that anonymous consumer instead of being rejected. This is how you run a plugin set that has “public” and “authenticated” access without maintaining two routes - but it means invalid tokens are silently downgraded, so use it only when the downstream policy genuinely permits anonymous access.
A rejected request returns 401 with a message that names the failure:
curl -i http://localhost:8000/orders -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJtb2JpbGUtYXBwIn0.bad"
# HTTP/1.1 401 Unauthorized
# {"message":"Invalid JWT"}
If you use asymmetric algorithms - RS256, ES256, EdDSA - the credential carries an rsa_public_key and Kong validates that the key parses before accepting the credential. The gateway then verifies signatures with the public key while clients sign with the private key, so the private key never has to reach Kong at all.
The OAuth 2.0 plugin does something different from the JWT plugin. The JWT plugin validates tokens that someone else issued. The OAuth 2.0 plugin issues them, acting as a lightweight authorization server for your APIs.
_format_version: "3.0"
services:
- name: orders
url: http://orders.internal:8080
routes:
- name: orders-route
paths:
- /orders
plugins:
- name: oauth2
service: orders
config:
enable_client_credentials: true
enable_authorization_code: true
enable_password_grant: false
enable_implicit_grant: false
mandatory_scope: true
scopes:
- read:orders
- write:orders
token_expiration: 3600
refresh_token_ttl: 1209600
auth_header_name: authorization
The four enable_ flags select which grant types are active, and each should be switched on only if you have a reason. The client credentials grant is for machine-to-machine calls and is the one you will use most; it issues a token directly for a client id and secret, with no user involved. The authorization code grant is the three-legged browser flow, where the user’s identity provider issues the code that Kong exchanges for a token. The password grant and the implicit grant are both deprecated in OAuth 2.1 because they expose user credentials to the client - leave them off unless a legacy client forces your hand.
token_expiration is the access token lifetime in seconds and defaults to 7200. refresh_token_ttl is the refresh token lifetime and defaults to 1209600, which is fourteen days. auth_header_name is the header the access token is read from, and defaults to authorization.
With mandatory_scope: true, a request whose token lacks the required scope is rejected. This is where OAuth 2.0 earns its keep over plain API keys: two clients can both be authenticated and still have different permissions, decided by the scopes attached to their tokens.
The rate-limiting plugin counts requests over a set of periods and rejects the ones that exceed the limit with a 429.
plugins:
- name: rate-limiting
service: orders
config:
minute: 100
hour: 1000
day: 10000
limit_by: consumer
policy: redis
fault_tolerant: true
hide_client_headers: false
error_code: 429
error_message: "API rate limit exceeded"
redis:
host: redis.internal
port: 6379
timeout: 2000
database: 0
The limit_by field is the most consequential setting, and the available values are consumer, credential, ip, service, header and path. consumer is the default and the right answer whenever callers are authenticated, because it gives each client its own budget rather than one shared bucket.
policy decides where the counters live, and this is where a single-node gateway and a fleet behave differently:
| Policy | Where counters live | Use when |
|---|---|---|
local | The node’s own memory | One node, or an approximate limit |
cluster | The Kong database | Database mode, a small number of nodes |
redis | An external Redis | Several nodes, or a limit that must be exact |
With local policy and three gateway nodes, a limit of 100 per minute becomes roughly 300 per minute in practice, because each node keeps its own counter. That is a real bug, not a rounding error, and it is invisible until you scale out.
Choosing redis means every node increments the same counter. The redis block needs host, port and timeout; username, password, database, ssl and ssl_verify are optional. And policy = redis requires a sync_rate that is either -1 (synchronous, the default) or a value of at least 0.02.
fault_tolerant decides what happens when Redis is unreachable. It defaults to true, which means requests are proxied anyway and the rate limit is not applied while Redis is down. That is the availability-first choice. Set it to false and a Redis outage becomes a total outage for everything behind the gateway - a single point of failure you introduced deliberately. The field hide_client_headers controls whether the X-RateLimit-* headers are returned to the caller; leaving them visible is usually helpful for clients that can back off gracefully.
Kong also ships rate-limiting-advanced, which adds sliding windows and arbitrary window sizes. The built-in rate-limiting plugin uses fixed windows, which means a client can send 100 requests at 11:00:59 and 100 more at 11:01:00 and still be within the limit - a burst of double the intended rate across the boundary.
Sometimes the gateway has to adapt a request to what the upstream expects: rename a header, add one, strip one, change the method, or rewrite the body.
plugins:
- name: request-transformer
service: orders
config:
add:
headers:
- "X-Gateway:kong"
querystring:
- "source:portal"
remove:
headers:
- "X-Internal-Debug"
rename:
headers:
- "X-Client-Id:X-Consumer-Id"
replace:
headers:
- "X-Client-Id:anonymous"
The four operations are add, remove, rename and replace, and each applies to headers, querystring and body. The add and replace operations take name:value strings - the colon separates key from value. remove takes bare names. rename takes old:new.
The replace operation on body deserves a warning: it replaces the entire request body with a fixed string, which is almost never what you want and is easy to do by accident. If you want to merge a field into a JSON body, use add with the body option instead.
The plugin also has an append operation, and an http_method field that rewrites the method.
A response transformer is the mirror image, and it is what you use to keep internal details internal:
plugins:
- name: response-transformer
service: orders
config:
remove:
headers:
- "X-Powered-By"
- "Server"
add:
headers:
- "X-API-Version:v2"
Stripping Server and X-Powered-By is worth doing as a matter of course: they tell an attacker the exact stack behind the gateway, and they leak through the gateway when an upstream sets them.
A gateway that behaves differently in production than in staging will eventually break in production. The defence is to keep the configuration in one file and change only the values that differ.
_format_version: "3.0"
services:
- name: orders
url: "{{ ORDERS_UPSTREAM }}"
routes:
- name: orders-route
paths:
- /orders
plugins:
- name: rate-limiting
service: orders
config:
minute: "{{ RATE_LIMIT_PER_MINUTE }}"
Then substitute per environment and validate the result before it goes anywhere:
envsubst < kong.yml.tmpl > kong.yml
kong config parse kong.yml
The kong config parse step is not optional in practice. It is a schema check, so it catches a typo in a plugin name, a missing required field, or a limit_by value that does not exist - all before the file reaches a running gateway.
For the drift check that keeps environments honest, decK compares the committed file against the live configuration:
deck gateway diff --state kong.yml
deck gateway sync --state kong.yml
Run deck gateway diff in CI against production. It exits non-zero when the live configuration has drifted from the file, which catches the most common cause of an unexplained environment difference: somebody changed something by hand through the Admin API.
If you need to know whether a particular plugin is even available on the node you are talking to, ask the node directly:
curl http://localhost:8001/plugins/enabled
A public API is served by the products service. Three kinds of caller use it:
- the web frontend, which has a JWT and needs to be limited to 60 requests per minute;
- the mobile app, which has a JWT and needs 600 per minute;
- an unauthenticated partner, which sends an API key and needs 30 per minute, and whose traffic must not count against either JWT consumer’s quota.
All three share the same upstream. Give the configuration, and explain where each rate limit is counted and why the partner’s traffic does not affect the others.
Approach:
The scenario is designed so that a single rate-limiting plugin cannot do the job. Three different callers with three different budgets means three independent counters, and the limit_by field is what decides what a counter is keyed on.
The partner is the hard case. They are unauthenticated, so they have no consumer identity to count against - but they must not consume the JWT consumers’ quota either. If you put one rate limiter on the Service with limit_by: consumer, the partner’s requests have no consumer to key on, and depending on how Kong treats an anonymous caller the partner either shares a bucket with everybody or lands in a bucket you did not intend. The clean answer is to give the partner their own route with their own limiter, and make the JWT consumers’ limit live on a scope that the partner’s route does not inherit.
Steps:
-
Service
productspointing at the upstream. One Service, shared by all three callers. -
Two Routes. The JWT callers and the partner need different plugin sets, so they need different Routes on the same Service.
products-jwt-routefor/products- the JWT callers.products-partner-routefor/partner/products- the partner.
-
JWT plugin on the Service, so both Routes inherit it. But the partner is unauthenticated, so on the partner Route you need to let unauthenticated requests through. The JWT plugin’s
anonymoussetting is the mechanism: set it to a dedicatedpartnerconsumer. Requests with a valid token are authenticated as their real consumer; requests without one become the anonymouspartnerconsumer instead of being rejected with401. -
Consumers.
web-frontend,mobile-appandpartnereach get a JWT credential (or a key-auth credential for the partner, if the partner sends a key rather than a token). These identities are what the rate counters are keyed on. -
Rate limit for the JWT callers - on the JWT Route,
limit_by: consumer, withminute: 60. But the two JWT callers need different budgets, and one plugin instance has one limit. So they need two Routes, or two plugin instances at different scopes.The clean way: two Routes -
/web/productsand/mobile/products- each with its own rate-limiting plugin,limit_by: consumer,minute: 60andminute: 600respectively. Becauselimit_byisconsumerand each caller has exactly one consumer identity, the limit is per consumer, not per route. -
Rate limit for the partner - on the partner Route,
limit_by: consumer,minute: 30. The partner is authenticated as the anonymouspartnerconsumer, so their counter is keyed on that consumer. -
Verify the isolation. The partner’s Route has its own rate-limiting plugin and does not inherit the JWT Routes’ limiters, because plugins do not cross Route boundaries. The JWT Routes’ limiters are keyed on
web-frontendandmobile-app, which the partner is not. So the three counters are genuinely independent. -
Choose
policy: redisif more than one gateway node serves this API, otherwise each node counts separately and the effective limit is multiplied by the node count.
Answer:
_format_version: "3.0"
services:
- name: products
url: http://products.internal:8080
routes:
- name: web-products-route
paths:
- /web/products
strip_path: false
- name: mobile-products-route
paths:
- /mobile/products
strip_path: false
- name: partner-products-route
paths:
- /partner/products
strip_path: false
consumers:
- username: web-frontend
jwt_secrets:
- algorithm: HS256
secret: 1a2b3c4d5e6f708192a3b4c5d6e7f809
- username: mobile-app
jwt_secrets:
- algorithm: HS256
secret: 9f8e7d6c5b4a39281706152433425160
- username: partner
jwt_secrets:
- algorithm: HS256
secret: 00112233445566778899aabbccddeeff
plugins:
- name: jwt
service: products
config:
claims_to_verify:
- exp
key_claim_name: iss
anonymous: partner
- name: rate-limiting
route: web-products-route
config:
minute: 60
limit_by: consumer
policy: redis
redis:
host: redis.internal
port: 6379
timeout: 2000
- name: rate-limiting
route: mobile-products-route
config:
minute: 600
limit_by: consumer
policy: redis
redis:
host: redis.internal
port: 6379
timeout: 2000
- name: rate-limiting
route: partner-products-route
config:
minute: 30
limit_by: consumer
policy: redis
redis:
host: redis.internal
port: 6379
timeout: 2000
| Caller | Route | Counter keyed on | Budget | Why the partner is unaffected |
|---|---|---|---|---|
| Web frontend | web-products-route | consumer web-frontend | 60/min | A separate Route with its own limiter; the partner’s Route does not inherit it |
| Mobile app | mobile-products-route | consumer mobile-app | 600/min | Same - the limiter is scoped to its own Route |
| Partner | partner-products-route | consumer partner (the anonymous consumer) | 30/min | Authenticated as partner, so it has its own counter and never appears in either JWT consumer’s bucket |
Two design decisions carry the whole solution. One Route per distinct budget, because a plugin instance has one limit and limit_by: consumer only separates callers who are already on the same Route. And anonymous: partner on the JWT plugin, which turns the partner’s missing token into a real consumer identity rather than a rejection - that identity is what makes limit_by: consumer able to count the partner at all.
