Kong Advanced Techniques

The bundled plugins cover most of what a gateway does. What they do not cover is the one rule your organisation has that nobody else’s rule set contains. At that point you write a plugin of your own. And once you have written one, you start thinking about where the gateway sits relative to the rest of your infrastructure - including the service mesh you may already run.

This chapter covers three things that separate an operator from a user of Kong: writing a custom plugin in Lua against the Plugin Development Kit, running Kong in front of a service mesh without the two fighting each other, and tuning the gateway so that it is not the bottleneck you blame for someone else’s slow service.

By the end you will be able to write a plugin that inspects and rewrites a request, place Kong correctly relative to a mesh, and read the handful of configuration values that actually determine how much traffic a Kong node can carry.

Writing a custom plugin

A Kong plugin is a directory with a handler, a schema, and optionally a rockspec. The handler implements named phases; the schema declares the configuration the plugin accepts.

kong/plugins/request-id/
  handler.lua
  schema.lua
  kong-plugin-request-id-0.1.0-1.rockspec

The handler returns a table of functions keyed by phase. The two you will use most are access, which runs on the way in, and response, which runs on the way out.

-- kong/plugins/request-id/handler.lua
local BasePlugin = require "kong.plugins.base_plugin"
local kong = kong

local RequestId = {}

-- Runs before the request is proxied upstream.
function RequestId:access(conf)
  local request_id = kong.request.get_header("X-Request-Id")

  if not request_id or request_id == "" then
    request_id = kong.id()
    kong.request.set_header("X-Request-Id", request_id)
  end

  kong.ctx.shared.request_id = request_id
  kong.log.debug("assigned request id ", request_id)
end

-- Runs after the upstream has responded.
function RequestId:response(conf)
  local request_id = kong.ctx.shared.request_id
  if request_id then
    kong.response.set_header("X-Request-Id", request_id)
  end
end

return RequestId

The plugin needs a schema so Kong knows what configuration it accepts and can validate it:

-- kong/plugins/request-id/schema.lua
local typedefs = require "kong.db.schema.typedefs"

return {
  name = "request-id",
  fields = {
    { protocols = typedefs.protocols },
    { config = {
        type = "record",
        fields = {
          { echo = { description = "Whether to echo the request id back in the response.",
                      type = "boolean", required = true, default = true } },
        },
      },
    },
  },
}

The schema must declare config as a record, and protocols restricts the plugin to HTTP. Because the schema is what the Admin API validates against, a plugin with a schema can be configured like any bundled one - including from declarative YAML.

Two details are easy to get wrong. First, kong.ctx.shared is per-request state; a value written in access is readable in response, but a value on the plugin instance itself would be shared across every request the worker handles, which is a data race. Second, a plugin that raises an error aborts the request with a 500; if the plugin is optional behaviour, catch the error and continue rather than failing the call.

Load it by placing it in the plugins directory and adding it to the plugins list, then restart:

KONG_PLUGINS=request-id \
KONG_CUSTOM_PLUGINS=/usr/local/kong/plugins \
kong start

Testing a plugin locally is much easier if you do it outside Kong entirely. The PDK can be driven from a plain Lua script, which lets you iterate in seconds rather than restarting the gateway:

luarocks install kong

For plugins that need heavy computation or a language you already know, Kong supports out-of-process plugins in Go, Java, Python and JavaScript. The Go implementation is the best supported, and the process is launched by Kong and spoken to over a local protocol, so the plugin still runs in the request path with the same lifecycle.

Kong and a service mesh

A service mesh and a gateway solve overlapping but different problems, and putting both in the path of the same request without a plan is how you end up with a request that is authenticated twice, retried twice, and logged twice.

The clean separation is:

ConcernBelongs to
Authenticating external clientsThe gateway
Quotas, request shaping, external loggingThe gateway
Mutual TLS between internal servicesThe mesh
Service-to-service authorisationThe mesh
Retry and circuit breaking between servicesThe mesh

With that split, the gateway sits at the north-south edge and the mesh handles east-west traffic. The mesh’s sidecar is injected into your services and mTLS the service-to-service hops; the gateway terminates the client’s TLS and speaks plain HTTP or mTLS to the mesh’s ingress.

The practical consequences are worth stating plainly:

  • Do not terminate TLS twice on the same hop. If the gateway re-encrypts to the mesh and the mesh re-encrypts again to the service, you pay two handshakes per request and lose the ability to inspect the plaintext at the gateway.
  • Decide where the trust boundary is. If the gateway authenticates the client and forwards an identity header, the mesh must be configured to trust that header from the gateway’s identity and to ignore it from anywhere else. Otherwise a compromised service can forge the header and bypass authentication entirely.
  • Decide where retries happen. A gateway that retries and a mesh that retries multiply: a request that fails can be attempted two, four or eight times, and the load it puts on a struggling upstream is far worse than a single failure would suggest. Pick one layer to retry.

Running Kong with a mesh also means the mesh must know about Kong. In Kubernetes the usual arrangement is that Kong runs as an Ingress controller, the mesh injects sidecars into the Kong data plane pods, and the gateway’s route configuration is applied through the ingress resources. The mesh then secures the hop from the gateway pod to the service pods, while the gateway secures the hop from the client.

Tuning for throughput

Kong’s performance is a function of a small number of settings, and the defaults are conservative in ways that suit a shared hosting environment more than a dedicated gateway.

Worker processes. Kong runs on NGINX, and NGINX uses a process-per-core model with an event-driven worker inside each process. The default is nginx_worker_processes = auto, which matches the number of CPU cores. On a container with a CPU limit, make sure the limit and the worker count agree - too many workers on a two-core limit causes contention rather than throughput.

KONG_NGINX_WORKER_PROCESSES=4

Event loop and connections. Each worker has its own event loop, and the number of simultaneous connections a worker can hold is bounded by the file descriptor limit:

KONG_NGINX_WORKER_CONNECTIONS=16384
KONG_NGINX_MAX_EVENTS=16384

Buffering. By default Kong buffers request and response bodies, which lets it retry a request safely and apply plugins that need the whole body. For large uploads and downloads that is pure overhead. KONG_PROXY_REQUEST_BUFFERING and KONG_PROXY_RESPONSE_BUFFERING can be turned off, but do it per route rather than globally, and only after confirming that no plugin on that route needs the buffered body.

Upstream keepalive. Kong opens a connection to the upstream per request unless keepalive is enabled, and on a high-latency network the handshake dominates. Keepalive needs both ends configured - Kong on one side, and the upstream service accepting and honouring Connection: keep-alive:

_format_version: "3.0"
services:
  - name: orders
    url: http://orders.internal:8080
    routes:
      - name: orders-route
        paths:
          - /orders

Timeouts. The defaults are connect_timeout, read_timeout and write_timeout, all 60000 milliseconds. A gateway that waits a full minute for a dead upstream holds a worker connection for a minute. Setting them to a few seconds makes failures fast, at the cost of failing slow-but-alive requests - which is usually the right trade for an API gateway.

The upstream’s health. The most common cause of a slow gateway is not the gateway. When the gateway is the only thing you can measure, it becomes the usual suspect, so measure the upstream directly and confirm where the time goes before tuning anything.

curl -s -o /dev/null -w "dns:%{time_namelookup} connect:%{time_connect} tls:%{time_appconnect} ttfb:%{time_starttransfer} total:%{time_total}\n" \
  http://localhost:8000/orders

The ttfb figure - time to first byte - is the one that tells you whether the delay is in your network or in the application.

Practice

Your organisation needs a plugin called require-header, which rejects any request that does not carry an X-Request-Source header, and adds the header X-Gateway-Policy: enforced to every request it accepts.

Write the plugin. Give the handler, the schema, and say which phase each part of the behaviour belongs in.

KongCustom PluginLuaPlugin SchemaRequest Phase

Approach:

Two behaviours, and the interesting question is which phase each belongs in. The behaviour that rejects must happen on the way in - access - because once the request has been proxied upstream you have already spent the upstream’s capacity and cannot un-send it. The behaviour that adds a header to an accepted request also belongs on the way in: you want the upstream to see the header, and the upstream only sees the request as it is being forwarded.

The subtlety is that “rejects” and “adds” are both decisions about the request, so both belong in access. The response phase is for modifying what goes back to the client, and it runs after the upstream has already handled the request - adding the header there would be too late for the upstream to see it.

Steps:

  1. Directory layout. kong/plugins/require-header/handler.lua and kong/plugins/require-header/schema.lua.

  2. The handler inherits from BasePlugin so it gets the standard lifecycle and the kong PDK handle.

  3. access(conf) does both jobs, in the right order. Check the header first and bail out with an error if it is missing - there is no point adding a header to a request you are about to reject. Then set the header on the outgoing request.

  4. Rejecting means raising an error with a status. kong.response.exit(401, { message = "..." }) ends the request immediately with the status you choose; the request is never proxied upstream. Using kong.log.err alone is not enough - logging does not stop the request.

  5. Adding the header uses kong.request.set_header, which modifies the request as it will be forwarded upstream.

  6. The schema declares config as a record so the Admin API can validate configuration for the plugin, and declares protocols so it applies to HTTP traffic.

  7. The schema should also declare the two header names as configuration, rather than hard-coding them in the handler. A plugin that hard-codes its header names cannot be reused for a different policy without editing code and redeploying.

  8. Load it by adding it to the plugins list and pointing KONG_CUSTOM_PLUGINS at the directory, then restarting Kong.

Answer:

-- kong/plugins/require-header/handler.lua
local BasePlugin = require "kong.plugins.base_plugin"
local kong = kong

local RequireHeader = {}

-- Runs on the way in, before the request is proxied upstream.
function RequireHeader:access(conf)
  local source = kong.request.get_header(conf.source_header)

  if not source or source == "" then
    kong.log.warn("rejected request without ", conf.source_header)
    kong.response.exit(401, {
      message = "missing required header: " .. conf.source_header,
    })
  end

  kong.request.set_header(conf.policy_header, conf.policy_value)
end

return RequireHeader
-- kong/plugins/require-header/schema.lua
local typedefs = require "kong.db.schema.typedefs"

return {
  name = "require-header",
  fields = {
    { protocols = typedefs.protocols },
    { config = {
        type = "record",
        fields = {
          { source_header = {
              description = "The header that must be present on the request.",
              type = "string", required = true, default = "X-Request-Source" } },
          { policy_header = {
              description = "The header added to accepted requests.",
              type = "string", required = true, default = "X-Gateway-Policy" } },
          { policy_value = {
              description = "The value written into the policy header.",
              type = "string", required = true, default = "enforced" } },
        },
      },
    },
  },
}

Load it with:

KONG_PLUGINS=require-header \
KONG_CUSTOM_PLUGINS=/usr/local/kong/plugins \
kong start

Why each piece is where it is:

BehaviourPhaseMechanismReason
Reject a request missing the headeraccesskong.response.exit(401, ...)Ends the request before it consumes upstream capacity; response would be far too late
Add the policy headeraccesskong.request.set_header(...)The upstream only observes the request while it is being forwarded, so the header must be set on the way in
Header names as configuration-schema config recordKeeps the plugin reusable; changing policy does not require a code change and redeploy

The one-line summary: everything that decides about the request belongs in access, and a rejection must use kong.response.exit rather than a log statement, because logging does not stop the request from being proxied.

Found an issue with this content?

Tell us what could be improved.