Bulkheads: Bound Concurrency Without Hiding Shared Limits

Isolate the resource that can run out

A bulkhead limits how much of a resource one workload can consume. Separate admission limits for search and checkout can prevent search requests from occupying every application slot. The boundary is useful only if it matches the scarce resource. Bulkhead pattern.

Two connection pools pointed at the same database still share its CPU, storage, locks and connection ceiling. They do not guarantee that checkout succeeds during database saturation. Count connections across all replicas and pools when defining the total budget.

A small local admission limit

This example admits at most a fixed number of concurrent operations and rejects excess work immediately. It deliberately has no waiting queue, retries or cancellation policy.

function createBulkhead(limit) {
  if (!Number.isInteger(limit) || limit < 1) {
    throw new RangeError("limit must be a positive integer");
  }
  let active = 0;

  return async function run(operation) {
    if (typeof operation !== "function") {
      throw new TypeError("operation must be a function");
    }
    if (active >= limit) {
      throw new Error("Capacity unavailable");
    }
    active += 1;
    try {
      return await operation();
    } finally {
      active -= 1;
    }
  };
}

Create separate instances for workloads that need separate budgets. The counter belongs to one JavaScript process. Ten replicas each configured with a limit of ten can admit roughly a hundred operations collectively.

The slot stays occupied until the operation settles. A Promise.race timeout that leaves the underlying request running must not be treated as proof that its resource was released. Use a client with supported cancellation and account for the actual operation lifecycle.

Understand pool settings

In node-postgres, idleTimeoutMillis concerns clients sitting idle in the pool. It is not a query execution deadline. Checked-out clients must be released, normally in finally; transactions require the same checked-out client for their statements. Pool API.

Similarly, a top-level maxSockets field passed to Axios is not its documented Node agent configuration. Choose the correct HTTP or HTTPS agent options for the selected adapter and runtime. Connection limits also differ from application-request limits, especially with multiplexed protocols. Axios request configuration.

Size with measurements

Arrival rate multiplied by average time in the system estimates average concurrency under stable conditions. It is a starting point, not a minimum-capacity guarantee for bursts or tail latency.

Measure active work, rejected requests, waiting time and downstream saturation. Increasing a pool can worsen performance when it pushes more work into an already saturated database. Prefer explicit overload responses over an unbounded in-memory queue.

Test containment

Hold one workload's operations open and verify that excess requests are rejected while a separate workload still progresses. Then reject and resolve operations to confirm their slots are released. Test the process-wide budget again with the intended replica count.

A bulkhead complements deadlines and circuit breakers. It provides a specific capacity boundary; the rest of the system must still handle failure and explain overload to callers.

Sardar Mukhshaf

Full-Stack & DevOps Engineer

I build scalable web applications, secure backend systems, and production-ready cloud infrastructure. My engineering focus spans interface design, API architecture, deployment pipelines, and observability. Follow for more deep dives into system architecture, performance engineering, and emerging technologies.

ShareTwitterLinkedInCopy Link