Why does a single 429 error cascade into dozens in a multi-agent system?

When one agent hits a 429, naive retry logic re-attempts at a fixed interval. If 8 agents share the same API key and all hit 429 simultaneously, they all retry at the same time — doubling the request burst. Without jitter, this synchronized wave continues until the rate limit window resets.

What is the difference between RPM and TPM rate limits from OpenAI?

OpenAI enforces two separate limits: Requests per Minute (RPM) and Tokens per Minute (TPM). A GPT-4o call with a 30,000-token context window can exhaust your TPM limit in just 2 requests, even if your RPM limit allows 10. Multi-agent systems with large shared conversation histories typically hit TPM limits first.

Does a circuit breaker help with 429 errors?

Yes. A circuit breaker halts all retry attempts for a configurable cooldown window after N consecutive 429 errors. This prevents the thundering herd — instead of all agents retrying simultaneously, the OPEN circuit rejects them locally before the API call is made, then probes with a single request after the cooldown.

Should I use separate API keys for each agent to avoid 429?

Separate keys help for RPM limits but do not help for account-level TPM limits enforced across all keys on the same organization. A better approach is per-agent token budget enforcement at the application layer combined with a shared circuit breaker that tracks aggregate spend across all agents.

Fix OpenAI 429 Too Many Requests in Multi-Agent Loops

Direct Answer

OpenAI 429 errors in multi-agent loops are caused by multiple agents sharing one API key without per-agent TPM tracking, combined with non-jittered retry strategies that create synchronized thundering herds. Fix with: jittered exponential backoff (tenacity / p-limit), per-agent concurrency semaphores, and a circuit breaker that halts retries during sustained OpenAI outage windows.

Why Multi-Agent Systems Amplify 429 Errors

A RateLimitError: 429 Too Many Requests is not a binary event — it is the beginning of a cascade when your retry strategy lacks jitter and per-agent coordination.

Single agent (safe):

T+0s → Request: 429 → wait 1s → retry: 200 OK

8 agents (thundering herd):

T+0s → All 8 agents hit TPM limit → all get 429

T+1s → All 8 retry simultaneously → 429 again (burst doubled)

T+3s → All 8 retry simultaneously → OpenAI rate-drops connection

The root cause: non-jittered backoff schedules all agents to retry at the same millisecond. The solution is two-part: prevent synchronized retries with jitter, and prevent simultaneous dispatch with a concurrency semaphore.

RPM vs TPM: Which Limit Are You Hitting?

Inspect the x-ratelimit-remaining-tokens and x-ratelimit-remaining-requests headers on your last successful response. If remaining-tokens hit 0 first, you have a TPM problem.

Limit	Trigger Pattern	Primary Fix
RPM	Many agents, short prompts	Concurrency semaphore
TPM	Few agents, large context windows	Per-session token budget + context pruning
Both	Recursive agents with growing history	Circuit breaker + execution budget

Fix 1 — Jittered Exponential Backoff (Python + tenacity)

Replace deterministic backoff with wait_exponential_jitter from tenacity. Each agent waits a random interval within the exponential window, preventing synchronized retry waves:

Python — jittered_retry.py

import logging
from openai import OpenAI, RateLimitError
from tenacity import (
    retry, retry_if_exception_type,
    wait_exponential_jitter, stop_after_attempt,
    before_sleep_log,
)

logger = logging.getLogger(__name__)
client = OpenAI()

@retry(
    retry=retry_if_exception_type(RateLimitError),
    wait=wait_exponential_jitter(initial=1, max=60, jitter=5),
    stop=stop_after_attempt(5),
    before_sleep=before_sleep_log(logger, logging.WARNING),
)
def call_with_jitter(messages: list[dict], model: str = "gpt-4o") -> str:
    response = client.chat.completions.create(model=model, messages=messages)
    return response.choices[0].message.content

Fix 2 — Concurrency Semaphore (Python asyncio)

Python — semaphore_pool.py

import asyncio
from openai import AsyncOpenAI

# Tier 2: ~5,000 RPM → max ~83/s → 10 concurrent is conservative
_SEMAPHORE = asyncio.Semaphore(10)
client = AsyncOpenAI()

async def call_agent(messages: list[dict]) -> str:
    async with _SEMAPHORE:
        response = await client.chat.completions.create(
            model="gpt-4o", messages=messages,
        )
    return response.choices[0].message.content

async def run_all(agent_messages: list[list[dict]]) -> list[str]:
    return await asyncio.gather(*[call_agent(m) for m in agent_messages])

Fix 3 — p-limit Concurrency (TypeScript)

TypeScript — rate-limited-pool.ts

import OpenAI from 'openai'
import pLimit from 'p-limit' // npm i p-limit

const client = new OpenAI()
const limit = pLimit(10) // max 10 inflight calls

async function callAgent(
  messages: OpenAI.ChatCompletionMessageParam[]
): Promise<string> {
  return limit(async () => {
    const res = await client.chat.completions.create({
      model: 'gpt-4o',
      messages,
    })
    return res.choices[0].message.content ?? ''
  })
}

async function runAll(
  allMessages: OpenAI.ChatCompletionMessageParam[][]
): Promise<string[]> {
  return Promise.all(allMessages.map(callAgent))
}

Fix 4 — Circuit Breaker for Sustained 429 Windows

Jitter handles momentary spikes. For sustained outage windows, a circuit breaker halts all retries and waits for confirmed recovery before resuming:

TypeScript — observyze-circuit-breaker.ts

import { ObservyzeClient } from '@observyze/sdk'
import OpenAI from 'openai'

const obs = new ObservyzeClient({
  apiKey: process.env.OBSERVYZE_API_KEY!,
  circuitBreaker: {
    failureThreshold: 3,        // OPEN after 3 consecutive 429s
    cooldownMs: 30_000,         // Wait 30s before HALF-OPEN probe
    tripOn: ['RateLimitError'], // Only trip on 429, not on 5xx
  },
})

// Circuit state is server-side — persists across distributed agent nodes
const openai = obs.wrapOpenAI(new OpenAI())

// Throws CircuitOpenError immediately if circuit is OPEN
// — no provider call, no token spend, no thundering herd
const res = await openai.chat.completions.create({ model: 'gpt-4o', messages })

429 Remediation Checklist

✓Check x-ratelimit-remaining-tokens vs x-ratelimit-remaining-requests to identify RPM vs TPM as root cause
✓Replace deterministic backoff with jittered exponential backoff (tenacity or p-retry)
✓Add asyncio.Semaphore (Python) or p-limit (TypeScript) sized to ~20% below your RPM cap
✓Implement a circuit breaker with 30–60s cooldown for sustained outage periods
✓Track cumulative token counts per session and abort before hitting TPM ceiling
✓For TPM issues: prune conversation history or use a model with a larger context window

AI Circuit Breaker Stop Runaway Retry Loops Cost Monitoring

Fixing OpenAI 429 Too Many Requests in Multi-Agent Loops

Why Multi-Agent Systems Amplify 429 Errors

RPM vs TPM: Which Limit Are You Hitting?

Fix 1 — Jittered Exponential Backoff (Python + tenacity)

Fix 2 — Concurrency Semaphore (Python asyncio)

Fix 3 — p-limit Concurrency (TypeScript)

Fix 4 — Circuit Breaker for Sustained 429 Windows

429 Remediation Checklist

Frequently Asked Questions

Q1.Why does a single 429 error cascade into dozens in a multi-agent system?

Q2.What is the difference between RPM and TPM rate limits from OpenAI?

Q3.Does a circuit breaker help with 429 errors?

Q4.Should I use separate API keys for each agent to avoid 429?

Related Technical Articles

Metadata-Only Telemetry: Reducing Sensitive Data in LLM Observability

The Rise of Autonomous Agentic Governance

Why Traditional Monitoring Is Not Enough for LLM Applications

Ready to Govern your Inference?