Rate limits

Two layers of rate limiting protect the platform and let you control your own usage.

Organization limits (TPM)

Each organization has a tokens-per-minute (TPM) limit measured over a rolling one-minute window across all keys in the org. The default allowance is 60,000 TPM; higher tiers — including unlimited — are available. Contact support@inferel.ai or your account manager to change yours.

Token usage is counted after each request completes, from the token counts the model reports. A single very large request can therefore push you past the limit rather than being refused up front; the next request is what gets throttled.

Per-key limits (TPM / RPM)

You can optionally attach TPM and/or RPM (requests-per-minute) limits to individual API keys on the console’s API Keys page. This is useful for capping a staging environment, an untrusted integration, or an individual teammate’s key. A key with no per-key limit set is bounded only by the organization limit.

When you exceed a limit

The API returns 429 Too Many Requests:

{
  "error": {
    "message": "Rate limit exceeded (org tpm)",
    "type": "rate_limit_exceeded"
  }
}

The message names the limit you hit — org tpm, org rpm, key tpm, or key rpm — so you can tell whether to slow down globally or raise that one key’s ceiling.

Responses do not include Retry-After or X-RateLimit-* headers. Retry with exponential backoff and jitter. Because the window is rolling rather than fixed, there is no window boundary to wait for: capacity returns gradually as older usage ages out of the trailing 60 seconds, so keep backing off until a request succeeds.

Tips

• Token limits count input + output tokens, so long prompts consume budget even on short replies.
• Spread bulk workloads across the minute instead of bursting.
• Use separate keys per application so one noisy workload can be capped without throttling everything else.
• Rate limiting fails open: if the limiter itself is unavailable, requests are allowed through rather than rejected.