Rate limits

Two layers of rate limiting protect the platform and let you control your own usage:

Organization limits (TPM)

Each organization has a tokens-per-minute (TPM) limit per model, measured over a sliding one-minute window across all keys in the org. The default tier allows 10,000 TPM per model; higher tiers are available — contact support@inferel.ai or your account manager to raise yours.

Per-key limits (TPM / RPM)

You can optionally attach TPM and/or RPM (requests-per-minute) limits to individual API keys on the console’s API Keys page. This is useful for capping a staging environment, an untrusted integration, or an individual teammate’s key. A key with no per-key limit set is bounded only by the organization limit.

When you exceed a limit

The API returns 429 Too Many Requests:

{
  "error": {
    "message": "Rate limit exceeded"
  }
}

Per-key violations are labeled explicitly ("Rate limit exceeded (per-key RPM)" / "(per-key TPM)") so you can tell which limit you hit.

Responses do not currently include Retry-After or X-RateLimit-* headers. Retry with exponential backoff and jitter; since limits are evaluated over a one-minute window, a backoff that reaches ~30–60 seconds will always clear a TPM window.

Tips

• Token limits count input + output tokens, so long prompts consume budget even on short replies.
• Spread bulk workloads across the minute instead of bursting.
• Use separate keys per application so one noisy workload can be capped without throttling everything else.