Guides
Rate limits
Each API key allows 300 requests per minute and 16 concurrent requests by default. Over either limit, you get a 429 with Retry-After.
On this page
Limits apply per API key. Two keys in the same organization have independent limits, so a busy batch job on one key does not slow down your production key.
Default limits
| Limit | Default per key | Error when exceeded |
|---|---|---|
| Requests per minute | 300 | 429 rate_limit_exceeded |
| Concurrent requests (in flight at once) | 16 | 429 concurrency_limit_exceeded |
Both responses carry a Retry-After header with the number of seconds to wait. A rejected request is not charged.
HTTP/1.1 429 Too Many Requests
Retry-After: 3
Content-Type: application/json
{"error":{"message":"…","type":"rate_limit_exceeded","code":"concurrency_limit_exceeded","param":null}}
Other things that can return 429
- IP-level protection. Very high request rates from a single IP address are rejected with
429 ip_rate_limited, independent of the key. - The model provider. If the provider rate-limits a request, its
429andRetry-Afterare forwarded to you unchanged.
In every case: wait for Retry-After, then retry. See Errors & retries.
Your balance is a limit too
Every request in flight holds part of your wallet balance until it finishes (see Billing). With a small balance, holds for running requests can use it up before you reach 16 concurrent requests; new requests then fail with 402 insufficient_credit until earlier ones settle or you top up.
Staying under the limits
- Bound your concurrency. Use a worker pool or semaphore of at most 16 per key.
- Spread batch work over time rather than firing everything at once.
- Use one key per environment or workload, so limits and budgets are isolated.
- Stream long answers. Streaming does not change the limits, but it lets you show progress while a request holds its concurrency slot.
If the defaults don't fit your workload, email [email protected] with your expected traffic.