open_keypool
A lightweight, thread-safe and async-compatible Python library for pooling and rotating API keys. Features AsyncKeyPool, high-level pool.call(), automatic recovery waiting, safe lifecycle callbacks, httpx authentication, provider presets, and cloud secret manager integration.
Installation #
Install the official package from PyPI using pip:
pip install open-keypool
Quickstart — High-Level pool.call() (Recommended) #
Runs the get_key → fn → handle_response retry loop automatically, collapsing your API integration down to 3 lines:
import httpx
from open_keypool import KeyPool
pool = KeyPool(keys=["sk-key1", "sk-key2"], provider="groq")
def fetch_models(key):
return httpx.get("https://api.groq.com/v1/models", headers={"Authorization": f"Bearer {key}"})
# Automatically handles rotation and retries internally up to max_retries
response = pool.call(fetch_models)
Quickstart — Async Usage (AsyncKeyPool) #
Full asyncio compatibility using AsyncKeyPool and httpx.AsyncClient:
import httpx
import asyncio
from open_keypool import AsyncKeyPool
async def main():
pool = AsyncKeyPool(keys=["sk-key1", "sk-key2"], provider="openai")
async def fetch_models(key):
async with httpx.AsyncClient() as client:
return await client.get("https://api.openai.com/v1/models", headers={"Authorization": f"Bearer {key}"})
response = await pool.call(fetch_models)
asyncio.run(main())
Provider Presets #
Pass provider="groq", "openai", "gemini", "together", "openrouter", "cerebras", "mistral", or "anthropic" to parse documented provider signals such as x-ratelimit-reset-requests, retry-after-ms, RESOURCE_EXHAUSTED, and overloaded_error:
pool = KeyPool(keys=["sk-1", "sk-2"], provider="groq")
httpx Authentication #
KeyPoolAuth integrates key rotation with any client that accepts a custom httpx authentication class. It retries rate-limited or disabled keys and returns caller errors immediately.
import httpx
from open_keypool import KeyPool, KeyPoolAuth
pool = KeyPool(["key-one", "key-two"], provider="openai")
with httpx.Client(auth=KeyPoolAuth(pool)) as client:
response = client.get("https://api.openai.com/v1/models")
Set scheme=None for raw-key headers such as x-goog-api-key. Use AsyncKeyPool with httpx.AsyncClient.
Recovery & Callbacks #
Opt into waiting for the soonest cooling-down key, with a per-call budget. Lifecycle callbacks receive masked keys only.
pool = KeyPool(
["key-one", "key-two"],
wait_for_recovery=True,
max_wait_seconds=30,
on_cooldown=lambda key, seconds, reason: print(key, seconds),
)
Client errors such as 400, 404, and 422 leave a key active and are returned immediately. 408 and 5xx responses use a short transient cooldown.
Cloud Secret Managers (AWS & GCP) #
Pull keys directly from AWS Secrets Manager or GCP Secret Manager:
from open_keypool import KeyPool
# AWS Secrets Manager (pip install open-keypool[aws])
aws_pool = KeyPool.from_aws_secrets("my-secret-name", key_prefix="GROQ_")
# GCP Secret Manager (pip install open-keypool[gcp])
gcp_pool = KeyPool.from_gcp_secrets("my-secret-id", project_id="my-project", key_prefix="GROQ_")
Quickstart — Local Keys Array #
Instantiate KeyPool with a list of API key strings and rotate through them inside your API call loop:
from open_keypool import KeyPool, AllKeysExhaustedError
# Initialize key pool with Round-Robin strategy
pool = KeyPool(
keys=["sk-key1", "sk-key2", "sk-key3"],
strategy="round_robin",
cooldown_seconds=60
)
for attempt in range(pool.max_retries):
try:
key = pool.get_key()
response = call_your_api(key)
if response.status_code == 429:
retry_after = float(response.headers.get("Retry-After", 0))
pool.mark_rate_limited(key, retry_after=retry_after or None)
elif response.status_code in (401, 403):
pool.mark_invalid(key)
else:
pool.mark_success(key)
break
except AllKeysExhaustedError:
print("All API keys are currently cooling down or disabled!")
break
Quickstart — Doppler Sync Integration #
Pull keys dynamically from Doppler secrets manager with in-memory TTL caching:
import os
from open_keypool import KeyPool
DOPPLER_TOKEN = os.getenv("DOPPLER_TOKEN", "dp.st.YOUR_SERVICE_TOKEN")
# Load all secrets matching prefix 'MY_APP_' into an LRU pool
pool = KeyPool.from_doppler(
token=DOPPLER_TOKEN,
project="refactor-ai",
config="dev",
key_prefix="MY_APP_",
strategy="lru",
)
# Get next active key
api_key = pool.get_key()
Interactive Live Playground #
Test rotation strategies, simulate HTTP 429 rate-limits, invalid 401 credentials, and watch live state recovery:
Live KeyPool Simulator
Strategy: Round-RobinAPI Reference Specification #
Complete class methods, signature details, parameters, and exceptions for open_keypool:
pool.call(fn, *args, **kwargs) -> Any
High-Level RunnerExecutes fn(key, *args, **kwargs) with an active key from the pool, handling rotation, response introspection, and automatic retries up to max_retries internally.
class AsyncKeyPool
Async ClassAn asyncio-compatible version of KeyPool offering async get_key(), async_get_key(), async call(), and async Doppler loading via httpx.AsyncClient.
class KeyPool
Core ClassA thread-safe pool of API keys with automatic cooldown and configurable rotation strategies (Round-Robin or LRU).
__init__(keys=None, max_retries=3, cooldown_seconds=60, strategy="round_robin", provider=None)
Initializes a new KeyPool instance.
| Parameter | Type | Default | Description |
|---|---|---|---|
| keys | list[str] | None | None | Initial list of API key strings. Must not be empty. |
| max_retries | int | 3 | Maximum attempts made by call() and KeyPoolAuth when rotating keys. |
| cooldown_seconds | int | 60 | Duration in seconds a key stays in COOLDOWN state after rate limit. |
| strategy | str | "round_robin" | Rotation strategy: "round_robin" or "lru". |
| provider | str | None | None | Provider preset: Groq, OpenAI, Gemini, Together, OpenRouter, Cerebras, Mistral, or Anthropic. |
| transient_cooldown_seconds | float | 5 | Cooldown for 408 and 5xx responses. |
| wait_for_recovery | bool | False | Allow call() to wait for a cooling-down key. |
| max_wait_seconds | float | 30.0 | Maximum recovery wait budget for one call. |
| on_cooldown | callable | None | None | Called with (masked_key, seconds, reason); keys are masked. |
| on_disabled | callable | None | None | Called with (masked_key, reason); keys are masked. |
| on_exhausted | callable | None | None | Called with the safe exhaustion message. |
@classmethod KeyPool.from_doppler(...)
Factory MethodCreates a KeyPool by fetching API keys from Doppler Secrets Manager via REST API. Caches keys in an in-memory 1-hour TTL cache.
| Parameter | Type | Description |
|---|---|---|
| token | str | Doppler service token (e.g. "dp.st.YOUR_TOKEN"). |
| project | str | Doppler project name. |
| config | str | Doppler config name (e.g. "dev", "prd"). |
| key_prefix | str | None | Optional prefix filter for secret names. |
@classmethod KeyPool.from_env(suffix, env_file=None, ...)
Factory MethodCreates a KeyPool from environment variables ending with the specified suffix. Automatically calls load_dotenv() if a .env file is present.
@classmethod KeyPool.from_aws_secrets(secret_name, region_name="us-east-1", ...)
Factory MethodFetches API keys from AWS Secrets Manager using boto3. Supports JSON dict parsing and line/prefix filtering.
@classmethod KeyPool.from_gcp_secrets(secret_id, project_id=None, ...)
Factory MethodFetches API keys from GCP Secret Manager using google-cloud-secret-manager. Supports JSON dict parsing and prefix filtering.
KeyPool.get_key() -> str
MethodReturns the next available ACTIVE API key. Automatically flips expired COOLDOWN keys back to ACTIVE before selection.
Raises AllKeysExhaustedError if no active keys are available in the pool.
KeyPool.handle_response(key, status_code, headers=None, body=None)
MethodInspects response metadata and returns the resulting KeyState. 429/413 and documented rate-limit signals enter COOLDOWN; 401/403 and invalid-key signals become DISABLED; 408 and 5xx use a short transient cooldown; other 4xx responses stay ACTIVE and are surfaced to the caller.
class KeyPoolAuth
httpx AuthAuthentication adapter for httpx.Client and httpx.AsyncClient. It selects a key, injects the configured header, feeds the response back to the pool, and retries cooldown or disabled keys up to max_retries.
Use KeyPoolAuth(pool, header="x-goog-api-key", scheme=None) when the provider expects a raw key instead of a Bearer token.
enum KeyState
Enum- ACTIVE — Key is healthy and ready for requests.
- COOLDOWN — Key hit rate limit (429) and is temporarily paused.
- DISABLED — Key failed authentication (401/403) and is permanently disabled.
class AllKeysExhaustedError(Exception)
ExceptionRaised when get_key() is invoked but all keys in the pool are either on COOLDOWN or DISABLED.