Projects open_keypool v0.4.1
PyPI v0.4.1 Python 3.9+ Async & Thread-Safe Provider Presets Doppler / AWS / GCP

open_keypool

A lightweight, thread-safe and async-compatible Python library for pooling and rotating API keys. Features AsyncKeyPool, high-level pool.call(), automatic recovery waiting, safe lifecycle callbacks, httpx authentication, provider presets, and cloud secret manager integration.

$ pip install open-keypool

Installation #

Install the official package from PyPI using pip:

bash
pip install open-keypool

Quickstart — High-Level pool.call() (Recommended) #

Runs the get_key → fn → handle_response retry loop automatically, collapsing your API integration down to 3 lines:

python
import httpx
from open_keypool import KeyPool

pool = KeyPool(keys=["sk-key1", "sk-key2"], provider="groq")

def fetch_models(key):
    return httpx.get("https://api.groq.com/v1/models", headers={"Authorization": f"Bearer {key}"})

# Automatically handles rotation and retries internally up to max_retries
response = pool.call(fetch_models)

Quickstart — Async Usage (AsyncKeyPool) #

Full asyncio compatibility using AsyncKeyPool and httpx.AsyncClient:

python
import httpx
import asyncio
from open_keypool import AsyncKeyPool

async def main():
    pool = AsyncKeyPool(keys=["sk-key1", "sk-key2"], provider="openai")

    async def fetch_models(key):
        async with httpx.AsyncClient() as client:
            return await client.get("https://api.openai.com/v1/models", headers={"Authorization": f"Bearer {key}"})

    response = await pool.call(fetch_models)

asyncio.run(main())

Provider Presets #

Pass provider="groq", "openai", "gemini", "together", "openrouter", "cerebras", "mistral", or "anthropic" to parse documented provider signals such as x-ratelimit-reset-requests, retry-after-ms, RESOURCE_EXHAUSTED, and overloaded_error:

python
pool = KeyPool(keys=["sk-1", "sk-2"], provider="groq")

httpx Authentication #

KeyPoolAuth integrates key rotation with any client that accepts a custom httpx authentication class. It retries rate-limited or disabled keys and returns caller errors immediately.

python
import httpx
from open_keypool import KeyPool, KeyPoolAuth

pool = KeyPool(["key-one", "key-two"], provider="openai")
with httpx.Client(auth=KeyPoolAuth(pool)) as client:
    response = client.get("https://api.openai.com/v1/models")

Set scheme=None for raw-key headers such as x-goog-api-key. Use AsyncKeyPool with httpx.AsyncClient.

Recovery & Callbacks #

Opt into waiting for the soonest cooling-down key, with a per-call budget. Lifecycle callbacks receive masked keys only.

python
pool = KeyPool(
    ["key-one", "key-two"],
    wait_for_recovery=True,
    max_wait_seconds=30,
    on_cooldown=lambda key, seconds, reason: print(key, seconds),
)

Client errors such as 400, 404, and 422 leave a key active and are returned immediately. 408 and 5xx responses use a short transient cooldown.

Cloud Secret Managers (AWS & GCP) #

Pull keys directly from AWS Secrets Manager or GCP Secret Manager:

python
from open_keypool import KeyPool

# AWS Secrets Manager (pip install open-keypool[aws])
aws_pool = KeyPool.from_aws_secrets("my-secret-name", key_prefix="GROQ_")

# GCP Secret Manager (pip install open-keypool[gcp])
gcp_pool = KeyPool.from_gcp_secrets("my-secret-id", project_id="my-project", key_prefix="GROQ_")

Quickstart — Local Keys Array #

Instantiate KeyPool with a list of API key strings and rotate through them inside your API call loop:

python
from open_keypool import KeyPool, AllKeysExhaustedError

# Initialize key pool with Round-Robin strategy
pool = KeyPool(
    keys=["sk-key1", "sk-key2", "sk-key3"], 
    strategy="round_robin",
    cooldown_seconds=60
)

for attempt in range(pool.max_retries):
    try:
        key = pool.get_key()
        response = call_your_api(key)

        if response.status_code == 429:
            retry_after = float(response.headers.get("Retry-After", 0))
            pool.mark_rate_limited(key, retry_after=retry_after or None)
        elif response.status_code in (401, 403):
            pool.mark_invalid(key)
        else:
            pool.mark_success(key)
            break
    except AllKeysExhaustedError:
        print("All API keys are currently cooling down or disabled!")
        break

Quickstart — Doppler Sync Integration #

Pull keys dynamically from Doppler secrets manager with in-memory TTL caching:

python
import os
from open_keypool import KeyPool

DOPPLER_TOKEN = os.getenv("DOPPLER_TOKEN", "dp.st.YOUR_SERVICE_TOKEN")

# Load all secrets matching prefix 'MY_APP_' into an LRU pool
pool = KeyPool.from_doppler(
    token=DOPPLER_TOKEN,
    project="refactor-ai",
    config="dev",
    key_prefix="MY_APP_",
    strategy="lru",
)

# Get next active key
api_key = pool.get_key()

Interactive Live Playground #

Test rotation strategies, simulate HTTP 429 rate-limits, invalid 401 credentials, and watch live state recovery:

Live KeyPool Simulator

Strategy: Round-Robin
[System] Initialized KeyPool with 3 active keys ("sk-key1", "sk-key2", "sk-key3").

API Reference Specification #

Complete class methods, signature details, parameters, and exceptions for open_keypool:

pool.call(fn, *args, **kwargs) -> Any

High-Level Runner

Executes fn(key, *args, **kwargs) with an active key from the pool, handling rotation, response introspection, and automatic retries up to max_retries internally.

class AsyncKeyPool

Async Class

An asyncio-compatible version of KeyPool offering async get_key(), async_get_key(), async call(), and async Doppler loading via httpx.AsyncClient.

class KeyPool

Core Class

A thread-safe pool of API keys with automatic cooldown and configurable rotation strategies (Round-Robin or LRU).

__init__(keys=None, max_retries=3, cooldown_seconds=60, strategy="round_robin", provider=None)

Initializes a new KeyPool instance.

Parameter Type Default Description
keys list[str] | None None Initial list of API key strings. Must not be empty.
max_retries int 3 Maximum attempts made by call() and KeyPoolAuth when rotating keys.
cooldown_seconds int 60 Duration in seconds a key stays in COOLDOWN state after rate limit.
strategy str "round_robin" Rotation strategy: "round_robin" or "lru".
provider str | None None Provider preset: Groq, OpenAI, Gemini, Together, OpenRouter, Cerebras, Mistral, or Anthropic.
transient_cooldown_seconds float 5 Cooldown for 408 and 5xx responses.
wait_for_recovery bool False Allow call() to wait for a cooling-down key.
max_wait_seconds float 30.0 Maximum recovery wait budget for one call.
on_cooldown callable | None None Called with (masked_key, seconds, reason); keys are masked.
on_disabled callable | None None Called with (masked_key, reason); keys are masked.
on_exhausted callable | None None Called with the safe exhaustion message.

@classmethod KeyPool.from_doppler(...)

Factory Method

Creates a KeyPool by fetching API keys from Doppler Secrets Manager via REST API. Caches keys in an in-memory 1-hour TTL cache.

Parameter Type Description
token str Doppler service token (e.g. "dp.st.YOUR_TOKEN").
project str Doppler project name.
config str Doppler config name (e.g. "dev", "prd").
key_prefix str | None Optional prefix filter for secret names.

@classmethod KeyPool.from_env(suffix, env_file=None, ...)

Factory Method

Creates a KeyPool from environment variables ending with the specified suffix. Automatically calls load_dotenv() if a .env file is present.

@classmethod KeyPool.from_aws_secrets(secret_name, region_name="us-east-1", ...)

Factory Method

Fetches API keys from AWS Secrets Manager using boto3. Supports JSON dict parsing and line/prefix filtering.

@classmethod KeyPool.from_gcp_secrets(secret_id, project_id=None, ...)

Factory Method

Fetches API keys from GCP Secret Manager using google-cloud-secret-manager. Supports JSON dict parsing and prefix filtering.

KeyPool.get_key() -> str

Method

Returns the next available ACTIVE API key. Automatically flips expired COOLDOWN keys back to ACTIVE before selection.

Raises AllKeysExhaustedError if no active keys are available in the pool.

KeyPool.handle_response(key, status_code, headers=None, body=None)

Method

Inspects response metadata and returns the resulting KeyState. 429/413 and documented rate-limit signals enter COOLDOWN; 401/403 and invalid-key signals become DISABLED; 408 and 5xx use a short transient cooldown; other 4xx responses stay ACTIVE and are surfaced to the caller.

class KeyPoolAuth

httpx Auth

Authentication adapter for httpx.Client and httpx.AsyncClient. It selects a key, injects the configured header, feeds the response back to the pool, and retries cooldown or disabled keys up to max_retries.

Use KeyPoolAuth(pool, header="x-goog-api-key", scheme=None) when the provider expects a raw key instead of a Bearer token.

enum KeyState

Enum
  • ACTIVE — Key is healthy and ready for requests.
  • COOLDOWN — Key hit rate limit (429) and is temporarily paused.
  • DISABLED — Key failed authentication (401/403) and is permanently disabled.

class AllKeysExhaustedError(Exception)

Exception

Raised when get_key() is invoked but all keys in the pool are either on COOLDOWN or DISABLED.