Retry

A Retry policy repeats a failing call with backoff and jitter. Use it for transient errors that are safe to retry: network blips, brief overloads, deadline-related failures.

Why

Survive flaky network paths without changing the call site.
Apply exponential backoff with jitter so retries do not stampede a recovering dependency.
Stay safe by default: retries trigger only on the exception classes you allow.

Usage

Decorate an async or sync function. Pass when= to say which outcomes should retry:

import httpx

from grelmicro.resilience import retry


@retry(when=httpx.HTTPError, attempts=3)
async def fetch(client: httpx.AsyncClient, url: str) -> bytes:
    response = await client.get(url)
    response.raise_for_status()
    return response.content


async def main() -> bytes:
    async with httpx.AsyncClient() as client:
        return await fetch(client, "https://example.com")

The decorator auto-detects whether it wraps an async or sync function.

For inline retries that span multiple statements, use the block form:

import httpx

from grelmicro.resilience import retrying


async def submit(client: httpx.AsyncClient, url: str, payload: dict) -> dict:
    async for attempt in retrying(when=httpx.HTTPError, attempts=3):
        async with attempt:
            response = await client.post(url, json=payload)
            response.raise_for_status()
            return response.json()
    return {}


async def main() -> dict:
    async with httpx.AsyncClient() as client:
        return await submit(client, "https://example.com", {"k": "v"})

when= is required. There is no default. Pass an exception class, a tuple of classes, a predicate callable, or a Match instance.

Choose a backoff

Retry ships five backoff algorithms. The default is exponential with full jitter. Pick by purpose.

Algorithm	Use when
`ExponentialBackoff` (default)	Network and HTTP retries. Doubling delay with jitter avoids retry storms.
`ConstantBackoff`	Polling-style retries (waiting for a job). Fixed interval is predictable.
`LinearBackoff`	Steady, predictable growth without the early-attempt cluster of exponential.
`FibonacciBackoff`	Smoother growth than exponential, faster than linear.
`RandomBackoff`	Uniform random delay in a fixed range. Maximum spread, no growth.

The factory classmethods build the right config for you:

policy = Retry.exponential("payments", when=httpx.HTTPError, attempts=5)
polling = Retry.constant("wait_job", when=NotReady, attempts=20, delay=1.0)

Add max_seconds= to cap the total time spent retrying. Retrying stops when either attempts or the time budget is reached, whichever comes first:

# Up to 5 attempts, but never more than 30 seconds in total.
policy = Retry.exponential("payments", when=httpx.HTTPError, max_seconds=30.0)

How backoff and jitter work

Exponential. The raw wait before retry N is min(base_delay * 2 ** (N - 1), max_delay). It doubles each attempt until it reaches the cap. With the defaults (base_delay=0.1, max_delay=30.0), the raw wait is 0.1s, 0.2s, 0.4s, 0.8s, 1.6s, ..., capped at 30.0s.

Jitter then transforms each raw wait into the actual sleep. The actual sleep may be smaller than the raw value.

Jitter is randomness added to the wait so concurrent clients do not retry at the same instant and overwhelm the recovering server. Pick a mode by how much spread you want. Default: full.

Jitter	Spread	When to use
`none`	none	Single client. Never with concurrency.
`full` (default)	maximum	The safe default.
`equal`	half	When you need predictable timing.
`decorrelated`	adaptive	High contention on a shared dependency.

Constant. A fixed delay between retries. One field: delay (seconds, default 1.0).

Filtering outcomes with `Match`

Match is the DSL every resilience strategy uses to decide whether an outcome (an exception OR a return value) should engage the strategy. The when= parameter on Retry accepts any Match instance, plus the bare-class shorthand for the simple case.

Here is the common case: retry on an exception type, or a tuple, or a predicate.

import httpx

from grelmicro.resilience import Match, retry


# Compose exception and result matchers with `|`.
# Retry on transient HTTP errors OR when the response carries a
# server-side soft-fail marker.
@retry(
    when=Match.exception(httpx.HTTPError)
    | Match.result(lambda r: r.headers.get("X-Soft-Fail") == "true"),
    attempts=5,
)
async def fetch(client: httpx.AsyncClient, url: str) -> httpx.Response:
    return await client.get(url)


# Polling-style: retry until the result is no longer ``None``.
@retry(when=Match.result(None), attempts=20)
async def poll_job(client: httpx.AsyncClient, job_id: str) -> dict | None:
    response = await client.get(f"/jobs/{job_id}")
    payload = response.json()
    return payload if payload["status"] == "ready" else None

Full Match DSL

Exception filtering

from grelmicro.resilience import Match, Retry

Retry("api", when=Match.exception(httpx.HTTPError))
Retry("api", when=Match.exception(httpx.HTTPError, OSError))
Retry("api", when=Match.exception(lambda e: e.status >= 500))
Retry("api", when=Match.exception_message(contains="timeout"))
Retry("api", when=Match.exception_message(regex=r"^5\d\d "))
Retry("api", when=Match.exception_cause(KeyError))

Result filtering

Retry("polling", when=Match.result(None))
Retry("polling", when=Match.result(False))
Retry("polling", when=Match.result(lambda r: r.status_code >= 500))

Match.result(callable) always treats the argument as a predicate. To match a function literal exactly, wrap with lambda r: r is my_fn.

Composition

# OR
Retry("api", when=Match.exception(httpx.HTTPError) | Match.result(None))

# AND
Retry("api", when=Match.exception(httpx.HTTPError) & Match.exception(lambda e: e.status >= 500))

# NOT (one symmetric `not_*` per primitive)
Retry("api", when=Match.not_exception(ValueError))
Retry("api", when=Match.not_result(None))
Retry("api", when=Match.not_exception_message(contains="ok"))
Retry("api", when=Match.not_exception_cause(KeyError))

Use | for OR and & for AND. Each primitive (exception, result, exception_message, exception_cause) has a not_* twin for the negated form.

What never retries

asyncio.CancelledError, KeyboardInterrupt, and SystemExit are BaseException subclasses outside Exception. They always propagate, regardless of the Match you pass. This is required for correct asyncio shutdown.

Behavior on exhaustion

When attempts is exhausted, the underlying exception is re-raised with a PEP 678 note attached:

try:
    await fetch(url)
except httpx.ConnectError as exc:
    print(exc.__notes__)
    # ['retry: 3/3 attempts exhausted in 1.40s (exponential backoff)']

Callers catch the underlying error type, unchanged. There is no RetryError wrapper class.

Composition with Circuit Breaker

Retry and Circuit Breaker compose by intent. When the breaker is OPEN, it raises CircuitBreakerError. Pick a narrow when= allowlist so the retry loop does not swallow that signal:

import httpx

from grelmicro.resilience import CircuitBreaker, retry

cb = CircuitBreaker("payments")


# A narrow allowlist that excludes CircuitBreakerError. When the
# breaker is open it raises CircuitBreakerError, which is not in
# `on`, so the retry loop aborts immediately.
@retry(when=(httpx.ConnectError, httpx.TimeoutException), attempts=3)
async def call_payments(
    client: httpx.AsyncClient, url: str, payload: dict
) -> dict:
    async with cb:
        response = await client.post(url, json=payload)
        response.raise_for_status()
        return response.json()


async def main() -> dict:
    async with httpx.AsyncClient() as client:
        return await call_payments(client, "https://example.com", {"k": "v"})

A broad allowlist (when=Exception) would retry through the open breaker. The narrow allowlist lets the breaker do its job.

Configuration

Build the policy with keyword arguments. Set when to choose which outcomes retry, attempts for the budget, and backoff for the wait strategy.

import httpx

from grelmicro.resilience import Retry

policy = Retry.exponential(
    "payments",
    when=httpx.HTTPError,
    attempts=5,
    base_delay=0.2,
    max_delay=10.0,
    jitter="full",
)


@policy
async def call_payments():
    pass

Environment variables

Prefix: GREL_RETRY_{NAME_UPPER}_. The default instance drops the name segment and reads GREL_RETRY_*.

Env var	Field	Type	Default
`GREL_RETRY_{NAME_UPPER}_ATTEMPTS`	`attempts`	`int` (>= 1)	`3`
`GREL_RETRY_{NAME_UPPER}_WHEN`	`when`	CSV or JSON list of FQN strings (e.g. `httpx.HTTPError`). Coerced to `Match.exception(...)`. Predicate forms cannot come from env.	required
`GREL_RETRY_{NAME_UPPER}_BACKOFF`	`backoff`	JSON object with a `kind` field (see below)	`{"kind":"exponential"}`

The full backoff config is a discriminated Pydantic union, so the env value is parsed as one JSON object. Each algorithm accepts the same fields it takes in code:

`kind`	Fields
`exponential`	`base_delay`, `max_delay`, `jitter` (`none` / `full` / `equal` / `decorrelated`)
`constant`	`delay`
`linear`	`base_delay`, `max_delay`
`fibonacci`	`base_delay`, `max_delay`
`random`	`min_delay`, `max_delay`

import httpx

from grelmicro.resilience import Retry

# Reads the config from environment variables. The backoff field is
# a discriminated union, pass it as a single JSON object.
#
# - GREL_RETRY_PAYMENTS_ATTEMPTS=5
# - GREL_RETRY_PAYMENTS_WHEN=httpx.HTTPError
# - GREL_RETRY_PAYMENTS_BACKOFF={"kind":"exponential","base_delay":0.2}
policy = Retry("payments", when=httpx.HTTPError)

The callable form of when cannot come from env. Use the FQN list for env-driven configs.

Advanced

For the from_config declarative path and pydantic-settings composition, see Declarative configuration.

Live reconfiguration

Retry inherits Reconfigurable[RetryConfig]. Calling policy.reconfigure(new_config) swaps the snapshot for future loops. An in-flight async for attempt in policy: keeps its snapshot until it completes. See Live reconfiguration.

Reference

See the API reference for every option.