Overview

Modern asynchronous backend applications frequently require executing background workloads outside the primary HTTP request/response cycle—generating reports, sending transactional notifications, resizing media assets, or syncing data with third-party webhooks. Heavy distributed task queues often introduce steep operational overhead, extensive broker dependencies, and complex serialization protocols.

redisjobs is a lightweight, strictly typed, asynchronous job queue built purely on top of modern Redis commands and native Python asyncio. It delivers production-grade resilience with zero background broker processes.

---

Guaranteed At-Least-Once Delivery

Traditional naive Redis queues rely on RPOP or BRPOP, where a message popped from the queue vanishes from the broker immediately. If the worker process experiences an unhandled panic, out-of-memory kill (OOM), or server crash during execution, that task is lost permanently.

redisjobs eliminates this risk by utilizing atomic reliable queue primitives:

1. Atomic Leasing via BLMOVE: Jobs are popped from the pending queue and atomically inserted into an in-flight processing list in a single, uninterruptible Redis operation.
2. Crash-Recovery Reaper: A lightweight sweeper task monitors active worker leases. If a worker dies while holding a leased job, the reaper detects the expired lease heartbeat and re-enqueues the job back to the active queue for healthy workers to pick up.
3. Safe Acknowledgement: Only upon verified task completion is the job removed from the in-flight list.

from redisjobs import Queue, Worker

queue = Queue("notifications", redis_client)

# Enqueue an asynchronous job with automatic retries and delay
await queue.enqueue(
    send_welcome_email,
    user_id=452,
    max_retries=5,
    delay_seconds=60,
)

---

Production Reliability Features

• Exponential Backoff with Full Jitter: Failed executions back off exponentially to avoid thundering-herd spikes against downstream third-party services.
• Dead-Letter Queue (DLQ): Tasks that exhaust all retry attempts are safely routed to a persistent DLQ for inspection, alerting, and manual replay rather than disappearing silently.
• Delayed and Scheduled Tasks: Jobs can be scheduled to execute at precise future timestamps using Redis sorted sets.
• Prioritization: Strict priority scheduling allows urgent jobs to preempt long-running background tasks.
• Cooperative Cancellation: Running jobs can inspect cancellation tokens and exit gracefully before performing expensive I/O.

---

Testing and Verification

The library is comprehensively tested against fakeredis, enabling complete integration test suites to execute inside continuous integration pipelines in fractions of a second, without requiring external network ports or active Redis daemons.