Using Redis as a Celery Result Backend

Your Celery workers run tasks fine, but you need to read back what a task returned — a report URL, a computed total, the outcome of a chord — and right now those return values vanish. Celery separates the broker (the queue that hands tasks to workers) from the result backend (the store that holds return values and states), and Redis is a strong fit for the latter when results are small and short-lived. This walkthrough configures Redis as the result backend, sets an expiry so finished results do not accumulate forever, fetches values safely, wires up a chord that structurally requires a backend, and keeps Redis memory bounded so your result store never crowds out the cache it may be sharing an instance with.

# Prerequisites

• Redis 6.2+ or 7.x reachable from every worker and from the process that dispatches tasks.
• Celery 5.3+ on Python 3.10+.
• pip install "celery[redis]>=5.3" "redis>=5,<6" — the redis extra pulls in the redis-py client Celery uses.
• A broker already chosen; this page uses Redis for both broker and backend but keeps them on separate db indices.
• An idea of your result payload sizes and how long consumers actually need to read them, which sets result_expires.

# Step-by-Step Implementation

# 1. Point result_backend at Redis

Configure the app with a broker URL and a separate result_backend URL. Keeping the backend on a different db index from the broker (and from any application cache-aside data) means a FLUSHDB on one never destroys the others.

# celeryconfig.py
from celery import Celery

app = Celery("reports")
app.conf.broker_url = "redis://127.0.0.1:6379/0"      # queue: db 0
app.conf.result_backend = "redis://127.0.0.1:6379/1"  # results: db 1

# 2. Bound every result with result_expires

By default Celery keeps results for a day; make the window explicit and as short as your consumers tolerate. result_expires sets a TTL on each result key, so finished results self-clean instead of accumulating — the same bounded-lifetime discipline as any TTL-based invalidation.

from datetime import timedelta

# Result keys expire 1 hour after the task finishes; tune to consumer needs.
app.conf.result_expires = timedelta(hours=1)

# 3. Store a result by returning from a task

Any value a task returns is serialized and written to the backend under the task's id. Keep return values small — an identifier or a short summary — and let consumers fetch the heavy payload from durable storage using that identifier.

@app.task
def build_report(account_id: int) -> dict:
    rows = crunch_report(account_id)          # expensive work
    url = upload_to_object_store(rows)        # heavy bytes go to S3, not Redis
    return {"account_id": account_id, "report_url": url}   # small result only

# 4. Fetch a result without blocking a worker

Read a result back through its AsyncResult. Check ready() before calling get(), or pass a timeout, so a consumer never blocks indefinitely on a task that has not finished — and never call get() from inside another task, which can deadlock a worker.

from celery.result import AsyncResult

def fetch_report(task_id: str) -> dict | None:
    res = AsyncResult(task_id, app=app)
    if not res.ready():                # non-blocking readiness check
        return None
    return res.get(timeout=1)          # safe: result is already available

# 5. Run a chord that requires the backend

A chord runs a group of tasks and then a callback over their collected results — which is impossible without a result backend, because the callback needs somewhere to read the group's outputs from. Redis is well suited to this short-lived coordination. When the callback's job is to purge caches, route it through async invalidation queues with Celery.

from celery import chord

@app.task
def tally(partials: list[int]) -> int:
    return sum(partials)              # callback reads all group results

def launch(account_ids: list[int]):
    # The chord's header group writes results the body callback then aggregates.
    return chord(build_partial.s(a) for a in account_ids)(tally.s())

# 6. Keep results from bloating Redis

Discard results you will never read. Setting task_ignore_result globally (or ignore_result=True per task) stops fire-and-forget tasks from writing to the backend at all, which is the single biggest lever on result-store memory. Combine it with a short result_expires and a bounded connection pool so the backend stays lean under load.

app.conf.task_ignore_result = True          # default: write nothing

@app.task(ignore_result=False)              # opt back in only where needed
def build_report(account_id: int) -> dict:
    ...

# 7. Enrich results with result_extended

When you need richer metadata than the return value — the task name, its arguments, the worker that ran it — enable result_extended. It stores those fields alongside the result so operational tooling can introspect a task by id, at the cost of a slightly larger result key.

# Store task name, args, kwargs, and worker alongside each result.
app.conf.result_extended = True

# Failure Modes

Redis memory climbs from accumulated results. used_memory grows in step with task volume because finished results are never cleaned up. Diagnose by counting result keys and confirming they carry a TTL:

redis-cli -n 1 --scan --pattern "celery-task-meta-*" | wc -l
redis-cli -n 1 TTL "celery-task-meta-<some-task-id>"   # expect positive, not -1

Fix by setting result_expires (Step 2) and turning on task_ignore_result for fire-and-forget tasks (Step 6). A -1 TTL means a result was written without an expiry — check that your Celery version applies result_expires and that no task overrides it.

Chord hangs and never fires its callback. The body task never runs because the header group's results cannot be collected — almost always a missing or misconfigured result backend. Diagnose by confirming the backend is reachable and that partial results are landing:

redis-cli -n 1 --scan --pattern "celery-task-meta-*" | head
redis-cli -n 1 PING   # backend must answer for chord synchronization to work

Fix by ensuring result_backend is set (Step 1); a chord is structurally impossible without one, unlike a plain task chain.

Consumer blocks a worker calling get(). A task that calls AsyncResult.get() on another task can deadlock when the pool has no free worker to run the awaited task. Diagnose by looking for get() inside task bodies:

grep -rn "\.get(" tasks/ | grep -i asyncresult

Fix by never synchronously waiting on a subtask inside a task — use a chord or chain to express the dependency (Step 5), and reserve get() for outside the worker pool.

# Verification

Confirm results are written with a bounded lifetime after a task completes:

python -c "from tasks import build_report; r = build_report.delay(42); print(r.id)"
redis-cli -n 1 EXISTS "celery-task-meta-<printed-id>"   # expect 1 shortly after
redis-cli -n 1 TTL "celery-task-meta-<printed-id>"      # expect ~3600, never -1

Confirm ignored results are not stored at all, so fire-and-forget tasks add zero backend pressure:

redis-cli -n 1 DBSIZE            # note the count
# ... dispatch a batch of ignore_result tasks ...
redis-cli -n 1 DBSIZE            # should be unchanged for ignored results

# FAQ

# What is the difference between the broker and the result backend?

The broker is the queue that transports tasks from producers to workers — it holds pending work and delivers it. The result backend is a separate store that holds the return value and state of a task after it finishes. They solve different problems: you always need a broker to run tasks, but you only need a result backend if something reads results back or you use primitives like chords. Redis can serve as both, but keep them on different db indices so their lifecycles stay independent.

# How does result_expires affect Redis memory?

result_expires sets a TTL on every result key, so finished results are reclaimed automatically once the window passes instead of living until a manual cleanup. A shorter expiry keeps used_memory flatter under high task volume; a longer one gives consumers more time to read a result. Because it is a real Redis TTL, expiry happens passively in the background, which is far cheaper than periodic bulk deletion. Pair it with task_ignore_result for tasks whose output no one reads.

# When should I set ignore_result?

Set ignore_result=True on any task whose return value no consumer will ever read — notifications, cache warmers, side-effect-only jobs. Ignoring the result skips the backend write entirely, which is the most effective way to keep the result store small and fast. Enable it globally with task_ignore_result = True and opt individual tasks back in with ignore_result=False only where a value is actually consumed or where a chord needs to collect the output.

# Why does a chord require a result backend when a chain does not?

A chain pipes each task's output directly into the next as an argument, so nothing needs to be stored between links. A chord is different: its callback runs only after a whole group of parallel tasks completes, and it aggregates all of their results — so those results must be collected somewhere while the group finishes. The result backend is that collection point. Without one, Celery has nowhere to accumulate the group's outputs and the chord can never fire its callback.

---

Up: Python Redis Integration Patterns

# Related

• Configuring the Django Redis Cache Backend
• Managing Redis Connections with the FastAPI Lifespan
• Python Redis Integration Patterns
• Building Async Invalidation Queues with Celery