Monitoring and Alerting for Redis Caches

This page identifies the handful of signals that tell you a Redis cache strategy is degrading — and how to write alert rules on them that fire before the degradation reaches users, instead of after.

A cache fails quietly. Latency creeps up a few milliseconds, the hit ratio slides from 0.97 to 0.94, evictions start where there were none — and none of it trips a page until the primary database is saturated and the incident is already underway. Effective monitoring turns those slow slides into early warnings by watching the rate of change of a small set of counters rather than their instantaneous values. The INFO-derived metrics and latency signals below are the same ones referenced throughout Redis Caching Architecture & Invalidation Fundamentals; this page is where they become alerts you can trust at 3 a.m.

# Signal Trade-offs

Five signals cover almost every cache degradation worth paging on. The engineering decision is not which to collect — collect all five — but how to alert on each: some are meaningful as absolute thresholds, most are only meaningful as a rate or derivative, and each carries a distinct false-positive profile. Alerting on the wrong form of a signal is the most common reason a Redis dashboard looks busy but never catches the real incident.

The rightmost column is where alert fatigue is born. An absolute hit-ratio threshold like "page if below 0.9" fires on every deploy that cold-starts the cache and stays silent through a slow month-long regression from 0.98 to 0.91 — exactly backwards. The signals split cleanly into two collection strategies: counters you scrape and differentiate (Approach A) and latency and event streams you observe directly (Approach B). Production monitoring uses both.

# Approach A — INFO Polling and a Prometheus Exporter

The INFO command returns a snapshot of monotonic counters — keyspace_hits, keyspace_misses, evicted_keys, expired_keys — plus point-in-time gauges from the memory section. The pattern is to scrape INFO on a fixed interval, expose the raw counters as Prometheus metrics, and let the time-series database compute rates and ratios at query time. Crucially, you do not compute the hit ratio inside the exporter: emit the two counters and derive the ratio in PromQL, so the alert can reason about the slope of the ratio rather than a single blended-since-boot number that flattens toward a meaningless lifetime average.

The exporter below uses redis.asyncio to poll INFO and mirror the relevant counters into Prometheus gauges. Counters are exported as Gauge set to the raw monotonic value; the rate() and division happen in the alerting layer.

import asyncio
import redis.asyncio as redis
from prometheus_client import Gauge, start_http_server

# Raw counters — differentiate these in PromQL, do NOT pre-divide here.
HITS = Gauge("redis_keyspace_hits_total", "Cumulative keyspace hits")
MISSES = Gauge("redis_keyspace_misses_total", "Cumulative keyspace misses")
EVICTED = Gauge("redis_evicted_keys_total", "Cumulative evicted keys")
EXPIRED = Gauge("redis_expired_keys_total", "Cumulative expired keys")
FRAG = Gauge("redis_mem_fragmentation_ratio", "used_memory_rss / used_memory")
USED_MEM = Gauge("redis_used_memory_bytes", "Bytes allocated by Redis")


async def scrape(client: redis.Redis, interval: float = 15.0) -> None:
    while True:
        # Two INFO sections cover every signal in the table above.
        stats = await client.info("stats")
        memory = await client.info("memory")

        HITS.set(stats["keyspace_hits"])
        MISSES.set(stats["keyspace_misses"])
        EVICTED.set(stats["evicted_keys"])
        EXPIRED.set(stats["expired_keys"])
        # mem_fragmentation_ratio is a point-in-time gauge, not a counter.
        FRAG.set(memory["mem_fragmentation_ratio"])
        USED_MEM.set(memory["used_memory"])

        await asyncio.sleep(interval)  # 15s matches a Prometheus scrape budget


async def main() -> None:
    client = redis.Redis(host="redis-primary", port=6379, decode_responses=True)
    start_http_server(9121)              # /metrics endpoint for Prometheus
    await scrape(client)


if __name__ == "__main__":
    asyncio.run(main())

With the raw counters exposed, the alert rules live in Prometheus, where rate() differentiates the monotonic counters over a window and the ratio is assembled from its parts. The hit-ratio rule below deliberately alerts on a sustained downward derivative, not a fixed floor, so a cold start (which reads low but recovers within minutes) never pages while a genuine week-long regression does. The eviction rule fires on the zero-to-nonzero transition that signals the working set has outgrown maxmemory — the churn that a frequency-biased LRU or LFU eviction policy is meant to contain.

groups:
  - name: redis-cache-health
    rules:
      # Hit ratio over a trailing 5m window, assembled from raw counters.
      - record: redis:hit_ratio:rate5m
        expr: |
          rate(redis_keyspace_hits_total[5m])
            / (rate(redis_keyspace_hits_total[5m])
               + rate(redis_keyspace_misses_total[5m]))

      # Alert on the SLOPE: ratio fell materially over 30m and stays low.
      - alert: RedisHitRatioDegrading
        expr: |
          redis:hit_ratio:rate5m
            < (redis:hit_ratio:rate5m offset 30m) - 0.05
          and redis:hit_ratio:rate5m < 0.9
        for: 15m
        labels: {severity: warning}
        annotations:
          summary: "Hit ratio dropped >5 points over 30m and is below 0.9"

      # Evictions were zero, now they are not — the working set overflowed.
      - alert: RedisEvictionOnset
        expr: rate(redis_evicted_keys_total[5m]) > 0
        for: 10m
        labels: {severity: warning}
        annotations:
          summary: "Sustained eviction — working set exceeds maxmemory"

      # Fragmentation band with hysteresis; slow evaluation avoids rewrite noise.
      - alert: RedisFragmentationHigh
        expr: redis_mem_fragmentation_ratio > 1.5
        for: 30m
        labels: {severity: warning}
        annotations:
          summary: "Fragmentation >1.5 — allocator overhead or jemalloc tuning"

This split — dumb exporter, smart alerting rules — is what makes the signals composable. The same raw counters power the derivative alert here and a long-horizon SLO burn-rate on a dashboard, without the exporter needing to know which. Keyspace-miss alerting follows the identical shape and is developed in full, with baselining against traffic, in Alerting on Redis Eviction Rate and Keyspace Misses.

# Approach B — Latency and Event-Driven Signals

Counters tell you that the cache degraded; they rarely tell you why. A hit-ratio dip caused by a blocking KEYS sweep looks identical, in the stats section, to one caused by a bad deploy. The latency and event signals close that gap. Redis maintains an internal latency monitor that records spikes by cause — command, fork, expire-cycle — reachable through LATENCY HISTORY and LATENCY LATEST, and a slow-command log reachable through SLOWLOG. Neither is a counter you scrape on a metronome; you pull them when a counter-based alert fires, or on a slower cadence, and correlate them with the spike.

The latency monitor must be armed first: CONFIG SET latency-monitor-threshold 100 tells Redis to record any event that blocks the event loop for more than 100 ms. The collector then reads LATENCY HISTORY per event class, exports the spikes, and resets the ring buffer with LATENCY RESET so the next window starts clean. The SLOWLOG pull names the exact offending command — the diagnostic that a percentile alone cannot give you.

import redis.asyncio as redis


async def poll_latency(client: redis.Redis) -> dict[str, list]:
    # Arm the monitor once; 100ms is a sane blocking threshold for a cache.
    await client.config_set("latency-monitor-threshold", 100)

    # LATENCY LATEST returns [event, timestamp, latest_ms, max_ms] per class.
    latest = await client.execute_command("LATENCY", "LATEST")
    spikes: dict[str, list] = {}
    for event, _ts, latest_ms, max_ms in latest:
        # Per-event history lets you see whether spikes are recurring.
        history = await client.execute_command("LATENCY", "HISTORY", event)
        spikes[event] = {"latest_ms": latest_ms, "max_ms": max_ms,
                         "samples": len(history)}

    # Reset the ring buffer so the next poll measures a fresh window.
    await client.execute_command("LATENCY", "RESET")
    return spikes


async def pull_slowlog(client: redis.Redis, n: int = 10) -> list[dict]:
    # Each entry: [id, unix_ts, exec_microseconds, [command, args...], ...].
    entries = await client.execute_command("SLOWLOG", "GET", n)
    return [
        {"micros": e[2], "command": " ".join(
            a.decode() if isinstance(a, bytes) else str(a) for a in e[3][:3])}
        for e in entries
    ]

The second event-driven signal is eviction itself. Rather than inferring eviction pressure from a counter delta, you can subscribe to the moment a key is evicted using keyspace notifications. Enabling the Eg flags (g generic, e evicted) makes Redis publish an __keyevent@0__:evicted message per eviction; a listener can sample the evicted key names to learn which key families are being sacrificed under pressure — information no counter carries.

import redis.asyncio as redis


async def watch_evictions(client: redis.Redis) -> None:
    # 'Kg' + 'e' publishes an event each time a key is evicted for memory.
    await client.config_set("notify-keyspace-events", "Ege")

    pubsub = client.pubsub()
    await pubsub.psubscribe("__keyevent@0__:evicted")
    async for msg in pubsub.listen():
        if msg["type"] == "pmessage":
            evicted_key = msg["data"]
            # Sample the key family (prefix) to see WHAT is being evicted —
            # a hot family here means the wrong keys are losing the memory race.
            family = evicted_key.split(":", 1)[0]
            record_eviction_sample(family)  # feed a low-cardinality metric

Keyspace notifications are best-effort fire-and-forget — a disconnected subscriber misses events — so they are a diagnostic sampler, never the source of truth for eviction volume. Use the evicted_keys counter from Approach A for the alert and the notification stream to explain it. The same duality applies to misses: the counter tells you a cache stampede is underway, while the latency and slowlog signals tell you whether it is your recompute path or a blocking command that turned the miss surge into user-visible slowness.

# When to Choose Which

Both approaches run in production together; the decision is which signal leads the alert for a given failure, so the page carries an actionable cause rather than a symptom. Resolve it against these criteria.

1. Alert on counters, diagnose with events. Make the paging alerts fire from the Approach A counter derivatives — they are cheap, always-on, and survive subscriber disconnects. Attach the Approach B latency and slowlog pulls as the first automated diagnostic the runbook runs, not as an independent alert source.
2. Match the alert form to the signal's physics. Eviction and miss rates cross a meaningful threshold at zero-to-nonzero, so alert on rate(). Hit ratio has no meaningful floor, so alert on its derivative. Fragmentation oscillates during background rewrites, so alert on an absolute band with a long for: clause to ride out the noise.
3. Let the false-positive column set the window. High-false-positive signals (absolute hit ratio, fast-sampled fragmentation) need long evaluation windows and hysteresis; low-false-positive signals (eviction onset) can page quickly. A signal is only as good as the window you evaluate it over.
4. Escalate by user proximity. Latency percentiles are the closest proxy for user pain — route them to a higher severity than hit-ratio drift, which the safety-net TTL behind explicit invalidation will partially absorb before users notice.

The pipeline below shows how the two approaches converge: raw signals on the left flow through a derivation stage that computes rates and derivatives, into rules that page on the right — with latency and events joining as diagnostic context rather than parallel noise.

# Failure Modes and Diagnostics

Three monitoring anti-patterns cause more missed incidents than any gap in coverage. Each has a fast diagnosis.

Alerting on absolute hit ratio instead of its derivative. A rule like "page if hit ratio < 0.90" is the classic false-signal generator: it screams through every legitimate cold start and stays mute through the slow regression that actually matters. A cache restarted during a deploy reads a low ratio for minutes while it warms, and a cache whose lifetime counters have accumulated over weeks reports an average that barely moves even as the current minute collapses. Diagnose by comparing the instantaneous windowed ratio against the since-boot ratio — a wide gap proves the absolute number is lying:

# Since-boot ratio hides the current window. Sample the counters twice,
# 10s apart, and compute the ratio over the DELTA to see the live picture.
redis-cli INFO stats | grep -E "keyspace_hits|keyspace_misses"
sleep 10
redis-cli INFO stats | grep -E "keyspace_hits|keyspace_misses"

If the delta-based ratio differs sharply from the raw cumulative one, your alert is reading the wrong number. The fix is the redis:hit_ratio:rate5m recording rule from Approach A, alerting on its change over 30 minutes rather than its value. The dashboard that makes this legible is built in Building a Cache Hit-Ratio SLO Dashboard.

Eviction masking expiry. When maxmemory is reached, Redis removes keys under its eviction policy before their TTL fires, so keys vanish on schedule-looking timing while expired_keys stays flat and evicted_keys climbs. Teams read the disappearing keys as healthy TTL decay and never realize memory pressure is silently shrinking their working set — until the hit ratio erodes. Diagnose by watching the two counters side by side:

redis-cli INFO stats | grep -E "expired_keys|evicted_keys"
redis-cli INFO memory | grep -E "used_memory:|maxmemory:"

A healthy TTL-driven cache shows expired_keys rising while evicted_keys stays near zero. The reverse — evicted_keys climbing while expired_keys stalls — means eviction is doing the work you attributed to expiry; revisit maxmemory sizing and the eviction policy rather than the TTL. This is precisely why the eviction alert keys on rate(evicted_keys[5m]) > 0 and not on the disappearance of keys.

Latency spikes from blocking commands. The event loop is single-threaded, so one KEYS *, a large HGETALL, or an unbounded SMEMBERS stalls every other client for the duration of that command — and a percentile alert flags the pain without naming the cause. Diagnose by cross-referencing the latency monitor against the slow log:

redis-cli CONFIG SET latency-monitor-threshold 100
redis-cli LATENCY LATEST          # which event class spiked: command, fork, expire
redis-cli LATENCY HISTORY command # recurring? or a one-off fork stall?
redis-cli SLOWLOG GET 5           # the exact command and its argument shape

LATENCY LATEST isolates the class of stall; SLOWLOG GET names the offending command and its arguments. If the culprit is enumeration, replace KEYS with a cursor-based SCAN; if it is a fork stall during BGSAVE, the spike is persistence, not your query path, and belongs on a different runbook.

# Verification

Before trusting any alert, confirm the underlying counters actually move under load — a rule wired to a metric that never changes is worse than no rule at all.

Generate synthetic traffic and watch the hit and miss counters advance in real time. redis-benchmark drives operations while a second shell samples INFO stats:

# Drive load in one shell...
redis-benchmark -h redis-primary -n 100000 -c 50 -t get,set -q
# ...and confirm the counters move in another.
redis-cli INFO stats | grep -E "keyspace_hits|keyspace_misses|instantaneous_ops_per_sec"

keyspace_hits, keyspace_misses, and instantaneous_ops_per_sec should all climb during the run. If they do not, the exporter is reading the wrong instance or the wrong INFO section.

Confirm the memory signals are populated and sane, so the fragmentation and eviction rules have real inputs:

redis-cli INFO memory | grep -E "used_memory:|used_memory_rss:|mem_fragmentation_ratio|maxmemory:"

A mem_fragmentation_ratio reported between roughly 1.0 and 1.5 is normal; a value pinned at exactly 1.0 with maxmemory:0 usually means the instance has no memory cap set, in which case the eviction alert can never fire because Redis will never evict. Set an explicit maxmemory and policy before relying on eviction-based alerting.

Finally, exercise the alert path end to end. Temporarily lower maxmemory to force eviction and verify the RedisEvictionOnset rule transitions to firing, then restore it:

redis-cli CONFIG SET maxmemory 8mb      # force pressure on a test instance
redis-cli INFO stats | grep evicted_keys # evicted_keys should now climb
redis-cli CONFIG SET maxmemory 0         # restore (test instances only)

An alert you have watched fire on demand is one you can trust to fire in an incident. Wire the same discipline into every rule: prove the signal moves, prove the derivative catches the slope, and prove the page carries a cause.

---

Up: Redis Caching Architecture & Invalidation Fundamentals

# Related

• Alerting on Redis Eviction Rate and Keyspace Misses — rate-based rules and baselining misses against traffic.
• Building a Cache Hit-Ratio SLO Dashboard — turning the derivative signal into a burn-rate SLO panel.
• LRU vs LFU Eviction Policies — the policy that decides what survives the memory pressure your eviction alert detects.
• TTL vs Explicit Invalidation: A Production Reliability Boundary — the safety-net TTL that absorbs hit-ratio drift before users notice.