redis-connections

Redis Connections

Client-side guidance for talking to Redis efficiently: how to share connections, how to batch commands, which commands not to call in production, when to turn on client-side caching, and how to set timeouts that fail fast without breaking healthy traffic.

When to apply

• Creating or reviewing a Redis client setup (redis-py, Jedis, Lettuce, go-redis, NRedisStack).
• Making many small Redis calls and wondering where the latency is going.
• Iterating large keyspaces, sets, hashes, or lists.
• Enabling client-side caching for hot keys.
• Tuning connect / read / write timeouts.

1. Pool or multiplex — never one connection per request

The single biggest mistake in Redis client code is opening a new TCP connection for every operation. Always either:

• Pool — keep N persistent connections that the application leases per call (redis-py ConnectionPool, Jedis JedisPooled, go-redis client).
• Multiplex — share a single connection across all requests (Lettuce, NRedisStack).

Set a bounded wait timeout for leasing a pooled connection, so callers fail fast instead of hanging when the pool is exhausted.

# redis-py — connection pool
pool = redis.ConnectionPool(host="localhost", port=6379, max_connections=50)
r = redis.Redis(connection_pool=pool)

These examples connect to a local development server. In production, enable authentication and TLS with certificate verification, and load the username and password from the environment or a secret store rather than source code.

See references/pooling.md for Python + Java + Lettuce examples.

2. Pipeline bulk work

For N commands that don't depend on each other's results, send them as a single batch with pipelining. One round-trip instead of N. In these snippets redis stands for a client instance, such as r from section 1, not the module.

pipe = redis.pipeline()
for user_id in user_ids:
    pipe.get(f"user:{user_id}")
results = pipe.execute()

Use non-transactional pipelining for performance, and pipeline(transaction=True) only when you actually need atomicity (see redis-core's transactions guidance).

See references/pipelining.md.

3. Avoid commands that scan everything

Anything that walks the whole keyspace (or a whole large container) blocks the server. Use incremental variants instead.

cursor = 0
while True:
    cursor, keys = redis.scan(cursor, match="user:*", count=100)
    for key in keys:
        process(key)
    if cursor == 0:
        break

Blocking commands (BLPOP, BRPOP, BLMOVE) are different — they intentionally wait for data and are fine for queue consumers, but always pass a timeout, and don't issue them on a multiplexed connection (Lettuce, NRedisStack).

See references/blocking.md.

4. Client-side caching for hot keys

For data that's read often and written rarely (config, feature flags), enable RESP3 client-side caching. The client keeps a local copy and the server invalidates it on writes — saving the round trip for hot reads.

client = redis.Redis(
    host="localhost",
    port=6379,
    protocol=3,                                    # RESP3 is required
    cache_config=redis.CacheConfig(max_size=1000),
)

Client-side caching needs a redis-py release that supports it, protocol 3, and a Redis server with RESP3 client tracking; check where your client version exports the cache config class. Keep session, authentication and revocation state out of the local cache, since it changes often and must stay current after a missed invalidation or reconnect.

Skip it for write-heavy workloads or data that changes constantly — the invalidation traffic overruns the savings.

See references/client-cache.md.

5. Set explicit timeouts

Defaults vary by client and may be too generous. Pick values that match the application's failure model:

r = redis.Redis(
    host="localhost",
    socket_connect_timeout=2.0,   # fail fast on dead nodes
    socket_timeout=5.0,           # tune to expected operation time
    retry_on_timeout=True,
)

Rule of thumb: connect timeout shorter than read/write timeout. Tight timeouts + retry-on-timeout for latency-sensitive paths; longer timeouts for batch jobs.

Use retry on timeout only for commands that are safe to repeat, such as reads. A timed out write may already have been applied, so for counters, list pushes and other writes that are not idempotent, turn retries off or use transactions. Newer redis-py versions favor an explicit retry policy with backoff over the simple flag; check your client version. These timeout examples also target a local server; apply the same production authentication and TLS settings described in section 1.

See references/timeouts.md.

References

• Redis: Connection Pools and Multiplexing
• Redis: Pipelining
• Redis: SCAN
• Redis: Client-side caching
• Redis: Clients