What you will learn
- Lists as FIFO queues (with blocking pops) and capped “recent items” feeds.
- Hashes for objects whose fields change independently.
- Sets for membership and set algebra, and HyperLogLog for cheap unique counts.
The code
"""Lists, hashes and sets: a job queue, a user profile and tag queries."""
import os
import redis
r = redis.Redis(port=int(os.getenv("REDIS_PORT", 6379)), decode_responses=True)
r.flushdb()
# List as a FIFO queue: LPUSH on one end, RPOP (or BRPOP to block) on the other
r.lpush("queue:emails", "welcome:42", "reset:7", "digest:13")
print("queue length:", r.llen("queue:emails"))
print("worker got:", r.rpop("queue:emails"), "then", r.brpop("queue:emails", timeout=1))
# Capped list: keep only the 3 most recent activity entries
for event in ["login", "view:sku1", "cart:add", "checkout", "logout"]:
r.lpush("user:42:recent", event)
r.ltrim("user:42:recent", 0, 2)
print("recent:", r.lrange("user:42:recent", 0, -1))
# Hash: one object, many fields, update fields independently
r.hset("user:42", mapping={"name": "Asha", "city": "Pune", "plan": "free"})
r.hset("user:42", "plan", "pro")
r.hincrby("user:42", "logins", 1)
print("profile:", r.hgetall("user:42"))
# Sets: membership and set algebra
r.sadd("tag:python", "post:1", "post:2", "post:5")
r.sadd("tag:redis", "post:2", "post:3", "post:5")
print("python AND redis:", sorted(r.sinter("tag:python", "tag:redis")))
print("python OR redis:", sorted(r.sunion("tag:python", "tag:redis")))
print("is post:3 python?", r.sismember("tag:python", "post:3"))
# HyperLogLog: approximate unique counts in ~12 KB regardless of size
r.pfadd("visitors:today", *[f"ip:{i % 5000}" for i in range(20000)])
print("unique visitors (approx):", r.pfcount("visitors:today"))
Output:
queue length: 3
worker got: welcome:42 then ('queue:emails', 'reset:7')
recent: ['logout', 'checkout', 'cart:add']
profile: {'name': 'Asha', 'city': 'Pune', 'plan': 'pro', 'logins': '1'}
python AND redis: ['post:2', 'post:5']
python OR redis: ['post:1', 'post:2', 'post:3', 'post:5']
is post:3 python? 0
unique visitors (approx): 5016
Sets are unordered, so the script sorts set results before printing.
Lists
A list is a sequence with fast pushes and pops at both ends.
- Queue:
LPUSHadds on the left,RPOPtakes from the right: first in, first out.BRPOPwaits up to a timeout for an item, so workers don’t busy-poll. - Capped feed:
LPUSHthenLTRIM 0 2keeps only the newest three entries. - Avoid
LINDEX/LINSERTdeep inside long lists: they’re O(n).
A plain list loses a job if a worker pops it and then crashes. LMOVE to a “processing” list helps, but Streams solve this properly with acknowledgements.
Hashes
A hash maps field → value under one key: ideal for objects. You can update one field (HSET user:42 plan pro) or increment one (HINCRBY) without rewriting the whole object, unlike a JSON string. Small hashes use a compact encoding, so many small hashes are memory-efficient.
Hash or JSON string? Use a hash when fields are read or updated individually; use a JSON string when you always read and write the whole object.
Sets
Unordered collections of unique strings with O(1) SADD/SISMEMBER/SREM.
- Tags and relationships: posts per tag, followers per user.
- Set algebra:
SINTER(posts tagged both),SUNION,SDIFF; the...STOREvariants save the result. - Deduplication: “has this user already voted?”
Note SISMEMBER returns 0/1 integers, as the output shows.
HyperLogLog
Counting unique visitors exactly needs memory proportional to the number of visitors. A HyperLogLog uses about 12 KB no matter how many items, with a standard error of about 0.81%. The run added 20,000 events from 5,000 distinct IPs and estimated 5,016. Use it for analytics, never for things that must be exact.
Key takeaways
- Lists: queues with
LPUSH/BRPOP, capped feeds withLTRIM; avoid deep index operations. - Hashes: objects with independently updated fields, memory-efficient when small.
- Sets: membership, dedup and intersections/unions.
- HyperLogLog: approximate unique counts in fixed small memory.
Exercises
- Model “users who liked a post” with a set and show mutual likes between two posts.
- Store a shopping cart as a hash of
sku → quantityand useHINCRBYto add items. - Add 1,000,000 unique ids to a HyperLogLog and measure its memory with
MEMORY USAGE.
Quiz
- How do you keep only the 100 newest items in a list?
- When is a hash better than a JSON string?
- What’s the trade-off of HyperLogLog?
Show answers
LPUSHthe new item, thenLTRIM key 0 99.- When individual fields are read or updated on their own, so you don’t rewrite the whole object.
- It uses tiny fixed memory (about 12 KB) but the count is approximate (about 0.81% standard error).