Lesson 3 of 10 Data Structures

Redis Lists, Hashes, Sets and HyperLogLog

Redis lists as queues and capped feeds, hashes for objects, sets for tags and membership, and HyperLogLog for approximate unique counts, in Python.

35-45 min beginner 0/10 lessons done
Course contents Lesson 3 of 10 · 0 done
On this page (9 sections)
  1. What you will learn
  2. The code
  3. Lists
  4. Hashes
  5. Sets
  6. HyperLogLog
  7. Key takeaways
  8. Exercises
  9. Quiz

What you will learn

  • Lists as FIFO queues (with blocking pops) and capped “recent items” feeds.
  • Hashes for objects whose fields change independently.
  • Sets for membership and set algebra, and HyperLogLog for cheap unique counts.

The code

"""Lists, hashes and sets: a job queue, a user profile and tag queries."""
import os

import redis

r = redis.Redis(port=int(os.getenv("REDIS_PORT", 6379)), decode_responses=True)
r.flushdb()

# List as a FIFO queue: LPUSH on one end, RPOP (or BRPOP to block) on the other
r.lpush("queue:emails", "welcome:42", "reset:7", "digest:13")
print("queue length:", r.llen("queue:emails"))
print("worker got:", r.rpop("queue:emails"), "then", r.brpop("queue:emails", timeout=1))

# Capped list: keep only the 3 most recent activity entries
for event in ["login", "view:sku1", "cart:add", "checkout", "logout"]:
    r.lpush("user:42:recent", event)
    r.ltrim("user:42:recent", 0, 2)
print("recent:", r.lrange("user:42:recent", 0, -1))

# Hash: one object, many fields, update fields independently
r.hset("user:42", mapping={"name": "Asha", "city": "Pune", "plan": "free"})
r.hset("user:42", "plan", "pro")
r.hincrby("user:42", "logins", 1)
print("profile:", r.hgetall("user:42"))

# Sets: membership and set algebra
r.sadd("tag:python", "post:1", "post:2", "post:5")
r.sadd("tag:redis", "post:2", "post:3", "post:5")
print("python AND redis:", sorted(r.sinter("tag:python", "tag:redis")))
print("python OR redis:", sorted(r.sunion("tag:python", "tag:redis")))
print("is post:3 python?", r.sismember("tag:python", "post:3"))

# HyperLogLog: approximate unique counts in ~12 KB regardless of size
r.pfadd("visitors:today", *[f"ip:{i % 5000}" for i in range(20000)])
print("unique visitors (approx):", r.pfcount("visitors:today"))

Output:

queue length: 3
worker got: welcome:42 then ('queue:emails', 'reset:7')
recent: ['logout', 'checkout', 'cart:add']
profile: {'name': 'Asha', 'city': 'Pune', 'plan': 'pro', 'logins': '1'}
python AND redis: ['post:2', 'post:5']
python OR redis: ['post:1', 'post:2', 'post:3', 'post:5']
is post:3 python? 0
unique visitors (approx): 5016

Sets are unordered, so the script sorts set results before printing.

Lists

A list is a sequence with fast pushes and pops at both ends.

  • Queue: LPUSH adds on the left, RPOP takes from the right: first in, first out. BRPOP waits up to a timeout for an item, so workers don’t busy-poll.
  • Capped feed: LPUSH then LTRIM 0 2 keeps only the newest three entries.
  • Avoid LINDEX/LINSERT deep inside long lists: they’re O(n).

A plain list loses a job if a worker pops it and then crashes. LMOVE to a “processing” list helps, but Streams solve this properly with acknowledgements.

Hashes

A hash maps field → value under one key: ideal for objects. You can update one field (HSET user:42 plan pro) or increment one (HINCRBY) without rewriting the whole object, unlike a JSON string. Small hashes use a compact encoding, so many small hashes are memory-efficient.

Hash or JSON string? Use a hash when fields are read or updated individually; use a JSON string when you always read and write the whole object.

Sets

Unordered collections of unique strings with O(1) SADD/SISMEMBER/SREM.

  • Tags and relationships: posts per tag, followers per user.
  • Set algebra: SINTER (posts tagged both), SUNION, SDIFF; the ...STORE variants save the result.
  • Deduplication: “has this user already voted?”

Note SISMEMBER returns 0/1 integers, as the output shows.

HyperLogLog

Counting unique visitors exactly needs memory proportional to the number of visitors. A HyperLogLog uses about 12 KB no matter how many items, with a standard error of about 0.81%. The run added 20,000 events from 5,000 distinct IPs and estimated 5,016. Use it for analytics, never for things that must be exact.

Key takeaways

  • Lists: queues with LPUSH/BRPOP, capped feeds with LTRIM; avoid deep index operations.
  • Hashes: objects with independently updated fields, memory-efficient when small.
  • Sets: membership, dedup and intersections/unions.
  • HyperLogLog: approximate unique counts in fixed small memory.

Exercises

  1. Model “users who liked a post” with a set and show mutual likes between two posts.
  2. Store a shopping cart as a hash of sku → quantity and use HINCRBY to add items.
  3. Add 1,000,000 unique ids to a HyperLogLog and measure its memory with MEMORY USAGE.

Quiz

  1. How do you keep only the 100 newest items in a list?
  2. When is a hash better than a JSON string?
  3. What’s the trade-off of HyperLogLog?
Show answers
  1. LPUSH the new item, then LTRIM key 0 99.
  2. When individual fields are read or updated on their own, so you don’t rewrite the whole object.
  3. It uses tiny fixed memory (about 12 KB) but the count is approximate (about 0.81% standard error).

Finished this lesson?

Mark it complete to track your progress (saved in this browser).