Skip to main content

diskcache-oxide

A shared, durable disk cache for Python, with a Rust engine, SQLite storage, and a diskcache-compatible API.

AI / Agents

Skills
llms.txt
llms-full.txt

Developers

Michael Kennedy

Author

Community

Full license MIT

Meta

Requires: Python >=3.14
Package Info

Python cache. Rust engine. SQLite storage.

Every Python web app arrives at the same crossroads. A query that was fine at launch is now the slowest thing on the page, so you cache its result in a dict, and for one afternoon you’re a hero. Then the app goes under gunicorn with four workers, and there are four dicts, each warming up alone and each forgetting everything the moment its worker is recycled. So you reach for Redis, and now there’s a server to deploy, secure, monitor and pay for, whose entire job is to remember the answer to one query.

python-diskcache showed there was a third road: a cache that is just a directory. Every thread and every process on the machine opens the same path and sees the same entries, and the entries are still there tomorrow, because a directory doesn’t restart. No daemon, no network hop, no second system to keep alive. It’s the kind of idea that sounds too simple to be fast, and for years that was the catch. diskcache itself has had no commits since 2023, and its issues and pull requests go unanswered.

diskcache-oxide removes the catch. It keeps the diskcache API you already know and rebuilds everything beneath it for where Python is going: many cores, many worker processes, free-threaded CPython, and async frameworks that can’t afford a slow call on the event loop. The engine is Rust. The storage is SQLite. The Python layer is as thin as it can be. One thread gets over 700,000 cache operations a second out of a Cache, and eight threads sharing it still get that rate while python-diskcache’s falls below 20,000. Nothing on your disk is a pickle unless you asked for one (increasing compatibility and security), and memory and disk use stay where you set them.

Source on GitHub: https://github.com/mikeckennedy/diskcache-oxide · Issues

Here is the whole idea on one screen. Four workers, and every one of them opens the same directories. The cache makes the slow query cheap, and add() elects exactly one worker to run the hourly job, because a write that’s atomic across processes is a lock if you squint:

import os

from diskcache_oxide import Cache

# Every worker process opens the same directory and sees the same cache.
cache = Cache('/var/cache/my-app', durable=True)  # rebuildable data: an outage is a miss
# Claims get their own plain cache: durable can drop a write, and a claim must not be dropped silently.
claims = Cache('/var/lib/my-app/state/claims', eviction_policy='none')

def popular_episodes() -> list[dict]:
    if (episodes := cache.get('episodes:popular')) is not None:
        return episodes
    episodes = db.load_popular_episodes()  # slow-ish
    cache.set('episodes:popular', episodes, expire=300)
    return episodes

# Only one worker in the whole fleet runs the hourly job: add() is atomic across processes.
if claims.add('job:refresh-feeds', os.getpid(), expire=3600):
    refresh_feeds()

cache.close()  # at shutdown
claims.close()

And here is what it does under load. Every thread shares one cache, the x axis adds threads, and the two libraries go in opposite directions:

Mix diskcache-oxide diskcache 5.6.3
99 reads to 1 write 718k / 715k / 711k / 709k 433k / 241k / 76k / 19k
1,000 reads to 1 write 820k / 812k / 810k / 805k 475k / 219k / 70k / 20k

An app with several caches

Most apps end up with more than one cache: rendered pages, API responses, sessions, rate limits, each with its own size and settings. Caches is the registry for them. You register each cache once at startup, and a lookup by name opens it on first use and returns the same object every time after that. If a cache can’t open (a full disk, a permissions problem), the lookup returns a NullCache instead of raising, so requests carry on without that cache while the error is logged, and the registry tries again every 10 seconds until the real one is back. One call switches a cache off for a load test, and stats() gives an admin page every cache’s entries, size and errors.

from diskcache_oxide import Cache, Caches

# Rebuildable data: durable, so an outage is a miss, never an error page.
caches = Caches('/var/lib/my-app/cache', durable=True, timeout=1.0)
caches.register('pages', size_limit=2 * 2**30)
caches.register('api', size_limit=256 * 2**20)

html = caches['pages'].get('/about')  # opens on first use, the same object after that
caches.disable('pages')  # a load test without the page cache: no call site changes

for name, stat in caches.stats().items():  # for an admin page
    print(name, stat.count, stat.volume, stat.error)

# State (sessions, claims, queues) is a plain Cache, opened directly: a lost write must raise.
sessions = Cache('/var/lib/my-app/state/sessions', timeout=3.0, eviction_policy='none')
sessions.set('session:9f2c', {'user': 42}, expire=3600)

caches.close_all()  # at shutdown
sessions.close()

An entry is gone the moment it expires, for every read, len() and iteration, and its space comes back on its own: a small thread in each process looks after every open cache every 5 minutes, removing expired entries, trimming a cache that’s over its size limit and keeping the write-ahead log small, with the workers taking turns so they don’t all do the same work. There’s nothing to schedule. Background maintenance has the details and the one setting, maintenance_interval.

Talk Python runs its eight caches this way. Named caches covers the registry in full.

The example app is this setup in a whole FastAPI app, with gunicorn, a background worker and Docker Compose: a durable registry for pages, slow lookups and rate limits, and plain caches for sessions, job claims and a work queue. Every page shows which cache reads served it.

Where this is going

diskcache-oxide is being built in three stages. The first is done and the second is mostly done.

  1. A better diskcache. The Cache and FanoutCache you already know, rebuilt in Rust: about half the read latency, many times the threaded throughput, safe across fork(), safe on free-threaded Python, and no pickle by default.
  2. The complete local caching toolkit. Keys of every native type, memoize, tags, work queues, transact() and the coordination recipes (locks, semaphores, throttles, stampede protection) are all here. Still to come in this stage are a Django cache backend, free-threaded scaling, and a native AsyncCache for FastAPI, Litestar and friends.
  3. A cache server. The same Rust engine served over the network: a Redis-style central cache for a fleet of machines, with SQLite underneath instead of RAM. As the only owner of its files, the server can use SQLite’s exclusive locking mode, which already measures about 40% faster reads than the shared-directory library.

See Vision and roadmap for how each stage breaks down.

Why diskcache-oxide

  • Fast where web apps spend their time. At a 99:1 read/write mix, one thread gets about 720,000 operations a second from a Cache, 1.7× diskcache’s rate and cheap enough to call straight from the event loop. Add threads and the gap opens: at 8 threads on a GIL build, diskcache’s per-thread connections pile into each other and its throughput falls to 19,000 a second, while oxide’s stays near 710,000, 38× ahead (the chart above).
  • Scales out across worker processes. A plain Cache has one write lock, so a server with a worker per core ends up queueing on it. FanoutCache spreads keys over several SQLite databases so writes to different keys stop waiting on each other. With 8 processes it runs 1.5 million operations a second, 5.7× a plain Cache and 1.2× diskcache’s FanoutCache, behind the same API and in one directory.
  • Built for free-threaded Python. The extension is compiled with gil_used = false and never leans on the GIL for correctness, which matters on a build where there’s no GIL to lean on. Every change is tested on 3.14, 3.14t, 3.15 and 3.15t.
  • Safe across fork(). gunicorn, Celery’s prefork pool and multiprocessing create workers by copying the parent, and the copy includes whatever SQLite lock a thread was holding at that instant. A process-wide fork barrier makes sure no child is ever born holding half a transaction.
  • No pickle on your disk. Values are msgpack (or JSON) with exact type tags, so tuples, sets, datetimes, dates, Decimals, big ints and NaN come back as what you stored, not as whatever the format could manage. Your own types are registered once and read back only where the name is registered. Pickle is opt-in, and a cache that didn’t opt in refuses to unpickle anything it finds, however it got there.
  • Predictable memory and disk. A bounded connection pool, a 1 MiB page cache per connection, a write-ahead log that trims itself back down, and a checkpoint() for when you want it empty now.
  • Correct under pressure. clear() never deletes writes that landed while it ran, a reader racing an overwrite gets the old value or the new one, never a spurious miss, and a queue never hands a worker another queue’s items or loses one it couldn’t read. Running the same test suite against both libraries turned up 19 bugs in diskcache, each with a reproduction in the repo. diskcache-oxide has none of them but one, which it keeps on purpose for compatibility (the list).
  • More than get and set. memoize caches a function’s results by its arguments. Tags group entries so evict(tag) removes a user’s or a page’s in one call. push and pull make a work queue every worker shares, handing each item to exactly one of them. Locks, semaphores and rate limits hold across every process on the machine, with no Redis.
  • A real off switch. NullCache turns caching off without a single if cache is not None: in your code, and can keep forwarding invalidations to the real cache while it’s off, so nothing stale is waiting when you turn it back on.
  • One place for all your caches. Caches is a registry for every cache in an app. Register each one once at startup, and code at any depth gets the cache it needs by name, caches['pages'], without passing cache objects down through constructors or a dependency-injection framework. Each cache opens on first use and every lookup after that returns the same object, cheap enough to do on every request.
  • Errors you can act on. Every exception says what went wrong, in words, and the errors page says what to do about it. Nothing is swallowed unless you turned on the setting that swallows it.
  • Typed. Ships py.typed and a CacheLike protocol that Cache, FanoutCache and NullCache all satisfy.

Status

Note

diskcache-oxide runs in production at Talk Python, Python Bytes and Talk Python Training. Every release runs three gauntlets first. A conformance suite runs the same tests against diskcache 5.6.3 itself, and every place the two differ is a deliberate decision, written down with its reason in Divergences. A sanitizer pass puts the whole codebase, bundled SQLite included, under Miri, ThreadSanitizer (with a free-threaded CPython built under it too), AddressSanitizer and LeakSanitizer. And a 24-hour soak throws processes, threads, forks, SIGKILL and clear() under load at one directory for a day, then checks it for corruption, lost writes and leaks.

Most of diskcache’s API is here: Cache and FanoutCache, keys of every native type, memoize, tags and evict(), transact(), the queue calls push, pull and peek, retry=, stats(), check(), eviction_policy='none', and the coordination recipes. Still to come: a Django cache backend, free-threaded scaling and AsyncCache. Deque and Index wait until someone asks for them, and reading a large value as a file (read=) is folded into a later large-file cache. Custom disk= classes and least-recently-used eviction aren’t coming; Vision and roadmap says why.

Install

uv pip install diskcache-oxide

It supports CPython 3.14 and 3.15, including the free-threaded builds (3.14t and 3.15t), on macOS, Linux and Windows. Wheels cover macOS on Apple Silicon, Linux (glibc) on x86_64 and aarch64, and Windows on x86_64 for all four Pythons, with the compiled engine and a bundled SQLite inside, so there’s nothing else to install. Anywhere else pip builds from source, which needs the latest stable Rust toolchain; Building from source walks through it. Windows has no fork(), so the fork guarantees are Unix-only there; Getting started lists what else differs.

Before 1.0, a release that changes the on-disk format bumps the minor version and says so first in its changelog entry. Upgrading to one doesn’t break anything: a release that meets a cache in another format opens a new, empty one in a subdirectory named for its format (diskcache-oxide-format-3) and logs a warning, and remove_old_formats() cleans up the old one once the old workers are gone. A release that keeps the format keeps your caches warm. See Upgrading across a format change.

Coming from diskcache?

Change the import and point it at a new directory. That’s the migration. diskcache-oxide keeps diskcache’s API but uses its own file format (diskcache-oxide.sqlite), so it never reads or touches an existing cache.db:

# from diskcache import Cache
from diskcache_oxide import Cache

cache = Cache('/var/cache/my-app-oxide')

Switching from diskcache covers every difference you might notice, which is mostly things getting better and now and then things getting stricter.

Where to next

Read them in order for the whole story, or jump straight to the one that bit you.

  • New to caching? Start here: what a cache is, why it’s worth it, the vocabulary, and the two hard things, invalidation and naming.
  • Getting started: create a cache, store and read values, expiry, counters, eviction, timeouts and durable=True.
  • Switching from diskcache: what changes, and why.
  • Errors and what to do about them: every exception the library raises, what it means and the fix.
  • Troubleshooting: start from the symptom, such as every lookup missing, a disk that keeps growing, slow writes or PermissionError.
  • Vision and roadmap: what exists today and what’s coming.
  • Serialization: msgpack and JSON with exact types, and why pickle is opt-in.
  • NullCache: caching switched off without changing call sites.
  • Caches: a registry of named caches for an app with several.
  • FanoutCache: sharding for many processes that write.
  • Transactions: several writes that land together.
  • Coordination: locks, semaphores, throttles and running averages that hold across every process.
  • Concurrency and forks: threads, processes, free-threading and the fork barrier.
  • Storage and WAL: what’s on disk, memory bounds, and the write-ahead log.
  • Performance: the measured numbers, and where the next speedups come from.
  • Divergences: every deliberate difference from diskcache, with the reason.
  • Recipes: caching expensive calls, stale-while-revalidate, upstream HTTP responses, invalidation by prefix or tag, your own types, job claims, rate limiting, counters, work queues, single-use keys, sessions, web workers, Docker and Compose, asyncio, monitoring, upgrading across a format change, and testing.
  • Example app: a FastAPI app built the way these docs recommend, with Docker Compose and tests.
  • Agent skill: a cheat sheet for AI coding agents, with llms.txt and llms-full.txt alongside.