---------------------------------------------------------------------- This is the API documentation for the diskcache_oxide library. ---------------------------------------------------------------------- ## Cache Disk-backed cache backed by SQLite, shared safely across threads and processes. Cache(directory: 'str | os.PathLike[str]', timeout: 'float | None' = None, *, size_limit: 'int | None' = None, cull_limit: 'int | None' = None, serializer: 'str' = 'msgpack', pickle: 'bool' = False, types: 'TypeRegistry | None' = None, clock: 'Callable[[], float] | None' = None, max_connections: 'int | None' = None, sqlite_cache_size: 'int | None' = None, wal_size_limit: 'int | None' = None, wal_checkpoint: "Literal['manual', 'auto']" = 'manual', on_decode_error: "Literal['raise', 'miss'] | Callable[[str, Exception], object] | None" = None, durable: 'bool' = False) -> 'Cache' A disk cache backed by SQLite, shared safely across threads and processes. Every thread and process that opens the same directory shares one cache. The API follows `diskcache.Cache`; the deliberate differences are listed in the Divergences guide. Args: directory: The cache directory. It's created (owner-only, `0700`) if it doesn't exist. timeout: Seconds a call waits for a lock or a pending fork before raising `Timeout` (default 60). It has to exceed work in flight plus a fork: see the Concurrency and Forks guide. size_limit: The disk budget in bytes, database plus value files (default 1 GiB, minimum 1 MiB). Stored in the cache, so it applies to every process. cull_limit: How many least-recently-stored entries a `set()` evicts when the cache is over `size_limit` (default 10). serializer: `'msgpack'` (the default) or `'json'`, for values that aren't plain `str` or `bytes`. pickle: Allow pickle for values the serializer can't represent exactly. Without it, such values raise `TypeError` and pickled values are never read. Once the encoder has used a registration (`types`, or the global `registry`) for any part of a value, that value is never pickled: it is stored through the registration, or raises. types: A `TypeRegistry` consulted before the process-wide `diskcache_oxide.registry`, for a cache that needs its own encoding of a type. Register types at import time, in every process that opens the directory, since a row written through a registration can only be read where it is registered. clock: A function returning the current time in seconds, used instead of the native clock. Tests pass `clock=lambda: time.time()` to follow a patched `time.time`. max_connections: The most SQLite connections this process opens for the directory, shared by every handle on it (default 8). sqlite_cache_size: SQLite's page cache per connection, in 4 KiB pages (default 256, 1 MiB). wal_size_limit: The write-ahead log is trimmed back to half this many bytes whenever it resets (default 64 MiB, minimum 1 MiB). wal_checkpoint: `'manual'` (the default) leaves checkpoints to `checkpoint()` and `close()`. `'auto'` also checkpoints after writes once the log is over `wal_size_limit`. on_decode_error: What a read does with a stored value it can't decode. `'raise'` (the default, unless `durable=True`, where it's `'miss'`) raises. `'miss'` treats it as a miss, logs a warning on the `diskcache_oxide` logger, and deletes the row if no build could ever read it (a class removed since it was pickled, malformed bytes). A row another version or host might read (`ForeignValueError`) is left in place. A callable does the same as `'miss'`, but is called with `(key, exc)` instead of the warning, after the row is dealt with; whatever it raises is logged. `UnsafeValueError`, `Timeout`, `MemoryError` and other errors that aren't about the stored bytes raise, though under `durable=True` a `Timeout`, `OSError` or `RecursionError` from a pickle hook is a miss, with the row kept, and so is a `RecursionError` from a value too deep for this thread's stack. Covers `get`, `cache[key]` and `pop`, which decodes before it deletes, so it keeps a row it can't read. Under `'raise'`, `pop` still deletes a value no build could read, then raises. durable: Keep the cache from taking a request down, for an app whose every caller can fall back to its source of truth (default False). An outage on a single-key call (a `Timeout`, a full disk, an I/O error, a damaged database) doesn't raise: `get`, `pop` and `in` give the default or False, `set`, `add`, `touch` and `delete` give False, and `incr` and `decr` give `default + delta`. It's reported on the `diskcache_oxide` logger, at most once a minute per kind of failure. Bugs still raise (`TypeError`, `ValueError`, `UnsafeValueError`), as do `MemoryError` and `KeyboardInterrupt`, a `RecursionError` from encoding a value too deep for this thread's stack, the constructor, admin calls (`len`, `volume`, `clear`, `expire`, `cull`, `checkpoint`, iteration) and anything inside a transaction held on this cache. This isn't durability in the database sense: this option can drop writes, and a dropped `set`, `delete`, `pop` or `incr` means stale data until the entry expires, or forever if it has no expiry. See "A cache that can't take a request down" in the Getting Started guide before you use it. Raises: ValueError: A setting is out of range, such as a `size_limit` under 1 MiB. TypeError: An argument has the wrong type, including a `bool` for `timeout`. FormatMismatchError: The directory holds a database in a format this version can't read. LayoutMismatchError: The directory is a `FanoutCache` root or one of its shards. Also a `ValueError`, and a `FormatMismatchError`. Timeout: Another process held the database locked, or a fork stayed pending, for longer than `timeout`. Examples: ```python from diskcache_oxide import Cache cache = Cache('/var/cache/my-app', timeout=1.0) # every worker opens the same directory cache.set('user:42', {'name': 'Sam', 'roles': {'admin'}}, expire=300) cache.get('user:42') # the dict, with roles still a set; None on a miss cache['greeting'] = 'hello' # never expires if cache.add('job:nightly', 1, expire=3600): # exactly one process wins run_nightly_job() cache.close() # or `with Cache(...) as cache:` ``` ## Cache Methods Methods for the Cache class directory The cache directory, as given to the constructor. timeout Seconds a call waits for a lock or a pending fork before raising `Timeout`. size_limit The disk budget in bytes. A `set()` that finds the cache over it evicts entries. cull_limit The most entries a single `set()` evicts when the cache is over `size_limit`. enabled Always True for a real cache; False for a `NullCache`. set(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' Store `value` under `key`, replacing any existing value. Values of 64 KiB or more (after encoding) are stored in their own file. Args: key: The key. value: The value: `str`, `bytes`, numbers, `Decimal`, `bool`, `None`, and lists, tuples, dicts, sets, frozensets, datetimes and dates of those, or anything picklable with `pickle=True`. expire: Seconds until the entry expires, or None for never. Returns: True. With `durable=True`, False if the write failed and was dropped: whatever was stored under `key` stays, and can be served stale. Raises: TypeError: The value can't be stored exactly without pickle, or `expire` is a `bool`. Timeout: Another process held the write lock for longer than `timeout`. Examples: ```python cache.set('episodes:popular', episodes, expire=300) # expires in 5 minutes cache.set('config', {'retries': 3}) # never expires ``` add(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' Store `value` only if `key` is missing or expired. Atomic across threads and processes: when several callers race to add one key, exactly one succeeds. That makes it a simple way to claim a job or take a lock. Args: key: The key. value: The value, as for `set()`. expire: Seconds until the entry expires, or None for never. Returns: True if the value was stored, False if the key already held a live value. With `durable=True`, also False if the write failed, so a claim fails closed. A `NullCache` returns True, so use a non-durable handle for claims and locks. Raises: TypeError: The value can't be stored exactly without pickle, or `expire` is a `bool`. Timeout: Another process held the write lock for longer than `timeout`. Examples: ```python if cache.add('job:hourly-cleanup', os.getpid(), expire=55 * 60): remove_stale_sessions() # only the worker whose add() returned True gets here ``` delete(self, key: 'str') -> 'bool' Remove `key`. Returns: True if a live entry was removed, False if the key was missing or expired. With `durable=True`, also False if the delete failed: that invalidation is lost, and the entry can be served stale until it expires, or forever if it has no expiry. The log line is the only signal. Raises: Timeout: Another process held the write lock for longer than `timeout`. Examples: ```python cache.delete('user:42') # True if it was there for key in cache.iterkeys(prefix='user:42:'): cache.delete(key) # safe while iterating ``` pop(self, key: 'str', default: 'Any' = None) -> 'Any' Remove `key` and return its value, delivering it to exactly one caller. Reads and decodes the value first, as `get` does, then deletes exactly that stored version, so a value it can't read stays for whoever can. A `pop` that loses that delete to other writers 8 times falls back to removing the row first and decoding afterwards. Only then can a failed decode cost the value: one from a newer version, or any error from a pickle hook. `timeout` covers the whole call. Args: key: The key. default: Returned when the key is missing or expired, or its value file is missing. Returns: The removed value, or `default`. With `durable=True`, a `pop` whose delete fails returns `default`, never the value, which could otherwise be delivered twice. The row stays. Raises: UnsafeValueError: The value was stored with pickle and this cache has `pickle=False`. The row stays. ForeignValueError: `on_decode_error='raise'`, and another version or host might read the value. The row stays. CorruptValueError: `on_decode_error='raise'`, and no build could read the value. The row is deleted first. Timeout: Another process held the write lock for longer than `timeout`, or `timeout` ran out after losing a race to another writer. This `pop` deletes nothing, and whatever the other writer left stays. Examples: ```python token = cache.pop('reset-token:abc') # None if it was missing or already used ``` touch(self, key: 'str', expire: 'float | None' = None) -> 'bool' Change the expiry of an existing entry without rewriting its value. Args: key: The key. expire: Seconds from now until the entry expires, or None to make it never expire. Returns: True if the entry existed and was updated, False if it was missing or expired, or with `durable=True`, if the call failed. Raises: Timeout: Another process held the write lock for longer than `timeout`. Examples: ```python cache.touch('session:xyz', expire=1800) # 30 more minutes, if it's still there ``` incr(self, key: 'str', delta: 'float' = Ellipsis, default: 'float | None' = Ellipsis) -> 'int | float' Add `delta` to the number stored under `key`, atomically, and return the result. Atomic across threads and processes, so concurrent increments are never lost. The entry keeps its existing expiry. Args: key: The key. delta: The amount to add (default 1). default: The starting value for a missing or expired key (default 0). Pass None to raise `KeyError` instead. Returns: The new value. With `durable=True`, `default + delta` if the call failed, whatever was stored, so a rate limiter fails open. Don't use that number as an id: two callers can get the same one. Raises: KeyError: The key is missing and `default` is None, or with `durable=True`, the call failed and `default` is None. TypeError: The stored value isn't a number. Timeout: Another process held the write lock for longer than `timeout`. Examples: ```python cache.incr('page_views') # 1 on a missing key, then 2, 3, ... cache.add('rate:203.0.113.7:1731', 0, expire=120) # a windowed counter: create it with an expiry, allowed = cache.incr('rate:203.0.113.7:1731') <= 100 # then count; incr keeps the expiry ``` decr(self, key: 'str', delta: 'float' = Ellipsis, default: 'float | None' = Ellipsis) -> 'int | float' Subtract `delta` from the number stored under `key`, atomically. See `incr()`. Returns: The new value. iterkeys(self, *, prefix: 'str | None' = None, reverse: 'bool' = False) -> 'Iterator[str]' Iterate over keys in sorted order, optionally only those starting with `prefix`. A prefix is an indexed range scan, so it costs time in proportion to the matches, not the size of the cache. Keys are fetched a page at a time with no lock held in between, so deleting keys while iterating is safe. Expired entries not yet removed are included. Like a generator, an iterator serves one thread at a time: `next()` raises `ValueError` while another thread is inside it. Args: prefix: Only yield keys that start with this string. reverse: Yield keys in descending order. Returns: An iterator of keys. Examples: ```python for key in cache.iterkeys(prefix='user:42:'): # an indexed range scan, sorted cache.delete(key) newest_first = cache.iterkeys(reverse=True) ``` expire(self, *, timeout: 'float | None' = None) -> 'int' Remove every expired entry, and its value file. Works in chunks and lets other writers in between them. Args: timeout: Seconds each chunk may wait for the write lock (another thread's or another process's), for this call only (default: the cache's `timeout`). It bounds each wait, not the whole call. Returns: How many entries were removed. Examples: ```python removed = cache.expire(timeout=10.0) # from a scheduled job, with its own timeout ``` clear(self, *, timeout: 'float | None' = None) -> 'int' Remove every entry that existed when the call started. Works in chunks and lets other writers in between them. Writes that land while it runs, including overwrites of existing keys, are kept. Afterwards it sweeps the large-value folders (D-042): a temp file no writer has touched for an hour (left by a process killed mid-write) is removed, and every empty folder goes, for at most 100 ms in all, unless this thread is inside a transaction (any cache's). Args: timeout: Seconds each chunk may wait for the write lock (another thread's or another process's), for this call only (default: the cache's `timeout`). It bounds each wait, not the whole call. Returns: How many entries were removed. Examples: ```python cache.clear(timeout=10.0) # safe while other processes keep writing ``` cull(self, *, timeout: 'float | None' = None) -> 'int' Remove expired entries, then evict the least recently stored until under `size_limit`. Args: timeout: Seconds each chunk may wait for the write lock (another thread's or another process's), for this call only (default: the cache's `timeout`). It bounds each wait, not the whole call. Returns: How many entries were removed. volume(self) -> 'int' The cache's size on disk in bytes: the database plus value files. checkpoint(self, *, timeout: 'float | None' = None) -> 'bool' Copy the write-ahead log into the database and truncate it to zero. With readers in other processes, a burst of writes can leave the log much larger than the database. Call this when a short pause suits the app: at shutdown, in a maintenance job, or after a large `clear()`. `close()` also does it for a log left large. Args: timeout: Seconds to keep retrying while other processes or a pending fork are busy (default: the cache's `timeout`). Returns: True if the log was reset, False if readers or writers in other processes, or a pending fork, kept it busy. It never raises for a busy database. Examples: ```python cache.checkpoint() # at shutdown, or after a big fill; False means "still busy, try later" ``` close(self, *, checkpoint: 'bool' = True) -> 'None' Close this handle and return its connections. When the last handle on a directory in the process closes, every connection to it closes, whichever threads used them, and the folders that large values use are swept (D-042): a temp file no writer has touched for an hour, left by a process killed mid-write, is removed, and a folder left empty goes (those not touched in the last 30 seconds), for at most 100 ms in all, unless this thread is inside a transaction. A handle dropped without `close()` does the same. Using the cache again afterwards reopens it. It never raises. Behind a fork that stays stuck it returns after about 2 seconds at most, whatever `timeout` is, and those connections close when the fork ends. Args: checkpoint: If this is the process's last handle and the write-ahead log is over half of `wal_size_limit`, checkpoint it first, waiting at most 1 second. A small log costs nothing extra. Examples: ```python with Cache('/var/cache/my-app') as cache: # closed at the end of the block cache.set('k', 'v') cache.close(checkpoint=False) # skip the checkpoint, for a quick exit ``` transact(self, retry: 'bool' = False) -> 'Transaction[Cache]' A write transaction, for `with cache.transact() as tx:`. `tx` is the cache itself. On the thread that entered the block, every call on this cache directory inside it, through any handle, runs in the transaction: it commits when the block exits and rolls back if the block raises. Other threads' and processes' writes wait for it. Entered on a thread that already holds a transaction on this directory, it joins that one: the inner block's exit doesn't commit, and an exception the caller catches inside doesn't undo the inner block's writes. A transaction belongs to a thread, not an asyncio task: another coroutine's writes on the same thread during an `await` inside the block join it and roll back with it. A block that exits with no exception but didn't commit (another thread ended it, or it was rolled back after an error inside it) raises `TransactionAborted` at its exit, so it never exits cleanly when its writes are gone. Args: retry: Keep trying to begin while it times out, instead of raising `Timeout`. Never retries `TransactionBusy`. Each attempt waits up to `timeout`. Raises: TypeError: `retry` isn't a `bool`. Examples: ```python with cache.transact() as tx: # all three land together, or none do tx.incr('requests') tx.incr(f'requests:{path}') tx.set('last_request', now) ``` __getitem__(self, key: 'str', /) -> 'Any' `cache[key]`: the value, or `KeyError` if it's missing or expired, or the read failed under `durable=True`. __setitem__(self, key: 'str', value: 'Any', /) -> 'None' `cache[key] = value`: store a value that never expires, like `set()`. Under `durable=True` a failed write is dropped. __delitem__(self, key: 'str', /) -> 'None' `del cache[key]`: remove the entry, or raise `KeyError` if it's missing or expired, or the delete failed under `durable=True`. __contains__(self, key: 'object', /) -> 'bool' `key in cache`: True if the key holds a live (unexpired) value. False if the check failed under `durable=True`. __len__(self) -> 'int' The number of entries, including expired entries not yet removed. __bool__(self) -> 'bool' Always True, even when empty, so `if cache:` means "caching is on". __iter__(self) -> 'Iterator[str]' Iterate over keys in insertion order. An overwrite keeps its key's place. As with `iterkeys()`, `next()` raises `ValueError` while another thread is inside it. __enter__(self) -> 'Self' Return the cache, for `with Cache(...) as cache:`. __exit__(self, exc_type: 'type[BaseException] | None', exc: 'BaseException | None', tb: 'TracebackType | None', /) -> 'None' Close the cache, as `close()` does. get(self, key: 'str', default: 'Any' = None, *, expire_time: 'Literal[False]' = False) -> 'Any' Return the value stored for `key`, or `default` if it's missing or expired. Reads never write, so any number of threads and processes can read at once. Args: key: The key to look up. default: Returned when the key is missing or expired. expire_time: Also return the entry's expiry time. Returns: The value, or `default`. With `expire_time=True`, a `(value, expire_time)` tuple, where `expire_time` is a Unix timestamp, or None for an entry that never expires (or a miss). Raises: UnsafeValueError: The value was stored with pickle and this cache has `pickle=False`. CorruptValueError: The stored bytes can't be decoded. Timeout: Reading a large value kept losing to concurrent overwrites until `timeout`, or a pending fork outlasted it. `durable=True` returns `default` instead. Examples: ```python user = cache.get('user:42') # None on a miss user = cache.get('user:42', default={}) # {} on a miss value, expires_at = cache.get('user:42', expire_time=True) ``` ## FanoutCache A cache sharded over several SQLite databases, so writers in many processes stop queueing on one write lock. FanoutCache(directory: 'str | os.PathLike[str]', timeout: 'float | None' = None, *, shards: 'int | None' = None, size_limit: 'int | None' = None, cull_limit: 'int | None' = None, serializer: 'str' = 'msgpack', pickle: 'bool' = False, types: 'TypeRegistry | None' = None, clock: 'Callable[[], float] | None' = None, max_connections: 'int | None' = None, sqlite_cache_size: 'int | None' = None, wal_size_limit: 'int | None' = None, wal_checkpoint: "Literal['manual', 'auto']" = 'manual', on_decode_error: "Literal['raise', 'miss'] | Callable[[str, Exception], object] | None" = None, durable: 'bool' = False) -> 'FanoutCache' A disk cache split over several SQLite databases (shards), so reads and writes on different keys don't queue on one file's locks. Each key lives in exactly one shard, chosen by a hash that is part of the on-disk format: every process, and every release, sends a key to the same shard. The root records its shard count, so every handle agrees on it. Single-key calls behave exactly as on `Cache`, on the key's shard. `len()` and `volume()` add up every shard under one `timeout`. `clear()`, `expire()`, `cull()` and `checkpoint()` run on every shard in turn, under one `timeout` for the whole call. If a shard fails, the others are still done, and then the first failure raises, as its own type (so `except Timeout` still works), with two extra attributes and a note naming them: - `failed_shards`: a tuple of the indexes of the shards that failed, in order. - `removed` (not on `checkpoint()`): how many entries the call removed in all, from every shard, including what a failed shard removed before it failed. Args: directory: The root directory. It holds the manifest (`diskcache-oxide-fanout`) and one directory per shard, `000` to `NNN`. It must not be a plain `Cache`'s directory. timeout: As for `Cache` (default 60). Opening, `len()`, `volume()`, `clear()`, `expire()`, `cull()`, `checkpoint()` and `close()` share one `timeout` across every shard, not one each. shards: How many shards, 1 to 256. `None` (the default) uses the count the root records, or 8 for a new root. A count that differs from the recorded one raises `LayoutMismatchError` (a `ValueError`): changing it needs a new directory. Each shard takes about 4 file descriptors to open, and 2 more per connection it opens in use, so 256 shards need over 1,024: more than Linux's default `ulimit -n`. Too few raises `OSError` (`EMFILE`) with the numbers. size_limit: The disk budget in bytes for the whole cache, split evenly (floored) across the shards, each of which culls on its own. Each shard needs at least 1 MiB, so the total must be at least `shards` MiB. With none given, a new root gets 1 GiB in all. cull_limit: As for `Cache`, per shard. serializer: As for `Cache`. pickle: As for `Cache`. clock: As for `Cache`. max_connections: The most SQLite connections this process opens for the whole cache, split across the shards (the first `max_connections % shards` get one more). At least `shards`. The default is two per shard, and never fewer than 8 in all. sqlite_cache_size: As for `Cache`, per connection. wal_size_limit: As for `Cache`, per shard (each shard has its own write-ahead log). wal_checkpoint: As for `Cache`. on_decode_error: As for `Cache`. durable: As for `Cache`: single-key calls on a shard that fails give their miss values. One report covers every shard, named by the root directory. Raises: ValueError: A setting is out of range, or `max_connections` is below `shards`. LayoutMismatchError: `shards` differs from the root's, or the directory belongs to a plain `Cache` (or a shard of it does). Also a `ValueError`, and a `FormatMismatchError`. TypeError: An argument has the wrong type, including a `bool` for `timeout` or `shards`. FormatMismatchError: The root's manifest, or a shard's database, is in a format this version can't read. OSError: The filesystem doesn't support hard links, which creating the manifest needs, or the process can't open enough files for every shard (`ulimit -n`). Timeout: A shard stayed locked, or a fork stayed pending, for longer than `timeout`. Examples: ```python from diskcache_oxide import FanoutCache cache = FanoutCache('/var/cache/my-app-counters', shards=8) # shards is keyword-only cache.incr('ip:203.0.113.7') # as on Cache, on the key's shard cache.shard_of('ip:203.0.113.7') # 0 to 7, the same in every process and release with cache.transact(key='totals') as tx: # one shard's write lock, not all of them tx.set('totals', tx.get('totals', 0) + 1) ``` ## FanoutCache Methods Methods for the FanoutCache class directory The root directory, as given to the constructor. timeout Seconds a call waits for a lock or a pending fork before raising `Timeout`. shards How many shards the root has. size_limit The disk budget in bytes: the sum of every shard's. cull_limit The most entries a single `set()` evicts from its shard when that shard is over its share. Shard 0's value: every shard's only while they were all created, and last set, under the same settings. enabled Always True for a real cache; False for a `NullCache`. shard_of(self, key: 'str') -> 'int' The shard `key` lives in, from 0 to `shards - 1`: the same in every process and release. Examples: ```python cache.shard_of('user:42') == cache.shard_of('user:42:profile') # usually False: no grouping by prefix ``` transact(self, *, key: 'str', retry: 'bool' = False) -> 'Transaction[FanoutCache]' A write transaction on the shard `key` lives in, for `with cache.transact(key='total') as tx:`. It is `Cache.transact()` on that one shard: `tx` is this FanoutCache, calls on this thread inside the block for keys on that shard run in the transaction (through any handle on the root), it commits when the block exits and rolls back if the block raises, and a nested `transact` for a key on the same shard joins it. Other threads' and processes' writes to that shard wait for it; the other shards are not locked. Inside the block, a call on this thread for a key on any other shard raises `ValueError`, reads included, through any handle on the root, and so does a `transact` for a key on another shard (its writes could not be in the transaction). Use `shard_of` to check that keys share a shard. Calls over every shard raise `ValueError` on this thread inside the block too: `len()`, `volume()`, `iter()`, `iterkeys()` (and `next()` on an iterator made before the block), `clear()`, `expire()`, `cull()` and `checkpoint()`. They would run inside the transaction on its shard and outside it on the others. The check costs one atomic load while no transaction is open anywhere in the process; while one is (any thread's, any cache's), keyed calls, calls over every shard and iterator `next()`s also look at each shard that holds one. As on `Cache`, a transaction belongs to a thread, not an asyncio task, a block that didn't commit raises `TransactionAborted` at its exit, and under `durable=True` a failure to begin, or inside the block, raises. Args: key: Keyword-only. Any key on the shard to hold; it needn't exist. retry: As for `Cache.transact()`. Raises: TypeError: `key` isn't a `str`, or `retry` isn't a `bool`. Examples: ```python with cache.transact(key='totals:2026-11') as tx: n = tx.get('totals:2026-11', 0) tx.set('totals:2026-11', n + 1) ``` set(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' As `Cache.set`, on the key's shard. add(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' As `Cache.add`, on the key's shard: atomic across threads and processes. delete(self, key: 'str') -> 'bool' As `Cache.delete`, on the key's shard. pop(self, key: 'str', default: 'Any' = None) -> 'Any' As `Cache.pop`, on the key's shard. touch(self, key: 'str', expire: 'float | None' = None) -> 'bool' As `Cache.touch`, on the key's shard. incr(self, key: 'str', delta: 'float' = Ellipsis, default: 'float | None' = Ellipsis) -> 'int | float' As `Cache.incr`, on the key's shard: atomic across threads and processes. decr(self, key: 'str', delta: 'float' = Ellipsis, default: 'float | None' = Ellipsis) -> 'int | float' As `Cache.decr`, on the key's shard. volume(self) -> 'int' The total disk use in bytes of every shard (databases plus value files). Raises on any shard's failure, under `durable=True` too. expire(self, *, timeout: 'float | None' = None) -> 'int' Remove every expired entry from every shard, and its value file, as `Cache.expire` does. Args: timeout: Seconds the whole call may spend waiting for write locks (another thread's or another process's) or a pending fork, across every shard, for this call only (default: the cache's `timeout`). It bounds waiting, not work: a chunk already running finishes, and once it's spent, each remaining chunk still gets one try with no wait, so shards no one else is writing are all done even at a short timeout. Returns: How many entries were removed. Raises: Timeout: Some shard stayed locked past `timeout`; the others were still done. It carries `removed` and `failed_shards` (see the class docs), as any failure here does, under `durable=True` too. Examples: ```python try: cache.expire(timeout=5.0) except Timeout as exc: exc.failed_shards, exc.removed # which shards were skipped, and how many entries went anyway ``` clear(self, *, timeout: 'float | None' = None) -> 'int' Remove every entry that existed when the call started, from every shard. Every shard's starting point is taken before anything is removed, so writes that land while it runs, on any shard, are kept, as on `Cache.clear`. Args: timeout: As for `expire`: one budget for the whole call. Returns: How many entries were removed. Raises: Timeout: As for `expire`, with `removed` and `failed_shards`. cull(self, *, timeout: 'float | None' = None) -> 'int' Remove expired entries, then evict the least recently stored from each shard until it's under its share of `size_limit`. Args: timeout: As for `expire`: one budget for the whole call. Returns: How many entries were removed. Raises: Timeout: As for `expire`, with `removed` and `failed_shards`. checkpoint(self, *, timeout: 'float | None' = None) -> 'bool' Checkpoint every shard's write-ahead log and truncate it to zero, as `Cache.checkpoint` does. Args: timeout: Seconds to keep retrying busy shards, for the whole call (default: the cache's `timeout`). A shard reached after it's spent still gets one try. Returns: True if every shard's log was reset, False if readers or writers in other processes, or a pending fork, kept any of them busy. Raises: OSError: A real error, such as a full disk, on some shard, raised after every shard was tried. It carries `failed_shards` (see the class docs). iterkeys(self, *, prefix: 'str | None' = None, reverse: 'bool' = False) -> 'Iterator[str]' Iterate over keys in sorted order, optionally only those starting with `prefix`. The same order `Cache.iterkeys` gives for the same keys (code-point order, like `sorted()`): a merge of every shard's sorted keys. Each shard keeps a small page of keys buffered, so the first `next()` reads every shard once, and later ones read a shard only when its page runs out. No connection or lock is held between `next()` calls. As on `Cache`, it isn't a snapshot: a key stored or deleted while it runs may or may not be yielded, but none is yielded twice and the order holds. Expired entries not yet removed are included. `next()` raises `ValueError` while another thread is inside it. Args: prefix: Only yield keys that start with this string. reverse: Yield keys in descending order. Raises: Timeout: A shard stayed locked, or a fork stayed pending, for longer than `timeout` during one `next()` (which shares that one budget across the shards it reads). __iter__(self) -> 'Iterator[str]' Iterate over every key once: shard 0's in insertion order, then shard 1's, and so on. That isn't the order keys were stored in across the cache. Otherwise as `iterkeys()`. close(self, *, checkpoint: 'bool' = True) -> 'None' Close this handle on every shard, as `Cache.close()` does for one. The whole close shares one budget: the checkpoints share at most 1 second, and behind a fork that stays stuck every release together waits at most 1 second more. It never raises. Using the cache again reopens the shards it touches. Args: checkpoint: Checkpoint each shard's write-ahead log first, as `Cache.close` does. __getitem__(self, key: 'str', /) -> 'Any' `cache[key]`, as on `Cache`. __setitem__(self, key: 'str', value: 'Any', /) -> 'None' `cache[key] = value`, as on `Cache`. __delitem__(self, key: 'str', /) -> 'None' `del cache[key]`, as on `Cache`. __contains__(self, key: 'object', /) -> 'bool' `key in cache`, as on `Cache`. __len__(self) -> 'int' The number of entries in every shard, including expired entries not yet removed. Raises on any shard's failure, under `durable=True` too. __bool__(self) -> 'bool' Always True, even when empty, so `if cache:` means "caching is on". __enter__(self) -> 'Self' Return the cache, for `with FanoutCache(...) as cache:`. __exit__(self, exc_type: 'type[BaseException] | None', exc: 'BaseException | None', tb: 'TracebackType | None', /) -> 'None' Close the cache, as `close()` does. get(self, key: 'str', default: 'Any' = None, *, expire_time: 'Literal[False]' = False) -> 'Any' As `Cache.get`, on the key's shard. ## NullCache Switched-off cache behaving as if every entry expires instantly. NullCache(directory: 'str | os.PathLike[str] | None' = None, timeout: 'float | None' = None, *, size_limit: 'int | None' = None, cull_limit: 'int | None' = None, serializer: "Literal['msgpack', 'json']" = 'msgpack', pickle: 'bool' = False, types: 'TypeRegistry | None' = None, clock: 'Callable[[], float] | None' = None, max_connections: 'int' = 8, sqlite_cache_size: 'int | None' = None, wal_size_limit: 'int | None' = None, wal_checkpoint: "Literal['manual', 'auto']" = 'manual', on_decode_error: "Literal['raise', 'miss'] | Callable[[str, Exception], object] | None" = None, durable: 'bool' = False, invalidate: 'CacheLike | None' = None) -> 'None' A switched-off cache, so app code needs no `if cache is None` checks. It behaves like a real cache in which every entry expires the instant it's written and is swept straight away: reads miss, writes succeed, and it's always empty. It's falsy, and its `enabled` is False. Args: directory: Accepted and ignored, like every other `Cache` argument, so switching between the two is a one-line change. invalidate: A real cache to forward removals to (`delete`, `pop`, `clear` and `iterkeys`), so invalidation by removal still lands while caching is off. Writes aren't forwarded: an overwriting `set` or a version-bump `incr` is dropped, and the old value is served once caching is back on. Examples: ```python from diskcache_oxide import Cache, NullCache cache = NullCache() cache.set('k', 1) # True, and nothing is stored cache.get('k', 'miss') # 'miss' cache.add('job', 1) # True: don't claim jobs on a cache that might be off real = Cache('/var/cache/my-app') off = NullCache(invalidate=real) off.delete('user:42') # removed from `real`; off.get('user:42') is still None ``` CacheLike(*args, **kwargs) What application code can rely on from a `Cache`, a `FanoutCache` or a `NullCache`. All three satisfy it, so a factory can return either and callers never need to check which one they got. A test keeps the three signatures identical. Examples: ```python from diskcache_oxide import Cache, CacheLike, NullCache def make_cache(enabled: bool) -> CacheLike: return Cache('/var/cache/my-app') if enabled else NullCache() def load_user(cache: CacheLike, user_id: str) -> dict: if (user := cache.get(f'user:{user_id}')) is not None: return user user = db.load_user(user_id) cache.set(f'user:{user_id}', user, expire=300) return user ``` TransactionalCacheLike(*args, **kwargs) A `CacheLike` with `transact()`, which `Cache` and `NullCache` both have. On a `Cache`, `with cache.transact() as tx:` runs the block's calls on this thread in one write transaction; `tx` is the cache. On a `NullCache` it's a no-op that yields the `NullCache`. Examples: ```python from diskcache_oxide import TransactionalCacheLike def count(cache: TransactionalCacheLike, path: str) -> None: with cache.transact() as tx: tx.incr('requests') tx.incr(f'requests:{path}') ``` ShardedCacheLike(*args, **kwargs) A `CacheLike` with a sharded cache's `transact(*, key, retry=False)`, which `FanoutCache` and `NullCache` both have. On a `FanoutCache`, `with cache.transact(key='total') as tx:` runs the block's calls on this thread for keys on `key`'s shard in one write transaction, and refuses keys on other shards (`ValueError`); `tx` is the cache. On a `NullCache` it's a no-op that yields the `NullCache`. `key` is keyword-only, so `transact('total')` is a type error rather than a `retry` of 'total'. Examples: ```python from diskcache_oxide import ShardedCacheLike def bump(cache: ShardedCacheLike, key: str) -> None: with cache.transact(key=key) as tx: tx.set(key, tx.get(key, 0) + 1) ``` ## NullCache Methods Methods for the NullCache class __repr__(self) -> 'str' Shows the directory and any `invalidate=` cache. directory The directory passed to the constructor, or None. Nothing is created there. timeout The timeout passed to the constructor (default 60). Unused. size_limit The size limit passed to the constructor (default 1 GiB). Unused. cull_limit The cull limit passed to the constructor (default 10). Unused. enabled Always False: caching is off. get(self, key: 'str', default: 'Any' = None, *, expire_time: 'bool' = False) -> 'Any' Return `default`, as every entry has already expired. With `expire_time=True`, `(default, None)`. __getitem__(self, key: 'str', /) -> 'Any' Always raises `KeyError`, as every entry has already expired. __contains__(self, key: 'object', /) -> 'bool' Always False. __len__(self) -> 'int' Always 0. __bool__(self) -> 'bool' Always False, so `if cache:` means "caching is on". __iter__(self) -> 'Iterator[str]' Yields nothing. volume(self) -> 'int' Always 0: a `NullCache` uses no disk. checkpoint(self, *, timeout: 'float | None' = None) -> 'bool' Always True: there is no write-ahead log to reset. set(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' Return True without storing anything: the write succeeds, then expires instantly. __setitem__(self, key: 'str', value: 'Any', /) -> 'None' Store nothing, as `set()` does. add(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' Return True without storing anything: the key is always absent, so the add succeeds. With caching off, every caller's `add()` succeeds, so don't rely on it for job claims or locks. A `Cache(durable=True)` does the opposite when a write fails: its `add()` returns False. touch(self, key: 'str', expire: 'float | None' = None) -> 'bool' Always False: the key never exists. incr(self, key: 'str', delta: 'float' = 1, default: 'float | None' = 0) -> 'int | float' Return `default + delta`, since every count starts again from `default`. Raises: KeyError: `default` is None. decr(self, key: 'str', delta: 'float' = 1, default: 'float | None' = 0) -> 'int | float' Return `default - delta`. See `incr()`. delete(self, key: 'str') -> 'bool' False, or with `invalidate=`, delete the key from that cache and return its result. __delitem__(self, key: 'str', /) -> 'None' Raise `KeyError`, or with `invalidate=`, delete the key from that cache. pop(self, key: 'str', default: 'Any' = None) -> 'Any' Return `default`. With `invalidate=`, also delete the key from that cache. clear(self, *, timeout: 'float | None' = None) -> 'int' 0, or with `invalidate=`, clear that cache (passing `timeout` on) and return its count. iterkeys(self, *, prefix: 'str | None' = None, reverse: 'bool' = False) -> 'Iterator[str]' Yield nothing, or with `invalidate=`, that cache's keys, so list-then-delete still works. transact(self, retry: 'bool' = False, *, key: 'str | None' = None) -> 'AbstractContextManager[Self]' A no-op context manager that yields this `NullCache`. It never opens a transaction on an `invalidate=` cache: removals forwarded inside the block land at once, and an exception in the block doesn't undo them. Args: retry: Accepted for `Cache.transact()`'s signature; it must be a `bool`. key: Accepted so a switched-off sharded cache's `transact(key=...)` works too; it must be a `str` or None. Raises: TypeError: `retry` isn't a `bool`, or `key` isn't a `str` or None. expire(self, *, timeout: 'float | None' = None) -> 'int' Always 0. cull(self, *, timeout: 'float | None' = None) -> 'int' Always 0. close(self, *, checkpoint: 'bool' = True) -> 'None' Do nothing. An `invalidate=` cache is never closed: it belongs to whoever created it. __enter__(self) -> 'Self' Return the cache, for `with NullCache() as cache:`. __exit__(self, exc_type: 'type[BaseException] | None', exc: 'BaseException | None', tb: 'TracebackType | None', /) -> 'None' Do nothing, as `close()` does. ## CacheLike Methods Methods for the CacheLike class directory The cache directory (None for a `NullCache` created without one). timeout Seconds a write waits for another process's lock before raising `Timeout`. size_limit The disk budget in bytes. cull_limit The most entries one `set()` evicts when over `size_limit`. enabled True if caching is on (`Cache`), False if it's off (`NullCache`). get(self, key: 'str', default: 'Any' = None, *, expire_time: 'bool' = False) -> 'Any' Return the value for `key`, or `default` if it's missing or expired. set(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' Store `value` under `key`, with an optional expiry in seconds. add(self, key: 'str', value: 'Any', expire: 'float | None' = None) -> 'bool' Store `value` only if `key` is missing or expired. True if it was stored. delete(self, key: 'str') -> 'bool' Remove `key`. True if a live entry was removed. pop(self, key: 'str', default: 'Any' = None) -> 'Any' Remove `key` and return its value, or `default`. touch(self, key: 'str', expire: 'float | None' = None) -> 'bool' Change an existing entry's expiry. True if the entry existed. incr(self, key: 'str', delta: 'float' = 1, default: 'float | None' = 0) -> 'int | float' Add `delta` to the number under `key`, atomically, and return the result. decr(self, key: 'str', delta: 'float' = 1, default: 'float | None' = 0) -> 'int | float' Subtract `delta` from the number under `key`, atomically, and return the result. iterkeys(self, *, prefix: 'str | None' = None, reverse: 'bool' = False) -> 'Iterator[str]' Iterate over keys in sorted order, optionally only those starting with `prefix`. expire(self, *, timeout: 'float | None' = None) -> 'int' Remove every expired entry. Returns how many were removed. clear(self, *, timeout: 'float | None' = None) -> 'int' Remove every entry that existed when the call started. Returns how many. cull(self, *, timeout: 'float | None' = None) -> 'int' Remove expired entries, then evict until under `size_limit`. Returns how many. volume(self) -> 'int' The size on disk in bytes. checkpoint(self, *, timeout: 'float | None' = None) -> 'bool' Empty the write-ahead log. True if it was reset, False if other processes kept it busy. close(self, *, checkpoint: 'bool' = True) -> 'None' Close this handle and return its connections. __getitem__(self, key: 'str', /) -> 'Any' `cache[key]`: the value, or `KeyError` if it's missing or expired. __setitem__(self, key: 'str', value: 'Any', /) -> 'None' `cache[key] = value`: store a value that never expires. __delitem__(self, key: 'str', /) -> 'None' `del cache[key]`: remove the entry, or raise `KeyError`. __contains__(self, key: 'object', /) -> 'bool' `key in cache`: True if the key holds a live value. __len__(self) -> 'int' The number of entries. __bool__(self) -> 'bool' True if caching is on, so `if cache:` reads naturally. __iter__(self) -> 'Iterator[str]' Iterate over every key once. A `Cache` yields them in insertion order. __enter__(self) -> 'Self' Return the cache, for use in a `with` block. __exit__(self, exc_type: 'type[BaseException] | None', exc: 'BaseException | None', tb: 'TracebackType | None', /) -> 'None' Close the cache. ## Caches A registry of named caches under one base directory. Caches(base_dir: 'str | os.PathLike[str]', *, on_open_error: 'Callable[[str, Exception], object] | None' = None, retry_open_after: 'float' = 10.0, _monotonic: 'Callable[[], float]' = , **defaults: 'Any') -> 'None' A registry of named caches under one base directory. `Caches[Literal['general', 'show_pages']]` makes a misspelled name a type error. Args: base_dir: Where the caches live: each one in `base_dir/` unless `register` says otherwise. Created when the first cache opens. on_open_error: Called as `on_open_error(name, exc)` when a cache can't open (default: log at ERROR on the `diskcache_oxide` logger, or at WARNING with no traceback for a `Timeout`). Whatever it raises is logged. retry_open_after: Seconds before a cache that failed to open is tried again (default 10). Lookups in between get a `NullCache` without trying. A `Timeout` means busy rather than broken, so that's tried again after at most 1 s. **defaults: `Cache` settings for every cache, such as `timeout=1.0`, `durable=True` or `on_decode_error='miss'`. `register` can override them. A callable `on_decode_error` here is called as `(name, key, exc)`, so one reporter can tell the caches apart. Examples: ```python from diskcache_oxide import Caches caches = Caches('/var/cache/my-app', timeout=1.0, on_decode_error='miss') caches.register('general') caches.register('pages', size_limit=2 * 2**30) caches.register('rate_limits', shards=8) caches['general'].set('greeting', 'hello') caches.fanout('rate_limits').incr('ip:203.0.113.7') caches.stats()['pages'].count ``` CacheStats(name: 'str', directory: 'str', enabled: 'bool', count: 'int', volume: 'int', error: 'str | None' = None) -> None One cache's numbers, for an admin page. `error` says why it couldn't be read, if it couldn't. ## Caches Methods Methods for the Caches class __repr__(self) -> 'str' Shows the base directory and the registered names. register(self, name: 'N', *, directory: 'str | os.PathLike[str] | None' = None, **settings: 'Any') -> 'None' Register a cache. Nothing opens until its first lookup. Args: name: The cache's name. Without `directory`, it's also the directory's name under `base_dir`, so it must be a plain name: no path separators, not `.` or `..`. directory: Put this cache somewhere else. **settings: `Cache` settings for this cache, over the registry's defaults. A callable `on_decode_error` is called as `(name, key, exc)`, as at the registry level. Adding `shards=` registers a `FanoutCache` instead, which takes the same settings; read it with `fanout(name)`. `shards=n` is the count for a new folder: an existing one keeps the count it was created with, and the registry warns when that isn't `n`. Passing the key at all is what picks the kind, so `shards=None` is NOT "not sharded": it is a `FanoutCache` with the stored shard count, or 8 for a new root. Leave `shards` out for a plain cache (a config file that writes `shards: null` for "none" would register a FanoutCache). Raises: ValueError: The name is taken or isn't a plain directory name, or another registered cache already uses that directory (two names would share one store, and the last one opened would quietly set its `size_limit`). TypeError: An unknown `Cache` (or `FanoutCache`) setting. Examples: ```python caches.register('pages', size_limit=2 * 2**30) # /var/cache/my-app/pages caches.register('objects', pickle=True, directory='/mnt/fast/objects') caches.register('rate_limits', shards=8) # a FanoutCache; read with fanout() ``` __contains__(self, name: 'object') -> 'bool' Whether `name` is registered. __iter__(self) -> 'Iterator[N]' The registered names, in registration order. __len__(self) -> 'int' How many caches are registered. directory(self, name: 'N') -> 'str' Where the named cache lives. __getitem__(self, name: 'N') -> 'TransactionalCacheLike' The named plain cache: opened on first use, then the same object every time. A `NullCache` while it can't be opened (see the class docs), or `NullCache(invalidate=real)` while it's disabled. The type is `TransactionalCacheLike`, so `caches[name].transact()` checks; a cache registered with `shards=` is read with `fanout()`, whose `transact` takes a key. Raises: KeyError: No cache by that name is registered. TypeError: The cache was registered with `shards=` (use `fanout(name)`), or its settings are invalid. ValueError: Its settings are invalid. fanout(self, name: 'N') -> 'FanoutCache | NullCache' The named sharded cache (one registered with `shards=`), looked up as `caches[name]` is. A `NullCache` while it can't be opened or is disabled, exactly as for a plain cache. That includes a directory whose layout doesn't match the registration (`LayoutMismatchError`: a plain cache's directory, or settings too small for the folder's shard count): reported through `on_open_error`, remembered, and retried after `retry_open_after`. A folder with another shard count than the registered one is not an error: it opens with its own count, and the registry logs a WARNING (see the class docs). Its `transact(retry=False, *, key=None)` accepts the `key=` a `FanoutCache` needs, so `caches.fanout(name).transact(key=k)` runs whether the cache is on or off. Both kinds satisfy `ShardedCacheLike`, so code (or a test fake) can be typed against that; the return stays the union so `shard_of` and `shards` still type-check on it when it's a `FanoutCache`. Raises: KeyError: No cache by that name is registered. TypeError: The cache is a plain one (use `caches[name]`), or its settings are invalid. ValueError: Its settings are invalid. disable(self, *names: 'N') -> 'None' Later lookups of these caches return `NullCache(invalidate=real)`. Examples: ```python caches.disable('pages') # reads miss, writes are dropped, deletes still land caches['pages'].get('/about') # None caches.enable('pages') ``` enable(self, *names: 'N') -> 'None' Later lookups of these caches return the real cache again. enabled(self, name: 'N') -> 'bool' Whether lookups of this cache return the real one (it may still fail to open). stats(self) -> 'dict[N, CacheStats]' Count and volume for every registered cache, opening each one. Never raises for a cache that can't be read: its `error` says why. Examples: ```python for name, stat in caches.stats().items(): print(name, stat.count, stat.volume, stat.error or 'ok') ``` clear(self, name: 'N', *, timeout: 'float | None' = None) -> 'int' Clear the real cache, even while it's disabled. Returns how many entries were removed. Raises: KeyError: No cache by that name is registered. OSError, RuntimeError: It can't be opened (an admin action shouldn't report 0 for that), or `Timeout` from the clear itself. expire_all(self, *, timeout: 'float | None' = None) -> 'dict[N, int | None]' Remove expired entries from every cache. Returns how many each removed, or None for a cache that couldn't be opened or expired (logged; the rest still run). close_all(self) -> 'None' Close every opened cache and forget it; a later lookup opens it again. A lookup that races this can return a handle that's no longer tracked. It still works: a closed handle reopens when it's used. __enter__(self) -> 'Self' Return the registry, for `with Caches(...) as caches:`. __exit__(self, exc_type: 'type[BaseException] | None', exc: 'BaseException | None', tb: 'TracebackType | None', /) -> 'None' Close every opened cache, as `close_all()` does. ## Errors Exceptions raised by cache operations. Timeout A lock stayed held, or a fork stayed pending, for longer than the cache's timeout. TransactionBusy A thread holding one cache's transaction couldn't begin another's, because another thread holds its write lock. It fails at once rather than waiting, since the holder could be waiting on a lock this thread holds. UnsafeValueError A pickle=False cache read a value that was stored with pickle. CorruptValueError A stored value couldn't be decoded: an unknown mode, or bytes that don't parse. ForeignValueError A stored value this build can't read but another might: an unknown mode, tag or extension (a newer version wrote it), or a time zone missing from this host. FormatMismatchError The cache directory holds a database in a format this version doesn't support. LayoutMismatchError The directory's layout doesn't match how it was opened: a plain `Cache` on a `FanoutCache` root or shard, a `FanoutCache` on a plain `Cache`'s directory, a shard opened as another index or count than it records, or a `shards=` that differs from the stored count. Also a `ValueError`, which is what these raised before. ## SQLite and System Information about the bundled SQLite and environment. sqlite_version() The SQLite version reported at runtime by `sqlite3_libversion()`. sqlite_compile_options() The compile-time options of the linked SQLite (`PRAGMA compile_options`). BUNDLED_SQLITE_VERSION str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.__str__() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'. ---------------------------------------------------------------------- This is the User Guide documentation for the package. ---------------------------------------------------------------------- ## Get started ### New to caching? Start here If you've never put a cache in front of anything, this page is the ten minutes that make the rest of the docs read easily. If you have, skip to [Getting started](getting-started.qmd); nothing here is specific to diskcache-oxide except the examples. ## What a cache is A cache is a copy of an answer, kept somewhere cheaper to reach than wherever the answer came from. The database can tell you the ten most popular episodes, but it takes a few milliseconds and some of its attention to do it, and the answer is the same for everyone who asks in the next five minutes. So you ask once, keep the answer, and hand out the copy. Something is worth caching when all three of these are true: - **It's asked for repeatedly.** An answer nobody asks for twice is a copy nobody reads. - **It costs something to produce.** A database query, a call to another service, rendering a page, a calculation. A cache lookup costs about a microsecond in diskcache-oxide, so anything slower than that is a candidate, and anything that takes milliseconds is a good one. - **A slightly old answer is acceptable, or you can tell when it stops being right.** This is the one that bites, and the rest of this page is mostly about it. Here it is in the smallest possible form. The first call does the work; the second gets the copy: ```python import tempfile import time from diskcache_oxide import Cache cache = Cache(tempfile.mkdtemp()) def popular_episodes() -> list[str]: episodes = cache.get('episodes:popular') # a hit returns the copy if episodes is None: # a miss: nothing cached, or it expired time.sleep(0.05) # stands in for the database query episodes = ['Rust for Python developers', 'Free-threaded Python', 'SQLite everywhere'] cache.set('episodes:popular', episodes, expire=300) return episodes for attempt in ('first call', 'second call'): start = time.perf_counter() popular_episodes() print(f'{attempt}: {(time.perf_counter() - start) * 1000:.2f} ms') ``` That shape, check the cache, do the work on a miss, store the result, is called **cache-aside**, and it's the one you'll use most. The [caching recipe](../recipes/caching-expensive-calls.qmd) turns it into a decorator. ## Why bother Three reasons, and you usually get all three at once: - **Latency.** The page that took 300 ms takes 3, because the slow parts were answers you already had. - **Load.** The database answers a question once a minute instead of once a request. That's the difference between a database that's fine and one you're upgrading at 2 a.m. - **Protection.** When the thing behind the cache is slow, flaky or rate-limited (a partner API, a search service), the cache keeps serving the last good answer while it recovers. The flip side: a cache is a copy, and copies go out of date. Every design decision in caching is a trade between how fresh the answer is and how cheap it is to serve. ## The words | Word | What it means | |---|---| | **Hit / miss** | A lookup found a live entry, or didn't. The **hit rate** is the fraction of lookups that hit; a cache with a low hit rate is costing you writes for nothing. | | **Key** | The name an entry is stored under. In diskcache-oxide a key is a `str`, like `user:42:profile`. | | **Expiry (TTL)** | How long an entry stays valid. After it, a lookup misses as if the entry were gone. `expire=300` means five minutes. | | **Stale** | Still cached, but no longer what the source would say. Expiry bounds how stale an entry can get. | | **Invalidation** | Removing or replacing an entry because its source changed, instead of waiting for it to expire. | | **Eviction** | The cache removing entries to stay under its size limit. diskcache-oxide evicts the least recently *stored* entries first. Eviction is why a cache can never be the only copy of something you need. | | **Cold / warm** | A cold cache is empty (just deployed, just cleared) and every lookup misses until it's warm. The first minutes after a deploy are the slow ones. | | **Stampede (thundering herd)** | A popular entry expires and every worker recomputes it at the same instant. See [Serving stale values while refreshing](../recipes/stale-while-revalidate.qmd). | | **Negative caching** | Caching "there's no such thing" so a missing record doesn't cost a database query every time someone asks. Use a short expiry. | ## The first hard thing: invalidation > There are only two hard things in Computer Science: cache invalidation and naming things. > > Phil Karlton (The popular extension adds "and off-by-one errors", which is funnier if you don't count.) Invalidation is hard because the cache can't know when the source changed. It has a copy of the answer; it has no idea the user just renamed themselves. Something has to tell it, and the three ways to do that are the whole of the art: 1. **Let it expire.** Give every entry an `expire`, chosen by how stale you can stand: five minutes for a popular list, an hour for a rendered page, a day for a thumbnail. It's the simplest strategy, it needs no knowledge of where the data changes, and it's the one safety net that catches every mistake in the other two. **Always set an expiry, even when you also invalidate explicitly.** An invalidation you forgot is otherwise forever. 2. **Invalidate at the write.** When your code saves the user's new name, it also deletes `user:42:profile`. The next read misses and rebuilds. This gives exact freshness, at the cost of knowing every key the change affects, which is where naming (below) comes in. When many keys depend on one thing, delete them by prefix, or change a generation number so the old keys are never read again: [Invalidating groups of keys](../recipes/invalidating-groups.qmd). 3. **Put the version in the key.** If the key is `episode:512:v7:page`, then publishing revision 8 means readers ask for `episode:512:v8:page`, miss, and rebuild, and the old entry is simply never asked for again. Nothing is deleted; the stale copy expires on its own. Most apps use all three: an expiry on everything, a delete where the write is obvious, and versioned keys for things that have a natural version. Here's the expiry half, with a fake clock so it runs instantly: ```python now = [1_000_000.0] cache = Cache(tempfile.mkdtemp(), clock=lambda: now[0]) cache.set('user:42:profile', {'name': 'Sam'}, expire=600) print('right away: ', cache.get('user:42:profile')) now[0] += 601 # ten minutes pass print('after expiry: ', cache.get('user:42:profile')) cache.set('user:42:profile', {'name': 'Sam'}, expire=600) cache.delete('user:42:profile') # the user renamed themselves: invalidate at the write print('after delete: ', cache.get('user:42:profile')) ``` One more rule that saves real grief: **never cache a value that depends on who's asking under a key that doesn't say who's asking.** A profile page cached as `page:profile` will be shown to the next user who asks for it. If the answer depends on the user, the locale, the plan, or the A/B group, the key includes them. ## The second hard thing: naming Keys are names, and the two problems turn out to be the same problem: an invalidation can only find the entries whose names tell it what they depend on. Some conventions that hold up: - **General to specific, separated by colons:** `user:42:profile`, `episode:512:transcript`. Everything about user 42 then shares the prefix `user:42:`, and a prefix scan (`iterkeys(prefix='user:42:')`) finds all of it in time proportional to how many there are, not the size of the cache. - **End a prefix with the separator.** `user:4` also matches `user:42`; `user:4:` doesn't. - **Include everything the value depends on, and nothing it doesn't.** A key with the request's timestamp in it never hits. A key without the locale serves French pages to English readers. - **Keys are cheap.** A long, descriptive key costs almost nothing; a short, clever one costs you a debugging session later. ## What goes wrong, and the fix - **Stale data.** Shorter expiry, or an invalidation at the write. If you can't say which writes affect an entry, the expiry is your answer. - **A stampede after a deploy or an expiry.** Refresh hot entries ahead of time, or serve the stale copy while one worker rebuilds: [Serving stale values while refreshing](../recipes/stale-while-revalidate.qmd). - **Cached errors.** Don't cache the result of a failed call as if it were an answer. Cache "not found" briefly if you must; never cache a 500. - **Treating the cache as storage.** An entry can be evicted or expire at any moment, and a deploy can clear the whole directory. Anything you can't rebuild from somewhere else (a session, a one-time token, a counter you haven't flushed) belongs in a cache of its own, sized so it never evicts, and ideally also somewhere permanent. The [sessions](../recipes/sessions.qmd) and [counters](../recipes/counters.qmd) recipes show the pattern. - **A bug that might be the cache.** Turn caching off without touching the code that uses it, with [`NullCache`](null-cache.qmd), and see whether the bug survives. ## Where diskcache-oxide fits There are three usual places to keep a cache, and the choice is about who can see it and what survives: | | A dict in the process | diskcache-oxide | A cache server (Redis) | |---|---|---|---| | Shared across worker processes | No, one copy per worker | Yes, every process on the machine | Yes, every machine | | Survives a restart | No | Yes, it's a directory | Depends on configuration | | Cost of a lookup | Nanoseconds | About a microsecond | A network round trip | | Something extra to run | No | No | Yes, a server | The in-process dict is unbeatable for a single process and useless the moment you have four gunicorn workers. The server is the right answer when the cache has to be shared across machines. diskcache-oxide is the middle: a cache that's just a directory, shared by every thread and process on the machine, still there after a restart, and fast enough that you stop thinking about the lookup. [Getting started](getting-started.qmd) is the next page. ### Getting started A `Cache` is a directory. That's the whole mental model, and it carries further than it looks: open the same directory from any number of threads and processes and they all share one cache, nobody has to be told about anybody else, and the entries are still there after a restart, because a directory doesn't restart. Hold on to that sentence; most of what follows is a consequence of it. This page covers installing diskcache-oxide, the everyday operations, the handful of things that only start to matter once several processes are involved, and how to shut down cleanly. ## Install ```bash pip install diskcache-oxide ``` - **Python:** CPython 3.14 or 3.15, including the free-threaded builds (3.14t and 3.15t). - **Platforms:** macOS and Linux. Wheels cover macOS on Apple Silicon and Linux (glibc) on x86_64 and aarch64; anywhere else pip builds from source, which needs the latest stable Rust toolchain. Windows support is planned. - **Dependencies:** none at runtime. SQLite is bundled and compiled in, so your system's SQLite version doesn't matter (`diskcache_oxide.sqlite_version()` reports the one in use). ## Open a cache ```python from diskcache_oxide import Cache cache = Cache('/var/cache/my-app') ``` The directory is required. If it doesn't exist, diskcache-oxide creates it, readable and writable only by your user (`0700`). That's deliberate: anyone who can write to a cache directory gets to decide what your app reads next, so the directory starts out as private as your home directory. So a cron job, an admin script or a container running as a different user can't open a directory your app created: it gets `PermissionError` ("Permission denied"). Run everything that shares a cache as the same user. Loosening the directory's mode yourself doesn't help, since the folders diskcache-oxide creates inside it for large values are owner-only too. Every process that should share the cache opens the same path. Inside one process, opening the same directory twice gives two handles on one shared engine, with one connection pool between them. ### Close it, or use `with` ```python with Cache('/var/cache/my-app') as cache: cache.set('greeting', 'hello') ``` `close()` returns the handle's SQLite connections. Once the last handle on a directory in a process closes, nothing in that directory is held open, so it's safe to move or delete. ## Store and read values ```python cache.set('user:42', {'name': 'Sam', 'roles': {'admin', 'editor'}}, expire=600) user = cache.get('user:42') # the dict, with roles still a set missing = cache.get('user:99', default={}) # {} value, expires_at = cache.get('user:42', expire_time=True) ``` - Keys are `str`. Other key types raise `TypeError` for now (they're planned). - Values can be `str`, `bytes`, `int`, `float`, `bool`, `None`, `Decimal`, and lists, tuples, dicts, sets, frozensets, datetimes and dates made of those. They come back as the types you stored. See [Serialization](serialization.qmd). - `expire` is in seconds. Leave it out for an entry that never expires. `touch(key, expire=...)` changes the expiry of an existing entry. ### Dictionary syntax ```python cache['alpha'] = 100 assert 'alpha' in cache assert cache['alpha'] == 100 del cache['alpha'] # KeyError if it's missing, like a dict ``` ### Remove values ```python cache.delete('user:42') # True if it was there value = cache.pop('session:7') # the value, or the default if it was missing ``` ## Atomic operations across processes Picture eight gunicorn workers, each with a scheduler that fires the same hourly job. You want it to run once. There's no leader, no Redis, and no way for the workers to talk to each other except through the cache. That turns out to be enough. Each call below is a single SQLite transaction, so it's atomic across every thread and process using the directory, and "store only if absent", done atomically, is a lock if you squint. These are what make a shared cache useful for coordination, not only for caching. ```python # Store only if the key is absent (or expired). Exactly one caller wins. if cache.add('job:refresh-feeds', os.getpid(), expire=3600): refresh_feeds() # Counters. incr() keeps the entry's existing expiry. cache.incr('page_views') # 1 cache.incr('page_views', 5) # 6 cache.decr('page_views', 2) # 4 ``` `incr()` on a missing key starts from `default` (0). Pass `default=None` to get a `KeyError` instead. A key holding a non-number raises `TypeError`. To make several calls atomic together, group them with `with cache.transact() as tx:`. See [Transactions](transactions.qmd). ## List keys ```python for key in cache.iterkeys(prefix='user:'): print(key) ``` `iterkeys()` yields keys in sorted order (`reverse=True` for descending), and with `prefix=` it's an indexed range scan: 400,000 keys scan in about 10 ms, against about 25 seconds for diskcache. Iterating the cache itself (`for key in cache`) yields keys in insertion order. ## Size limits and eviction ```python cache = Cache( '/var/cache/my-app', size_limit=500 * 1024 * 1024, # 500 MiB (default 1 GiB, minimum 1 MiB) cull_limit=10, # entries evicted per set() when over the limit (default 10) ) ``` When the cache's on-disk volume goes over `size_limit`, each `set()` evicts up to `cull_limit` of the least recently stored entries. `cache.volume()` reports the current size in bytes. Maintenance calls, all safe to run while other processes use the cache: - `cache.expire()` removes every expired entry and returns how many. - `cache.clear()` removes every entry that existed when it started, and keeps anything written while it runs. - `cache.cull()` evicts down to the size limit. ## Timeouts One process at a time commits to a database file. When another one is in the middle of doing so, your write waits its turn, up to the cache's `timeout` (60 seconds by default), then gives up and raises `diskcache_oxide.Timeout`, a subclass of the built-in `TimeoutError`: ```python from diskcache_oxide import Cache, Timeout cache = Cache('/var/cache/my-app', timeout=1.0) try: cache.set('key', 'value') except Timeout: ... # the cache was too busy; carry on without it ``` Maintenance calls work in chunks, so they take the write lock many times. A request-sized timeout can cut an admin `clear()` off partway through, so `clear()`, `expire()` and `cull()` each take their own `timeout=` for that call. It bounds each chunk's wait for the lock, not the whole call: ```python cache = Cache('/var/cache/my-app', timeout=1.0) # for requests cache.clear(timeout=10.0) # for the admin page ``` What else `Timeout` can mean, and what to do about every other exception the library raises, is in [Errors and what to do about them](errors.qmd). ## Values that can't be read A cache outlives the code that filled it. Deploy a rename of a class you pickled, or have a disk hand back damaged bytes, and a row that read fine yesterday raises today. By default `get` raises, as diskcache does. But a cache is only a copy of data you can rebuild, so most apps would rather treat that row as a miss and move on. `on_decode_error` does that: ```python cache = Cache('/var/cache/my-app', pickle=True, on_decode_error='miss') cache.get('homepage') # None if the stored value can't be decoded; the bad row is deleted ``` - `'miss'` returns the default, deletes the bad row, and logs a warning on the `diskcache_oxide` logger. Only that exact stored version is deleted, so a fresh value another process wrote in the meantime survives, and the read never waits for the write lock to do it. If the lock is busy, the row stays and a later read tries again. - A value this version can't read but another might, such as one written by a newer diskcache-oxide or one that needs a time zone this host lacks, is a miss too, but its row is kept. Reading it with the default `'raise'` raises `ForeignValueError`, a subclass of `CorruptValueError`. - Pass a function instead to report the failure yourself, for example to Sentry. It's called as `report(key, exc)` in place of the warning, after the row has been dealt with. Anything it raises is logged, and the read is still a miss. - Errors that aren't about the stored bytes still raise: `Timeout`, `MemoryError`, an I/O error, and `UnsafeValueError` (a pickled value read by a cache without `pickle=True`, which is a configuration mistake, not damage). [`durable=True`](#a-cache-that-cant-take-a-request-down) makes a `Timeout`, an I/O error or a `RecursionError` from a pickle hook a miss too. It covers `get`, `cache[key]` (which raises `KeyError`) and `pop`. A row that stays (a foreign value, or a bad one whose delete had to be skipped) fails on every read, so each stored version is reported once per cache handle, not on every read. `pop` decodes the value before it deletes the row, so a value that another version or host might read survives it. This table is the usual path; the fallback described below it removes the row first: | What went wrong | `'raise'` | `'miss'` or a function | `durable=True`, with its default `'miss'` or a function | |---|---|---|---| | A foreign value | raises `ForeignValueError`, row kept | returns the default and reports, row kept | the same as `'miss'` | | `UnsafeValueError` | raises, row kept | raises, row kept | raises, row kept | | An unreadable value (malformed bytes, a failing unpickle, a registered type's `decode` raising about the row) | deletes that version, then raises | deletes that version, returns the default and reports | the same as `'miss'` | | An unreadable value, and that delete fails | raises the delete's error, row kept | raises the delete's error, row kept | returns the default and reports both, row kept | | `Timeout`, an I/O error or `RecursionError` from a pickle hook or a registered type's `decode` | raises, row kept | raises, row kept | returns the default, row kept, reported once per stored version | | `RecursionError` from a value too deep for this thread's stack | raises, row kept | raises, row kept | returns the default, row kept, reported once per stored version | | `MemoryError` or a `BaseException` from a pickle hook or a `decode` | raises, row kept | raises, row kept | raises, row kept | | The value decoded, and the delete fails | raises the delete's error, row kept | raises the delete's error, row kept | returns the default, never the value, row kept | `durable=True` with an explicit `on_decode_error='raise'` follows the `'raise'` column, except that when an unreadable value's delete fails it raises the decode error rather than the delete's. If that delete itself fails, for example with `Timeout`, its error is raised under every policy with the decode error as its `__cause__`, unless `durable=True`, and the row stays. A value still goes to exactly one caller, because `pop` deletes only the version it decoded. A `pop` that loses that delete to another writer reads again, and after 8 losses it falls back to removing the row first and decoding afterwards, so it can't starve. If its `timeout` has run out by the time it loses, it raises `Timeout` instead: this `pop` deletes nothing, and whatever the other writer left stays. There are two limits: - That fallback is the one place a failed decode can cost the value, because the row is already gone: a foreign value in a known mode, or any error from a pickle hook (`Timeout`, `MemoryError`, an I/O error). It's possible on a heavily overwritten key, since a `pop` can lose while it queues for the write lock. How often it happens hasn't been measured yet. - Like `get`, `pop` doesn't wait to clean up a row whose large value file has gone missing. It returns the default, and if the write lock is busy the row can stay until a later `pop` or write removes it. ## A cache that can't take a request down It's 3 a.m., the disk under `/var/cache` is full, and every request that writes to the cache is now a 500. Nothing the cache held was precious: it was all a copy of something the database still has. If every caller can fall back to its source of truth, an outage in the cache should cost you speed, not requests. `durable=True` makes that the rule for the calls a request makes: ```python cache = Cache('/var/cache/my-app', timeout=1.0, durable=True) page = cache.get('homepage') # None if the cache is locked, full, broken or the disk failed ``` "Durable" here means the cache keeps your app running. It doesn't mean what it means in a database, where a durable write survives a crash. This option does the opposite: it can drop writes. It's off by default. An operational failure is a `Timeout`, an `OSError` (disk full, I/O), a SQLite error including a damaged database, or a child forked inside a transaction that can't write. On a single-key call it's answered like this: | Call | On an operational failure | |---|---| | `get`, `pop` | the default; `get(key, expire_time=True)` gives `(default, None)` | | `key in cache` | `False` | | `cache[key]` | raises `KeyError` | | `set`, `add`, `touch` | `False` | | `cache[key] = value` | drops the write | | `incr`, `decr` | `default + delta` or `default - delta`, as `NullCache` does. `default=None` raises `KeyError` | | `delete` | `False` | | `del cache[key]` | raises `KeyError` | These always raise: - **Bugs.** A non-`str` key, a value that can't be stored without pickle, a non-numeric `incr`, an `incr` past 64 bits, `UnsafeValueError`, and misusing a transaction. - **`MemoryError` and `KeyboardInterrupt`.** That includes SQLite running out of memory. - **Admin calls.** `clear`, `expire`, `cull`, `checkpoint`, `len`, `volume` and iteration, because their callers need the truth, and most have their own `timeout=`. `checkpoint` still returns `False` when it's only busy. - **The constructor, and starting a [transaction](transactions.qmd).** Use [`Caches`](caches.qmd) if opening shouldn't raise: it gives you a `NullCache` until the cache can open. - **Anything inside a transaction you hold on this cache.** A dropped write there would break the transaction's atomicity. - **Errors from your own code**, other than the pickle hooks on a read described above. A `clock=` function that raises, or a pickle hook on a write, raises as usual, even if what it raises is a `Timeout`. `on_decode_error` defaults to `'miss'` under `durable=True`, and a value you pass wins. ### What a dropped call costs A failure reaches you as "wasn't there", so know what each one means before you turn this on. - **A dropped `set`, `delete`, `pop` or version-bump `incr` serves stale data.** The old entry stays. It's served until it expires, and with no expiry that means forever, or until a later write replaces it. `delete` returns `False`, the same as for a key that wasn't there, so your code can't tell. The log line is the only signal. If stale data would be a real problem, give those entries an expiry. - **A `pop` whose delete fails returns the default, not the value**, since the row is still there and a later `pop` could deliver the value a second time. - **A fake `incr` isn't a count.** It's what a `NullCache` returns, so a rate limiter fails open. Don't use one as an id: two callers can get the same number. - **`add` fails closed.** It returns `False` under `durable=True`, but `True` on a `NullCache`, so `Caches` can give you either answer for one key. Use a separate handle without `durable` for claims and locks, and treat its exceptions as "not claimed". ### It doesn't bound time A call still waits up to its `timeout` before it gives up, so pair `durable=True` with a small `timeout=`. Not too small: `timeout` still has to exceed the work in flight, plus a fork, plus the interpreter handing over, or ordinary forks turn into misses. Beside busy threads on a GIL build, that hand-over alone takes tens to hundreds of milliseconds. See [How long a call waits for a fork](concurrency-and-forks.qmd#how-long-a-call-waits-for-a-fork). ### Reporting Every swallowed failure is counted in one of three groups, and logged on the `diskcache_oxide` logger, so one group can't hide another: | Group | Level | What's in it | |---|---|---| | Busy | `WARNING` | `Timeout`: a lock or a fork outlasted `timeout` | | Failing | `ERROR` | Everything else: a full disk, an I/O error, a damaged database | | Lost invalidations | `ERROR` | Any failed `delete` | Each group logs its first failure at once, then at most once a minute per cache handle, with counts by operation since its last line: `get: 1204, set: 37`. An outage can be thousands of failures a second, and that's one line a minute. Failures after a group's last line stay counted until its next one, or until you call `close()`, which logs what's left. Dropping a handle without `close()` doesn't. An outage that ends within the minute can therefore show only its first failure until then, which is why each line says more may follow. A hook outage on a read is reported once per stored version, as in the `pop` table above, and counted here as busy for a timeout and failing otherwise, with no line of its own: the group's next line includes it. Time comes from the failing call's clock, so `clock=` drives it in tests. `cache._durable_stats()` has the running totals by group and operation, the place to look for "how many invalidations did we lose?". It's a private hook for now, like `_wal_stats()`. The limit is per handle, so a `with Cache(...)` per request logs on every request during an outage. A handle from `Caches` is opened once, so it isn't affected. ## More than one process writing A single `Cache` has one write lock per directory. Reads never wait for it, but every write does, so eight worker processes that all bump counters are eight writers lining up for one lock. If your app runs several worker processes that write often (counters, rate limits, job claims), open a [`FanoutCache`](fanoutcache.qmd) instead: it spreads the keys over several databases, each with its own lock, and in the repository's benchmark it holds its rate at 8 processes where a plain `Cache` falls to about 40% of its one-process rate. For one process, or mostly reads, a plain `Cache` is the simpler choice. ## Shutting down Call `cache.checkpoint()` at shutdown if you can. It isn't required. With readers in other processes, a burst of writes (a big fill, or a `clear()`) can leave SQLite's write-ahead log much larger than the database, and SQLite only shrinks it when it gets a moment with no readers. `checkpoint()` makes that moment: it waits briefly for in-flight reads, copies the log into the database, and truncates it. It returns `False` instead of raising if other processes keep it busy. `close()` does the same by default, but only when the log is large, and it waits at most a second, so closing stays cheap. [Storage and WAL](storage-and-wal.qmd) has the details. ## Testing code that uses a cache Two tools help here: - **`NullCache`** is a cache that's switched off: every write succeeds and every read misses. Swap it in for tests or local development without touching call sites. See [NullCache](null-cache.qmd). - **`clock=`** controls the cache's clock. diskcache-oxide reads time natively in Rust, so patching `time.time` doesn't affect it on its own. Pass `clock=lambda: time.time()` and the cache follows your patched clock: ```python cache = Cache(tmp_path, clock=lambda: time.time()) ``` ### Switching from python-diskcache If you're here, the question is a practical one: what breaks? Very little, and on purpose. diskcache-oxide is **API compatible** with [python-diskcache](https://github.com/grantjenks/python-diskcache) 5.6.3's `Cache`, and deliberately **not data compatible**. Keeping diskcache's API makes switching a one-line change. Not keeping its on-disk format is what made it possible to fix the design instead of inheriting it, pickle and all. So the switch itself is small, and the rest of this page is about what you'll notice afterwards, which is mostly things getting better and now and then things getting stricter. For most apps, switching looks like this: ```python # from diskcache import Cache, Timeout from diskcache_oxide import Cache, Timeout cache = Cache('/var/cache/my-app-oxide') ``` ## Start with a fresh cache diskcache-oxide stores its data in `diskcache-oxide.sqlite`, not diskcache's `cache.db`, and never opens or converts a diskcache database. The two can even share a directory without seeing each other's data. Your cache starts cold after the switch, which for a cache is usually fine. The cleanest switch gives diskcache-oxide a new directory and deletes the old one once every worker has moved. If the two share a directory, diskcache's files are `cache.db*` and its value folders, which nest two levels deep (`3f/a2/….val`); diskcache-oxide's folders are one level deep and hold `.val` files directly, so a two-hex-digit folder that contains folders is diskcache's. ### Upgrading diskcache-oxide During 0.x, a release that changes the on-disk format bumps the minor version and says so first in its changelog entry. Delete existing caches when you upgrade to it, with every worker stopped. Until you do, opening one raises `FormatMismatchError`, which a `Caches` registry reports and answers with a `NullCache`. A release that keeps the format needs nothing. ## What you might notice Most of these are fixes. Each one has a conformance test, or an entry in [Divergences](divergences.qmd) that explains why. If the switch turns up an exception you haven't seen before, [Errors and what to do about them](errors.qmd) covers each one. | | python-diskcache 5.6.3 | diskcache-oxide | |---|---|---| | Values | Pickled | msgpack (default) or JSON with exact type tags. Pickle is opt-in with `pickle=True`. | | Unrepresentable values | Pickled | `TypeError` at `set()`, unless `pickle=True` | | Keys | Anything picklable | `str` only for now | | `bool(cache)` | False when empty | Always True; `NullCache` is False | | `Timeout` | Subclasses `Exception` | Subclasses `TimeoutError` | | `directory` | Optional (makes a temp dir) | Required | | Clock | Reads `time.time()` | Reads the clock in Rust; pass `clock=` to control it | | `size_limit` | Anything (tiny values silently empty the cache) | At least 1 MiB | | `expire=True` / `timeout=True` | Treated as `1` | `TypeError` | | `disk=`, `eviction_policy=` | Supported | Not supported (eviction policies are coming) | | `transact()` | `tx` is `None` | `tx` is the cache; see [`transact()`](#transact) below | | `FanoutCache` | Supported | Supported, with differences; see [`FanoutCache`](#fanoutcache) below | | Tags, `memoize` | Supported | Coming in later phases | ## Values: no pickle unless you ask diskcache pickles every value. That's convenient, but anyone who can write to the cache directory can make your app run arbitrary code the next time it reads, and objects that were never meant to be cached get stored without complaint. diskcache-oxide stores values as msgpack (or JSON), and tags the types those formats would otherwise change, so tuples, sets, frozensets, datetimes, dates, Decimals, big ints, NaN, nested bytes and non-`str` dict keys all come back exactly as stored. Something that can't be represented raises `TypeError` when you store it, not later when you read it: ```python cache.set('job', MyJob()) # TypeError: values of type MyJob can't be stored without pickle (only exact built-in types round-trip) ``` If you really do need to cache arbitrary objects, opt in for that cache: ```python objects = Cache('/var/cache/my-app-objects', pickle=True) ``` A cache opened without `pickle=True` never unpickles. If it finds a pickled value written by a `pickle=True` peer, it raises `UnsafeValueError` and leaves the value alone. More in [Serialization](serialization.qmd). ## String keys Only `str` keys are accepted today; anything else raises `TypeError`. Keys are already stored with a type tag, so other key types can be added later without changing the file format (see the [roadmap](roadmap.qmd)). In the meantime, build string keys yourself (`f'user:{user_id}'`). ## `transact()` `with cache.transact():` works as it does in diskcache: the block's calls on the entering thread commit together on exit and roll back on an exception, a nested block joins the outer one, its exit commits nothing, and an exception caught inside the outer block keeps the inner writes. What differs (details in [Transactions](transactions.qmd)): - **`tx` is the cache** instead of `None`, and a second `Cache` handle on the same directory joins the transaction instead of waiting for it (D-029). - **A transaction on a second cache whose write lock another thread holds** raises `TransactionBusy`, a `Timeout` subclass, at once, and `retry=True` retries only `Timeout`, never `TransactionBusy`. diskcache waits, and its `retry=True` loops on any `OperationalError` (D-030). - **Another thread may end it.** Its commit or rollback runs within the transaction's `timeout`, and the owner's own exit then raises `TransactionAborted` (a `RuntimeError`) unless it committed (D-031). The object `transact()` returns can be entered once. - **A block that didn't commit raises at its exit** with `TransactionAborted`, a `RuntimeError`, for example after a write inside it failed on a full disk. - **`durable=True`**, which diskcache doesn't have, still raises when a transaction can't begin and for any failure inside one (D-028). - **Forking inside your own transaction** poisons the child's copy, whose writes raise `RuntimeError` at once, instead of stalling for the timeout (D-020). - **It belongs to a thread, not an asyncio task**, as in diskcache: don't `await` inside the block. ## `FanoutCache` `FanoutCache` is covered in full in [FanoutCache: sharded caches](fanoutcache.qmd). The differences from diskcache's, each with an entry in [Divergences](divergences.qmd): - **`shards` is keyword-only** (`FanoutCache(path, shards=4)`). `FanoutCache(path, 4)` is a four-second `timeout` here. The constructor is `FanoutCache(directory, timeout=None, *, shards=None, ...)`, and `directory` is required (D-039). - **The shard count is recorded**, and a different `shards=` raises `LayoutMismatchError` instead of silently sending keys to other shards. `shards=None` uses the stored count, or creates 8 (D-032). - **A plain `Cache` and a `FanoutCache` refuse each other's directories** with `LayoutMismatchError`, and nothing is created by the refused open (D-038). - **Shards are chosen by XXH3, not adler32**, so which shard a key lands in differs from diskcache's, and `shard_of(key)` tells you. Iteration goes shard by shard, and `iterkeys(prefix=, reverse=)` gives the merged sorted order. There's no `reversed()` yet (D-033). - **A busy shard raises `Timeout`** after `timeout` (60 seconds by default) instead of silently giving False or the default after 10 ms. `durable=True` gives the miss answers, and logs them (D-034). - **Maintenance shares one `timeout`** across the shards, doesn't retry forever, and reports partial progress: a failure raises with `failed_shards` and `removed` (D-035). - **`size_limit` is a total**, split evenly with at least 1 MiB per shard, and it reads back as the total, not shard 0's share (D-036). - **`max_connections` is a total** across the shards, so file descriptors are bounded and not one per thread per shard. It must be at least `shards` (D-037). - **`transact()` takes a key** and covers one shard: `with cache.transact(key=k):` where diskcache's `with cache.transact():` locks every shard. Calls for keys on other shards inside the block raise `ValueError`, reads included. A ported bare `transact()` raises `TypeError` (D-040). - **Not there yet:** `cache()`, `deque()` and `index()`, `stats()`, `evict()` and tags, `check()`, `reset()`, `read()`, `reversed()`, `retry=`, `read=` and `tag=` arguments, settings as attributes, and pickling (D-039). ## Truthiness In diskcache, an empty cache is falsy, so `if cache:` quietly skips caching right after every deploy, when the cache is empty. In diskcache-oxide, `bool(Cache)` is always True and `bool(NullCache)` is always False, so `if cache:` means "is caching on?". Both have an `enabled` property that says the same thing. ## Tests that patch `time.time` diskcache reads `time.time()` in Python, so tests could advance its clock with `unittest.mock.patch` or a freezegun-style fixture. diskcache-oxide reads the clock natively, which keeps a Python call off every operation. To make it follow a patched clock, pass `clock=`: ```python cache = Cache(tmp_path, clock=lambda: time.time()) with patch('time.time', return_value=1000.0): cache.set('k', 'v', expire=10) with patch('time.time', return_value=1015.0): assert cache.get('k') is None ``` ## Things that just get better You don't have to change anything for these: - Reads take about half the time, and threaded throughput no longer collapses as threads are added. See [Performance](performance.qmd). - A child process forked while another thread was writing can always write. In diskcache it can stall for the full 60-second timeout. - `clear()` keeps writes that land while it runs, `expire()` removes every expired entry, and a read during an overwrite never misses. - File descriptors and memory are bounded by a connection pool, not by how many threads touched the cache. A `FanoutCache` bounds them for the whole root, not per shard. - Opening a cache and reading from it never writes to the database. ### Errors and what to do about them Sooner or later a traceback ends in `diskcache_oxide`, and the question is whether it's your bug, an outage, or two processes disagreeing about what a directory is for. diskcache-oxide raises when something is wrong rather than guessing, and each exception is chosen so that `except` clauses you already have keep working: `Timeout` is a `TimeoutError`, `TransactionAborted` is a `RuntimeError`, `UnsafeValueError` is a `TypeError`, `LayoutMismatchError` is a `ValueError`. This page is the field guide: every exception, what it means, and what to do. The last section covers the log lines the library writes when it has been told not to raise, because then a log line is the only evidence you'll get. | Exception | Base classes | Where | |---|---|---| | `Timeout` | `TimeoutError` (so `OSError`) | any call that waits for a lock or a fork | | `TransactionBusy` | `Timeout` | `transact()` on a second cache | | `TransactionAborted` | `RuntimeError` | a `transact()` block that didn't commit | | `UnsafeValueError` | `TypeError` | reading a pickled value without `pickle=True` | | `CorruptValueError` | `ValueError` | reading a value no build could decode | | `ForeignValueError` | `CorruptValueError` | reading a value another version or host could decode | | `FormatMismatchError` | `RuntimeError` | opening a database or manifest this version can't read | | `LayoutMismatchError` | `FormatMismatchError`, `ValueError` | opening a directory as the wrong kind of cache | | `TypeError`, `ValueError`, `KeyError`, `RuntimeError`, `OSError`, `MemoryError` | built-in | described below | ## `TypeError` at `set()`: a value that can't be stored ```text TypeError: values of type Episode can't be stored without pickle (only exact built-in types round-trip) ``` The value isn't one the msgpack or JSON format can represent exactly, so it's refused where it was created rather than three requests later when something reads it. Only exact built-in types round-trip: `str`, `bytes`, `int`, `float`, `bool`, `None`, and `list`, `tuple`, `dict`, `set` and `datetime` made of those. A dataclass, an ORM row, a `dict` subclass or a class of your own doesn't. Fix it in this order: 1. Register the type: `diskcache_oxide.register_type(Episode, name='tp.Episode')` for a dataclass, or with `encode=` and `decode=` functions for anything else (an `ObjectId`, a namedtuple, a `str` subclass). The value is stored without pickle and comes back as the type you stored. See [Your own types](serialization.qmd#your-own-types). 2. Or store a plain structure instead: a dict from a model's `model_dump()`, and rebuild the object on the way out. 3. If you really need arbitrary objects, open a separate cache for them with `Cache('/var/cache/my-app-objects', pickle=True)`. Don't turn `pickle=True` on for a cache that other code shares just to make the error go away: every reader of that directory then runs `pickle.loads` on whatever is written there. See [Serialization](serialization.qmd). A registered type that still raises here is one whose `encode` function raised, or returned something the codec can't store, or a dataclass with a field it can't store. That raises even on a cache with `pickle=True`: a registered type is never pickled. ## `TypeError` for keys and booleans ```text TypeError: cache keys must be str, not int TypeError: expire must be a number of seconds or None, not bool ``` Keys are `str` only for now; build them yourself (`f'user:{user_id}'`). Other key types are planned and won't need a format change. A `bool` where a number of seconds belongs (`expire=True`, `timeout=True`) is almost always an argument in the wrong position, so it's refused rather than treated as `1`, which is what diskcache does. ## `Timeout`: a lock or a fork outlasted `timeout` ```text diskcache_oxide.Timeout: timed out after 1 s waiting for a lock or a pending fork ``` The call waited the cache's `timeout` (60 s by default, or the `timeout=` passed to that call) and gave up. Something held what it needed, and there are three suspects: - **Another writer held the write lock** that long. One process at a time commits to a database file, and a burst of writes, a `clear()` from a maintenance script, or a long `transact()` block holds it. Reads never wait for writers, so a `Timeout` on a `get` is one of the next two cases. - **A fork was pending.** Before a fork, every cache call in the process waits for in-flight writes and then for the fork itself, and that counts against `timeout`. A fork takes about a millisecond in a quiet process and tens to hundreds of milliseconds beside busy threads on a GIL build. A `timeout` below that sees `Timeout` during ordinary forks. See [How long a call waits for a fork](concurrency-and-forks.qmd#how-long-a-call-waits-for-a-fork). - **Reading a large value kept losing to overwrites** of the same key until `timeout`. Rare, and only on a key that's rewritten flat out. What to do: - **In a request path, catch it and skip the cache.** Give the cache a short `timeout` (`Cache(path, timeout=1.0)`), and treat `Timeout` as "don't cache this time": the value is a copy, and the request can go to the source of truth. [Sharing a cache across web workers](../recipes/web-workers.qmd) has the pattern. - **Give maintenance its own timeout.** `clear()`, `expire()` and `cull()` take `timeout=` for that call, and bound each chunk's wait, so an admin page can pass `clear(timeout=10)` on a cache whose requests use 1 second. Run them from one scheduled job rather than from every worker. - **Keep `transact()` blocks short**, and never `await` inside one. - **If many processes write often, use a [`FanoutCache`](fanoutcache.qmd).** A plain `Cache` has one write lock per directory, and eight workers that all write queue on it. Sharding gives each shard its own. - **Let the library answer it as a miss** with [`durable=True`](getting-started.qmd#a-cache-that-cant-take-a-request-down): single-key calls then return their miss value instead of raising, and the failure is counted and logged. Pair it with a small `timeout`, since it doesn't bound time on its own. - `checkpoint()` never raises `Timeout`: it returns `False` when other processes keep it busy. Call it again later, or at shutdown. ## `TransactionBusy`: two caches in one transaction Inside one cache's `transact()` block, beginning a transaction on a second cache whose write lock another thread holds raises `TransactionBusy` at once instead of waiting. That thread could be waiting for the lock you hold, and then neither would ever finish. It's a `Timeout` subclass, so existing handlers catch it, and `retry=True` never retries it. Nest the two caches in the same order everywhere in your app. When `TransactionBusy` reaches you, leave the outer block, so your lock is released, and try the whole thing again. See [Transactions](transactions.qmd#two-caches-at-once-transactionbusy). ## `TransactionAborted`: a block that didn't commit ```text TransactionAborted: the write transaction was rolled back after an error; nothing in it can commit ``` A `with cache.transact():` block never exits cleanly when its writes are gone. This one comes from a write inside the block, or from the block's exit, after one of its operations failed in a way that left the transaction unusable (a full disk, an I/O error). ```text TransactionAborted: the transaction did not commit: another thread or an earlier exit ended it, or it was rolled back after an error ``` This one comes from the block's exit, when the transaction had already ended without committing: another thread rolled it back, or an earlier `__exit__` call on the same object ended it (including one made from inside the block's own operations, such as a pickle hook, which raises `RuntimeError` where it's called and rolls the transaction back when that operation returns). Either way nothing in the block was kept. Treat it as you would the original failure: log it, and run the block again if the work is idempotent. `TransactionAborted` is a `RuntimeError`, so an `except RuntimeError` you already have still catches it; catch `TransactionAborted` to handle this case alone. ## `RuntimeError` from a transaction ```text RuntimeError: this database can't be written in a process forked inside a transaction ``` A process forked while the forking thread itself held a `transact()` block, and this is the child. The parent's transaction carries on and commits; the child's copy is poisoned, and its writes to that database raise this instead of stalling for a minute. Don't fork from inside a transaction: start worker processes before the block, or after it. See [Forking from inside your own transaction](concurrency-and-forks.qmd#forking-from-inside-your-own-transaction). ## `UnsafeValueError`: a pickled value, read without `pickle=True` ```text UnsafeValueError: this value was stored with pickle; open the cache with pickle=True to read it ``` Two openers of one directory disagree about what it's allowed to hold. A handle opened without `pickle=True` found a row that a `pickle=True` handle wrote, and refused to unpickle it, because unpickling runs whatever code the bytes describe. The row is left in place for the handle that can read it. This is a configuration mismatch, not damage, so it raises under every `on_decode_error` policy and under `durable=True` too. Decide which the directory is for. If it holds pickled objects on purpose, open every handle on it with `pickle=True`. If it shouldn't, find the writer that passed `pickle=True` and give it its own directory, then `clear()` this one. It's a `TypeError`, like the error at `set()` above, because both come from the same decision: what this cache is allowed to hold. ## `CorruptValueError` and `ForeignValueError`: a value that can't be decoded ```text ForeignValueError: ... CorruptValueError: ... ``` `CorruptValueError` means no build of diskcache-oxide could read the stored bytes: they're malformed, a pickled class has been removed since it was stored, or a [registered type's](serialization.qmd#your-own-types) `decode` function raised about the row (a dataclass row written before a field was removed, say), in which case your exception is its `__cause__`. `ForeignValueError`, its subclass, means this build can't read them but another might: a newer version wrote a mode or tag this one doesn't know, the value needs a time zone this host doesn't have, or it holds a registered type this process hasn't registered (the message names the tag: register it, at import time). By default a read raises these, as diskcache does. A cache holds copies, so most apps would rather treat them as a miss: - `Cache(..., on_decode_error='miss')` returns the default, logs a warning on the `diskcache_oxide` logger, and deletes a row no build could read. A foreign row is kept, since the build that wrote it can still read it. `durable=True` turns this on by default. - Pass a function instead to report it yourself: `on_decode_error=lambda key, exc: sentry.capture(exc)`. From a `Caches` registry it's called as `(name, key, exc)`. - For a `ForeignValueError` that keeps coming back, upgrade the older readers to the version that wrote the value, or stop the newer writer from sharing the directory. For a missing time zone, install the host's zone data. The full table of what each policy does with each failure is in [Values that can't be read](getting-started.qmd#values-that-cant-be-read). ## `FormatMismatchError`: a database this version can't read ```text FormatMismatchError: cache format version 3 is not supported (this build uses version 2) at /var/cache/my-app ``` The directory holds a database, or a `FanoutCache` manifest, written in a format this build doesn't know, usually by a newer diskcache-oxide. Nothing is read or changed. Upgrade this process to match the writer, or point it at a different directory. If the directory should be reset, stop every process using it and delete it; a cache starts cold without harm. ## `LayoutMismatchError`: the wrong kind of cache for the directory ```text LayoutMismatchError: the root has 8 shards, not 16 ``` The directory's layout doesn't match how it was opened: a plain `Cache` on a `FanoutCache` root or one of its shard directories, a `FanoutCache` on a plain `Cache`'s directory, or a `shards=` count that differs from the one the root records. The refused open creates nothing, so a typo doesn't leave a directory neither kind can open. It's both a `ValueError` and a `FormatMismatchError`. Use the class the directory was created with, and for a `FanoutCache` either leave `shards` out (the stored count is used) or pass the stored one. A different shard count means a new directory: the count decides where every key lives, so it can't change in place. See [The layout and the hash](fanoutcache.qmd#the-layout-and-the-hash). In a [`Caches`](caches.qmd) registry, a lookup that hits a role mismatch returns a `NullCache` and reports it like any failed open. Another shard count is not an error there: the registry opens the folder with the count on disk and logs a warning (below). It still fails with `LayoutMismatchError`, and so a `NullCache`, when a `size_limit` or `max_connections` that fits the registered count is too small to split over the folder's count; delete the folder or raise the setting. ## `RuntimeError`: the filesystem can't run SQLite in WAL mode ```text RuntimeError: the cache requires WAL journal mode, but SQLite is using "delete" ``` SQLite refused write-ahead logging, which diskcache-oxide requires for readers and a writer to work at the same time. That happens on network filesystems (NFS, SMB) and some FUSE mounts. The cache has to live on a local disk: `/var/cache/`, or a Docker volume on the host's disk. See [Pick a directory](../recipes/web-workers.qmd#pick-a-directory). ## `OSError` - **Too many open files (`errno` `EMFILE`) opening a `FanoutCache`.** Each shard takes about 4 file descriptors to open, so 256 shards need more than Linux's usual `ulimit -n` of 1,024. The message gives the shard count, the descriptors needed, the limit and how many are already open. Raise the limit, or use fewer shards; eight is plenty for most apps. A failed open leaves nothing behind. - **No hard links.** Creating a `FanoutCache` manifest uses a hard link, so a filesystem without them (FAT, exFAT, SMB, some FUSE mounts) raises `OSError` saying so. Use a local disk. - **Disk full, or an I/O error.** The write didn't happen and the cache is unchanged. Under `durable=True` a single-key call answers it as a miss or `False` and logs it; otherwise it raises. `Timeout` is an `OSError` too, so an `except OSError` around a cache call catches both. ## `ValueError` - `size_limit` below 1 MiB. In diskcache a tiny limit silently evicts every value, including the one just stored; here it's refused. For a `FanoutCache` the total must be at least `shards` MiB, since each shard needs 1 MiB. - `max_connections` below `shards` on a `FanoutCache`. Each shard needs at least one connection. - A key on another shard inside `transact(key=...)`. The block covers one shard, and a call for a key elsewhere, reads included, can't be atomic with it. Check with `shard_of()` that the keys share a shard, or keep them in one value. - A `Caches` name that's taken, isn't a plain directory name, or would share a directory with another registered cache. ## `KeyError` - `cache[key]` and `del cache[key]` for a missing or expired key, as with a dict. Use `get()` and `delete()` for the "might not be there" cases. - `incr(key, default=None)` and `decr(key, default=None)` on a missing key. With the default `default=0`, a missing key starts counting from 0 instead. - Every `NullCache[key]`, since nothing is ever present. ## `MemoryError` It always raises, under `durable=True` too, including when SQLite runs out of memory. A `Caches` lookup raises it as well, rather than handing out a `NullCache`. ## Log lines, when the library was told not to raise Three settings turn failures into log lines on the `diskcache_oxide` logger instead of exceptions. Make sure that logger reaches your logging setup, since the line is then the only signal. - **`on_decode_error='miss'`** logs a warning for each stored version it couldn't read, once per cache handle, and deletes the row if no build could read it. - **`durable=True`** counts every swallowed failure in one of three groups and logs each group's first failure at once, then at most once a minute per cache handle, with counts by operation since its last line (`get: 1204, set: 37`). Busy (a `Timeout`) logs at `WARNING`, failing (a full disk, an I/O error, a damaged database) at `ERROR`, and lost invalidations (any failed `delete`, whose entry can now be served stale) at `ERROR`. An outage that ends within the minute shows only its first failure until the next line, so each line says more may follow. `close()` logs what's left. - **`Caches`** logs `could not open cache 'pages' at /var/cache/my-app/pages; using a NullCache, retrying in 10 s` at `ERROR` with the traceback when a cache can't open, and the same at `WARNING` without one when the open merely timed out (that's retried after at most 1 s). Pass `on_open_error=` to send it to Sentry instead. - **`Caches`** also logs `cache 'rate_limits' at /var/cache/my-app/rate_limits has 8 shards but is registered with shards=16; using the 8 on disk. To apply shards=16, delete /var/cache/my-app/rate_limits while no process has it open` at `WARNING` each time it opens a sharded cache whose folder has another count than the registration, and once per process `a FanoutCache is registered with a shard count its folder does not have` at `ERROR`, a constant message with the details in the record's `cache`, `directory`, `stored_shards` and `registered_shards` attributes, so an error tracker groups every worker's report into one issue. The cache works throughout; this one doesn't go through `on_open_error`, since the open succeeded. ## If it looks like a bug in diskcache-oxide If you've read this far and the traceback still doesn't fit, it may be one. Open an [issue](https://github.com/mikeckennedy/diskcache-oxide/issues) with `diskcache_oxide.__version__`, `diskcache_oxide.sqlite_version()`, the interpreter (`python -VV` says whether it's free-threaded), the platform and filesystem, how many processes and threads share the directory, and the full traceback. Every deliberate difference from diskcache is listed in [Divergences](divergences.qmd), so check there first if the surprise is a behavior rather than an error. ## The vision A shared cache is one of the first pieces of infrastructure a Python web app grows into, and today the choice is awkward. An in-process dict is fast, but each worker has its own copy and it's gone on restart. Redis is shared and durable, but it's a separate server, a network hop on every read, and one more thing to run. [python-diskcache](https://github.com/grantjenks/python-diskcache) showed there's a lot of room between those two: a cache that is just a directory, shared by every thread and process on the machine, built on SQLite. diskcache-oxide is a bet that this middle option can be much bigger than it has been. With a Rust engine underneath, a directory-backed cache can be fast enough to call on the event loop, safe enough for pre-forking servers and free-threaded Python, and complete enough to cover the jobs people now reach for Redis to do: caching, counters, rate limits, locks, job claims and queues. And when one machine isn't enough, the same engine can be served over the network. It's being built in three stages. ### Stage 1: a better diskcache (now) Keep diskcache's API, fix what's wrong underneath it, and make it fast. Behavior comes first: every phase keeps a conformance suite green against real diskcache 5.6.3 before chasing speed, and every deliberate difference is written down in [Divergences](divergences.qmd). ### Stage 2: the complete local caching toolkit Everything diskcache offers that real apps use, plus the things a modern Python stack needs that it never had: an async API, sharding designed for many threads, and coordination recipes that are atomic across every process on the machine. ### Stage 3: a cache server The same Rust core, served over HTTP to many clients: a Redis-style central cache with SQLite underneath, for when your app outgrows one machine. A server that is the only owner of its files can run SQLite in exclusive locking mode, which keeps the WAL index in heap memory instead of a shared file. Measured on the development machine: | Storage mode | Read | Write | Shared by several processes? | |---|---|---|---| | WAL (the library today) | 1.18 µs | 13.9 µs | Yes | | WAL with exclusive locking (the server) | 0.69 µs | 16.5 µs | No, one owner | ## Where things stand ### Done: the core `Cache` - `get`, `set`, `add`, `delete`, `pop`, `touch`, `incr`, `decr`, with expiry. - The dictionary interface: `cache[k]`, `k in cache`, `del cache[k]`, `len(cache)`, iteration. - Indexed prefix scans: `iterkeys(prefix=..., reverse=...)`. - Maintenance: `expire()`, `clear()`, `cull()`, `volume()`, and `checkpoint()` for the write-ahead log. - Least-recently-stored eviction with `size_limit` and `cull_limit`. - Large values spilled to files, with readers that never see a spurious miss during an overwrite. - msgpack (default) and JSON serializers with exact type tags; pickle only when you opt in. - Thread-, process- and fork-safe, on GIL and free-threaded builds of 3.14 and 3.15. - Bounded connections and memory, and a target size for the write-ahead log. - `NullCache`, the `CacheLike` protocol, `py.typed`. - [`transact()`](transactions.qmd), ahead of the rest of Phase 6, with `TransactionBusy` and the `TransactionalCacheLike` protocol. - [`FanoutCache`](fanoutcache.qmd) (Phase 6c), sharding across several databases, with `Caches.register(name, shards=...)` and `ShardedCacheLike`. It removes the multi-process write-lock ceiling; making threads that share one handle scale on free-threaded Python is part of Phase 4. ### Done: concurrency review and soak (Phase 2.5) The concurrency core is where serious bugs hide, so it got its own phase before anyone runs it in production: independent deep reviews by several AI coding agents, model-checker tests for the lock state machines, and day-long soak runs with many processes and threads, forks, SIGKILLs and `clear()` under load, checking for corruption, lost writes, leaks and latency drift. The 24-hour soak passed, and it paid for itself along the way: it found the unbounded WAL growth that `wal_size_limit` and `checkpoint()` now rein in. The limit is a target, not a bound: several processes writing flat out can still take the log well past it (see [Storage and WAL](storage-and-wal.qmd)). ### Done: release (Phase 3) `0.1.2` is on PyPI, after a sanitizer pass over the whole codebase (Miri, ThreadSanitizer including a free-threaded CPython built under it, AddressSanitizer and LeakSanitizer, with the bundled SQLite built under them) and a fresh 24-hour soak on the released code. Next it replaces diskcache at [Talk Python](https://talkpython.fm), with production latency watched against the old numbers. ### Next | Phase | What it delivers | |---|---| | **4. Free-threaded performance** | Make throughput scale with threads on 3.14t and 3.15t. Today a cached read spends about 60% of its time in one system call SQLite makes for its shared-memory lock, and that's the first target. | | **5. `AsyncCache`** | An async API for asyncio frameworks that can't slow down the sync one, including `transact_async()`. | | **6a. Cache options** | More eviction policies (LRU, LFU, none), tags and `evict(tag)`, statistics, `memoize`, `peek`/`pull`/`push`, `check()`, `retry=`, and non-`str` keys. | | **6b. `transact()`** | Multi-operation transactions. Done early: see [Transactions](transactions.qmd). | | **6c. `FanoutCache`** | Done: see [FanoutCache](fanoutcache.qmd). | | **6d. `Deque` and `Index`** | diskcache's persistent collections. | | **6e. Recipes** | `Lock`, `RLock`, `BoundedSemaphore`, `throttle`, `barrier`, and stampede-proof `memoize`. | | **7. Network server** | Stage 3, above. | | **Windows** | Supported as a development platform once macOS and Linux are solid. | ## Further out Ideas being considered, not yet committed to: - **Expiry that's exact everywhere.** Lookups already treat an expired entry as gone at the moment it expires. The plan is to make `len()` and iteration do the same, and move the physical cleanup to a small background sweeper written in Rust, coordinated across processes so eight workers don't all sweep at once. - **Encryption at rest**, if a real need appears: SQLCipher for the database, authenticated encryption for spill files. ## Non-goals - Reading existing diskcache directories. diskcache-oxide uses its own on-disk format, so caches start cold when you switch. - A module named `diskcache` (it would collide with the real package). - `DjangoCache`. - Python older than 3.14. ## Core Features ### Serialization A cache has a quiet second job: it's a program that reads bytes it didn't write, left on disk by whatever process got there first. That makes the serializer the most security-relevant piece of the whole library, and it has three jobs. What goes into the cache should come out as exactly what you put in. Encoding and decoding should be fast, because they happen on every call. And reading the cache should never run code that someone else left there. diskcache's default, pickle, does the first two well and fails the third by design: unpickling runs whatever the bytes tell it to. diskcache-oxide gets all three, and pickle becomes something you opt into, one cache at a time. ## Two formats Each cache picks a structured format with `serializer=`: ```python cache = Cache('/var/cache/my-app') # msgpack, the default readable = Cache('/var/cache/my-app-json', serializer='json') ``` - **msgpack** (the default) is compact and fast. On a 450 KB page-cache entry it's about 25× faster than tagged JSON, and it stores nested `bytes` without base64. - **json** is human-readable: open `diskcache-oxide.sqlite` in any SQLite browser and read your values. It uses `serde_json` with exact float round-tripping. Plain `str` and `bytes` values skip both and are stored raw. Every row records how it was encoded, so if you change a cache's `serializer`, existing entries stay readable. ## Exact types JSON turns tuples into lists and can't store sets, datetimes, NaN, very large ints, or dicts with non-`str` keys. msgpack has some of the same gaps. The usual answer is to coerce quietly and let the bug show up later. diskcache-oxide's encoder, written in Rust, notices anything its format would change and tags it instead, so it decodes back to the same type: | Python value | Comes back as | |---|---| | `tuple` | a `tuple`, not a list | | `set` | a `set` | | `frozenset` | a `frozenset` | | `datetime.datetime` | a `datetime`, with its timezone and microseconds | | `datetime.date` | a `date` | | `decimal.Decimal` | a `Decimal`, exact | | `int` wider than 64 bits | the same int | | `float('nan')`, `inf`, `-inf` | the same float | | `bytes` nested in a container | `bytes` | | `dict` with `int` (or other non-`str`) keys | the same keys | ```python import datetime cache.set('record', { 'created': datetime.datetime.now(datetime.UTC), 'point': (37.77, -122.42), 'ids': {101, 102}, 'big': 2**128, 'by_id': {1: 'one', 2: 'two'}, }) record = cache.get('record') assert isinstance(record['point'], tuple) assert record['by_id'][1] == 'one' ``` User data can't be mistaken for a tag: a dict that happens to look like one is escaped on the way in and decoded as an ordinary dict on the way out. ## What isn't stored A value that neither format can represent exactly, such as an instance of your own class, raises `TypeError` at `set()`: ```python cache.set('service', MyService()) # TypeError: values of type MyService can't be stored without pickle (only exact built-in types round-trip) ``` Only exact built-in types count. A subclass of `dict` or `str` isn't stored as its base type, because it wouldn't come back as the class you stored. Failing at `set()` time means you find out where the value was created, not three requests later when something reads it. For a type of your own, or one from a library, [register it](#your-own-types). ## Your own types Most of what an app stores that isn't a built-in is a type it doesn't own: a bson `ObjectId` inside a session dict, a namedtuple a library hands back, the `str` subclass a Markdown renderer returns. Converting those by hand at every call site works, but it's the kind of code that drifts. Instead, tell the codec once how to store the type: ```python from bson import ObjectId import diskcache_oxide diskcache_oxide.register_type(ObjectId, name='bson.ObjectId', encode=str, decode=ObjectId) cache.set('session', {'id': ObjectId(), 'user': ObjectId()}) cache.get('session')['id'] # an ObjectId again ``` `encode` turns an instance into a value the codec can already store, built-ins or other registered types, nested as deep as you like. `decode` turns that value back into an instance. The registration is process-wide, so make it at import time, in every process that opens the directory. A dataclass needs only a name. Its `init=True` fields are read straight off the instance, with no intermediate dict, and on the way out it's rebuilt with `cls(**fields)`, so defaults and `__post_init__` still run: ```python @dataclasses.dataclass class Episode: id: int title: str published: datetime.datetime | None = None diskcache_oxide.register_type(Episode, name='tp.Episode') cache.set('episode:491', Episode(491, 'Rust and Python')) ``` The write path skips `dataclasses.asdict()`, which deep-copies: on a 20-field dataclass a `set()` goes from about 23 µs to 18. A read still builds one keyword dict and calls the constructor, so it saves almost nothing (4.0 µs against 3.8), and that's fine: a cache is mostly reads, and what you gain is the call sites. What the registry guarantees: - **Nothing is imported by name.** A registered value is stored as `{"$t:bson.ObjectId": "..."}`, and a reader looks that name up in its own registry. A process that hasn't registered it gets `ForeignValueError`, and the row stays for one that has. That is the whole difference from pickle. - **The name is yours.** Rename or move the class and the stored rows still read, because the tag is the name you chose, not the class's path. - **Lookup is by exact type.** A subclass of a registered class is its own registration. The codec's own types (`str`, `dict`, `datetime` and the rest) can't be registered, since their wire form is a contract across versions; their subclasses can, which is how a namedtuple or a `str` subclass gets in. - **A registered type is never pickled**, even on a cache with `pickle=True`. Once the encoder has used a registration for any part of a value, that value is stored through it or `set()` raises: if its `encode` raises, or returns something the codec can't store. You registered it to stay off pickle, and a silent fallback would hide the bug. - **Last registration wins**, by class and by name, so a module that a dev server reloads can register its new class object again. - **A decode function's errors are classified as a pickle hook's would be.** A `Timeout` or `OSError` raises with the row kept (a miss under `durable=True`). An error about the row, such as a `TypeError` from a dataclass row written before a field was removed, becomes `CorruptValueError` with yours as `__cause__`, and `on_decode_error='miss'` deletes that row so the writer replaces it. Decode runs while the cache holds a connection, as unpickling does, so a slow `decode` delays forks and checkpoints for as long as it runs; one that re-enters the cache is fine. A cache that needs a different encoding of a type than the rest of the process gets its own registry, consulted before the global one: ```python reg = diskcache_oxide.TypeRegistry() reg.register(ObjectId, name='bson.ObjectId', encode=lambda o: o.binary, decode=ObjectId) cache = Cache('/var/cache/my-app-binary-ids', types=reg) ``` Two processes with different registrations on one directory write rows the other can't read, the same way two processes with different `serializer=` settings would. Register at import time, the same way everywhere. ## Why pickle is off by default - **Unpickling runs code.** Anyone who can write to the cache directory, including a compromised neighbor process or a shared volume, can make every reader run arbitrary Python. - **Pickle hides mistakes.** Database sessions, open files and other objects that were never meant to be cached get pickled without complaint and fail later in confusing ways. ## Opting in to pickle When you really do need to cache arbitrary objects, turn pickle on for that cache. It's used only for values the structured format can't represent; everything else is still stored as msgpack or JSON. ```python objects = Cache('/var/cache/my-app-objects', pickle=True) objects.set('service', MyService()) ``` A good habit is to give pickled values their own cache directory, and keep everything else in a cache without pickle. ## Reading safely: `UnsafeValueError` Several processes can open one directory with different settings. A handle opened without `pickle=True` never unpickles. If it reads a value that a `pickle=True` handle stored, it raises `diskcache_oxide.UnsafeValueError` (a `TypeError` subclass) and leaves the row in place, so the `pickle=True` handle can still read it. A value whose stored bytes can't be decoded at all raises `CorruptValueError` (a `ValueError` subclass). ### NullCache Sooner or later you need to turn caching off. In tests, so each one starts clean. In local development, so you see your change instead of yesterday's. In production, at the worst possible moment, while you chase a bug that might be a stale entry and might not. Without help, that means `if cache is not None:` around every call, or a hand-written fake that drifts a little further from the real API with every release. `NullCache` is a cache that's switched off. It has the same methods and signatures as `Cache`, so swapping one for the other is a one-line change, and the code that uses it never knows. ```python from diskcache_oxide import Cache, CacheLike, NullCache def make_cache(settings) -> CacheLike: if settings.cache_enabled: return Cache(settings.cache_dir) return NullCache() ``` ## One rule > **A `NullCache` behaves like a real cache in which every entry expires the instant it's written and is swept straight away.** Every row of this table follows from that sentence, so you never have to guess, and once it clicks you never have to look it up again: | Call | Result | Because | |---|---|---| | `get(key, default)` | `default` | the entry already expired | | `cache[key]` | raises `KeyError` | the entry already expired | | `set(...)` | `True` | the write succeeded, then expired | | `add(...)` | `True` | the key was absent, so the add succeeded | | `incr(key, delta, default)` | `default + delta` | it starts from the default each time; `KeyError` if `default=None` | | `delete`, `touch`, `key in cache` | `False` | there's nothing there | | `pop(key, default)` | `default` | there's nothing there | | `len(cache)`, `cache.volume()` | `0` | it's empty and uses no disk | | `expire()`, `clear()`, `cull()` | `0` | nothing to remove | | iteration, `iterkeys()` | nothing | it's empty | | `bool(cache)`, `cache.enabled` | `False` | caching is off | The constructor accepts the same arguments as `Cache` and ignores them, and the directory is optional. Bad argument *types* still raise as they would on `Cache` (a non-`str` key, `expire=True`), so code that passes with a `NullCache` in tests also works with a real one. ### Watch `add()` The rule means `add()` always succeeds on a `NullCache`. If you use `add()` to claim a job so only one worker runs it, then with caching off every worker's claim succeeds and the job runs once per worker. That's harmless for idempotent work and wrong for work with side effects, like sending email. Keep claims like that on a cache that's never switched off. `Cache(durable=True)` goes the other way: its `add()` returns `False` when a write fails, so a claim fails closed there and open here. Keep claims on a handle that isn't durable and handle its exceptions. See [A cache that can't take a request down](getting-started.qmd#a-cache-that-cant-take-a-request-down). ## `if cache:` means "caching is on" `bool(Cache(...))` is always True and `bool(NullCache())` is always False, and both have an `enabled` property. (In diskcache, an empty real cache is falsy, which makes this check unreliable.) ```python if cache: log.info('Caching is on') ``` ## Keep invalidating while caching is off: `invalidate=` Turning caching off creates a trap. While it's off, your app keeps changing data, and the code that would have deleted the now-stale cache entries is talking to a `NullCache`. When you turn caching back on, the real cache serves the stale values. `invalidate=` closes the trap. Give the `NullCache` the real cache, and it forwards the operations that *remove* data to it, while everything else stays switched off: ```python real = Cache('/var/cache/my-app') cache = NullCache(invalidate=real) cache.set('user:42', profile) # not cached cache.get('user:42') # None cache.delete('user:42') # deleted from the real cache cache.pop('user:42') # deleted from the real cache; returns the default cache.clear() # clears the real cache cache.iterkeys(prefix='user:') # the real cache's keys, so list-then-delete still works ``` When caching comes back on, nothing you invalidated by removing it is waiting. Only removals are forwarded, so invalidating by *writing* is lost while caching is off. Overwriting the stale value with a fresh `set`, or bumping a version key with `incr` so readers build a new key, does nothing, and the old value is served once caching is back on: ```python cache = NullCache(invalidate=real) # real holds 'user:42' = old_profile, and 'users:version' = 3 cache.set('user:42', new_profile) # dropped: real still has old_profile cache.incr('users:version') # dropped: still 3, so readers keep reading the old keys ``` If your app invalidates that way, delete the entry instead (or as well), or `clear()` the real cache when you turn caching back on. ## `CacheLike` `CacheLike` is a `typing.Protocol` that both `Cache` and `NullCache` satisfy. Annotate with it and ty or pyright will accept either: ```python from diskcache_oxide import CacheLike def load_user(user_id: str, cache: CacheLike) -> dict: if (user := cache.get(f'user:{user_id}')) is not None: return user user = db.load_user(user_id) cache.set(f'user:{user_id}', user, expire=300) return user ``` A test in diskcache-oxide's own suite checks that `Cache`, `NullCache` and `CacheLike` keep identical public signatures, so they can't drift apart. ## `transact()` and `TransactionalCacheLike` `NullCache.transact()` is a no-op context manager that yields the `NullCache`, so code written as `with cache.transact() as tx:` runs unchanged with caching off. There's nothing to commit or roll back. It accepts `retry=` as `Cache.transact()` does, and a keyword-only `key=`, which it ignores. With `invalidate=`, it never opens a transaction on the real cache: a switched-off cache must never take a write lock or raise `Timeout`. So removals forwarded inside the block land at once, and an exception in the block doesn't undo them. `CacheLike` has no `transact()`. To accept either class and call it, annotate with `TransactionalCacheLike`, which adds it. See [Transactions](transactions.qmd#typing-transactionalcachelike). A [`FanoutCache`](fanoutcache.qmd)'s `transact` takes a key, so for code that accepts a `FanoutCache` or a `NullCache` use `ShardedCacheLike` instead (see [Typing](fanoutcache.qmd#typing)); the `key=` a `NullCache` ignores is what lets it satisfy both. ### Caches: named caches Every app with two caches eventually grows a file called something like `cache_service.py`. It maps names to directories, opens each cache once and hands out the same object afterwards, carries on with caching switched off when a directory can't be opened, lets you turn one off for a load test, and feeds counts and sizes to an admin page. It's the same file in every app, written slightly differently each time. `Caches` is that file, so you don't have to write it again. ```python from diskcache_oxide import Caches caches = Caches('/var/cache/my-app', timeout=1.0, on_decode_error='miss') caches.register('general') caches.register('pages', size_limit=2 * 2**30) caches['general'].set('greeting', 'hello') caches['pages'].get('/about') ``` Settings passed to `Caches` apply to every cache, and `register` can override them for one. Each cache lives in a directory named after it under the base directory, unless you pass `directory=`. One setting works differently here: a function passed as `on_decode_error` is called as `report(name, key, exc)`, with the cache's name first, so one reporter can serve every cache. (On a plain `Cache` it's `report(key, exc)`.) A function that doesn't take three arguments is rejected when you create the registry or register the cache. ## Lookups A cache opens on its first lookup, and every lookup after that returns the same object. A lookup is a single dict read with no lock, so it's fine to do on every request rather than keeping your own references. Lookups are safe from any thread, on free-threaded Python too, and in a child process after a fork. `Caches` is generic in its names. Give it a `Literal` and a misspelled name becomes a type error: ```python from typing import Literal CacheName = Literal['general', 'pages'] caches: Caches[CacheName] = Caches('/var/cache/my-app') caches['genral'] # ty: invalid argument ``` ## When a cache can't open A cache can fail to open: a permissions problem, a disk that went away, a database written by a newer version, or another process holding the lock for longer than `timeout`. A lookup then returns a [NullCache](null-cache.qmd) instead of raising, so the request carries on without the cache. The error goes to `on_open_error(name, exc)`, which logs it at `ERROR` on the `diskcache_oxide` logger by default. Pass your own function to send it to Sentry or your logging setup. The failure is remembered for `retry_open_after` seconds (10 by default), so a broken cache costs one failed open every 10 seconds, not one per request, and the lookup after that tries again. A cache that recovers is picked up without a restart. Under load, several threads may make that retry at the same moment, and each one that fails reports it. `MemoryError` isn't an open failure, so it raises from the lookup, as it would from `Cache(...)`. That includes SQLite running out of memory while opening (`SQLITE_NOMEM`), which used to be an `OSError` and got a `NullCache`. A `Timeout` at open means busy, not broken: another process holding the lock, or a first lookup that collided with a fork. It's remembered for at most 1 second, and the default report is a `WARNING` with no traceback. A custom `on_open_error` still gets the exception. Lookups of a cache that's already open never touch any of this. A bad *setting* (`size_limit=True`, `serializer='yaml'`) is a bug, not an outage, so it raises on lookup instead. What each open error means, and the fix, is in [Errors and what to do about them](errors.qmd). ## Sharded caches Pass `shards=` to `register` to open a [`FanoutCache`](fanoutcache.qmd) instead of a `Cache`. It takes the same settings, and `max_connections` is then a total across the shards. ```python caches.register('rate_limits', shards=8) caches.fanout('rate_limits').incr('ip:203.0.113.7') ``` A plain cache is read with `caches[name]` and a sharded one with `caches.fanout(name)`, typed `FanoutCache | NullCache`, because its `transact` takes a key. Reading a name the wrong way raises `TypeError`. Passing `shards` at all picks the kind, even `shards=None`, so a config file that writes `shards: null` for "none" would register a `FanoutCache`: leave it out. A directory that holds the other kind of cache fails to open with `LayoutMismatchError`, and the lookup returns a `NullCache` like any failed open. A sharded cache keeps the shard count it was created with. `register('rate_limits', shards=16)` on a folder that already has 8 shards opens it with 8, logs a `WARNING` saying so and how to apply 16, and logs one `ERROR` per process so an error tracker hears of it. Every process then routes keys by the same count, so a deploy that changes `shards=` keeps working while old and new workers overlap. The registered count applies only to a new folder: to change it, delete the folder with every worker stopped, for example at deploy. A folder deleted while old workers still run is re-created by whichever process opens it first, with that process's count. One case is still a `NullCache`: a `size_limit` or `max_connections` that fits the registered count but is too small to split over the folder's count (at least one connection and 1 MiB a shard) fails with `LayoutMismatchError` naming both counts. A setting that's wrong for the registered count itself still raises `ValueError` from the lookup, as any bad setting does. A direct `FanoutCache(path, shards=16)` on that folder still raises `LayoutMismatchError`: only the registry is lenient. Everything else on this page (`disable`, `stats`, `clear`, `expire_all`, `close_all`) works the same for both kinds. ## Switching caches off ```python caches.disable('pages') # later lookups return NullCache(invalidate=real) caches.enable('pages') caches.enabled('pages') # False while it's off ``` While a cache is disabled, its lookups return `NullCache(invalidate=real)`: reads miss and writes are dropped, but `delete` and `clear` still reach the real cache, so entries you invalidate by removing them aren't served stale when it comes back on. Invalidating by writing doesn't survive: an overwriting `set` or a version-bump `incr` is dropped like any other write, and the old value is served again (see [Keep invalidating while caching is off](null-cache.qmd#keep-invalidating-while-caching-is-off-invalidate)). Switches only change later lookups. Code that already holds the real `Cache` keeps using it, so set them at startup, before you hand caches out. If your app has its own names for what it caches (Talk Python calls them purposes), map them onto registered caches in your own code. It's a few lines: ```python PURPOSES = {'rss_feed': 'general', 'episode_html': 'episodes'} off = {'rss_feed'} # from your settings file def cache_for(purpose: str) -> CacheLike: real = caches[PURPOSES[purpose]] return NullCache(invalidate=real) if purpose in off else real ``` Build the wrapper on each call rather than memoizing it: `caches[...]` is already cheap, and a memoized `NullCache(invalidate=...)` wrapped around a cache that had failed to open would never pick up its recovery. ## Admin and shutdown ```python caches.stats() # {name: CacheStats(name, directory, enabled, count, volume, error)} caches.clear('pages', timeout=10) # works on a disabled cache too caches.expire_all(timeout=10) # {name: removed, or None if it failed} caches.close_all() # at shutdown; `with Caches(...) as caches:` does it too ``` - `stats()` opens every registered cache and never raises: a cache that can't be read has its `error` set. - `clear(name)` raises if the cache can't be opened, so an admin page doesn't report "0 cleared" for a failure. - `expire_all()` runs every cache, logging and reporting `None` for one that fails. Run it from a scheduled job rather than inside a request: a large expire takes seconds. - Maintenance calls hold the write lock once per chunk, so give them a longer `timeout=` than the request path's. ## Coming from your own diskcache wrapper Apps that run diskcache in production tend to grow the same wrapper around it: a class that turns errors into misses, a null cache for turning caching off, and a module that creates, lists and maintains the app's caches. Talk Python's, for one, are a `DurableCache` class and a `cache_service` module, and the left column uses their names. Each piece now has a home in the library: | A typical wrapper | diskcache-oxide | |---|---| | `bool()` is always True | `Cache` is always truthy (D-009); `NullCache` is falsy | | `CacheProtocol` | `CacheLike` | | `NullCache`, `PerfTestNullCache` | `NullCache`, and `NullCache(invalidate=real)` | | `get` that logs, deletes and misses on any error | `Cache(on_decode_error='miss')` or a reporting function ([details](getting-started.qmd#values-that-cant-be-read)) | | A cache that can't take a request down | `Caches(base_dir, durable=True)`, which also covers `set`, whose failures such a wrapper usually raises ([details](getting-started.qmd#a-cache-that-cant-take-a-request-down)) | | `create_cache(name, size_limit)` | `caches.register(name, size_limit=...)`, then `caches[name]` | | `timeout=1.0` on every open | `Caches(base_dir, timeout=1.0)` | | A `NullCache` when opening fails | the same, reported through `on_open_error` and retried after 10 s | | `get_all_caches()`, `clear_*()`, `close_all_caches()` | `caches.stats()`, `caches.clear(name, timeout=...)`, `caches.close_all()` | | A separate maintenance handle with a 10 s timeout | `timeout=` on `clear()`, `expire()` and `cull()` | | Hourly `expire_all_caches()` inside `create_cache` | `caches.expire_all()` from a scheduled job | The library's self-healing read is safer than the wrapper's in three ways: - **It deletes only what failed.** The wrapper's `get`-then-`delete` can remove a fresh value another process wrote in between. The library deletes exactly the stored version it couldn't read. - **It never waits to clean up.** If the write lock is busy, the bad row stays and a later read tries again. - **It doesn't hide outages.** The wrapper catches every exception, so a lock `Timeout` also became a silent miss. In the library, `Timeout`, I/O errors and `MemoryError` still raise by default, and so does reading a pickled value from a cache without `pickle=True`, which is a configuration mistake. `durable=True` is how you choose a miss for an outage, and it counts each one and logs them, at most once a minute, rather than staying silent; `MemoryError` and that mistake raise regardless. What stays in the app: reading its settings file and environment variables (pass the result as `base_dir`), its purposes and their on/off flags, descriptions for the admin page, and wiring the reporting functions to Sentry or logbook. ### FanoutCache: sharded caches A plain `Cache` has one write lock per database file, which makes it a shop with one checkout lane: readers walk straight through, but every writer queues, and eight worker processes that all bump counters are eight carts in one lane. A `FanoutCache` opens more lanes. It's one cache spread over several SQLite databases, called shards. Each key lives in exactly one shard, chosen by a fixed hash of the key, so writes to different keys go to different files and stop queueing on one lock. It has the same methods as `Cache` for single keys, and it's one directory, shared by any number of threads and processes. ```python from diskcache_oxide import FanoutCache cache = FanoutCache('/var/cache/my-app-rate-limits', shards=8) cache.set('ip:203.0.113.7', 1, expire=60) cache.incr('ip:203.0.113.7') cache.shard_of('ip:203.0.113.7') # 0 to 7, the same in every process and every release ``` ## Should you use it? Use `FanoutCache` when **several processes write to the same cache**. Use a plain [`Cache`](getting-started.qmd) when they don't. The honest picture, from the benchmark in the repository (`bench/fanout.py`, release builds, a 99:1 get:set mix with 1 KB values on a Mac with 18 cores): | Processes | Plain `Cache` | `FanoutCache(shards=8)` | Defunct diskcache `FanoutCache(8)` | |---|---|---|---| | 1 | 660k ops/s | 714k | 396k | | 4 | 607k | 1,553k | 987k | | 8 | 262k | 1,496k | 1,238k | - **It removes the multi-process write-lock ceiling.** A plain `Cache` peaks at two processes and then falls, because the 1% of operations that are writes all queue on one database file's write lock. A `FanoutCache` keeps climbing and holds its rate: 5.7x a plain `Cache` at 8 processes, and 1.2x the defunct diskcache's `FanoutCache`. If your app runs a pre-forking server with a worker per core and writes often (counters, rate limits, job claims), this is the case it's for. - **It does not yet make threads sharing one handle scale on free-threaded Python.** Four threads on one `FanoutCache` handle read no faster than, and on Linux slower than, four threads on a plain `Cache`, and well short of four separate handles. A larger `max_connections` helps but doesn't close the gap. This is a limit in what threads on one handle share, it's being worked on, and no numbers are promised here. If your concern is read throughput from many threads in one process, a `FanoutCache` doesn't fix it today. - **On GIL builds, threads are flat for every diskcache-oxide cache,** as the GIL says they must be, and a `FanoutCache` is about 10 to 15% ahead of a plain `Cache` on that mix at one thread. It's well ahead of stock diskcache, which collapses as threads are added. - **A single process with a few threads and few writes gains little.** You pay the cost of more files (below) for a benefit you won't see. Single-key reads and writes cost the same as on `Cache` plus the hash, which is about the cost of one more string hash, while no transaction is open in the process (see `transact(key=)` below). ## The layout and the hash A `FanoutCache` directory looks like this: ``` my-app-rate-limits/ diskcache-oxide-fanout the manifest 000/diskcache-oxide.sqlite shard 0 001/diskcache-oxide.sqlite shard 1 ... ``` Each shard is an ordinary diskcache-oxide database. The manifest records the shard count and the hash. A key's shard is XXH3-64 (seed 0) of its type byte and its UTF-8 bytes, mapped onto the shard count. **That hash is part of the on-disk format and is frozen**: pinned by golden vectors in the test suite, so every process and every release sends a key to the same place. A future scheme would get a new format number, never a change in place. Because the shard count decides where every key lives, it's recorded, and a different `shards=` is an error instead of a quiet loss of data: ```python FanoutCache('/var/cache/my-app-rate-limits', shards=16) # LayoutMismatchError: the root has 8 shards, not 16 ``` `shards=None` (the default) uses the stored count, or creates 8. Changing the count means a new directory. Creating the manifest uses a hard link, so a filesystem without them (FAT, exFAT, SMB, some FUSE mounts) raises `OSError` saying so. ### Plain `Cache` and `FanoutCache` directories don't mix A `Cache` opened on a `FanoutCache`'s directory, or on one of its shard directories, raises `LayoutMismatchError`, and so does a `FanoutCache` opened on a plain `Cache`'s directory. The refused open creates nothing, so a typo doesn't leave a directory that neither kind can open. `LayoutMismatchError` is both a `ValueError` and a `FormatMismatchError`. If a registry hits it, the lookup returns a [`NullCache`](null-cache.qmd) and reports it like any failed open. A registry that meets another shard count than its registration's opens the folder with the count on disk and warns instead ([Caches](caches.qmd)). ## Choosing `shards` and `max_connections` **`shards`** is 1 to 256. Eight is a good start: the benchmark's write scaling is already flat by 4 processes at 8 shards. More shards spread writes further but cost file descriptors (below) and give each shard a smaller share of `size_limit`. Pick it once, because it's permanent for the directory. **`max_connections`** is the most SQLite connections the process opens for the whole `FanoutCache`, split across the shards (`max_connections // shards` each, with the first `max_connections % shards` getting one more). It must be at least `shards`, and the default is two per shard with a floor of 8 in all: 16 at the default 8 shards. With one per shard, threads sharing a handle waited for a shard's only connection and read 16-31% below a plain `Cache`; two gets 97-98% of the read rate of 32. SQLite's page caches for the whole cache are then about `16 × sqlite_cache_size` (16 MiB at the defaults, against a plain `Cache`'s 8). A busy shard only gets its share, so if a measurement shows one hot shard waiting for a connection, raise it. It doesn't matter across processes, since each process has its own connections. **File descriptors.** Each connection holds its shard's database and WAL open, and each shard adds its `-shm` file and a pin. The descriptors inside the root stay within `2 × max_connections + 2 × shards`, however many threads you run: 48 at the defaults, measured at 32-34 with 64 threads on 3.14t. That's a ceiling, reached only when that many threads hit the same shards at once, since a shard opens its second connection only then, where stock diskcache's FanoutCache holds 264. `close()` leaves none. Opening needs about 4 per shard, so 256 shards need more than 1,024, Linux's usual default `ulimit -n`. If there aren't enough, opening raises `OSError` with `errno` `EMFILE`, and a message giving the shard count, the descriptors needed, the limit and how many are already open. A failed open leaves no descriptor behind. **`size_limit`** is a total. Each shard gets `floor(size_limit / shards)` and culls on its own, and each needs at least 1 MiB, so the total must be at least `shards` MiB (`ValueError` otherwise). A new root with none given gets 1 GiB in all. `cache.size_limit` reads back the total (the sum of the shards' floored shares). A key that happens to land in a full shard evicts from that shard, even if the others have room. ## `shard_of` and `transact(key=)` `cache.shard_of(key)` says which shard a key lives in. It's how you find out which keys can share a transaction. `transact` on a `FanoutCache` is not stock's. Stock's takes every shard's lock, so one transaction stalls every writer in every process. Here it's a transaction on **one shard**, named by a key: ```python with cache.transact(key='totals:2026-11') as tx: n = tx.get('totals:2026-11', 0) tx.set('totals:2026-11', n + 1) ``` - **Only that shard is locked.** Writes to other shards, from other threads and processes, carry on. - **`key` is required and keyword-only.** A ported `with cache.transact():` raises `TypeError`, and so does `transact('k')`, so a key can't be mistaken for `retry`. - **Other shards' keys are refused, reads included.** On the thread inside the block, a call for a key on any other shard raises `ValueError` naming both shards, through any handle on the same directory. Entering a second `transact` for a key on another shard is refused too. A nested `transact` for a key on the same shard joins the outer one. A read on another shard is refused because it would see a value the block's writes can't be atomic with, so a block that mixes shards is a mistake either way, and it's caught on the first call. - **Whole-cache calls are refused too.** `len()`, `volume()`, iteration (`iter()`, `iterkeys()`, and `next()` on an iterator made before the block), `clear()`, `expire()`, `cull()` and `checkpoint()` raise `ValueError` inside the block, since they would run inside the transaction on one shard and outside it on the rest. Inside the block, only keyed calls on the transaction's shard reach the cache. Call the others before or after it. `close()` isn't refused, since it never raises. - Everything else is as in [Transactions](transactions.qmd): commit on exit, roll back on an exception, `retry=`, `TransactionBusy`, and `durable=True` still raising. If you need related keys in one transaction, give them a shared prefix scheme and check with `shard_of` that they land together, or keep them in one value. There is no transaction across shards. A `FanoutCache` isn't a `TransactionalCacheLike`, because its `transact` needs a key. It and `NullCache` are both [`ShardedCacheLike`](#typing). ## Maintenance: one budget, partial failures `clear()`, `expire()`, `cull()` and `checkpoint()` run over every shard. Their `timeout=` (default: the handle's) bounds **the whole call**, not each shard. The handle's timeout also bounds opening, `len()` and `volume()` as one budget, and `close()` uses at most 1 s of it for checkpoints. A shard reached after the deadline still gets one try with no wait, so idle shards are always done. A shard that fails doesn't stop the others. Afterwards the call raises the first failure's own type, so `except Timeout` works, with attributes saying how far it got: ```python try: cache.clear(timeout=5) except Timeout as exc: exc.failed_shards # (3,): the indexes of the shards that failed exc.removed # entries removed in all, including what a failed shard removed before it failed ``` `removed` is the total removed in all, including what a failed shard removed before it failed, except for `checkpoint()`, which has none; `checkpoint()` returns True only if every shard's log was reset. Writes made while a removal runs survive on every shard, as on `Cache`. `durable=True` doesn't swallow any of these. ## Timeouts on single keys On a `FanoutCache`, the defunct diskcache silently turns a busy shard into a dropped write: `set` gives False after 10 ms. diskcache-oxide behaves as `Cache` does: a busy shard raises `Timeout` after `timeout` (60 seconds by default), and so do `cache[k]`, `cache[k] = v` and `del cache[k]`. If you want a miss instead, say so with [`durable=True`](getting-started.qmd#a-cache-that-cant-take-a-request-down), which also logs and counts the failures, with one report for the whole `FanoutCache`. ## Iteration order `iter(cache)` yields every key once, shard 0 to the last, each shard in its insertion order. That is not the cache's insertion order, and not stock's, since the shards hold different keys. `iterkeys(prefix=, reverse=)` merges the shards into one sorted order, the same order a plain `Cache` gives, and holds no connection between yields. Neither is a snapshot: a concurrent write may or may not be seen, never twice. There's no `reversed()` yet. ## In a `Caches` registry Pass `shards=` to `register` and the name opens a `FanoutCache`, with the same settings as any other cache: ```python caches = Caches('/var/cache/my-app', timeout=1.0, durable=True) caches.register('general') caches.register('rate_limits', shards=8, max_connections=16) caches['general'].get('greeting') caches.fanout('rate_limits').incr('ip:203.0.113.7') ``` Plain caches are read with `caches[name]` and sharded ones with `caches.fanout(name)`, because their `transact` takes a key. Reading a name the wrong way raises `TypeError`. Passing `shards` at all is what picks the kind, so `shards=None` still registers a `FanoutCache`; leave it out for a plain cache. A failed open becomes a `NullCache`, `disable()`, `stats()`, `clear()` and `close_all()` work the same, and `fanout(name)` is typed `FanoutCache | NullCache`. See [Caches](caches.qmd). ## Typing `ShardedCacheLike` is a protocol for "a `CacheLike` whose `transact` takes `key=`". Both `FanoutCache` and `NullCache` satisfy it, so code that works with either a sharded cache or a switched-off one can be annotated with it: ```python from diskcache_oxide import ShardedCacheLike def bump(cache: ShardedCacheLike, key: str) -> None: with cache.transact(key=key) as tx: tx.set(key, tx.get(key, 0) + 1) ``` ## What `FanoutCache` doesn't have yet Stock's `FanoutCache` also has `cache()`, `deque()` and `index()` sub-collections, `stats()`, `evict()` and tags, `check()`, `reset()`, `read()`, `reversed()`, the `retry=`, `read=` and `tag=` arguments, settings read as attributes, and pickling. None of those exist yet; the ones that belong to a feature come with that feature (see the [roadmap](roadmap.qmd)), each designed across shards rather than copied. `directory` is required and there's no `disk=`, as for `Cache`. The constructor takes `shards` as a keyword: stock's `FanoutCache(path, 4)` means four shards, but here the second argument is `timeout`, so write `shards=4`. ### Transactions Every write is already its own transaction, atomic across every thread and process using the directory. Sometimes one write at a time isn't enough. A request bumps a total, bumps a per-path counter and records when it happened, and a reader that sees two of those three has seen something that was never true. `transact()` groups several calls into one: they all commit together when the block exits, or none of them do. ```python with cache.transact() as tx: tx.incr('requests') tx.incr(f'requests:{path}') tx.set('last_request', now) ``` That's also faster. Each commit costs a disk sync, so grouping writes pays for one instead of several: three counter bumps measured 79 µs one by one and 48 µs in one transaction. ## What the block does - **Commit on exit, roll back on an exception.** If the block raises, none of its writes are kept, and the exception carries on as usual. - **`tx` is the cache itself.** On the thread that entered the block, every call on that cache directory inside it runs in the transaction, whether you call it on `tx`, on `cache`, or on another `Cache` handle for the same directory. diskcache's `transact()` yields `None`, so code written for it keeps working. - **The block reads its own writes.** Other threads and processes see nothing until it commits, and their writes to this directory wait for it, up to their own `timeout`. Keep blocks short. - **Nesting joins the outer block.** A `transact()` inside another one on the same directory, on the same thread, joins the outer transaction instead of starting its own. As in diskcache, the inner block's exit commits nothing, and an exception you catch inside the outer block doesn't undo the inner block's writes. Only the outer block's exit commits. ```python with cache.transact(): cache.set('a', 1) try: with cache.transact(): cache.set('b', 2) raise ValueError except ValueError: pass # Both 'a' and 'b' are committed: the inner block had no savepoint of its own. ``` The object `transact()` returns is used once: entering it a second time, or while another thread is entering it, raises `RuntimeError`. ## Beginning one: `timeout` and `retry=` Beginning a transaction takes the directory's write lock, and waits for it up to the cache's `timeout`, then raises `Timeout`, as a write does. With `retry=True` it keeps trying instead. Each attempt waits up to `timeout` with the interpreter released, and `Ctrl-C` is checked between attempts, so a `KeyboardInterrupt` gets through within one `timeout`. ```python with cache.transact(retry=True) as tx: ... ``` ### Two caches at once: `TransactionBusy` Inside one cache's transaction, beginning a transaction on a second cache takes that cache's write lock if it's free. If another thread holds it, `transact()` raises `TransactionBusy` at once instead of waiting: that thread could be waiting for the lock you hold, and then neither would ever finish. `TransactionBusy` is a subclass of `Timeout`, so existing handlers still catch it, and `retry=True` never retries it, since trying again while you still hold the first lock can't help. To use two caches in one block, always nest them in the same order everywhere in your app, and if `TransactionBusy` reaches you, leave the outer block and try the whole thing again. ## When a block doesn't commit A `with` block never exits cleanly when its writes are gone. Its exit raises `TransactionAborted`, a `RuntimeError`, when the transaction didn't commit: - an operation inside it failed in a way that left the transaction unusable (a full disk, an I/O error). Later writes in the block fail at once, and nothing in it commits; - it was ended from another thread, which rolled it back (below); - it was ended from inside one of its own operations, such as a pickle hook that calls the block's `__exit__`. That raises `RuntimeError` where it's called, and the transaction rolls back when that operation returns. A commit that can't finish raises too: a busy or failed `COMMIT` raises `Timeout` or an `OSError`, and nothing is kept. ### Ending it from another thread diskcache says a transaction "may not be shared between threads". In diskcache-oxide, any thread may call the transaction's `__exit__`, to commit (no exception) or roll back the owner's transaction. The rules: - While the owner is inside one of its own operations (decoding a value, say), a rollback is left to that operation, and happens when it returns. A commit waits for it, up to the cache's `timeout`. A commit that can't run by then raises `Timeout` and rolls back. - The owner's own exit afterwards raises `TransactionAborted` unless the transaction committed. - An owner's exit that comes while the other thread is still committing waits for the outcome, up to the cache's `timeout`, and reports it. Past that, it raises `Timeout` ("whether it committed is unknown"). An exit with an exception passing doesn't wait. - Writes the owner makes after another thread ended the transaction run on their own, outside it. ## `durable=True` On a [`durable=True`](getting-started.qmd#a-cache-that-cant-take-a-request-down) cache, beginning a transaction still raises, and so does any failure inside the block: dropping one write and committing the rest would break the transaction's atomicity. Only calls outside a transaction are answered as misses. ## A transaction belongs to a thread, not a task The calls that join a transaction are the ones made on the thread that entered the block. In asyncio, every task on a loop runs on that same thread. So if you `await` inside the block, another coroutine that runs meanwhile and writes to the same cache joins your transaction, and rolls back with it if your block raises. Don't `await` inside a `transact()` block. An `async with cache.transact_async()` form is planned for `AsyncCache` (see the [roadmap](roadmap.qmd)), where waiting for the write lock won't block the event loop. ## Transactions and forks A fork waits for every transaction other threads hold, with no timeout, so the child never inherits half of one (D-012). A long transaction holds up every fork in the process while it's open, and calls behind that fork time out (see [How long a call waits for a fork](concurrency-and-forks.qmd#how-long-a-call-waits-for-a-fork)). Forking from inside your own transaction can't be waited for: the child's copy is poisoned and raises `RuntimeError` (see [Forking from inside your own transaction](concurrency-and-forks.qmd#forking-from-inside-your-own-transaction)). ## Typing: `TransactionalCacheLike` `Cache` and [`NullCache`](null-cache.qmd) both have `transact()`. Annotate with `TransactionalCacheLike` to accept either and call it: ```python from diskcache_oxide import TransactionalCacheLike def count(cache: TransactionalCacheLike, path: str) -> None: with cache.transact() as tx: tx.incr('requests') tx.incr(f'requests:{path}') ``` On a `NullCache`, `transact()` is a no-op that yields the `NullCache`. A [`FanoutCache`](fanoutcache.qmd) is not a `TransactionalCacheLike`: its `transact(*, key)` covers only the shard that `key` lives in, and refuses calls for keys on other shards inside the block. Annotate with `ShardedCacheLike` to accept a `FanoutCache` or a `NullCache`. `CacheLike` itself has no `transact()`, so a cache whose transactions need more than `retry=` (a sharded cache's `transact(key=...)`) can still be a `CacheLike`. A [`Caches`](caches.qmd) lookup is a `TransactionalCacheLike`, so `caches['general'].transact()` type-checks, and a disabled cache's `transact()` is the `NullCache` one. ### Concurrency and forks A production Python web app is a crowd. Several worker processes, often several threads in each, a pre-forking server that creates and recycles those workers while they're busy, and a cron job or an admin script opening the same cache on the side. More and more, it's also free-threaded Python, where the threads really do run at the same time. diskcache-oxide is designed for all of that at once, and its test suite runs each of these cases on every change, because this is where cache bugs hide: not in `get` and `set`, but in what happens when two of them collide. ## Threads One `Cache` can be shared by any number of threads. There's nothing to lock and no per-thread setup. - **Free-threaded ready.** The extension declares `gil_used = false` and never relies on the GIL for correctness. Its shared state is guarded by Rust's own locks and atomics. Every change is tested on 3.14, 3.14t, 3.15 and 3.15t. - **Waits don't hold up Python.** Anything that can wait, such as a busy write lock, reading a large value file, or a chunk of `clear()`, releases the interpreter while it waits, so other Python threads keep running on GIL builds too. - **Threads queue fairly.** Writers in one process queue on an in-process lock for each database file instead of all retrying against SQLite's busy handler, and long maintenance calls like `clear()` and `expire()` step aside between chunks so a waiting write gets in. ### A bounded connection pool diskcache opens a SQLite connection for every thread that touches the cache and keeps it. With 64 threads that's 131 open files in the cache directory, and a server with enough threads runs out of file descriptors. diskcache-oxide shares a pool of connections per database file across every handle in the process: ```python cache = Cache('/var/cache/my-app', max_connections=8) # the default ``` The pool never holds more than `max_connections` connections, so at most `2 × max_connections + 1` files are open in the cache directory, however many threads you run. A thread that finds every connection busy waits for one, up to the cache's `timeout`. When the last handle on a directory closes, every connection closes with it, whichever threads used them. ## Processes Any number of processes can open the same directory, each with its own `Cache`. That's the normal deployment: every gunicorn or uvicorn worker opens the cache at startup. - **WAL mode, always.** SQLite runs in write-ahead-log mode, so readers never block writers and writers never block readers. One writer at a time commits to each database file. diskcache-oxide checks that WAL mode is really on when it opens a cache, and fails loudly if it isn't (on some network filesystems, for example). - **Atomic across processes.** `add()`, `incr()`, `decr()` and every other write is a single transaction. Two processes can't both win an `add()`, and two `incr()` calls never lose an update. `pop()` is a read and then a transaction that deletes exactly the version it read, so two processes can't both get the same value. - **Maintenance that doesn't destroy work.** Every write gets a new sequence number. `clear()` and `expire()` note the latest number when they start and only touch older rows, so writes from other processes that land while they run are kept. - **Large values are always whole.** A process reading a large value while another overwrites it gets the old value or the new one, never a miss. (diskcache's standalone reproduction of that race sees about 40 misses in every 325,000 reads.) - **One write lock per database file.** Every writer to a directory queues on its lock, so with many processes writing often a plain `Cache` stops scaling at a couple of processes. A [`FanoutCache`](fanoutcache.qmd) has one lock per shard and keeps scaling (5.7x a plain `Cache` at 8 processes on a 99:1 read/write mix). It doesn't change the thread picture below: threads sharing one handle on free-threaded Python are not lifted by it today. - **Opening never writes.** Opening a cache with unchanged settings and reading from it never takes the write lock or touches the database file, so a read-mostly worker costs the writers nothing. ## Forks Pre-forking servers (gunicorn, Celery's prefork pool, `multiprocessing` with the fork start method) create workers with `fork()`, which copies the whole process. Picture a thread halfway through a write at the instant the server forks. The child is a photocopy of the parent, SQLite's note that this process holds the write lock included. But only the forking thread exists in the child, so the thread that was going to release that lock isn't there. In diskcache, that child's first write waits for a ghost: it stalls for the full timeout (60 seconds by default). Under a busy read/write load, that hits 20-50% of forks. diskcache-oxide prevents it with a **fork barrier**, a crossing guard installed when `diskcache_oxide` is imported. It stops traffic in a specific order: 1. **Before the fork,** the barrier waits for any write transactions already running in other threads to commit, and holds off new ones. It waits with the interpreter released, so the threads it waits for can finish. Then it closes the idle pooled connections, so the child doesn't inherit SQLite's lock bookkeeping for those files. 2. **In the parent,** it lets the held-off writes continue. 3. **In the child,** it drops any connection still inherited without closing it (closing a connection inherited across a fork is unsafe in SQLite), and opens fresh ones on first use. The result: whatever other threads were doing at the moment of the fork, the child is born with no locks and no debts, and can read and write right away. The barrier covers every `Cache` in the process, not just the one you're holding. Your own `os.register_at_fork(after_in_child=...)` hooks can use a cache too, whether they were registered before or after `diskcache_oxide` was imported. Closing the idle connections matters for more than file descriptors. SQLite keeps one lock record per database file per process, and a new connection that finds a record saying the process already holds a shared lock takes no lock of its own. A child that inherited such a record would run with no real locks, and once the parent closed its cache, another process's last close could delete the write-ahead log under the child and lose its writes. The cost is that the parent reopens connections after each fork. The only connections a child can still inherit are the ones the forking thread itself was using: a transaction it holds (see below), or an operation it forked from inside, such as from a callback. The child leaves those file descriptors open by design. ### How long a call waits for a fork A fork counts against `timeout`. A cache call behind a pending fork waits for it through the call's own `timeout` and then raises `Timeout`, so `timeout` has to exceed the work in flight, plus the fork, plus the interpreter hand-off. A fork takes about a millisecond in a quiet small process, but tens to hundreds of milliseconds beside busy threads on a GIL build (the forking thread has to get the interpreter back), longer in a large process, and it waits for work in flight such as a forced checkpoint. A `timeout` under that sees `Timeout` during ordinary forks, and a loop on `timeout=0` adds to the problem by being one of those busy threads. A fork waits for every [transaction](transactions.qmd) another thread holds, with no timeout of its own, so a long transaction delays every fork in the process. Code that holds a transaction or runs inside a pickle hook (or a registered type's `decode` function, which runs in the same place), and waits for another thread's cache call, can keep a fork from finishing: the fork waits for that code, and the other thread waits for the fork. That other thread's call now raises `Timeout`, and the fork finishes once your code lets go. Until it does, `os.fork()` doesn't return, a second `os.fork()` queues behind it, and every thread with no transaction or hook of its own gets `Timeout` on every `Cache` in the process, after using up its whole `timeout` (60 seconds by default). Other processes aren't affected. If the code never reacts (a hook that retries forever, or one blocked on something that isn't a cache call), the fork stays stuck. A hook stuck in C code that holds the GIL stops everything, as it would without a cache. Two calls don't raise. `checkpoint()` returns `False` when a fork keeps it from running. `close()` takes at most about 2 seconds behind a stuck fork, 1 for the checkpoint and 1 for the release, and `del` about 1, whatever `timeout` is. An ordinary fork ends well inside that, so `close()` waits it out as usual. If the fork outlasts it, `close()` returns anyway and the connections close when the fork ends. Before-fork hooks run in reverse registration order, and `logging` and `concurrent.futures` each take a lock in theirs and keep it until the fork is over. `diskcache_oxide` makes sure its own hook runs before those two, so they stay usable while a fork waits for your code: a pickle hook or a `decode` function can still log or submit work. Two things it can't fix. Another library's hook that takes a lock and is registered after `diskcache_oxide` stays held while a fork is stuck. And a thread that calls the cache while holding `logging`'s module lock can hold up a fork until that call's `timeout`. With [`durable=True`](getting-started.qmd#a-cache-that-cant-take-a-request-down), a `Timeout` behind a fork is a miss or `False` rather than an exception on a single-key call, so a `timeout` shorter than this makes ordinary forks into misses. A pickle hook or a registered type's `decode` that re-raises the `Timeout` it got from another thread's call lets go of its level under any policy, which clears the stuck fork. What `durable=True` adds is that the request doesn't fail: that `Timeout` is a miss, with the row kept. ### Forking from inside your own transaction CPython won't let a library refuse a fork, so the barrier can't wait for a [transaction](transactions.qmd) held by the very thread that is forking. The child's copy of that transaction is poisoned: anything done inside it raises `RuntimeError` and nothing commits, and the child's writes to that database raise `RuntimeError` straight away instead of stalling for a minute. A `transact()` in the child doesn't join the inherited transaction either: it raises `RuntimeError` too. The parent's block carries on and commits as usual, and the parent gets a `RuntimeWarning`. A clear error at the call site beats a mysterious 60-second hang in a fresh worker. ## Free-threaded performance today On free-threaded 3.14t, diskcache-oxide is already faster than diskcache at every thread count, and at a 1,000:1 read/write mix 8 threads reach about 1.4 million operations per second. But that's only about 1.6× its single-thread number, so threads don't scale the way they should yet. Profiling shows where the time goes: about 60% of a cached read is one system call SQLite makes to take its shared-memory read lock, under a process-wide mutex. Removing that wall is the next stage of the [roadmap](roadmap.qmd). A [`FanoutCache`](fanoutcache.qmd) doesn't remove it for threads sharing one handle either; that is being worked on. See [Performance](performance.qmd). ### Storage and WAL Nobody restarts a cache. It sits inside a worker process for weeks, and whatever it does to memory and disk, it does slowly and out of sight, until an alert fires at a bad hour. So how much a cache uses has to be something you set, not something you discover. This page covers what diskcache-oxide puts on disk, and the settings that keep memory and disk where you put them. ## What's in the directory Open the directory and this is what you find: ``` /var/cache/my-app/ ├── diskcache-oxide.sqlite # keys, metadata and small values ├── diskcache-oxide.sqlite-wal # SQLite's write-ahead log ├── diskcache-oxide.sqlite-shm # SQLite's shared-memory index for the log └── 3f/ └── 3fa2…c91e.val # one large value ``` The database file carries a format version, and a build that can't read it raises `FormatMismatchError` rather than guessing. During 0.x, a release that changes the format bumps the minor version and says so first in its changelog entry; existing caches are deleted on that upgrade, not migrated (see [Upgrading](switching-from-diskcache.qmd#upgrading-diskcache-oxide)). ## Small values inline, large values in files - **Under 64 KiB** (after encoding), a value is stored in its database row. Reading it is one indexed lookup. - **64 KiB or more**, it's written to its own file, and the row points to it. This keeps the database small and its page cache hot, and large values stream straight from the file. Writing a large value never holds the database lock while the bytes go to disk: the value goes to a temporary file first, then inside a short transaction the file is renamed into place and the row updated. The old file is removed after the commit, and file names are never reused. Large values live one folder deep, in up to 256 folders named by the first two characters of the file's name (D-042). Deleting a value removes only its file. When the process closes its last handle on the cache, it sweeps those folders: a temp file that no writer has touched for an hour, left by a process killed mid-write, is removed, and a folder left empty goes too, skipping any touched in the last 30 seconds so a busy folder isn't removed and re-created, all within 100 ms; a close that runs out of time leaves the rest to a later one. `clear()` runs the same sweep and removes the folders it emptied at once. So a cache never has more than 256 value folders, and a closed one has none sitting empty for long. A `FanoutCache`'s shards don't sweep yet: each keeps at most 256. A reader racing an overwrite always gets a whole value, old or new. If a value's file really is gone (deleted by hand, or cut short by a crash, since value files aren't fsynced), `get` returns the default and the orphaned row is cleaned up, so `in` and `len` don't keep reporting an entry that can't be read. ## Size limit and eviction ```python cache = Cache('/var/cache/my-app', size_limit=500 * 1024 * 1024, cull_limit=10) ``` - `size_limit` is the cache's disk budget in bytes: the database plus value files. The default is 1 GiB and the minimum is 1 MiB. - When a `set()` finds the cache over its limit, it evicts up to `cull_limit` of the least recently stored entries (default 10). `cull()` evicts down to the limit on demand. - `volume()` reports the current size. Least-recently-stored eviction means a `get` never has to write anything, which keeps reads fast and lets any number of processes read in parallel. LRU and LFU policies are planned (see the [roadmap](roadmap.qmd)). ## Memory: `max_connections` and `sqlite_cache_size` diskcache gives each thread its own connection with a 32 MiB page cache plus a 64 MiB memory map, so its memory grows with your thread count. diskcache-oxide bounds it: ```python cache = Cache( '/var/cache/my-app', max_connections=8, # connections per database file, per process (default 8) sqlite_cache_size=256, # 4 KiB pages per connection: 1 MiB (default) ) ``` SQLite's memory for a directory stays around `max_connections × sqlite_cache_size` pages, 8 MiB per process by default, however many threads use it. The 1 MiB default is the smallest size that left hot reads, writes and the threaded benchmarks unchanged. If your cache is large and read uniformly at random, raising it helps: 2,000 pages measured about 17% faster on that workload. ## The write-ahead log SQLite's write-ahead log (WAL) is a ledger of recent changes kept beside the database, and it's what lets readers and a writer work at the same time: the writer appends to the ledger while readers keep reading the book. Every so often SQLite folds the ledger back into the book and starts over. The catch in a multi-process app is that it can only start over when it finds a moment with no readers, and with readers in other processes that moment may never come. A burst of writes (a big fill, or a `clear()`) grows the log to fit the burst, and then it stays that size. diskcache-oxide's soak tests saw logs over 1 GiB next to databases of a few MiB. diskcache-oxide handles this in layers, from free and always on to explicit. ### Always on: `wal_size_limit` ```python cache = Cache('/var/cache/my-app', wal_size_limit=64 * 1024 * 1024) # the default ``` This tells SQLite to trim the log back to half the limit (32 MiB by default) whenever it resets, and scales how often SQLite checkpoints automatically. These are connection settings, so they never pause anything. At normal load they keep the log at a few MiB. ### On demand: `checkpoint()` ```python cache.checkpoint() # True if the log was reset and truncated ``` `checkpoint()` copies the log into the database and truncates it to zero. It waits briefly for in-flight reads, retrying up to the cache's `timeout`, and returns `False` instead of raising if other processes keep the database busy. Call it when a short pause suits your app: at shutdown, in a maintenance job, or after a large `clear()`. ### At close `close()` runs the same checkpoint when this is the process's last handle on the cache and the log is over half the limit, waiting at most a second. A small log, the usual case, costs nothing extra. Pass `close(checkpoint=False)` to skip it. ### Opt-in: automatic checkpoints ```python cache = Cache('/var/cache/my-app', wal_checkpoint='auto') ``` With `'auto'`, writes check the log (at most every 250 ms or 64 writes) and checkpoint it when it's over the limit, with the interpreter released. Readers in other processes can get in the way for a moment, so it tries the final truncation up to 8 times, a couple of milliseconds each, before backing off for a second. It never fails a write. It isn't the default because on a saturated disk the write that triggers the checkpoint can still pause for a long time, and no write should pay for housekeeping unless you asked for it. The plan is to move this into a background maintenance thread so writes never pay for it at all. Writes queue behind a checkpoint for tens of milliseconds, so a `timeout` under about 35 ms can time out behind one. ### The limit is a target, not a bound `wal_size_limit` says where the log should stay, not where it's guaranteed to stay. Sustained writes from several processes near flat out can take it to tens of times the limit: we measured 6 GiB on free-threaded 3.15 and 21 GiB on 3.14 in 10 minutes, against a 64 MiB limit. It's much smaller below flat out, for example 250 MiB at about 2,600 writes per second across 3 writers in `auto`. In the default `manual` mode, with busy readers in other processes, the log grows until something checkpoints. If your app has that shape, use `wal_checkpoint='auto'` or call `checkpoint()` periodically. A script should call `checkpoint()` or `close()` after a large fill. ## Maintenance calls All of these work in chunks, release the database between chunks so other writers get in, and are safe to call while other processes use the cache: - `expire()` removes every expired entry and its value file. - `clear()` removes every entry that existed when it started, and keeps writes that land while it runs. - `cull()` evicts down to `size_limit`. ## Background & Reference ### Performance A microsecond is nothing until you multiply it. Web apps call their cache on almost every request, often straight from the asyncio event loop, where every microsecond the cache spends is a microsecond every other request on that loop waits. So diskcache-oxide's performance work starts with read latency, then throughput under many threads, then predictable resource use. Correctness always comes first: every speedup has to keep the conformance suite green, or it doesn't ship. The numbers below are the ones that survived. ## The numbers Measured with `bench/gate.py`, which runs the defunct python-diskcache (5.6.3, its last release, from August 2023) and diskcache-oxide in the same session on the same machine, in four interleaved rounds. These numbers come from Linux (Ubuntu 26.04 LTS) in a container on an Apple Silicon Mac. Both libraries ran in the same container, so the ratios compare like with like, but the absolute numbers on an x86-64 server will differ. Latencies are p50 in microseconds. Ratios are oxide's time divided by diskcache's, so lower is better. | Workload | 3.14 | 3.14t (free-threaded) | |---|---|---| | `get`, small value, hit | **0.76 µs** (0.42×) | **0.75 µs** (0.46×) | | `get`, miss | **0.65 µs** (0.45×) | **0.62 µs** (0.46×) | | `get`, small dict | **1.18 µs** (0.47×) | **1.26 µs** (0.54×) | | `get`, random keys over a large database | **2.21 µs** (0.65×) | **2.41 µs** (0.79×) | | `set`, small value | **22.6 µs** (0.77×) | **24.0 µs** (0.82×) | Threaded throughput, every thread sharing one `Cache`, in operations per second at 1/2/4/8 threads (ratios are oxide's throughput over diskcache's, so higher is better): | Mix | 3.14 | 3.14t (free-threaded) | |---|---|---| | 99:1 reads to writes | **750k / 745k / 723k / 723k** (2.0-16.5×) | **701k / 639k / 680k / 729k** (1.7-1.8×) | | 1,000:1 reads to writes | **900k / 886k / 876k / 875k** (2.1-18.9×) | **903k / 680k / 1,181k / 1,416k** (1.25-2.2×) | Two things stand out: - **On GIL builds, diskcache collapses as threads are added and oxide doesn't.** At 99:1, diskcache drops from about 375k to 44k operations per second going from 1 to 8 threads; oxide stays flat around 720-750k. diskcache's per-thread connections end up fighting over the GIL and SQLite's locks. - **On free-threaded builds, oxide is faster at every thread count, but it doesn't scale yet.** 8 threads reach about 1.4M operations per second at 1,000:1, only 1.6× one thread. That's the next target, below. Run them yourself with `python -m bench.gate`. Numbers vary by machine; the ratios are what the gate enforces. ## Where the speed comes from - **Rust from the call to the disk.** A `get` goes from Python straight into compiled Rust: one prepared SQLite statement, decode, done. There's no Python wrapper in between. - **Statements prepared once.** Each pooled connection caches its prepared statements, so a lookup never recompiles SQL. - **The clock is read in Rust.** Checking expiry doesn't call back into Python's `time.time()`. - **Fast codecs.** msgpack and tagged JSON are encoded and decoded in Rust, building Python objects directly. - **Reads never write.** Least-recently-stored eviction means a `get` never updates anything, so any number of readers proceed in parallel. - **A smarter write lock.** Writers in one process queue on an in-process lock per database file, instead of every thread retrying against SQLite's busy handler. - **SQLite tuned for it.** The bundled SQLite is built with its global memory statistics turned off, which removes a process-wide mutex from every allocation: 2-10% more threaded throughput. ## Prefix scans at scale diskcache's `iterkeys()` gets slower faster than the cache grows: about 1.7 seconds at 100,000 keys, and over 25 seconds at 400,000. diskcache-oxide's `iterkeys(prefix=...)` is an indexed range scan, paged in batches so no read transaction is held between them. **400,000 keys scan in about 10 ms.** ## Memory that stays flat Memory is bounded by configuration, not by how many threads touch the cache: at most `max_connections` connections per database file (default 8), each with a 1 MiB page cache. A [`FanoutCache`](fanoutcache.qmd) defaults to two per shard for the whole root, about 16 MiB at 8 shards. A separate memory-instrumented build runs `bench/memory.py` and the memory tests under heavy load to hold that bound. See [Storage and WAL](storage-and-wal.qmd). ## Known slow spots Two oversubscribed cases measure slower than before the SQLite mutex spin and the connection-slot spin went in. Both are known, and neither is in the path of a web app running a few threads per process. - **A plain `Cache` with far more threads than cores**, all sharing one handle: 7% slower on an M-series Mac at 36 threads, and 5% in a Linux VM at 32. Faster hand-off lets more threads into SQLite at once. - **A `FanoutCache` at `max_connections=32` with 8 threads, on macOS only:** 5-12% slower. The same case on Linux is 41-56% faster. ## Next: free-threaded scaling Profiling a cached `get` shows that about 60% of its time is one system call: SQLite taking its shared-memory read lock on the `-shm` file, under a process-wide mutex. It's the biggest single cost on every read, and it's also what keeps free-threaded throughput from scaling. The next stage goes after it, along with: - a per-process lock-caching layer under SQLite, or longer-lived read snapshots; - thread-affine connections; - cheaper reference counting on shared `Cache` objects in free-threaded builds; - Making threads that share one handle scale. [`FanoutCache`](fanoutcache.qmd) is built and removes the multi-process write-lock ceiling, but at its default settings it doesn't lift this read ceiling, so the shared per-handle path is being worked on. Past that, the [network server](roadmap.qmd) can go further still: as the only owner of its files, it can use SQLite's exclusive locking mode, which keeps that lock in ordinary memory. Measured reads drop from 1.18 µs to 0.69 µs. ### Divergences from python-diskcache diskcache-oxide's test suite is a conformance suite: the same black-box tests run against real diskcache 5.6.3 and against diskcache-oxide. Where they behave differently, it's on purpose, and it's written down here. Running one suite against two implementations turned out to be a bug detector pointed in both directions: most of the differences below are diskcache bugs the suite found, each with a reproduction in the repository's `upstream/` folder. A divergence is never a way to skip a failing test. The full record, including test names and measurements, is [tests/DIVERGENCES.md](https://github.com/mikeckennedy/diskcache-oxide/blob/main/tests/DIVERGENCES.md). ## Summary | Ref | Topic | diskcache 5.6.3 | diskcache-oxide | |---|---|---|---| | D-001 | Values | Pickles everything | msgpack or JSON with type tags; pickle opt-in; never unpickles unless opted in | | D-002 | Database file | `cache.db` | `diskcache-oxide.sqlite` | | D-003 | Clock | `time.time()` | Native; `clock=` to control it | | D-004 | Keys | Anything picklable | `str` only, for now | | D-005 | Minimum `size_limit` | None; tiny limits silently empty the cache | 1 MiB, else `ValueError` | | D-006 | Empty-cache `volume()` | Its own value | A different value | | D-007 | `Timeout` | Subclasses `Exception` | Subclasses `TimeoutError` | | D-008 | Top-level NaN | Reads back as `None` | Round-trips | | D-009 | Truthiness | Empty cache is falsy | `Cache` always truthy, `NullCache` falsy | | D-010 | `expire()` | Can stop after 100 rows | Removes every expired entry | | D-011 | `clear()` | Also deletes writes made while it runs | Removes only what existed when it started | | D-012 | Fork during another thread's write | Child's writes stall for the timeout | Child can always write | | D-013 | Missing value file | Miss, but the key still reads as present | Miss, and the row is removed | | D-014 | `expire=True`, `timeout=True` | Treated as `1` | `TypeError` | | D-015 | New directory permissions | Follow the umask | `0700` | | D-016 | Reading a large value during an overwrite | Occasional spurious miss | Always the old or new value | | D-017 | Connections and fds | One connection per thread | A bounded pool | | D-018 | `close()` | Closes only the calling thread's connection | Closes every connection | | D-019 | Opening an existing cache | Writes to the database | Never writes unless settings change | | D-020 | Forking inside your own transaction | Child stalls for the timeout | Fails fast with `RuntimeError` | | D-021 | `directory` | Optional (temp dir) | Required | | D-022 | `disk=`, `eviction_policy=` | Supported | Not supported (policies coming) | | D-023 | SQLite memory | 32 MiB cache + 64 MiB mmap per thread | 1 MiB per pooled connection | | D-024 | WAL file size | Can grow past 1 GiB and stay there | Trimmed; `checkpoint()` on demand | | D-025 | A key iterator used by two threads at once | `ValueError` from the generator | `ValueError`, from a native iterator with no `send()` or `close()` | | D-026 | `pop` of a value it can't read | Row is deleted, then the decode fails | Decodes first, so the row stays unless no build could read it | | D-027 | A call behind a pending fork | Never waits: there's no fork barrier | Waits within its `timeout`, then raises `Timeout` | | D-028 | An outage on a single-key call | `Cache` raises; `FanoutCache` answers `Timeout` alone as a miss | Raises by default; `durable=True` answers it as a miss or `False`, and reports it | | D-029 | `with cache.transact() as tx:` | `tx` is `None`; a second handle's `transact()` inside waits out the timeout | `tx` is the cache; every handle on the directory joins the outer transaction | | D-030 | A transaction on a second, busy cache | Waits for the timeout; `retry=True` loops on any `OperationalError` | `TransactionBusy` at once; `retry=True` retries only `Timeout` | | D-031 | Ending a transaction on another thread | "May not be shared between threads" | Allowed, within its `timeout`; the owner's exit raises unless it committed | | D-032 | `FanoutCache` shard count | Not recorded; a different `shards=` quietly misroutes keys | Recorded in a manifest; a different `shards=` raises `LayoutMismatchError` | | D-033 | Which shard a key lives in | adler32; no `iterkeys()` | XXH3-64, frozen in the format; `shard_of(key)`; `iterkeys()` merges the shards in sorted order | | D-034 | A busy shard on a single-key call | Swallows `Timeout` (10 ms default); `get` swallows any `OperationalError` | Raises `Timeout`; `durable=True` answers it as a miss and reports it | | D-035 | `clear()`, `expire()`, `cull()` over the shards | Retries each shard forever | One `timeout` for the call; raises with `failed_shards` and `removed` | | D-036 | `size_limit` | Divided as a float, no minimum; reads shard 0's share | Split as integers, 1 MiB per shard minimum; reads back as the total | | D-037 | `max_connections` | One connection per thread per shard (264 fds at 64 threads) | A total across the shards; fds within `2 × max_connections + 2 × shards` | | D-038 | `Cache` and `FanoutCache` on each other's directory | Quietly creates or reuses a database | `LayoutMismatchError`, creating nothing | | D-039 | `FanoutCache` constructor and features | `FanoutCache(path, 4)`; sub-collections, tags, `stats()`, `check()` and more | `shards` is keyword-only; those features aren't there yet | | D-040 | `FanoutCache.transact()` | Locks every shard; `tx` is `None` | `transact(*, key)` locks one shard and refuses other shards' keys | | D-041 | Your own types | A `Disk` subclass, or pickle | `register_type()` per type; stored tagged, read by an allowlist; never pickled | | D-042 | Large values' folders | Two levels (65,536 folders); a folder goes with its last file | One level (at most 256); empty ones go at the last close and in `clear()` | | D-043 | A large write after the cache's folder was deleted | Re-creates the folders and succeeds | `OSError`; nothing is re-created | ## Safety ### D-001: no pickle by default diskcache pickles every value. diskcache-oxide stores `str` and `bytes` raw and everything else as msgpack or tagged JSON. A value the format can't represent exactly raises `TypeError` at `set()` unless the cache was opened with `pickle=True`. A cache opened without it never unpickles: reading a pickled value written by a `pickle=True` peer raises `UnsafeValueError` and leaves the row alone. Unpickling runs code from whatever is in the cache directory, and silently pickling hides values that were never meant to be cached. See [Serialization](serialization.qmd). ### D-015: owner-only directories A cache directory diskcache-oxide creates, and its value-file subdirectories, are `0700`. Existing directories are never changed. Anyone who can write to a cache directory can poison the cache, and its contents (keys with IP addresses in them, for example) can be private. ### D-021: `directory` is required diskcache creates a temporary directory if you don't give one. A cache directory is shared state and a security boundary, and an implicit temp directory is almost never what a multi-process app wants. ## Correctness ### D-008: NaN round-trips In diskcache, `set('k', float('nan'))` succeeds but `get('k')` returns `None`, ignoring any default. In diskcache-oxide it's a float like any other. ### D-010: `expire()` removes everything that has expired diskcache's `expire()` works in chunks of 100, and when expired entries share one expiry time it stops after the first chunk. With 250 such entries, one call removes 100. diskcache-oxide removes all of them, still in chunks so writers can get in between. ### D-011: `clear()` keeps concurrent writes diskcache's `clear()` keeps deleting until the cache is empty, including entries written while it runs. In a reproduction with 50,000 entries and a concurrent writer, it deleted 76,000 writes that had already succeeded, and its running time grew with the write rate. diskcache-oxide's `clear()` removes only entries that existed when it started, and always finishes. ### D-013 and D-016: large values under pressure If a large value's file disappears, diskcache returns the default but the key still reads as present in `in` and `len`. diskcache-oxide treats it as a miss and removes the orphaned row. When one process overwrites a large value while others read it, diskcache occasionally returns the default for a key that was never deleted, which makes callers recompute a hot value for nothing. diskcache-oxide's readers detect the overwrite and retry, so they always get a whole value, old or new. ### D-014: no booleans for numbers `bool` is an `int` subclass, so in diskcache `expire=True` is a one-second TTL and `expire=False` stores an already-expired entry. diskcache-oxide raises `TypeError`: a bool there is almost always an argument in the wrong position. ## Processes and forks ### D-012: forks while another thread writes When a process forks while another of its threads is in a write transaction, diskcache's child inherits a lock nobody in the child will release, and its first write stalls for the full timeout (60 seconds by default). Under a busy read/write load that hits 20-50% of forks. diskcache-oxide's fork barrier waits for in-flight transactions before the fork, so the child can always write. See [Concurrency and forks](concurrency-and-forks.qmd). ### D-020: forking inside your own transaction CPython won't let a library stop a fork, and a transaction held by the forking thread itself can't be waited for. In diskcache the child stalls behind it. In diskcache-oxide, the child's copy of the transaction is poisoned and its writes to that database raise `RuntimeError` immediately, and the parent gets a `RuntimeWarning`. ## Transactions ### D-029: `tx` is the cache, and every handle shares it diskcache's `with cache.transact() as tx:` binds `tx` to `None`, and a second `Cache` on the same directory has its own connection, so its `transact()` inside the first one's block waits for the write lock until it times out. In diskcache-oxide `tx` is the cache itself, and on the thread that entered the block every call on that directory runs in the transaction, through any handle: a `transact()` on a second handle joins the outer one instead of waiting. Other threads' calls never join it. See [Transactions](transactions.qmd). ### D-030: `TransactionBusy`, and what `retry=True` retries Inside one cache's transaction, diskcache's `transact()` on a second cache waits for its write lock until it times out, and `retry=True` loops on any `sqlite3.OperationalError`. When another thread holds that lock, diskcache-oxide raises `TransactionBusy`, a `Timeout` subclass, at once: that thread could be waiting for a lock this one holds, and no timeout setting fixes that. `retry=True` retries a plain `Timeout` and never `TransactionBusy`, and checks for `Ctrl-C` between attempts. ### D-031: a transaction can end on another thread diskcache says a transaction "may not be shared between threads". In diskcache-oxide any thread may commit or roll back the owner's transaction through its `__exit__`, within the transaction's `timeout`. A commit that can't run in time raises `Timeout` and rolls back, and the owner's own exit afterwards raises `TransactionAborted` (a `RuntimeError`) unless the transaction committed, so a `with` block never exits cleanly when its writes are gone. An owner's exit during another thread's commit waits for the outcome. ## Resources ### D-017 and D-018: bounded connections diskcache opens a connection for every thread that uses a cache, so 64 threads hold 131 files open in the cache directory, and `close()` closes only the calling thread's connection. diskcache-oxide uses a pool per database file, at most `max_connections` (default 8), and the last `close()` closes every connection, so the directory is safe to move or delete afterwards. ### D-019: opening never writes Opening an existing diskcache commits a write transaction even when nothing changed. diskcache-oxide's open, and reads after it, never write, so a read-mostly worker or a read-only copy of the directory never takes the write lock. ### D-023: bounded SQLite memory diskcache's default is a 32 MiB page cache and a 64 MiB memory map per connection, with one connection per thread. diskcache-oxide's `sqlite_cache_size` defaults to 256 pages (1 MiB) per pooled connection, so SQLite's memory per directory stays around 8 MiB per process. ### D-024: a WAL that shrinks With readers in other processes, a burst of writes makes SQLite's write-ahead log grow to fit the burst and stay that size: hundreds of MiB to over 1 GiB beside a database of a few MiB. diskcache-oxide trims the log whenever it resets (`wal_size_limit`), offers `checkpoint()` to empty it on demand, checkpoints a large log at `close()`, and can checkpoint automatically with `wal_checkpoint='auto'`, which tries a few times in a row so that a reader in flight in another process doesn't defeat it. See [Storage and WAL](storage-and-wal.qmd). ## API differences - **D-002:** the database file is `diskcache-oxide.sqlite`, so the two libraries never misread each other's files. - **D-003:** the clock is read natively; pass `clock=` to control it in tests. - **D-004:** keys are `str` only for now (other key types are on the [roadmap](roadmap.qmd)). They're stored with a type tag, so other key types won't need a format change. - **D-005:** `size_limit` must be at least 1 MiB. In diskcache, a limit below the empty database's size silently evicts every value, including the one just stored. - **D-006:** an empty cache's `volume()` differs, so a `size_limit` leaves slightly different room. - **D-007:** `Timeout` subclasses `TimeoutError`, so `except TimeoutError` catches it too. - **D-009:** `bool(Cache)` is always True and `bool(NullCache)` is False, so `if cache:` means "caching is on". - **D-022:** there's no `disk=` (a Python `Disk` class would put Python code on every read and write) and no `eviction_policy=` until the other policies arrive (see the [roadmap](roadmap.qmd)). - **D-041:** a type the codec doesn't know is registered per type with `register_type(cls, name=..., encode=..., decode=...)` (a dataclass needs only a name), process-wide or per cache with `types=`. It's stored as a tagged value and read back only where the name is registered, so nothing is imported by name; a registered type is never pickled; a `decode` error about the row is a `CorruptValueError` with yours as `__cause__`. See [Your own types](serialization.qmd#your-own-types). - **D-028:** `Cache(..., durable=True)` has no counterpart in diskcache. It answers an outage on a single-key call as a miss or `False`, and it can drop writes. Beginning a transaction, and anything inside one, still raises. See [A cache that can't take a request down](getting-started.qmd#a-cache-that-cant-take-a-request-down). - **D-042:** a large value's file sits one folder deep, in at most 256 folders. A delete removes only the file; the last close in a process removes folders left empty and untouched for 30 seconds, and `clear()` removes the ones it emptied. A `FanoutCache`'s shards don't sweep yet. Before this, empty folders were never removed and a long-lived cache could hold 65,536 of them, about 257 MiB on ext4. See [Storage and WAL](storage-and-wal.qmd). - **D-043:** if a cache's folder is deleted while a process has it open, that process's next large write raises `OSError` instead of re-creating the folder. Delete a cache's folder only with every process stopped. - **D-025:** `iter(cache)` and `iterkeys()` return a native iterator, not a generator. Two threads calling `next()` on the same one at once get `ValueError` at once, as with a shared generator. ### FanoutCache (D-032 to D-040) These are covered together in [FanoutCache: sharded caches](fanoutcache.qmd), and listed against diskcache's in [Switching from python-diskcache](switching-from-diskcache.qmd#fanoutcache). - **D-032:** the shard count is part of where every key lives, so it's recorded, and a different `shards=` is an error instead of a quiet loss of data and atomicity. - **D-033:** the hash is fixed by its definition and part of the format, and `iterkeys(prefix=)` gives `FanoutCache` the ordered listing `Cache` has. - **D-034:** a silently dropped write and a hang are both worse than an error the caller chose how to handle; `durable=True` is that choice. - **D-035:** a maintenance job must finish in bounded time, and when it can't, it must say what it did. - **D-036 and D-037:** budgets (size, connections, file descriptors) follow configuration, not the number of threads or shards. - **D-038:** a typo shouldn't leave a directory that neither kind of cache can open, or a shard written behind the `FanoutCache`'s back. - **D-039:** the missing stock features each need their own design across shards, so they arrive with those features on the [roadmap](roadmap.qmd) rather than as copies. - **D-040:** holding every shard's lock would serialize the whole cache behind one transaction, which is what sharding exists to avoid. ## diskcache-community [diskcache-community](https://github.com/secondwindoss/diskcache-community) is an actively maintained fork of diskcache, and the conformance suite can run against it too. It has fixed some of the same bugs (D-012, D-018, D-019, and the `0700` directories of D-015) and keeps others (D-008, D-016, D-017). The full comparison is at the end of [tests/DIVERGENCES.md](https://github.com/mikeckennedy/diskcache-oxide/blob/main/tests/DIVERGENCES.md). ## Recipes ### Caching expensive calls The cache-aside pattern, and a decorator that caches any function's results. The page takes 900 ms, and 850 of them are one query that returns the same ten rows it returned a second ago. Everyone has met this page. Caching is how you stop paying for the same answer twice: before doing something slow (a database query, an API call, rendering a big page), ask the cache. If it knows, you're done in a microsecond or two. If not, do the work and leave the answer for whoever asks next, which with a shared directory means every worker process on the machine, today and after the next restart. ## Cache-aside ```python import tempfile import time from diskcache_oxide import Cache cache = Cache(tempfile.mkdtemp()) def load_popular_episodes() -> list[dict]: time.sleep(0.2) # stands in for a slow database query return [{'id': 512, 'title': 'Rust for Python developers'}] def popular_episodes() -> list[dict]: episodes = cache.get('episodes:popular') if episodes is None: episodes = load_popular_episodes() cache.set('episodes:popular', episodes, expire=300) return episodes start = time.perf_counter() popular_episodes() print(f'first call: {(time.perf_counter() - start) * 1000:.1f} ms') start = time.perf_counter() for _ in range(1000): popular_episodes() print(f'cached: {(time.perf_counter() - start) * 1000:.1f} µs per call') # 1000 calls, so ms total = µs each ``` Always give cached values an `expire`. It bounds how stale a value can get if an invalidation is ever missed. ## When `None` is a real value If the function can legitimately return `None`, `cache.get(key)` can't tell "cached as `None`" from "not cached", and you'll recompute every time. Use a sentinel default: ```python MISSING = object() value = cache.get(key, default=MISSING) if value is MISSING: value = compute() cache.set(key, value, expire=300) ``` ## A caching decorator The same pattern wrapped up for reuse. The key is the function's full name plus its arguments, encoded as JSON so that different arguments always give different keys. ```python import functools import json from collections.abc import Callable from typing import Any from diskcache_oxide import CacheLike MISSING = object() def cached(cache: CacheLike, expire: float | None = None) -> Callable: """Cache a function's results, keyed by its arguments.""" def decorator(func: Callable) -> Callable: prefix = f'{func.__module__}.{func.__qualname__}:' @functools.wraps(func) def wrapper(*args: Any, **kwargs: Any) -> Any: key = prefix + json.dumps([args, kwargs], sort_keys=True, default=str) value = cache.get(key, default=MISSING) if value is MISSING: value = func(*args, **kwargs) cache.set(key, value, expire=expire) return value return wrapper return decorator calls = 0 @cached(cache, expire=600) def episode_details(episode_id: int, *, include_transcript: bool = False) -> dict: global calls calls += 1 return {'id': episode_id, 'transcript': include_transcript} episode_details(512) episode_details(512) episode_details(512, include_transcript=True) print(f'{calls} real calls for 3 lookups') ``` A few things to keep in mind: - **Arguments become part of the key**, so keep them simple: ids, strings, numbers. `default=str` covers values like dates, but two different objects with the same `str()` would share a key. - **Return values must be storable**, which means built-in types (dicts, lists, tuples, sets, datetimes and so on). A dataclass or ORM object raises `TypeError` at `set()`: return a dict instead, or use a cache opened with `pickle=True`. See [Serialization](../user-guide/serialization.qmd). - **Long keys are fine.** A key is just an indexed string. ## Thundering herds When a popular entry expires, every worker that asks for it in the next moment misses at the same time, and all of them recompute it. For most values that's harmless. If the computation is expensive enough that doing it eight times at once hurts, refresh it ahead of expiry from a background job (see [Running a job once across workers](run-once.qmd)), so readers never see it missing. A `memoize` with built-in stampede protection is planned (see the [roadmap](../user-guide/roadmap.qmd)). ### Serving stale values while refreshing Give an entry a freshness deadline shorter than its expiry, so readers never wait for a recompute and only one worker does it. The newsletter goes out at 9:00. At 9:00:01, four hundred readers land on the front page, and the cached copy expired at 8:59:58. Every worker misses at the same instant, every one of them starts rendering the same page, and the database spends its morning answering one question four hundred times. The [caching recipe](caching-expensive-calls.qmd#thundering-herds) calls this a thundering herd. For most values it's harmless. When the value is slow to build (a front page that takes 400 ms, a report that queries a warehouse), there are two ways to make sure readers never see the gap, and only one worker does the work. ## Stale-while-revalidate Store the value with its own freshness deadline inside, and give the entry a longer hard expiry. A reader that finds the entry fresh returns it. One that finds it stale returns it anyway, and the first of them to win a claim starts a refresh in the background. Readers never wait, and the herd is one worker. ```python import tempfile import threading import time from concurrent.futures import ThreadPoolExecutor from typing import Any from diskcache_oxide import Cache now = [1_000_000.0] # a fake clock; in a real app use time.time() cache = Cache(tempfile.mkdtemp(), clock=lambda: now[0]) refresher = ThreadPoolExecutor(max_workers=2) # one per process, for background refreshes loads: list[float] = [] def load_front_page() -> dict: time.sleep(0.05) # stands in for 400 ms of rendering loads.append(now[0]) return {'rendered_at': now[0]} def store(key: str, value: Any, fresh_for: float, grace: float) -> Any: cache.set(key, {'value': value, 'fresh_until': now[0] + fresh_for}, expire=fresh_for + grace) return value def refresh(key: str, fresh_for: float, grace: float) -> None: try: store(key, load_front_page(), fresh_for, grace) finally: cache.delete(f'refresh:{key}') def front_page(fresh_for: float = 60, grace: float = 300) -> dict: key = 'page:front' entry = cache.get(key) if entry is None: # a real miss: nothing to serve, compute inline return store(key, load_front_page(), fresh_for, grace) if now[0] >= entry['fresh_until'] and cache.add(f'refresh:{key}', 1, expire=30): refresher.submit(refresh, key, fresh_for, grace) # stale, and this worker won the refresh return entry['value'] # fresh or stale, the reader gets it now front_page() # the first request renders the page now[0] += 61 # a minute later it's stale results = [] start = threading.Barrier(8) def request() -> None: start.wait() results.append(front_page()['rendered_at']) threads = [threading.Thread(target=request) for _ in range(8)] for t in threads: t.start() for t in threads: t.join() refresher.shutdown(wait=True) # in the example only: let the refresh land print(f'{len(results)} readers got the stale page from t={results[0]:.0f}, {len(loads)} renders in total') print(f"the next reader gets the fresh page from t={front_page()['rendered_at']:.0f}") ``` The three numbers to choose: - **`fresh_for`** is how old a value may get before a refresh starts. It plays the role `expire` plays in a plain cache. - **`grace`** is how much longer the stale value stays available. If no refresh lands within it (the refresher crashed, the source was down), the entry expires and the next reader computes inline, herd and all. Make it generous: minutes, not seconds. - **The claim's `expire`** (30 s here) bounds how long a crashed refresher blocks the next one. Make it longer than a refresh ever takes. In an asyncio app, start the refresh with `asyncio.create_task(asyncio.to_thread(refresh, ...))` instead of a thread pool; see [Using the cache from asyncio](asyncio.qmd). With `durable=True`, a failed `add()` returns `False`, so nobody refreshes until the cache is healthy again, which is the right outcome. ## Refreshing ahead from a job When the hot keys are known in advance (the front page, the top ten episodes, the pricing table), it's simpler to never let them go stale: a scheduled job, [claimed by one worker](run-once.qmd), recomputes them every few minutes and stores each with an `expire` of at least twice the interval. ```python HOT_PAGES = ['front', 'episodes', 'pricing'] def refresh_hot_pages() -> None: # runs every 5 minutes, on one worker if not coordination.add('job:refresh-hot-pages', os.getpid(), expire=4 * 60): return for name in HOT_PAGES: cache.set(f'page:{name}', render_page(name), expire=15 * 60) ``` Readers do a plain `get()` and never miss; a missed run or two costs nothing because the entries outlive several intervals. The cost is that the job renders pages nobody may be asking for, which is fine for a handful of keys and wrong for a long tail. Stale-while-revalidate handles the tail, since it only refreshes what's being read. A `memoize` decorator with this built in is planned (see the [roadmap](../user-guide/roadmap.qmd)). ### Caching responses from other services Cache what an upstream API returns, honor its max-age, revalidate with ETags, and keep serving the last good answer when it's down. Your dashboard endpoint takes 1.2 seconds, and 1.1 of them are spent waiting for a partner API to return the same JSON it returned to the previous request. An app that calls other services (a payments API, a headless CMS, your own microservices) spends most of its latency waiting on them, and most of those answers don't change between calls. Caching them in a directory every worker shares means one request to the upstream per machine instead of one per worker, and a last known good answer to serve when the upstream is slow, or down, or has started returning the wrong thing with a 500. ## A cached fetch The entry holds the response body, its `ETag` and a freshness deadline taken from the upstream's `Cache-Control: max-age`, with a default when there isn't one. A fresh entry is returned without a request. A stale one is revalidated with `If-None-Match`, so an unchanged resource costs a `304` with no body. And if the request fails while there's a stale entry, the stale entry is the answer. ```python import json import re import tempfile import httpx from diskcache_oxide import Cache now = [1_000_000.0] # a fake clock; in a real app use time.time() cache = Cache(tempfile.mkdtemp(), clock=lambda: now[0]) def max_age(response: httpx.Response) -> float | None: m = re.search(r'max-age=(\d+)', response.headers.get('cache-control', '')) return float(m.group(1)) if m else None def fetch_json(client: httpx.Client, url: str, *, default_ttl: float = 300, keep: float = 24 * 3600) -> object: key = f'http:GET:{url}' entry = cache.get(key) if entry is not None and now[0] < entry['fresh_until']: return json.loads(entry['body']) # fresh: no request at all headers = {'If-None-Match': entry['etag']} if entry and entry['etag'] else {} try: response = client.get(url, headers=headers) if response.status_code != 304: response.raise_for_status() except httpx.HTTPError: if entry is not None: return json.loads(entry['body']) # upstream trouble: the last good answer raise if response.status_code == 304: # unchanged: keep the body, extend its freshness entry['fresh_until'] = now[0] + (max_age(response) or default_ttl) else: entry = { 'body': response.content, 'etag': response.headers.get('etag'), 'fresh_until': now[0] + (max_age(response) or default_ttl), } cache.set(key, entry, expire=keep) return json.loads(entry['body']) ``` Here it runs against a mock upstream that counts its requests: ```python seen: list[str | None] = [] def upstream(request: httpx.Request) -> httpx.Response: seen.append(request.headers.get('If-None-Match')) headers = {'ETag': '"v1"', 'Cache-Control': 'max-age=60'} if request.headers.get('If-None-Match') == '"v1"': return httpx.Response(304, headers=headers) return httpx.Response(200, json={'episodes': [512, 513]}, headers=headers) client = httpx.Client(transport=httpx.MockTransport(upstream)) url = 'https://api.example.com/episodes' fetch_json(client, url) fetch_json(client, url) # fresh: served from the cache now[0] += 61 fetch_json(client, url) # stale: one conditional request, answered 304 print(f'{len(seen)} upstream requests; If-None-Match headers sent: {seen}') ``` Why it's shaped this way: - **Freshness lives in the value, not in `expire`.** The entry's `expire` (`keep`) is how long a stale copy is worth holding on to for revalidation and outages. If freshness were the expiry, a stale entry would vanish just when it's useful. - **`304` costs one small write.** The body is kept and only `fresh_until` changes. A `set()` is needed because the deadline is inside the value; `touch()` only moves the entry's expiry. - **The key names the request.** Method and URL here. If the response depends on who's asking (an `Authorization` header, a tenant), put that identity in the key too, or one user's data will be served to another. ## Variations - **Serving the bytes straight through.** When your endpoint relays the upstream's body, store `response.content` and the `content-type`, and return them as they are, without parsing. Bodies of 64 KiB or more are stored in their own file, so a large payload costs no more than a small one. - **Negative caching.** A `404` that costs a round trip every time is worth caching for a short while: store `{'status': 404, 'fresh_until': ...}` and return your own not-found from it. Don't cache `5xx` as answers; the stale-on-error path above already covers an upstream outage. - **Headers not to keep.** Drop `Set-Cookie` and anything else tied to the one connection before storing a response's headers. - **Only one worker should refetch.** When many requests find the same stale entry at once, they all revalidate. `304`s are cheap, so that's usually fine. If the upstream charges per call, wrap the refetch in a claim as the [stale-while-revalidate](stale-while-revalidate.qmd) recipe does. ### Invalidating groups of keys Clear everything cached for one user or one object, by key prefix or by generation number. A user changes their display name, and for the next five minutes half the site still uses the old one: the profile page, the sidebar widget, three API responses, each cached on its own schedule and each wrong in its own way. A single change often makes many cache entries stale, and deleting them one by one means knowing every key that was ever derived from that user. There are two good ways to handle this. Both start with a naming convention. ## Name keys hierarchically Put the thing a key depends on at the front of the key, from the general to the specific: ``` user:42:profile user:42:sidebar user:42:api:/v1/me episode:512:page episode:512:transcript ``` Everything about user 42 now shares the prefix `user:42:`. Note the trailing colon: without it, `user:4` would also match `user:42`. ## Option 1: delete by prefix `iterkeys(prefix=...)` is an indexed range scan, so finding a user's keys costs time in proportion to how many there are, not the size of the cache. It fetches keys a page at a time with no lock held in between, so deleting while iterating is safe. ```python import tempfile from diskcache_oxide import Cache cache = Cache(tempfile.mkdtemp()) for key in ['user:42:profile', 'user:42:sidebar', 'user:420:profile', 'episode:512:page']: cache.set(key, '...', expire=3600) def invalidate_prefix(prefix: str) -> int: removed = 0 for key in cache.iterkeys(prefix=prefix): removed += cache.delete(key) return removed print(invalidate_prefix('user:42:'), 'removed') print(sorted(cache)) ``` This is simple and frees the space right away. It costs one small write per key, so it's best when a group has tens or hundreds of keys, not hundreds of thousands. ## Option 2: generation numbers Instead of deleting anything, keep a generation number per group and put it in every key. To invalidate the group, change the generation. Every old key becomes unreachable at once, in a single write, however many keys there were. The old entries are never read again, and they leave the cache when they expire or are evicted. ```python import time def generation(user_id: int) -> int: key = f'gen:user:{user_id}' gen = cache.get(key) if gen is None: cache.add(key, time.time_ns()) # if several workers race, one value wins gen = cache.get(key) return gen def user_key(user_id: int, name: str) -> str: return f'user:{user_id}:g{generation(user_id)}:{name}' def invalidate_user(user_id: int) -> None: cache.set(f'gen:user:{user_id}', time.time_ns()) cache.set(user_key(7, 'profile'), {'name': 'Sam'}, expire=3600) print(cache.get(user_key(7, 'profile'))) invalidate_user(7) print(cache.get(user_key(7, 'profile'))) ``` Generation numbers cost one extra read per lookup (the generation, which is tiny and hot) and make invalidation free. Give the real entries an `expire`, since nothing deletes them. The generations are timestamps rather than a counter starting at 0 for a reason. A generation key is written rarely, and the cache evicts the least recently *stored* entries first, so under size pressure it can be evicted. A counter would then restart at 0 and bring old entries back to life. A new timestamp is always larger than any generation used before, so a lost generation only costs a few cache misses. ## Which to use - **Prefix deletes** when groups are small and you want the space back immediately. - **Generation numbers** when groups are large, or invalidation is frequent, or many workers might invalidate the same group at the same time. ## While caching is switched off If you turn caching off with `NullCache`, pass the real cache as `invalidate=`. Prefix deletes still reach the real cache (`iterkeys` and `delete` are both forwarded), so nothing stale is waiting when caching comes back on. See [NullCache](../user-guide/null-cache.qmd). Tags, which group keys without a naming convention, are planned (see the [roadmap](../user-guide/roadmap.qmd)). ### Storing your own types UUIDs, enums, timedeltas, Paths, Pydantic models and dataclasses in the cache with register_type(), and no pickle. The first `TypeError` from `set()` is a small shock. You've been storing dicts for weeks, and today one of them has a `UUID` in it and the cache says no. That refusal is the design: the cache stores the built-in types exactly ([Serialization](../user-guide/serialization.qmd#exact-types) has the table: `str`, numbers, `bytes`, `bool`, `None`, lists, tuples, sets, dicts, `datetime`, `date`, `Decimal`) and refuses anything else rather than pickling it behind your back. For everything else, you tell the codec once how to store the type, and the shock never comes back. This recipe collects the registrations a typical web app needs. ## The standard library's types `uuid.UUID`, `datetime.timedelta` and `pathlib.Path` aren't built-in types to the codec. Each needs an `encode` that turns an instance into something storable and a `decode` that turns it back: ```python import datetime import pathlib import tempfile import uuid import diskcache_oxide from diskcache_oxide import Cache diskcache_oxide.register_type(uuid.UUID, name='std.uuid', encode=lambda u: u.bytes, decode=lambda b: uuid.UUID(bytes=b)) diskcache_oxide.register_type(datetime.timedelta, name='std.timedelta', encode=datetime.timedelta.total_seconds, decode=lambda s: datetime.timedelta(seconds=s)) diskcache_oxide.register_type(type(pathlib.Path()), name='std.path', encode=str, decode=pathlib.Path) cache = Cache(tempfile.mkdtemp()) cache.set('upload', {'id': uuid.uuid4(), 'ttl': datetime.timedelta(hours=2), 'file': pathlib.Path('/srv/uploads/a.png')}) print({k: type(v).__name__ for k, v in cache.get('upload').items()}) ``` The `Path` registration says `type(pathlib.Path())`, not `pathlib.Path`, for a reason: lookup is by exact type, and `pathlib.Path()` builds a `PosixPath` or a `WindowsPath`. Registering the base class would match nothing. The same applies to any factory that returns a subclass. ## Enums Store the value, and let the enum class turn it back into a member. A `StrEnum` or `IntEnum` is a subclass of `str` or `int`, and a subclass is never stored as its base, so it needs this too: ```python import enum class Plan(enum.StrEnum): FREE = 'free' PRO = 'pro' diskcache_oxide.register_type(Plan, name='app.Plan', encode=lambda p: p.value, decode=Plan) cache.set('account:42:plan', Plan.PRO) print(repr(cache.get('account:42:plan'))) ``` ## Dataclasses A dataclass needs only a name. Its fields are stored and it's rebuilt with `cls(**fields)`, so defaults and `__post_init__` run. Fields can hold other registered types, nested as deep as you like: ```python import dataclasses @dataclasses.dataclass class Invoice: id: uuid.UUID plan: Plan issued: datetime.date paid_within: datetime.timedelta | None = None diskcache_oxide.register_type(Invoice, name='app.Invoice') cache.set('invoice:latest', Invoice(uuid.UUID('2b7e1c4a-5d3f-4e8b-9a6c-0f1d2e3a4b5c'), Plan.PRO, datetime.date(2026, 11, 8))) print(cache.get('invoice:latest')) ``` ## Pydantic models Pydantic can produce and consume a plain dict, so a model registers with its own two methods. `mode='json'` makes `model_dump()` emit only JSON-compatible values (UUIDs and dates as strings), which `model_validate()` parses back, so the model's field types need no registrations of their own: ```python from pydantic import BaseModel class User(BaseModel): id: uuid.UUID name: str plan: Plan diskcache_oxide.register_type(User, name='app.User', encode=lambda u: u.model_dump(mode='json'), decode=User.model_validate) ``` attrs classes and msgspec structs follow the same pattern with `attrs.asdict()` and `msgspec.to_builtins()`, and the class as the decoder. ## Where the registrations live Registrations are process-wide and must exist in every process that opens the directory: the web workers, the Celery workers, the management command that warms the cache. A row written with a tag the reading process doesn't know is a `ForeignValueError`, and the row is left for a process that does. Put them in one module and import it early: ```python # myapp/cache_types.py: imported from myapp/__init__.py, so every entry point gets it import diskcache_oxide from myapp.models import Invoice, Plan, User diskcache_oxide.register_type(Plan, name='app.Plan', encode=lambda p: p.value, decode=Plan) diskcache_oxide.register_type(Invoice, name='app.Invoice') diskcache_oxide.register_type(User, name='app.User', encode=lambda u: u.model_dump(mode='json'), decode=User.model_validate) ``` ## When the shape changes The name is the contract. Renaming or moving the class changes nothing, because rows carry the name you chose. Changing the fields does: a dataclass row written before a required field was added raises `CorruptValueError` when it's read, and `on_decode_error='miss'` turns that into a miss and deletes the row, so the next writer replaces it. For a change the old rows can't survive, give the type a new name (`app.Invoice.v2`). The old rows become `ForeignValueError`, treated as misses under the same policy, and expire on their own. See [Errors](../user-guide/errors.qmd#corruptvalueerror-and-foreignvalueerror-a-value-that-cant-be-decoded). ### Running a job once across workers Use add() to make sure only one worker process runs a scheduled job. At the top of the hour, eight gunicorn workers wake up at the same instant with the same idea: time for the hourly cleanup. Each one has its own copy of the scheduler, none of them knows the others exist, and the only thing they share is a cache directory. You want the job to run once, and it turns out the directory is enough. `add()` stores a value only if the key is missing or expired, and it's atomic across every thread and process using the cache directory. When several callers race to add the same key, exactly one gets `True`. That makes the key a claim on the job. ## Claim the run ```python import os from diskcache_oxide import Cache cache = Cache('/var/cache/my-app') def hourly_cleanup() -> None: if not cache.add('job:hourly-cleanup', os.getpid(), expire=55 * 60): return # another worker has it this hour remove_stale_sessions() ``` The `expire` is the important part. It decides when the job can be claimed again, and it's also what saves you if the worker that claimed the job crashes halfway through: the claim expires and the next run goes ahead. Set it a little shorter than the schedule's interval, so the next run is never blocked by the previous claim. Storing `os.getpid()` (or a hostname and pid) as the value costs nothing and makes it easy to see who claimed the job: `cache.get('job:hourly-cleanup')`. ## See it work Here eight threads race for the same claim. Processes behave the same way. ```python import tempfile import threading from diskcache_oxide import Cache cache = Cache(tempfile.mkdtemp()) winners = [] start = threading.Barrier(8) def worker(n: int) -> None: start.wait() if cache.add('job:hourly-cleanup', n, expire=3300): winners.append(n) threads = [threading.Thread(target=worker, args=(n,)) for n in range(8)] for t in threads: t.start() for t in threads: t.join() print(f'{len(winners)} winner: worker {winners[0]}') print('claim holds:', cache.get('job:hourly-cleanup')) ``` ## Jobs that should run once, then again as soon as they're done For work that isn't on a fixed schedule (process the queue, rebuild the search index), use `expire` as a lease: a time limit after which a crashed worker's claim lapses. Delete the key when the work finishes, so the next request can claim it right away. ```python def rebuild_search_index() -> None: if not cache.add('job:search-index', os.getpid(), expire=10 * 60): return try: do_rebuild() finally: cache.delete('job:search-index') ``` Make the lease comfortably longer than the job ever takes. If the job outlives it, a second worker can start a second copy. ## Give claims their own small cache Two things can make a claim vanish early, and both argue for keeping claims out of your main cache: - **Switching caching off.** A `NullCache` follows the rule "every entry expires the instant it's written", so its `add()` always returns `True`. With caching off, every worker's claim succeeds and the job runs once per worker. That's harmless for idempotent jobs and wrong for jobs with side effects, like sending email. - **Eviction.** When a cache is over its `size_limit`, it evicts the least recently stored entries, and a claim is exactly the kind of small, rarely written entry that goes first. If it's evicted before it expires, a second worker can claim the job. A separate cache directory just for coordination, which is never swapped for a `NullCache` and holds only a handful of tiny keys, avoids both: ```python coordination = Cache('/var/cache/my-app-coordination') ``` Proper cross-process `Lock` and `BoundedSemaphore` recipes are planned (see the [roadmap](../user-guide/roadmap.qmd)). ### Rate limiting Fixed-window request limits shared by every worker on the machine, with add() and incr(). Someone has pointed a script at your search endpoint. It's a polite script: it stays under your limit of 100 requests a minute. Per worker. With eight workers and an in-memory counter in each, the limit you meant is 100 and the limit you have is 800. To limit each client for real, every worker process has to see the same count, and Redis is the usual fix. A diskcache-oxide counter is shared by every worker on the machine, and `incr()` is atomic across processes, so concurrent requests never lose a count. ## A fixed-window limiter Each client gets a counter for the current window, with the window's number in the key. When the window ends, requests move on to a new key and the old one simply expires. ```python import tempfile from diskcache_oxide import Cache now = [1_000_000.0] # a fake clock, so this example can move time forward cache = Cache(tempfile.mkdtemp(), clock=lambda: now[0]) def allow_request(client: str, limit: int = 5, window: int = 60) -> bool: current = int(now[0] // window) # in a real app: int(time.time() // window) key = f'rate:{client}:{current}' cache.add(key, 0, expire=window * 2) # start the window's counter, once return cache.incr(key) <= limit print([allow_request('203.0.113.9') for _ in range(7)]) now[0] += 60 # the next window print(allow_request('203.0.113.9')) ``` Why it's built this way: - **`add()` then `incr()`.** `add()` creates the counter with an expiry the first time and does nothing after that. `incr()` keeps a key's existing expiry, so the counter expires on schedule however many times it's incremented. Calling `incr()` alone would create a counter that never expires. - **The window number is in the key.** A new window means a new key, so no reset is needed, and there's no race where one request resets the counter while another increments it. - **The expiry is two windows long,** so a counter can't expire while its window is still current. Expired counters are cleaned up by `expire()` or eviction like any other entry. In a real app, use `time.time()` and leave out `clock=`; the fake clock is only here so the example can jump to the next minute. ## Using it in a view ```python from fastapi import FastAPI, HTTPException, Request app = FastAPI() @app.get('/api/search') def search(request: Request, q: str) -> dict: if not allow_request(request.client.host, limit=100, window=60): raise HTTPException(status_code=429, detail='Too many requests') return run_search(q) ``` ## Things to know - **Fixed windows allow short bursts at the boundary.** A client can make 100 requests at the end of one minute and 100 more at the start of the next. For most APIs that's fine. If it isn't, check both the current and the previous window and weight the previous one by how much of it overlaps the last 60 seconds (a sliding-window estimate). - **It's per machine.** Every worker on one machine shares the count, but two machines behind a load balancer each have their own. The planned network server will share one cache across machines. - **Fail open on a busy cache.** If you set a short `timeout` on the cache so that requests never wait long, catch `Timeout` and allow the request rather than failing it: ```python from diskcache_oxide import Timeout try: allowed = allow_request(client) except Timeout: allowed = True ``` - **Don't rate limit with a cache that might be a `NullCache`.** Its `incr()` always returns `default + delta`, so every request would be allowed. That's often what you want with caching switched off, but make it a deliberate choice. A ready-made `throttle` recipe is planned (see the [roadmap](../user-guide/roadmap.qmd)). ### Counting things View counts, download totals and usage quotas with incr() and decr(), shared by every worker and flushed to the database without losing a count. A download counter is the simplest feature on the site, and the first one to start lying once there's more than one worker: each process counts what it saw, and the number on the page depends on which worker rendered it. Moving the count to the database fixes the lie and makes every download a write to your busiest table. A cache counter sits in between: `incr()` is atomic across every thread and process on the machine, costs tens of microseconds, and the totals can be moved to the database on a schedule, by one worker, without losing a single count on the way. ## Count from every worker ```python import tempfile import threading from diskcache_oxide import Cache cache = Cache(tempfile.mkdtemp()) def record_view(episode_id: int) -> None: cache.incr(f'views:episode:{episode_id}') def viewer() -> None: for _ in range(250): record_view(512) threads = [threading.Thread(target=viewer) for _ in range(8)] for t in threads: t.start() for t in threads: t.join() print(cache.get('views:episode:512'), 'views, none lost') ``` `incr()` starts a missing counter at `default` (0) and adds `delta` (1), so there's no setup step. Processes behave exactly like the threads here. ## Flush the counts to the database Totals belong in the database eventually: for reports, and so they survive a cleared cache. `pop()` reads a counter and removes it in one atomic step, so a view recorded while the flush runs lands in a fresh counter and is picked up next time. Nothing is counted twice or lost. ```python database: dict[int, int] = {} # stands in for UPDATE episodes SET views = views + ? WHERE id = ? def flush_views() -> None: for key in cache.iterkeys(prefix='views:episode:'): pending = cache.pop(key, 0) if pending: episode_id = int(key.rsplit(':', 1)[1]) database[episode_id] = database.get(episode_id, 0) + pending record_view(513) flush_views() print('database:', database, '| pending in cache:', cache.get('views:episode:512', 0)) ``` Run the flush from one place, a scheduled job that [one worker claims](run-once.qmd), every minute or so. To show a live total, add the pending count to the stored one: `database[id] + cache.get(key, 0)`. ## Counters that clean themselves up For per-day or per-month figures, put the period in the key and give each counter an expiry. Create it with `add()` first: `incr()` keeps whatever expiry the key already has, and on a key it has to create, it sets none. ```python import datetime def record_download(file_id: str, day: datetime.date) -> int: key = f'downloads:{file_id}:{day.isoformat()}' cache.add(key, 0, expire=40 * 24 * 3600) # keep 40 days of history return cache.incr(key) today = datetime.date(2026, 11, 8) record_download('talk-python-491.mp3', today) print(record_download('talk-python-491.mp3', today), 'downloads today') ``` To report a week, read the seven keys. Old days expire on their own, so the cache never fills with history. ## Quotas with `decr()` A quota is a counter that starts at the allowance and counts down: ```python def consume_quota(user_id: int, units: int = 1, allowance: int = 1000) -> bool: key = f'quota:{user_id}:2026-11' # the billing period, in a real app from the date cache.add(key, allowance, expire=40 * 24 * 3600) # the first call this period sets the allowance return cache.decr(key, units) >= 0 print(consume_quota(42, units=999), consume_quota(42), consume_quota(42)) ``` Once the counter is below zero every later call is denied too, until the period's key is replaced. To let an admin grant more, `incr()` the key by the extra amount. `delta` can be a float, for fractional units. ## Several counters at once Counting a request under several keys (total, per path, per status) is three `incr()` calls, and a reader can see the first land before the third. If that matters, put them in one [transaction](../user-guide/transactions.qmd): the block commits all of them together. ## Things to know - **A missing counter reads as `default`.** When a missing counter is a bug rather than a zero, pass `default=None` and `incr()` raises `KeyError` instead. - **Counters need a real cache.** A `NullCache` has nothing to count in: its `incr()` returns `default + delta` every time, so every view would be the first. Keep counters in a cache that's never switched off, which [`Caches`](../user-guide/caches.qmd) lets you do per name. - **`durable=True` drops counts silently.** A failed `incr()` returns `default + delta` so a rate limiter fails open, but for a quota or a view count that's a lost increment with only a log line to show for it. Let these raise, and catch `Timeout` where you call them. - **Not an id generator.** Two callers can get the same number back under `durable=True`, and a cleared cache restarts at zero. Ids come from the database. - **One machine.** Every worker on a host shares the counts; two hosts behind a load balancer each have their own, so flush each host's counts to the database and sum there. ### Single-use keys One-time codes, reset links and idempotency keys: pop() hands a value to exactly one caller, and add() accepts a key exactly once. At 2 a.m. a tired user double-clicks a password reset link, and two requests arrive on two workers a few milliseconds apart. Both find the token. Both accept it. Some things must happen exactly once however many times they're asked for: a reset link is good for one click, a verification code for one entry, a payment request for one charge even when the client retries. Two cache calls are atomic across every worker on the machine, `pop()` and `add()`, and between them they cover all of these. ## One-time tokens with `pop()` Store the token when you issue it, with the lifetime the link should have. To redeem it, `pop()` it: the value comes back and the entry is gone in one step, so two requests presenting the same token can't both succeed, whichever workers they land on. ```python import secrets import tempfile import threading from diskcache_oxide import Cache tokens = Cache(tempfile.mkdtemp()) def issue_reset_token(user_id: int) -> str: token = secrets.token_urlsafe(32) tokens.set(f'reset:{token}', user_id, expire=15 * 60) return token def redeem_reset_token(token: str) -> int | None: """The user id the token was issued to, or None if it's unknown, expired or already used.""" return tokens.pop(f'reset:{token}') token = issue_reset_token(42) # Eight requests present the same link at the same moment. results = [] start = threading.Barrier(8) def click() -> None: start.wait() results.append(redeem_reset_token(token)) threads = [threading.Thread(target=click) for _ in range(8)] for t in threads: t.start() for t in threads: t.join() print(f'{results.count(42)} redeemed, {results.count(None)} rejected') ``` A few details that matter: - **The token is the key, and it's random.** `secrets.token_urlsafe(32)` can't be guessed, so a key that exists is proof the link was issued. Never build the key from the user id alone. - **Unknown, expired and used all look the same.** `pop()` returns `None` for each, which is what the user should see too: one message, with no hint about which it was. - **Issuing a new link should cancel the old one.** Keep a pointer from the user to their current token, `reset-for:{user_id}`, and `pop()` it (then delete the token it names) before issuing a new one. ### Short codes need an attempt limit A six-digit verification code is guessable, so pair it with a counter, created with an expiry the way the [rate limiting](rate-limiting.qmd) recipe does, and burn the code after a few wrong guesses: ```python def check_code(user_id: int, entered: str) -> bool: attempts_key = f'verify-attempts:{user_id}' tokens.add(attempts_key, 0, expire=10 * 60) if tokens.incr(attempts_key) > 5: tokens.delete(f'verify-code:{user_id}') # too many tries: the code is dead return False if tokens.get(f'verify-code:{user_id}') != entered: return False tokens.delete(f'verify-code:{user_id}') # right, and good for one entry only return True ``` ## Idempotency keys with `add()` A client that times out on a `POST` retries it, and the second attempt must not charge the card twice. The client sends the same idempotency key with both, and `add()` makes the first one to arrive the only one that does the work: ```python def charge(idempotency_key: str, amount_cents: int) -> dict: key = f'idem:{idempotency_key}' if not requests_seen.add(key, {'status': 'in-progress'}, expire=24 * 3600): previous = requests_seen.get(key) if previous is None or previous['status'] == 'in-progress': raise Conflict('this request is still being processed; retry in a moment') return previous['result'] # the retry gets the first attempt's answer try: result = payments.charge(amount_cents) except Exception: requests_seen.delete(key) # let a retry try again raise requests_seen.set(key, {'status': 'done', 'result': result}, expire=24 * 3600) return result ``` The sequence: `add()` claims the key, and exactly one caller gets `True`. A retry that arrives while the first attempt is still running sees `in-progress` and is told to come back; one that arrives after it finished gets the stored result, which should carry everything needed to send the same response again (the status code too). If the work fails, the claim is deleted so that a retry really retries. `previous is None` covers the rare case where the entry expired, or was evicted, between the `add()` and the `get()`. The stored result has to be something the cache can hold: a dict of built-in types, or a type you've [registered](custom-types.qmd). ## Keep them out of your main cache Tokens and idempotency keys aren't a copy of something you can recompute, so two things that are fine for cached values are wrong for them: - **Eviction.** A cache at its `size_limit` evicts the least recently stored entries, and an unused token is a tiny, old entry, exactly what goes first. An evicted reset link is a dead link; an evicted idempotency key is a double charge. - **Switching caching off.** A `NullCache` never stores, so `pop()` returns `None` and no token can ever be redeemed, and its `add()` always returns `True`, so every retry charges again. Give them a directory of their own, with a `size_limit` they'll never reach and no `NullCache` in front of it, as the [job claims](run-once.qmd#give-claims-their-own-small-cache) recipe does. [`Caches`](../user-guide/caches.qmd) can register it under its own name so the rest of the app's caches can still be switched off. `durable=True` is safe here, if you check return values: a `pop()` whose delete fails returns `None` rather than risk delivering the token twice (the user clicks again), `add()` fails closed, and a `set()` that was dropped returns `False`, so `issue_reset_token` should raise rather than hand out a link that was never stored. See [errors](../user-guide/errors.qmd). ### Server-side sessions Sessions every worker can read, with a sliding idle timeout from touch(), an absolute lifetime, and log-out-everywhere by key prefix. Every request arrives with a cookie and a question: who is this? A session is the one piece of per-user state almost every web app keeps server side to answer it: who's logged in, since when, with which roles. Keeping it in a cache directory means every worker sees the same session the instant it's written, a login on one worker is a login on all of them, and there's no database row per request and no Redis to run. The cookie holds only a random id. ## Start, load, end ```python import secrets import tempfile from diskcache_oxide import Cache now = [1_000_000.0] # a fake clock; in a real app use time.time() sessions = Cache(tempfile.mkdtemp(), clock=lambda: now[0]) IDLE = 30 * 60 # a session ends after 30 minutes without a request ABSOLUTE = 12 * 3600 # and 12 hours after login, however active def start_session(user_id: int, roles: list[str]) -> str: sid = secrets.token_urlsafe(32) sessions.set(f'session:{sid}', {'user_id': user_id, 'roles': roles, 'started': now[0]}, expire=IDLE) sessions.set(f'user-sessions:{user_id}:{sid}', True, expire=ABSOLUTE) return sid def load_session(sid: str) -> dict | None: data, expires_at = sessions.get(f'session:{sid}', expire_time=True) if data is None or now[0] - data['started'] > ABSOLUTE: return None if expires_at - now[0] < IDLE / 2: # slide the idle timeout, but not on every request sessions.touch(f'session:{sid}', expire=IDLE) return data def end_session(sid: str) -> None: data = sessions.pop(f'session:{sid}') if data is not None: sessions.delete(f'user-sessions:{data["user_id"]}:{sid}') sid = start_session(42, ['member']) now[0] += 20 * 60 print('after 20 min:', load_session(sid) is not None) # active, and the timeout slides now[0] += 20 * 60 print('after 40 min:', load_session(sid) is not None) # still active: the last request was 20 min ago now[0] += 31 * 60 print('after 71 min:', load_session(sid) is not None) # idle too long ``` - **`touch()` slides the idle timeout** without rewriting the session. It's a small write, so it's done only when less than half the idle time is left, which `get(expire_time=True)` tells you in the same read as the data. A busy user costs one write every 15 minutes instead of one per request. - **The absolute lifetime is inside the value.** `expire` can only say when the session dies if it's left alone; the login time says when it dies regardless. - **The id is the secret.** `secrets.token_urlsafe(32)` in an `HttpOnly`, `Secure`, `SameSite` cookie, and nothing else in the cookie. The session dict holds ids and roles, not the whole user object, which belongs in the database. ## Log out everywhere A password change or a stolen laptop should end every session a user has. The second key written at login, `user-sessions:{user_id}:{sid}`, is an index for exactly this: a prefix scan finds them all. ```python other = start_session(42, ['member']) sid = start_session(42, ['member']) def end_all_sessions(user_id: int) -> int: ended = 0 for key in sessions.iterkeys(prefix=f'user-sessions:{user_id}:'): sid = key.rsplit(':', 1)[1] ended += sessions.delete(f'session:{sid}') sessions.delete(key) return ended print(end_all_sessions(42), 'sessions ended;', 'still valid:', load_session(other) is not None) ``` Index entries for sessions that idled out stay until their own `ABSOLUTE` expiry, which is why they have one. To show a user their active sessions, read each `session:` key the index names and skip the ones that are gone. ## Sessions are not cache entries A lost session is a forced logout, so sessions get a cache of their own, configured unlike the one for page fragments: - **A `size_limit` it never reaches.** Eviction removes the least recently stored entries, and a long-lived session is exactly that. Sessions are small (a few hundred bytes each), so even 1 GiB, the default, holds millions. Watch `volume()` if you store more per session. - **Never a `NullCache`.** A `NullCache` stores nothing, so nobody could log in. If the app's other caches can be switched off through [`Caches`](../user-guide/caches.qmd), register the session cache by name and leave it enabled. - **Not `durable=True`.** A dropped write there means a login that silently didn't happen. Let `set()` raise and show the user an error. - **Run `expire()` from a scheduled job**, [claimed by one worker](run-once.qmd), so ended sessions free their space. Expired sessions are already invisible to `get()`; this is housekeeping. - **One machine.** Every worker on the host shares the sessions. Behind a load balancer, use sticky sessions or wait for the [network server](../user-guide/roadmap.qmd). ### Sharing a cache across web workers Directory, startup, request-path timeouts and shutdown for gunicorn, uvicorn, FastAPI, Flask, Django and Celery. Deployment is where a cache meets the real world: several worker processes that don't know each other's names, a server that forks and recycles them while they're busy, a cron job that opens the same directory on the side. diskcache-oxide is built for exactly this, so there isn't much to it. This page covers the few choices that matter and leaves out the ones that don't. ## Pick a directory - **Somewhere persistent and private:** `/var/cache/`, owned by the user the app runs as. diskcache-oxide creates the directory `0700` if it doesn't exist; if you create it yourself, keep it that way. Anyone who can write to a cache directory can change what your app reads from it. - **On a local disk.** The cache relies on SQLite's write-ahead log and shared memory, which don't work on network filesystems like NFS. diskcache-oxide checks this when it opens the cache and fails loudly instead of risking corruption. - **In Docker,** mount a volume for it if the cache should survive container restarts, and give every worker container on a host the same volume if they should share it. - **One directory per purpose.** Separate directories for page fragments, API responses, and coordination keys (job claims, rate limits) keep their size limits and eviction independent. ## Open it once per process Open the cache once, at startup, and share it across requests and threads. A `Cache` is safe to use from any number of threads. ### FastAPI ```python from contextlib import asynccontextmanager from fastapi import FastAPI from diskcache_oxide import Cache cache = Cache('/var/cache/my-app', timeout=1.0) @asynccontextmanager async def lifespan(app: FastAPI): yield cache.checkpoint() cache.close() app = FastAPI(lifespan=lifespan) ``` ### Flask ```python import atexit from flask import Flask from diskcache_oxide import Cache app = Flask(__name__) cache = Cache('/var/cache/my-app', timeout=1.0) @atexit.register def close_cache() -> None: cache.checkpoint() cache.close() ``` ### Django diskcache-oxide isn't a Django cache backend (`DjangoCache` is a [non-goal](../user-guide/roadmap.qmd#non-goals)), so `django.core.cache` keeps whatever it has. Open the shared cache in a module of its own and import it where it's needed; the settings module says where it lives. ```python # myapp/caching.py from django.conf import settings from diskcache_oxide import Cache cache = Cache(settings.SHARED_CACHE_DIR, timeout=1.0) ``` Register the [types](custom-types.qmd) you store in the same module, so every process that imports the cache gets them too. ### Celery and other background workers Celery's prefork pool, RQ and `multiprocessing` workers are pre-forking servers like gunicorn: open the cache at import time, and each child gets its own connections on first use. Register your types the same way the web workers do, or a worker that reads a web worker's value gets `ForeignValueError`. Closing at shutdown is optional; Celery's `worker_process_shutdown` signal is the place for it. ```python from celery.signals import worker_process_shutdown @worker_process_shutdown.connect def close_cache(**kwargs) -> None: cache.close() ``` ### Pre-forking servers With gunicorn (including `--preload`), uvicorn `--workers`, or Celery's prefork pool, it doesn't matter whether you open the cache before or after the fork. diskcache-oxide's fork barrier makes sure no worker starts life holding half a database transaction, and each child opens its own fresh connections on first use. See [Concurrency and forks](../user-guide/concurrency-and-forks.qmd). ## Keep the request path fast Reads never wait for writers, so a `get` stays fast however busy the cache is. Writes are different: only one process can commit to a cache file at a time, and a write waits up to the cache's `timeout` for its turn (60 seconds by default, matching diskcache). In a web request you'd rather skip a cache write than make the user wait, so give the cache a short timeout and treat `Timeout` as "don't cache this time": ```python from diskcache_oxide import Timeout def cache_quietly(key: str, value: object, expire: float) -> None: try: cache.set(key, value, expire=expire) except Timeout: log.warning('cache busy; skipped storing %s', key) ``` Under normal load a write takes about 25 µs and never comes near the timeout. The timeout matters during bursts, such as a big `clear()` from a maintenance script, and even then `clear()` steps aside between chunks to let waiting writes in. ## Maintenance Run maintenance from one place, such as a scheduled job (see [Running a job once across workers](run-once.qmd)), not from every worker: - `cache.expire()` removes expired entries and frees their space. A cache that's mostly read can accumulate expired entries, because the cleanup each `set()` does is small. Once an hour is plenty for most apps. - `cache.checkpoint()` empties the write-ahead log after a write burst. It returns `False` instead of waiting long if workers are busy, so it's safe to call any time. ## At shutdown Calling `checkpoint()` and `close()` at shutdown, as in the examples above, is optional but tidy. `checkpoint()` copies the write-ahead log into the database and truncates it, so a large log left by a burst of writes doesn't sit on disk. `close()` does the same automatically when the log is large, waiting at most a second. See [Storage and WAL](../user-guide/storage-and-wal.qmd). ## Switching caching off To run with caching off (local development, or chasing a stale-data bug in production), build the cache in one factory function and return a `NullCache` when it's disabled. Nothing else in the app changes. Pass `invalidate=` so deletes still reach the real cache while it's off. See [NullCache](../user-guide/null-cache.qmd). ```python from diskcache_oxide import Cache, CacheLike, NullCache def make_cache(settings) -> CacheLike: real = Cache(settings.cache_dir, timeout=1.0) return real if settings.cache_enabled else NullCache(invalidate=real) ``` ### Using the cache from asyncio Read inline, write off the loop, and keep transactions on one thread: the three rules for FastAPI, Starlette, aiohttp and friends until AsyncCache lands. An async framework makes one promise to every request it's serving: nothing blocks the loop. A synchronous cache call looks like a small risk to that promise, and whether it is depends entirely on which call. diskcache-oxide's API is synchronous. An `AsyncCache` is on the [roadmap](../user-guide/roadmap.qmd), but the sync cache already works well in an asyncio app, provided you know which calls can block the event loop and for how long. Here are the three rules. ## Reads: call them inline A cached read costs a few microseconds (the [performance](../user-guide/performance.qmd) page has the measurements), less than the event loop spends scheduling a task. Reads never take the write lock and never wait for writers, so `cache.get()` on the loop is the right call, and `await asyncio.to_thread(cache.get, ...)` would cost more than the read it protects. The exception is a read of a large value that keeps losing to concurrent overwrites, which can wait up to the cache's `timeout` before raising `Timeout`. That's rare; `durable=True` turns it into a miss. ## Writes: move them off the loop A write has to take the cache's write lock, and if another process holds it (a `clear()` in a maintenance script, a long transaction, a fork in flight) the write waits up to `timeout`. Under normal load that wait is nothing. In the worst case it's `timeout` seconds during which every request on that loop stops. Two ways to make sure that never happens: - **Run writes in a worker thread** with `asyncio.to_thread()`. The thread does the waiting; the loop keeps serving. The thread hop costs tens of microseconds, which is fine for a write. - **Or keep writes inline with a short `timeout` and `durable=True`.** Then the worst case is a dropped cache write logged once a minute, not a stalled loop. ```python import asyncio import tempfile from diskcache_oxide import Cache cache = Cache(tempfile.mkdtemp(), timeout=1.0) async def load_user(user_id: int) -> dict: await asyncio.sleep(0.001) # stands in for a database query return {'id': user_id} async def handle_request(request_id: int) -> dict: key = f'user:{request_id % 10}' user = cache.get(key) # a read, on the event loop if user is None: user = await load_user(request_id % 10) await asyncio.to_thread(cache.set, key, user, expire=300) # a write, off the loop return user results = await asyncio.gather(*(handle_request(i) for i in range(100))) print(f'{len(results)} requests served, {len(cache)} users cached') ``` Some of the 100 requests for one user miss together and each stores the same value; that's harmless, and the [stale-while-revalidate](stale-while-revalidate.qmd) recipe shows how to make only one of them do the work when it isn't. ### Fire and forget When the response shouldn't wait for the write at all, schedule it as a task. Keep a reference to the task until it finishes, or the loop may drop it half done: ```python background: set[asyncio.Task] = set() def cache_later(key: str, value: object, expire: float) -> None: task = asyncio.create_task(asyncio.to_thread(cache.set, key, value, expire=expire)) background.add(task) task.add_done_callback(background.discard) ``` ## Transactions: one thread, no `await` inside A [transaction](../user-guide/transactions.qmd) belongs to the thread that opened it, and it holds the write lock for the length of the block. Never `await` inside one: the loop would run other requests on that thread while the lock is held, and one of them may try to write. Put the whole block in a function and run that function in a thread: ```python def record_request(path: str, status: int) -> None: with cache.transact() as tx: tx.incr('requests:total') tx.incr(f'requests:{path}') tx.incr(f'requests:{path}:{status}') async def middleware(request, call_next): response = await call_next(request) await asyncio.to_thread(record_request, request.url.path, response.status_code) return response ``` ## In a FastAPI app Open the cache once, in the lifespan, and use it from any endpoint. Starlette's `run_in_threadpool` does the same job as `asyncio.to_thread()`; under Trio or AnyIO use `anyio.to_thread.run_sync()`. ```python from contextlib import asynccontextmanager from fastapi import FastAPI from starlette.concurrency import run_in_threadpool from diskcache_oxide import Cache cache = Cache('/var/cache/my-app', timeout=1.0) @asynccontextmanager async def lifespan(app: FastAPI): yield cache.close() app = FastAPI(lifespan=lifespan) @app.get('/episodes/{episode_id}') async def episode(episode_id: int) -> dict: key = f'episode:{episode_id}' if (cached := cache.get(key)) is not None: return cached data = await fetch_episode(episode_id) await run_in_threadpool(cache.set, key, data, expire=600) return data ``` See [Sharing a cache across web workers](web-workers.qmd) for where the directory goes and what to do at shutdown. ### Testing code that uses a cache pytest fixtures for a real cache, a NullCache, and a fake clock for testing expiry. Code that uses a cache has a talent for passing its tests for the wrong reason: the second assertion was green because the first one left the answer behind, and the test that "covers" expiry sleeps for two seconds and hopes. Such code needs tests of two kinds: tests where caching shouldn't get in the way, and tests of the caching behavior itself (is the value cached, does it expire, is it invalidated). diskcache-oxide gives you a tool for each, and neither involves sleeping. ## A real cache per test A `Cache` in pytest's `tmp_path` is fast and isolated: each test gets its own empty cache, and it's thrown away afterwards. ```python import pytest from diskcache_oxide import Cache @pytest.fixture def cache(tmp_path): with Cache(tmp_path / 'cache') as cache: yield cache ``` ## No caching at all: `NullCache` For tests that exercise your app rather than its caching, a cached value from an earlier test can hide a bug. Swap in a `NullCache`: every read misses and every write succeeds, so every call goes to the real code path. It has the same methods and signatures as `Cache`, so nothing else changes. ```python from diskcache_oxide import NullCache @pytest.fixture def app_cache(monkeypatch): null = NullCache() monkeypatch.setattr('my_app.services.cache', null) return null ``` ## Testing expiry without sleeping diskcache-oxide reads the clock natively in Rust, so patching `time.time` doesn't affect it on its own. Pass `clock=` instead. A small fake clock lets a test jump forward in time: ```python import tempfile from diskcache_oxide import Cache class FakeClock: def __init__(self, now: float = 1_000_000.0) -> None: self.now = now def __call__(self) -> float: return self.now def advance(self, seconds: float) -> None: self.now += seconds clock = FakeClock() cache = Cache(tempfile.mkdtemp(), clock=clock) cache.set('session:abc', {'user': 42}, expire=30) clock.advance(29) print('after 29 s:', cache.get('session:abc')) clock.advance(2) print('after 31 s:', cache.get('session:abc')) ``` As a fixture: ```python @pytest.fixture def clock(): return FakeClock() @pytest.fixture def cache(tmp_path, clock): with Cache(tmp_path / 'cache', clock=clock) as cache: yield cache def test_sessions_expire(cache, clock): cache.set('session:abc', {'user': 42}, expire=30) clock.advance(31) assert cache.get('session:abc') is None ``` If your tests already patch `time.time` (with `unittest.mock`, freezegun or time-machine) and you want the cache to follow along, pass `clock=lambda: time.time()`. The lambda looks up `time.time` on every call, so it sees the patched version. ## Testing code that runs in several processes Code that relies on the cache being shared, like a job claim or a rate limit, can be tested for real: start processes with `multiprocessing` and point them at the same `tmp_path` directory. diskcache-oxide is safe with every start method, including fork. ```python import multiprocessing def claim(directory: str) -> bool: with Cache(directory) as cache: return cache.add('job:nightly', 'claimed', expire=60) def test_only_one_worker_claims_the_job(tmp_path): with multiprocessing.Pool(8) as pool: results = pool.map(claim, [str(tmp_path)] * 8) assert results.count(True) == 1 ```