Building from source
Most people never build diskcache-oxide. PyPI has wheels for macOS on Apple Silicon, Linux (glibc) on x86_64 and aarch64, and Windows on x86_64, for CPython 3.14, 3.14t, 3.15 and 3.15t, and pip picks the right one. This page is for everyone else:
- No wheel for your platform. An Intel Mac, Alpine or another musl-based Linux, Linux on an architecture other than x86_64 or aarch64, Windows on ARM. pip falls back to the source distribution on PyPI and compiles it, and it needs a Rust toolchain and a C compiler to do that.
- You’d rather not run prebuilt binaries. Some teams build every native package themselves.
- You want to read, change or test the code. That means a git checkout and an editable install.
The engine is Rust, and it bundles its own copy of SQLite, written in C. So a build needs three things beyond Python: Rust, a C compiler, and (for a checkout) git. Python’s side of the build is maturin, which pip installs into a throwaway build environment by itself.
1. Install the tools
Rust comes from rustup, the official installer, on every platform. diskcache-oxide tracks the latest stable Rust (the minimum is in rust-version in Cargo.toml, 1.99 today), and a Rust packaged by your OS is usually older than that, so use rustup even where your package manager offers Rust.
Apple’s command line tools give you the C compiler (clang) and git:
xcode-select --installThen Rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shAccept the default installation, then open a new terminal (or run source "$HOME/.cargo/env") so cargo and rustc are on your PATH. If you use Homebrew, brew install rustup followed by rustup-init does the same thing.
Install a C compiler, git and curl with your distribution’s package manager:
# Debian, Ubuntu
sudo apt install build-essential git curl
# Fedora, RHEL
sudo dnf install gcc git curl
# Alpine
apk add build-base git curlThen Rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shAccept the default installation, then open a new shell (or run . "$HOME/.cargo/env") so cargo and rustc are on your PATH.
In a Docker image, the non-interactive form is curl -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal, followed by . "$HOME/.cargo/env" in the same RUN step. The compilers make an image a lot bigger, so build in one stage and copy the virtual environment into a slim final stage; Docker has the layout.
Rust on Windows uses Microsoft’s C++ build tools (MSVC) as its linker and C compiler. Install rustup with winget, in PowerShell:
winget install --id Rustlang.Rustupor download rustup-init.exe from rustup.rs and run it. If the C++ build tools aren’t installed, the installer offers to install them through the Visual Studio installer; accept it, or install them yourself with the Desktop development with C++ workload of Visual Studio Build Tools. Building from a checkout also needs git:
winget install --id Git.GitOpen a new terminal afterwards so cargo, rustc and git are on your PATH.
Check the toolchain is there and recent enough:
rustc --versionIf it reports a version below the minimum, rustup update brings it up to the latest stable.
2. Install from PyPI, building it yourself
On a platform with no wheel, a plain install already builds from source:
uv pip install diskcache-oxideOn a platform that has a wheel, ask for the source distribution explicitly. --no-binary diskcache-oxide applies to this one package, so your other dependencies still install from wheels:
uv venv --python 3.14
uv pip install --no-binary diskcache-oxide diskcache-oxideuv venv --python 3.14
uv pip install --no-binary diskcache-oxide diskcache-oxideuv venv --python 3.14
uv pip install --no-binary diskcache-oxide diskcache-oxideWith pip instead of uv, it’s pip install --no-binary diskcache-oxide diskcache-oxide in an activated virtual environment. For the free-threaded build, create the environment with --python 3.14t. Each Python needs its own build: a build for 3.14 won’t load on 3.14t or 3.15.
The build compiles in release mode. It took about 30 seconds on a recent 18-core Mac and about a minute in an Alpine container with 6 CPUs; a machine with fewer cores takes longer. Nothing is printed while Rust compiles, so a quiet stretch is normal. Add -v to watch it. uv and pip cache the built wheel, so a second environment on the same Python reuses it instead of compiling again.
3. Or work from a git checkout
To change the code or run the tests, clone the repository and install it in editable mode:
git clone https://github.com/mikeckennedy/diskcache-oxide.git
cd diskcache-oxide
uv venv --python 3.14 venv
uv pip install --python venv/bin/python --group dev -e .git clone https://github.com/mikeckennedy/diskcache-oxide.git
cd diskcache-oxide
uv venv --python 3.14 venv
uv pip install --python venv/bin/python --group dev -e .git clone https://github.com/mikeckennedy/diskcache-oxide.git
cd diskcache-oxide
uv venv --python 3.14 venv
uv pip install --python venv\Scripts\python.exe --group dev -e .The checkout pins its exact Rust version in rust-toolchain.toml, and rustup installs that version the first time you build inside the directory. If yours doesn’t, rustup toolchain install (run in the checkout, with no arguments) installs the one the file names.
--group dev adds the test tools (pytest, ruff, ty, and diskcache 5.6.3 itself, which the conformance tests compare against). After you change any Rust code, run the same uv pip install -e . again: the project tells uv to rebuild whenever a .rs or Cargo.toml file changes. Python changes need no rebuild.
Then run the tests:
venv/bin/pytest # oxide's own tests, plus conformance against diskcache
venv/bin/pytest --backend=oxide tests/conformance # the conformance tests against your buildSome tests open thousands of files at once. If they fail with “Too many open files”, raise the limit first with ulimit -n 10240.
venv/bin/pytest # oxide's own tests, plus conformance against diskcache
venv/bin/pytest --backend=oxide tests/conformance # the conformance tests against your buildSome tests open thousands of files at once. If they fail with “Too many open files”, raise the limit first with ulimit -n 10240.
venv\Scripts\pytest # oxide's own tests, plus conformance against diskcache
venv\Scripts\pytest --backend=oxide tests/conformance # the conformance tests against your buildTests for fork() are skipped on Windows, which has no fork.
Plain pytest runs the conformance tests against diskcache, the library diskcache-oxide matches; --backend=oxide runs the same tests against your build. Both should pass.
4. Check the build
Whichever way you built it, this confirms the engine loads and shows the bundled SQLite:
python -c "import diskcache_oxide as d; print(d.__version__, d.sqlite_version())"The wheels compile SQLite with a few flags of our own, set in .cargo/config.toml, which both the source distribution and the checkout carry, and a source build picks them up when it runs from the unpacked directory, as pip and uv do. To confirm they reached your build:
python -c "import diskcache_oxide as d; print([o for o in d.sqlite_compile_options() if 'MEMSTATUS' in o or 'STAT4' in o])"It should print ['DEFAULT_MEMSTATUS=0'] and nothing about STAT4. If ENABLE_STAT4 appears, the flags were lost, usually because something set LIBSQLITE3_FLAGS in the environment, which overrides them. The build still works, but writes and reads run noticeably slower; Storage and WAL and the comments in .cargo/config.toml explain what each flag does.
How a source build differs from a wheel
The code and the on-disk format are the same, so a cache written by a source build and one written by a wheel are interchangeable. One thing differs: the published wheels are built with profile-guided optimization (PGO), which compiles the engine, runs a training workload, and compiles it again laid out for the paths that workload took. A source build skips that, so its hot paths are somewhat slower; on one Windows machine a cached read measured about 10% faster on the PGO wheel. If speed matters and your platform has wheels, use them.
When the build fails
cannot be built because it requires rustc 1.99 or newer: your Rust is older than the minimum. Runrustup update.cargoorrustcnot found: the shell started before rustup finished. Open a new one, or runsource "$HOME/.cargo/env"(Windows: open a new terminal).linker `cc` not found, orfailed to run custom build command for libsqlite3-syswithFailed to find tool: there’s no C compiler. Install the tools in step 1 for your platform.link.exenot found (Windows): the MSVC build tools are missing. Install the Desktop development with C++ workload, then open a new terminal.- The build is killed partway, with no error from Rust: the machine, or the container, ran out of memory. Give the container more memory, or build with fewer parallel jobs by setting
CARGO_BUILD_JOBS=2.
If a build fails for a reason not listed here, please open an issue with the platform, the Python version (python -VV), rustc --version, and the output of the install with -v.