# Building from source Most people never build diskcache-oxide. PyPI has wheels for macOS on Apple Silicon, Linux (glibc) on x86_64 and aarch64, and Windows on x86_64, for CPython 3.14, 3.14t, 3.15 and 3.15t, and pip picks the right one. This page is for everyone else: - **No wheel for your platform.** An Intel Mac, Alpine or another musl-based Linux, Linux on an architecture other than x86_64 or aarch64, Windows on ARM. pip falls back to the source distribution on PyPI and compiles it, and it needs a Rust toolchain and a C compiler to do that. - **You'd rather not run prebuilt binaries.** Some teams build every native package themselves. - **You want to read, change or test the code.** That means a git checkout and an editable install. The engine is Rust, and it bundles its own copy of SQLite, written in C. So a build needs three things beyond Python: Rust, a C compiler, and (for a checkout) git. Python's side of the build is [maturin](https://www.maturin.rs/), which pip installs into a throwaway build environment by itself. # 1. Install the tools Rust comes from [rustup](https://rustup.rs/), the official installer, on every platform. diskcache-oxide tracks the latest stable Rust (the minimum is in `rust-version` in `Cargo.toml`, 1.99 today), and a Rust packaged by your OS is usually older than that, so use rustup even where your package manager offers Rust. - macOS - Linux - Windows Apple's command line tools give you the C compiler (clang) and git: ``` bash xcode-select --install ``` Then Rust: ``` bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Accept the default installation, then open a new terminal (or run `source "$HOME/.cargo/env"`) so `cargo` and `rustc` are on your `PATH`. If you use Homebrew, `brew install rustup` followed by `rustup-init` does the same thing. Install a C compiler, git and curl with your distribution's package manager: ``` bash # Debian, Ubuntu sudo apt install build-essential git curl # Fedora, RHEL sudo dnf install gcc git curl # Alpine apk add build-base git curl ``` Then Rust: ``` bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Accept the default installation, then open a new shell (or run `. "$HOME/.cargo/env"`) so `cargo` and `rustc` are on your `PATH`. In a Docker image, the non-interactive form is `curl -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal`, followed by `. "$HOME/.cargo/env"` in the same `RUN` step. The compilers make an image a lot bigger, so build in one stage and copy the virtual environment into a slim final stage; [Docker](../recipes/docker.md) has the layout. Rust on Windows uses Microsoft's C++ build tools (MSVC) as its linker and C compiler. Install rustup with winget, in PowerShell: ``` powershell winget install --id Rustlang.Rustup ``` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/) and run it. If the C++ build tools aren't installed, the installer offers to install them through the Visual Studio installer; accept it, or install them yourself with the **Desktop development with C++** workload of [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/). Building from a checkout also needs git: ``` powershell winget install --id Git.Git ``` Open a new terminal afterwards so `cargo`, `rustc` and `git` are on your `PATH`. Check the toolchain is there and recent enough: ``` bash rustc --version ``` If it reports a version below the minimum, `rustup update` brings it up to the latest stable. # 2. Install from PyPI, building it yourself On a platform with no wheel, a plain install already builds from source: ``` bash uv pip install diskcache-oxide ``` On a platform that has a wheel, ask for the source distribution explicitly. `--no-binary diskcache-oxide` applies to this one package, so your other dependencies still install from wheels: - macOS - Linux - Windows ``` bash uv venv --python 3.14 uv pip install --no-binary diskcache-oxide diskcache-oxide ``` ``` bash uv venv --python 3.14 uv pip install --no-binary diskcache-oxide diskcache-oxide ``` ``` powershell uv venv --python 3.14 uv pip install --no-binary diskcache-oxide diskcache-oxide ``` With pip instead of uv, it's `pip install --no-binary diskcache-oxide diskcache-oxide` in an activated virtual environment. For the free-threaded build, create the environment with `--python 3.14t`. Each Python needs its own build: a build for 3.14 won't load on 3.14t or 3.15. The build compiles in release mode. It took about 30 seconds on a recent 18-core Mac and about a minute in an Alpine container with 6 CPUs; a machine with fewer cores takes longer. Nothing is printed while Rust compiles, so a quiet stretch is normal. Add `-v` to watch it. uv and pip cache the built wheel, so a second environment on the same Python reuses it instead of compiling again. # 3. Or work from a git checkout To change the code or run the tests, clone the repository and install it in editable mode: - macOS - Linux - Windows ``` bash git clone https://github.com/mikeckennedy/diskcache-oxide.git cd diskcache-oxide uv venv --python 3.14 venv uv pip install --python venv/bin/python --group dev -e . ``` ``` bash git clone https://github.com/mikeckennedy/diskcache-oxide.git cd diskcache-oxide uv venv --python 3.14 venv uv pip install --python venv/bin/python --group dev -e . ``` ``` powershell git clone https://github.com/mikeckennedy/diskcache-oxide.git cd diskcache-oxide uv venv --python 3.14 venv uv pip install --python venv\Scripts\python.exe --group dev -e . ``` The checkout pins its exact Rust version in `rust-toolchain.toml`, and rustup installs that version the first time you build inside the directory. If yours doesn't, `rustup toolchain install` (run in the checkout, with no arguments) installs the one the file names. `--group dev` adds the test tools (pytest, ruff, ty, and diskcache 5.6.3 itself, which the conformance tests compare against). After you change any Rust code, run the same `uv pip install -e .` again: the project tells uv to rebuild whenever a `.rs` or `Cargo.toml` file changes. Python changes need no rebuild. Then run the tests: - macOS - Linux - Windows ``` bash venv/bin/pytest # oxide's own tests, plus conformance against diskcache venv/bin/pytest --backend=oxide tests/conformance # the conformance tests against your build ``` Some tests open thousands of files at once. If they fail with "Too many open files", raise the limit first with `ulimit -n 10240`. ``` bash venv/bin/pytest # oxide's own tests, plus conformance against diskcache venv/bin/pytest --backend=oxide tests/conformance # the conformance tests against your build ``` Some tests open thousands of files at once. If they fail with "Too many open files", raise the limit first with `ulimit -n 10240`. ``` powershell venv\Scripts\pytest # oxide's own tests, plus conformance against diskcache venv\Scripts\pytest --backend=oxide tests/conformance # the conformance tests against your build ``` Tests for `fork()` are skipped on Windows, which has no fork. Plain `pytest` runs the conformance tests against diskcache, the library diskcache-oxide matches; `--backend=oxide` runs the same tests against your build. Both should pass. # 4. Check the build Whichever way you built it, this confirms the engine loads and shows the bundled SQLite: ``` bash python -c "import diskcache_oxide as d; print(d.__version__, d.sqlite_version())" ``` The wheels compile SQLite with a few flags of our own, set in `.cargo/config.toml`, which both the source distribution and the checkout carry, and a source build picks them up when it runs from the unpacked directory, as pip and uv do. To confirm they reached your build: ``` bash python -c "import diskcache_oxide as d; print([o for o in d.sqlite_compile_options() if 'MEMSTATUS' in o or 'STAT4' in o])" ``` It should print `['DEFAULT_MEMSTATUS=0']` and nothing about STAT4. If `ENABLE_STAT4` appears, the flags were lost, usually because something set `LIBSQLITE3_FLAGS` in the environment, which overrides them. The build still works, but writes and reads run noticeably slower; [Storage and WAL](storage-and-wal.md) and the comments in `.cargo/config.toml` explain what each flag does. # How a source build differs from a wheel The code and the on-disk format are the same, so a cache written by a source build and one written by a wheel are interchangeable. One thing differs: the published wheels are built with profile-guided optimization (PGO), which compiles the engine, runs a training workload, and compiles it again laid out for the paths that workload took. A source build skips that, so its hot paths are somewhat slower; on one Windows machine a cached read measured about 10% faster on the PGO wheel. If speed matters and your platform has wheels, use them. # When the build fails - **`cannot be built because it requires rustc 1.99 or newer`**: your Rust is older than the minimum. Run `rustup update`. - **`cargo` or `rustc` not found**: the shell started before rustup finished. Open a new one, or run `source "$HOME/.cargo/env"` (Windows: open a new terminal). - **`` linker `cc` not found ``, or `failed to run custom build command for libsqlite3-sys` with `Failed to find tool`**: there's no C compiler. Install the tools in [step 1](#install-the-tools) for your platform. - **`link.exe` not found** (Windows): the MSVC build tools are missing. Install the **Desktop development with C++** workload, then open a new terminal. - **The build is killed partway, with no error from Rust**: the machine, or the container, ran out of memory. Give the container more memory, or build with fewer parallel jobs by setting `CARGO_BUILD_JOBS=2`. If a build fails for a reason not listed here, please [open an issue](https://github.com/mikeckennedy/diskcache-oxide/issues) with the platform, the Python version (`python -VV`), `rustc --version`, and the output of the install with `-v`.