# Building from source
Most people never build diskcache-oxide. PyPI has wheels for macOS on Apple Silicon, Linux (glibc) on x86_64 and aarch64, and Windows on x86_64, for CPython 3.14, 3.14t, 3.15 and 3.15t, and pip picks the right one. This page is for everyone else:
- **No wheel for your platform.** An Intel Mac, Alpine or another musl-based Linux, Linux on an architecture other than x86_64 or aarch64, Windows on ARM. pip falls back to the source distribution on PyPI and compiles it, and it needs a Rust toolchain and a C compiler to do that.
- **You'd rather not run prebuilt binaries.** Some teams build every native package themselves.
- **You want to read, change or test the code.** That means a git checkout and an editable install.
The engine is Rust, and it bundles its own copy of SQLite, written in C. So a build needs three things beyond Python: Rust, a C compiler, and (for a checkout) git. Python's side of the build is [maturin](https://www.maturin.rs/), which pip installs into a throwaway build environment by itself.
# 1. Install the tools
Rust comes from [rustup](https://rustup.rs/), the official installer, on every platform. diskcache-oxide tracks the latest stable Rust (the minimum is in `rust-version` in `Cargo.toml`, 1.99 today), and a Rust packaged by your OS is usually older than that, so use rustup even where your package manager offers Rust.
- macOS
- Linux
- Windows
Apple's command line tools give you the C compiler (clang) and git:
``` bash
xcode-select --install
```
Then Rust:
``` bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
Accept the default installation, then open a new terminal (or run `source "$HOME/.cargo/env"`) so `cargo` and `rustc` are on your `PATH`. If you use Homebrew, `brew install rustup` followed by `rustup-init` does the same thing.
Install a C compiler, git and curl with your distribution's package manager:
``` bash
# Debian, Ubuntu
sudo apt install build-essential git curl
# Fedora, RHEL
sudo dnf install gcc git curl
# Alpine
apk add build-base git curl
```
Then Rust:
``` bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
Accept the default installation, then open a new shell (or run `. "$HOME/.cargo/env"`) so `cargo` and `rustc` are on your `PATH`.
In a Docker image, the non-interactive form is `curl -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal`, followed by `. "$HOME/.cargo/env"` in the same `RUN` step. The compilers make an image a lot bigger, so build in one stage and copy the virtual environment into a slim final stage; [Docker](../recipes/docker.md) has the layout.
Rust on Windows uses Microsoft's C++ build tools (MSVC) as its linker and C compiler. Install rustup with winget, in PowerShell:
``` powershell
winget install --id Rustlang.Rustup
```
or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/) and run it. If the C++ build tools aren't installed, the installer offers to install them through the Visual Studio installer; accept it, or install them yourself with the **Desktop development with C++** workload of [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/). Building from a checkout also needs git:
``` powershell
winget install --id Git.Git
```
Open a new terminal afterwards so `cargo`, `rustc` and `git` are on your `PATH`.
Check the toolchain is there and recent enough:
``` bash
rustc --version
```
If it reports a version below the minimum, `rustup update` brings it up to the latest stable.
# 2. Install from PyPI, building it yourself
On a platform with no wheel, a plain install already builds from source:
``` bash
uv pip install diskcache-oxide
```
On a platform that has a wheel, ask for the source distribution explicitly. `--no-binary diskcache-oxide` applies to this one package, so your other dependencies still install from wheels:
- macOS
- Linux
- Windows
``` bash
uv venv --python 3.14
uv pip install --no-binary diskcache-oxide diskcache-oxide
```
``` bash
uv venv --python 3.14
uv pip install --no-binary diskcache-oxide diskcache-oxide
```
``` powershell
uv venv --python 3.14
uv pip install --no-binary diskcache-oxide diskcache-oxide
```
With pip instead of uv, it's `pip install --no-binary diskcache-oxide diskcache-oxide` in an activated virtual environment. For the free-threaded build, create the environment with `--python 3.14t`. Each Python needs its own build: a build for 3.14 won't load on 3.14t or 3.15.
The build compiles in release mode. It took about 30 seconds on a recent 18-core Mac and about a minute in an Alpine container with 6 CPUs; a machine with fewer cores takes longer. Nothing is printed while Rust compiles, so a quiet stretch is normal. Add `-v` to watch it. uv and pip cache the built wheel, so a second environment on the same Python reuses it instead of compiling again.
# 3. Or work from a git checkout
To change the code or run the tests, clone the repository and install it in editable mode:
- macOS
- Linux
- Windows
``` bash
git clone https://github.com/mikeckennedy/diskcache-oxide.git
cd diskcache-oxide
uv venv --python 3.14 venv
uv pip install --python venv/bin/python --group dev -e .
```
``` bash
git clone https://github.com/mikeckennedy/diskcache-oxide.git
cd diskcache-oxide
uv venv --python 3.14 venv
uv pip install --python venv/bin/python --group dev -e .
```
``` powershell
git clone https://github.com/mikeckennedy/diskcache-oxide.git
cd diskcache-oxide
uv venv --python 3.14 venv
uv pip install --python venv\Scripts\python.exe --group dev -e .
```
The checkout pins its exact Rust version in `rust-toolchain.toml`, and rustup installs that version the first time you build inside the directory. If yours doesn't, `rustup toolchain install` (run in the checkout, with no arguments) installs the one the file names.
`--group dev` adds the test tools (pytest, ruff, ty, and diskcache 5.6.3 itself, which the conformance tests compare against). After you change any Rust code, run the same `uv pip install -e .` again: the project tells uv to rebuild whenever a `.rs` or `Cargo.toml` file changes. Python changes need no rebuild.
Then run the tests:
- macOS
- Linux
- Windows
``` bash
venv/bin/pytest # oxide's own tests, plus conformance against diskcache
venv/bin/pytest --backend=oxide tests/conformance # the conformance tests against your build
```
Some tests open thousands of files at once. If they fail with "Too many open files", raise the limit first with `ulimit -n 10240`.
``` bash
venv/bin/pytest # oxide's own tests, plus conformance against diskcache
venv/bin/pytest --backend=oxide tests/conformance # the conformance tests against your build
```
Some tests open thousands of files at once. If they fail with "Too many open files", raise the limit first with `ulimit -n 10240`.
``` powershell
venv\Scripts\pytest # oxide's own tests, plus conformance against diskcache
venv\Scripts\pytest --backend=oxide tests/conformance # the conformance tests against your build
```
Tests for `fork()` are skipped on Windows, which has no fork.
Plain `pytest` runs the conformance tests against diskcache, the library diskcache-oxide matches; `--backend=oxide` runs the same tests against your build. Both should pass.
# 4. Check the build
Whichever way you built it, this confirms the engine loads and shows the bundled SQLite:
``` bash
python -c "import diskcache_oxide as d; print(d.__version__, d.sqlite_version())"
```
The wheels compile SQLite with a few flags of our own, set in `.cargo/config.toml`, which both the source distribution and the checkout carry, and a source build picks them up when it runs from the unpacked directory, as pip and uv do. To confirm they reached your build:
``` bash
python -c "import diskcache_oxide as d; print([o for o in d.sqlite_compile_options() if 'MEMSTATUS' in o or 'STAT4' in o])"
```
It should print `['DEFAULT_MEMSTATUS=0']` and nothing about STAT4. If `ENABLE_STAT4` appears, the flags were lost, usually because something set `LIBSQLITE3_FLAGS` in the environment, which overrides them. The build still works, but writes and reads run noticeably slower; [Storage and WAL](storage-and-wal.md) and the comments in `.cargo/config.toml` explain what each flag does.
# How a source build differs from a wheel
The code and the on-disk format are the same, so a cache written by a source build and one written by a wheel are interchangeable. One thing differs: the published wheels are built with profile-guided optimization (PGO), which compiles the engine, runs a training workload, and compiles it again laid out for the paths that workload took. A source build skips that, so its hot paths are somewhat slower; on one Windows machine a cached read measured about 10% faster on the PGO wheel. If speed matters and your platform has wheels, use them.
# When the build fails
- **`cannot be built because it requires rustc 1.99 or newer`**: your Rust is older than the minimum. Run `rustup update`.
- **`cargo` or `rustc` not found**: the shell started before rustup finished. Open a new one, or run `source "$HOME/.cargo/env"` (Windows: open a new terminal).
- **`` linker `cc` not found ``, or `failed to run custom build command for libsqlite3-sys` with `Failed to find tool`**: there's no C compiler. Install the tools in [step 1](#install-the-tools) for your platform.
- **`link.exe` not found** (Windows): the MSVC build tools are missing. Install the **Desktop development with C++** workload, then open a new terminal.
- **The build is killed partway, with no error from Rust**: the machine, or the container, ran out of memory. Give the container more memory, or build with fewer parallel jobs by setting `CARGO_BUILD_JOBS=2`.
If a build fails for a reason not listed here, please [open an issue](https://github.com/mikeckennedy/diskcache-oxide/issues) with the platform, the Python version (`python -VV`), `rustc --version`, and the output of the install with `-v`.