Rust and Docker: Improve build time

Why does the docker build time of a rust project are slow, and how to improve it

A snail on a branch
Photo by Михаил Павленко / Unsplash

Disclaimer: This is my first post in English, so it will probably be full of grammatical errors.

Last time, I wrote an article (fr) about how to reduce Rust project docker image size.

Now it's time to tackle another concern: build time.

It's not a secret, rust build time can be long. For those like me who are used to work with stuff like Java, it's a bit unusual to have to wait for minutes to complete a build from scratch: a relatively small project with a lot of dependencies like this one (wink wink) take up to 3 min to build in release mode on my laptop (core i7-1185G7, 8 cores, 32 Go, "Balanced" power-mode, MTV unplugged).

$ cargo build --release
[... compiling dependencies ...]
  Compiling rss-aggregator v0.1.0 (/home/eric/dev/rss-aggregator)
    Finished release [optimized] target(s) in 2m 56s

Also, please remind that this build use the release profile optimizations from this post (fr)

Of course, it's only for the first build: all the dependencies are compiled, and cargo is smart enough to not rebuild them, although you update/change them. But if you modify something like one line in your code and want to cargo build --release it, it will take something like 2:05 min.

$ cargo build --release
  Compiling rss-aggregator v0.1.0 (/home/eric/dev/rss-aggregator)
   Finished release [optimized] target(s) in 2m 04s

"Are you dumb?", you will say, "nobody use --release on a day-to-day basis!". And you're right! So, to be fare, the build time on debug from scratch is more like 2:15 min

$ cargo build
[... compiling dependencies ...]
  Compiling rss-aggregator v0.1.0 (/home/eric/dev/rss-aggregator)
    Finished dev [unoptimized + debuginfo] target(s) in 2m 15s

and 6 seconds once the dependencies are built.

$ cargo build
   Compiling rss-aggregator v0.1.0 (/home/emercier/perso/pedro/rss-aggregator)
    Finished dev [unoptimized + debuginfo] target(s) in 6.63s

The docker issue

OK, so build time may be long, but what about docker? Well docker will complicate things a bit. Let take a "dummy" Dockerfile, targeting musl

FROM rust:latest AS builder
ARG DEBIAN_FRONTEND=noninteractive

RUN rustup target add x86_64-unknown-linux-musl
RUN apt update && apt install -y musl-tools musl-dev
RUN update-ca-certificates

ENV USER=rss-aggregator
ENV UID=10001

RUN adduser \
    --disabled-password \
    --gecos "" \
    --home "/nonexistent" \
    --shell "/sbin/nologin" \
    --no-create-home \
    --uid "${UID}" \
    "${USER}"

WORKDIR app
COPY . .

RUN cargo build --target x86_64-unknown-linux-musl --release

### Our runtime image
FROM alpine

COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
COPY --from=builder /app/target/x86_64-unknown-linux-musl/release/rss-aggregator /usr/local/bin
COPY static/ static/

USER rss-aggregator:rss-aggregator
EXPOSE 8080

ENTRYPOINT ["rss-aggregator"]

The issue here is that we won't be able to leverage on Docker build cache: if you change even one comma in your code, the RUN cargo build ... step will rebuild all the dependencies. Yes, for one comma.

So, you may think "Maybe there is a way to build only the dependencies in a step, and then build our code in another one? This way, if we don't touch our dependencies, docker will cache the dependencies building layer, our code compilation will take only a few seconds, and we will be happy ever after. Right? It's possible? Please."

I spent way too much making this

Well... no. Not officially. There is an issue open since before my son's birth on the cargo GitHub, and it's complicated. There is a lot of work around proposed in the comments, and one of it is cargo-chef from Luca Palmieri, author of Zero To Production In Rust, a great book (please buy and read it)

In short, cargo chef will plane the dependencies to build in a step, and build them (and only them) in another one. Then you can build your source in a reasonable time, as long as you don't change all your dependencies all at one.

FROM rust:latest AS chef
ARG DEBIAN_FRONTEND=noninteractive

RUN rustup target add x86_64-unknown-linux-musl
RUN cargo install cargo-chef
RUN apt update && apt install -y musl-tools musl-dev
RUN update-ca-certificates

WORKDIR /app

FROM chef AS planner
COPY entity/Cargo.toml ./entity/Cargo.toml
COPY Cargo.* ./
RUN cargo chef prepare --recipe-path recipe.json


FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json
# Build dependencies
RUN cargo chef cook --release --target x86_64-unknown-linux-musl --recipe-path recipe.json
# Build application
COPY entity/src entity/src
COPY src/ src/
RUN cargo build --release --target x86_64-unknown-linux-musl --bin rss-aggregator

FROM alpine
LABEL maintainer=eric@pedr0.net
RUN addgroup -S rss-aggregator && adduser -S rss-aggregator -G rss-aggregator

RUN apk --no-cache add curl # Needed for the docker health check

COPY --from=builder /app/target/x86_64-unknown-linux-musl/release/rss-aggregator /usr/local/bin
COPY static/ static/

EXPOSE 8080
USER rss-aggregator
ENTRYPOINT ["rss-aggregator"]

"OK, you're cute, but does it really worth it?" Well...

dummy Dockerfile build from scratch 4:14 minutes
dummy Dockerfile subsequent build 5:04 minutes
cargo-chef Dockerfile build from scratch 6:37 minutes
cargo-chef Dockerfile subsequent build 1:34 minutes

Even if the first build take a bit more time, each subsequent build will take a lot less time.

So when you're paying your CI (gitlab ci, github actions for exemple) by the minute, gains some minutes from here to there can help a lot to reduce the bill, you may even stay in the free tier.

To conclude, it's not a magical fix, but it helps a lot!