⑂ Open source

Reproducible builds and signed releases in open source projects

Why public code is not enough: checksums, release signatures, signed tags and reproducible builds to prove the binary really comes from the source.

One of the arguments for open source is that anyone can read the code. But almost nobody installs software by building it themselves: people download packages, binaries and ready-made images. So a simple and uncomfortable question arises. How do I know that the file I downloaded really matches the public code I could have read? Signed releases and reproducible builds are the two complementary answers.

The problem of trusting the binary

Between the code in the repository and the file on your disk there are several steps: the machine of whoever builds it, the build tools, the server you download from, any mirrors. Each is a point where an attacker could slip in changes. If the binary contains malicious code absent from the source, reading the source is useless.

Defences sit at different levels:

Level Question it answers Tool
Download integrity Did the file arrive intact? Checksum
Authenticity Does the file really come from the project? Release signature
Code origin Does the tag match what maintainers approved? Signed tags and commits
Source-binary match Does the binary derive from that source? Reproducible build

Checksums: useful but not enough

Many projects publish a list of hashes, for example SHA-256, alongside their files. Checking it is immediate:

sha256sum -c SHA256SUMS --ignore-missing

A checksum guarantees that the file was not corrupted during download. It does not, however, guarantee that it is authentic: if an attacker controls the server, they can replace both the file and the list of hashes. That is why a checksum must be accompanied by a signature.

Signed releases

The project signs the list of checksums, or the files themselves, with a private key. Whoever downloads verifies the signature with the project's public key, obtained beforehand from an independent source.

gpg --verify SHA256SUMS.asc SHA256SUMS

There are also newer approaches that tie the signature to the identity of the build system or the maintainer's account, recording it in a public transparency log, so as to avoid managing long-term keys. Whatever the method, two critical points remain:

  • key custody: if the key is stolen, signatures are worthless; it should be kept offline or in a hardware device;
  • public key distribution: it must be published in several places and stay stable over time, otherwise users have no way to tell a legitimate key from a substituted one.

Signed tags and commits

Git lets you sign tags and commits. A signed release tag proves that point in history was approved by whoever holds the key:

git tag -s v2.4.0 -m "Release 2.4.0"
git tag -v v2.4.0

This protects against tampering with the repository or with the account on the hosting platform. Many platforms also show a verification badge next to signed commits.

Reproducible builds

A build is reproducible when, starting from the same source with the same declared environment, anyone obtains a bit-for-bit identical file. At that point the comparison becomes simple: if my hash matches that of the official binary, the binary derives from that source.

Achieving this means eliminating the most common sources of variability:

  • dates and times embedded in the binary or in archives;
  • absolute paths of the build machine;
  • file order in archives, which depends on the file system;
  • unpinned versions of compilers and dependencies;
  • locale and time zone of the environment.

A widespread practice is the SOURCE_DATE_EPOCH environment variable, which many tools honour in order to use a fixed date instead of the build time. Several Linux distributions have worked for years to make most of their packages reproducible.

Where to start in a project

  1. Publish checksums for every release file.
  2. Sign the checksums and document how to obtain and verify the public key.
  3. Sign release tags.
  4. Build releases in an automated, declared environment, not on a maintainer's laptop.
  5. Pin the versions of dependencies and tools.
  6. Check reproducibility by building twice in different environments and comparing hashes.

In short

  • Reading the source is not enough if you cannot verify that the binary derives from it.
  • Checksums guarantee integrity, signatures guarantee authenticity.
  • Signed tags and commits protect the repository's history.
  • Reproducible builds let anyone confirm that source and binary match.
MM
Marco Marzaduri

Marco Marzaduri works in cybersecurity, digital forensics and web development. A computer science graduate, he combines rigorous technical analysis with attention to the people who use technology.

Read also