Nix CI: evaluation, builds, and output contracts

The GitHub Actions workflow for example-theme-project is a single-job Nix CI setup with read-only permissions. It installs Nix on ubuntu-latest, evaluates the flake, builds six theme variants, and finally runs a derivation that checks the output contract. It can establish that a particular source snapshot passes Nix evaluation, builds, and file-contract checks on this runner. It cannot establish that CI has built aarch64-linux outputs, or that Fcitx5 or any desktop compositor is actually running.

This article uses an anonymized theme project to demonstrate how to record three kinds of evidence separately. The repository, configuration, and records below are teaching examples, not reports of real runs from a personal project:

1. Local checkout/flake evidence: the source code, outputs declared in flake.nix, and results of local commands.
2. GitHub Actions run evidence: the event, job, steps, and conclusion of a particular workflow run on GitHub's servers.
3. Desktop runtime evidence: startup, configuration loading, and visual behavior of Fcitx5, fcitx5-gtk, and the compositor in a real user session.

Conflating these leads to common mistakes: treating nix flake check --all-systems --no-build as a cross-architecture build, a package build as a desktop compatibility test, or a release page as the result of automated publishing by the workflow.

1. The example's goals and evidence boundaries#

1.1 Project context and source identity

Assume example-theme-project is a theme flake for Fcitx5, offering light, dark, and combined themes, each in blur/solid styles. The project declares x86_64-linux and aarch64-linux and also exports Home Manager modules. Names, theme directories, and the repository owner are all example placeholders.

A review must record the checkout's full revision, whether there are uncommitted files, flake.lock, and the build system. Matching origin/main identifies only the source snapshot under review. It does not show that the theme has been activated in a user session or establish compatibility with every compositor.

If the README distinguishes “tested on a particular desktop” from “theoretical protocol support only on other desktops,” preserve that distinction and inspect the evidence. Welcoming Issues/PRs does not expand the supported scope. The absence of CONTRIBUTING, CODEOWNERS, SECURITY, or templates does not establish an absence of contributor activity either. This teaching example contains no findings from actual desktop testing.

1.2 A three-layer evidence matrix

1.3 Local entry points for verification

In the checkout being verified, the minimum review path is as follows. These are command examples only:

cd <repo-path>
git status --short --branch
git rev-parse HEAD
nix flake check --all-systems --no-build

If nix flake check --all-systems --no-build succeeds but reports unknown flake output 'homeManagerModules', record “evaluation passed with a warning.” The specific warning varies with the Nix version and flake structure; this example does not claim that these commands have been run. Record the warning separately rather than rewriting the result as “completely warning-free.” It is not a runtime test result, either. To check the package implementation itself, also run the workflow's six nix build --no-link commands and the variants check, rather than simply repeating --no-build.

2. A section-by-section workflow walkthrough#

2.1 The complete teaching YAML

The following is the complete teaching configuration for nix.yml. There are no hidden matrix, cache, artifact, or release steps:

name: Nix checks

on:
  push:
  pull_request:

permissions:
  contents: read

jobs:
  flake:
    runs-on: ubuntu-latest
    steps:
      - name: Check out source
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

      - name: Install Nix
        uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
        with:
          extra_nix_config: |
            experimental-features = nix-command flakes

      - name: Evaluate the flake
        run: nix flake check --all-systems --no-build

      - name: Build all theme variants
        run: >-
          nix build --no-link
          .#both-blur .#light-blur .#dark-blur
          .#both-solid .#light-solid .#dark-solid

      - name: Check package output contracts
        run: nix build --no-link .#checks.x86_64-linux.variants

2.2 Triggers and permissions

• Neither push: nor pull_request: has branches, paths, or paths-ignore filters. This example can therefore trigger on branch/tag pushes and the default PR activity types; PR triggers do not respond to every activity type by default. There is no branch filter restricting it to main, nor any directory-based filtering to reduce runs.
• permissions: contents: read limits GITHUB_TOKEN permissions to reading repository contents. The example workflow does not need to write tags, write releases, comment on PRs, or access a package registry.
• There is only one job, flake, running on ubuntu-latest. There is no strategy.matrix for operating systems, architectures, Nix versions, or theme variants.
• For cachix/install-nix-action, extra_nix_config enables nix-command and flakes, making the subsequent nix flake/nix build commands available. It is not Nix binary-cache configuration.

“All pushes” here includes tag pushes. A push of the example v0.2.0 tag can therefore trigger the same check, but this does not mean the workflow publishes a release.

2.3 The supply-chain value of pinned SHAs

actions/checkout is pinned to 3d3c42e5aac5ba805825da76410c181273ba90b1, with a comment identifying v7.0.1; cachix/install-nix-action is pinned to 13d8dd58da0234aa297dedd986986ccb8e7f3e24, labeled v31.11.1. Both full SHAs were checked against their version tags through the upstream tag API on 2026-10-08. The selected versions are retained here, without a guarantee that they are the latest or free of vulnerabilities.

Pinning a full commit SHA is a supply-chain practice: it prevents a mutable tag from silently pointing to different code in the future and helps with auditing and reproducibility. It does not prove that an Action is safe. The Action's repository, source code, permissions, and upgrade changes still need review. Nor is SHA pinning being overstated here as signature verification, an SBOM, or dependency scanning.

3. Flake outputs and the output contract#

3.1 Inputs, systems, and package outputs

This example assumes that flake.nix uses github:NixOS/nixpkgs/nixos-unstable and declares two systems, x86_64-linux and aarch64-linux. The forAllSystems helper maps package and checks definitions to both systems.

Assume the core package variants in flake.nix are defined as follows:

Both default and example-theme-package point to both-blur, while the six named variants remain available for consumers to choose from. This is a Nix output-naming contract, not a list of GitHub release assets.

3.2 Home Manager outputs and the boundaries of CI

The flake also exports homeManagerModules.default and homeManagerModules.example-theme, corresponding to flake.nix. The workflow has no dedicated Home Manager consumer, module-option, NixOS dry-build, or configuration-instance tests. “Exports a Home Manager module” and “CI has verified Home Manager integration” are therefore not interchangeable statements.

If the local evaluation produces an unknown flake output 'homeManagerModules' warning, record it separately from successful packages/checks. A nonfatal warning does not justify claiming that the module has been activated in a consumer configuration, much less that Actions performed dedicated Home Manager testing.

3.3 The content contract of checks.x86_64-linux.variants

The six variants under checks are defined in flake.nix. The check script is generated in flake.nix using pkgs.runCommand, coreutils, and gnugrep. It checks file and text contracts in the final Nix store outputs:

The following content-contract pseudocode omits the actual Nix expressions and shell implementation. It cannot be run directly as a .nix file:

selectedThemes =
  # theme.conf、panel.svg、highlight.svg、theme.conf.tpl、highlight.svg.tpl 存在

unselectedThemes =
  # 未选主题的两个目录不存在

styleChecks =
  if variant.blur then
    # EnableBlur=True、BlurMask=blur-mask.svg、fill-opacity:0.25、mask 存在
  else
    # EnableBlur=False、BlurMask=、fill-opacity:1、mask 不存在

This is therefore a lightweight output-contract test. It uses shell test and grep to constrain the consumer-visible paths, theme selection, blur fields, SVG opacity, and mask files. It is not a unit-testing framework and does not simulate Fcitx5, render GTK candidate windows, or start a compositor.

3.4 Evaluation is not a cross-architecture build

This is the easiest part of the example to misread:

1. nix flake check --all-systems --no-build evaluates/discovers checks across the flake's system outputs, but --no-build explicitly prevents this command from actually building derivations.
2. The six nix build --no-link .#... commands without an explicit system resolve to the runner's own system on the example ubuntu-latest runner, so the outputs actually built are x86_64-linux.
3. The final step explicitly specifies .#checks.x86_64-linux.variants, selecting the x86_64 check even more directly.

Thus, when flake.nix declares aarch64-linux, it only means that output definitions exist for that system; it does not mean GitHub Actions has built aarch64 outputs. The example workflow has no matrix, native aarch64 runner, remote builder, or cross-architecture emulation, so it cannot be described as “CI has verified both architectures.”

4. Local, Actions, and runtime verification: keep three separate records#

4.1 Local checkout/flake evidence

Local evidence answers “does this checkout's Nix structure work?” The following is a blank record format, with no actual execution results:

checkout: <repo-path>
revision: <完整源码提交 SHA>
branch: <实际分支及远程对齐情况>
command: nix flake check --all-systems --no-build
result: <实际退出码与结果>
note: <实际 warning；无则明确记录无>

To connect local and remote results, additionally run all six package builds and the variants check, recording the Nix version, nixpkgs lock, system architecture, and whether uncommitted files were used. A successful local run cannot replace an Actions run, and successful evaluation cannot replace an actual build.

4.2 How to record GitHub Actions run evidence

Using main pushes and tag pushes as examples, verify each run's event, head SHA, job, and steps rather than relying on a green badge. This table is a recording template; it contains no actual successful results:

The following commands can read the relevant runs. Viewing logs does not require publishing permissions:

gh run list --repo OWNER/REPO --workflow nix.yml --limit 10
gh run view RUN_ID --repo OWNER/REPO
gh run view RUN_ID --repo OWNER/REPO --log-failed

A real run establishes only which steps the runner executed against the checkout for that event; it does not establish the local GUI state. Fcitx5, GTK, Wayland/X11, or compositor tests that are not written into the workflow do not happen automatically when a job turns green. This table does not claim that any run succeeded.

4.3 Desktop runtime evidence

Desktop evidence should come from a real user session. Record software versions, the loaded theme, Fcitx5/GTK candidate-window behavior, and the compositor's protocols and visual results. Whether a reload/restart is required depends on the actual installation method and software documentation; these actions are outside this article's CI.

• CI success does not prove that Fcitx5 ClassicUI has loaded the theme.
• CI success does not prove that fcitx5-gtk's GTK-embedded candidate window displays correctly.
• CI success does not prove that the blur protocols in Hyprland, niri, KWin, or GNOME are working.
• If you have tested only one desktop, mark the others as untested or theoretically supported, and specify separate conditions for subsequent verification.

Do not put source declarations, inferred protocol compatibility, the author's README descriptions, and behavior you actually measured into a single “verified” column. This anonymized example contains no findings of actual desktop testing.

5. Supply chain, releases, and current gaps#

5.1 Capabilities declared by the example configuration

5.2 Capabilities not declared by the example

The following describes the limits of the complete example workflow above, not the results of an audit of a real repository's contents:

• No strategy.matrix: this is not a multi-system/multi-architecture matrix.
• No actions/cache, Cachix cache configuration, or other caching steps: dependency caching cannot be claimed for this CI.
• No upload-artifact: there is no basis for claiming that failure logs, packages, or test reports are retained as artifacts.
• No concurrency: there is no policy to cancel outdated runs on the same branch.
• No workflow_dispatch, schedule, or release automation: manual/nightly checks and automatic publishing cannot be claimed.
• No dedicated Home Manager consumer tests, NixOS dry-build, lint/format job, or desktop runtime job.
• No Secrets, OIDC, deployment permissions, or publishing token; the example's read-only CI does not need them.

The absence of Actions cache configuration or Cachix upload steps does not mean Nix cannot fetch already-built outputs from default substituters. Distinguish Actions directory caches, Nix binary caches, and actual local builds.

5.3 Tags and Releases do not imply automated publishing

Use the ref API to distinguish lightweight and annotated tags. If the ref's object type is commit, it points directly to a commit; if it is tag, also read the tag object and verify its target.

gh api repos/OWNER/REPO/git/ref/tags/v0.2.0 --jq '.object.type'
gh release view v0.2.0 --repo OWNER/REPO

A Release is a platform publishing record associated with a tag; whether it contains assets must be checked separately. This example workflow only checks out the code, installs Nix, evaluates, builds, and checks contracts. It has no gh release create, release Action, or equivalent publishing step.

Therefore, if the check triggered by a tag push succeeds, you can say only “Nix CI passed for this tag.” A separate Release in the repository does not mean this workflow created it. Without upload/release steps, you cannot claim that artifacts were uploaded either. No factual claim is made here about a real repository's releases, Latest designation, or tag types.

5.4 Optional enhancements, not implemented in the example

If future requirements become “faster, broader, or publishable,” add the relevant capabilities separately. The following are design directions only and must not be retroactively described as the current state of this example:

1. Architecture coverage: Create a matrix for x86_64-linux and available aarch64-linux runners, or explicitly configure a remote builder. Each matrix cell must select and actually realize the corresponding system output, with a usable builder of the same architecture. For example, run nix build .#packages.aarch64-linux.both-blur; simply leaving --all-systems in a --no-build evaluation is insufficient.
2. Caching: Design cache keys around lock files, systems, and Nix inputs, and define cache invalidation and trust boundaries. A cache hit can improve speed, but cannot replace a build result.
3. Artifacts: Upload logs, nix path-info output, or reviewed packages only when genuinely needed for debugging or distribution. An artifact is not a release and should not contain credentials.
4. Home Manager checks: Add a minimal consumer flake to verify module imports, option evaluation, and a dry-build, and name it separately from the package output check.
5. lint/format: Add a separate job if the project establishes a Nix/Markdown formatting contract. The example has no such job, and its shell contract check should not be called lint.
6. Release automation: To have tags create releases, add a separate, protected release workflow that declares write permissions, tag conditions, asset sources, and approvals. Do not mistake the current tag-push check for a release pipeline.
7. Runtime smoke tests: Add Fcitx5/GTK/compositor verification to CI only when a controlled graphical runner, virtual compositor, or real test host is ready. Otherwise, continue recording it as a manual/host test after deployment.

6. Troubleshooting checklist#

6.1 First identify which layer failed

1. No trigger occurred: Confirm whether the event was a push or PR, and whether the commit includes .github/workflows/nix.yml. The example has no branch filters, so do not first assume it is restricted to main.
2. Checkout/Action failure: Inspect the Action version at the pinned SHA, the runner image, and permissions. Do not switch to a mutable tag to work around the failure.
3. Evaluation failure: Run nix flake check --all-systems --no-build locally at the same revision. First distinguish syntax/input/system-attribute errors from nonfatal warnings.
4. Build failure: Run the six outputs individually on the local machine, using nix build --no-link .#both-blur and the other output names, to determine whether the issue lies in a variant, dependency download, platform, or source file. Do not look only at flake check.
5. Contract-check failure: Compare the realized target derivation against flake.nix. Check for missing theme files, unintentionally exposed unselected themes, and incorrect blur/solid fields or SVG opacity.
6. Suspected aarch64 issue: A single green run cannot settle it. Select the corresponding system outputs in an environment with an aarch64 builder and actually build them, recording the builder, revision, and result separately.
7. Incorrect desktop appearance: First verify package contents and Fcitx5's search paths, then reload/restart Fcitx5 in a real session. Finally, record protocol and visual results separately for Hyprland, niri, KWin, and GNOME.
8. Treating a warning as a failure: unknown flake output 'homeManagerModules' is a possible evaluation warning. Track its implications for flake compatibility, but do not let it obscure a genuine nonzero exit code.
9. Asking whether publishing is automatic: Check the workflow for a release Action and write permissions. This example has neither, so query and report the tag check and Release separately. Do not assume that a Release exists or report automatic uploads.
10. Credentials involved: Do not copy tokens, Secrets, complete environment variables, or private runner information into logs, notes, or artifacts. This example has no evidence of Secrets/OIDC and does not need invented evidence of them.

7. How to interpret the results#

The value of this example is that it demonstrates a small but auditable Nix CI contract: pinned Action SHAs, a read-only token, unified push/PR triggers, evaluation followed by builds of six variants, and test/grep checks on the consumer-visible output structure. Its limits are equally clear: a single ubuntu-latest job that selects only x86_64-linux outputs on the assumed x86_64 runner, with no cache, artifacts, matrix, release automation, dedicated Home Manager tests, or desktop runtime verification.

When reading or reusing this configuration, the shortest accurate conclusion is:

A green run establishes that Nix CI passed. It does not establish that every system was built, that a release was created automatically, or that Fcitx5 or any desktop compositor has run.

For related concepts, revisit 04: Actions Fundamentals and Execution Model, 05: Configuring Actions CI, and 06: Actions Security and Troubleshooting. Consult GitHub Actions workflow syntax for workflow syntax, and GitHub Actions security hardening for the security rationale behind pinning third-party Actions to SHAs.

Version and source boundaries: This example preserves the source material's six-variant structure, Home Manager modules, and output-contract approach, while removing personal project identity and run records. The tag mappings for the pinned Action SHAs were checked; the workflow itself was not run during this edit. The third-party Nix installation Action's behavior is defined by its upstream documentation and the pinned commit. The Nix CLI is an evolving interface; this article's explanations of --no-build/--all-systems refer to the official manual consulted during verification.