Use this guide when changing .gitlab-ci.yml, the job definitions under ci/, the test selector scripts/affected_tests.py, the pass cache ci/scripts/test_cache.py, or the clang-tidy driver. ci.md is the consumer's view of the same machinery; the checked-out files are authoritative where the two disagree.
Both selector scripts, affected_tests.py and test_cache.py, fail closed: an error or an unknown case either fails the job or widens the selection, never silently narrows it. A wrong answer, however, goes unnoticed: a job that skips too many tests still reports success. Their unit tests are therefore blocking and run on every merge request in checkformat:lint:
python3 scripts/test_affected_tests.py python3 ci/scripts/test_test_cache.py
Build jobs publish the configured build directory as an artifact. Their paired test jobs consume that artifact and run CTest without rebuilding. When changing either side, keep the test job's needs, CTest label or filter, and the corresponding build target consistent; otherwise CTest can discover tests whose executables are absent.
Publishing is opt-in per job rather than inherited: the bases .common:linux:cross and .common:windows have no artifacts: key. To make a job publish, add .artifacts:linux:builddir, .artifacts:windows:builddir or .artifacts:test:results as a second extends: parent. Give a test job the results template, not a build-directory template. A test job links nothing, and the build directory it downloaded is already the build job‘s artifact. The results template also registers JUnitTestResults_*.xml through artifacts:reports:junit:. That registration puts failures in the job’s Tests tab and in the merge request widget, not only in the log. A job that needs neither, such as test:linux:buildsystem, extends the base alone and publishes nothing.
A job that failed and then passed on retry still reports the failed first attempt. The report is converted from the dashboard file Test.xml, and the retry runs --rerun-failed without -T test, so it never rewrites that file. The job exits 42 to mark the soft failure. The merge request widget can compare results only against a base-branch report from a job with the same name. Default-branch pushes run only a small subset of jobs, so most jobs show a summary without a comparison.
In merge request pipelines, the Linux test jobs keep a content-addressed pass cache in .testcache/, with one GitLab cache per job name. test.linux.script.sh skips a test when an earlier merge request pipeline recorded a pass on the first attempt for the same executable, emulator, CTest definition, and environment fingerprint. The fingerprint covers the image, the lib* package state, ci/scripts/ and ci/docker/, and variables that change behavior, such as EIGEN_REPEAT and QEMU_CPU. After the run, the script calls test_cache.py to record the tests that passed on the first attempt. It reads their statuses from the dashboard run's Test.xml.
Scheduled and web pipelines always run every test their job selects. Each of their runs draws fresh random seeds from the clock, and those seeds are part of what they test. Sharded jobs never skip, and EIGEN_CI_TEST_CACHE: "off" opts a job out. Skipped tests are absent from that run's JUnit report.
The fingerprint hashes ci/scripts/ and ci/docker/ but not the ci/*.gitlab-ci.yml files. Every YAML setting that can change a test's outcome already enters the key by value:
KEYED_ENV_PREFIXES;CI_JOB_IMAGE;If the key included the YAML files, any edit to them would discard all recorded passes. Leaving them out has two consequences:
KEYED_ENV_PREFIXES. Setting it in the YAML alone does not put it in the key.tags:. The key cannot tell two hosts in the same tag pool apart either.The affected-tests and all-tests labels each turn off the smoke jobs (.rules:libeigen:smoketest). Both tiers test more deeply than the fixed smoke list, on the same native runners, so the smoke jobs would add cost but no coverage. The smoke jobs are turned off only in the libeigen namespace, because the affected and full tiers have no jobs in a fork. In merge request pipelines, .rules:libeigen:nvhpc runs the NVHPC build and test pair only when the nvhpc-tests label is set. When the pair was part of the all-tests matrix, its compiler frontend used roughly a quarter of the project's hosted-runner minutes.
Under affected-tests, either of two independent triggers adds a platform beyond the four unconditional jobs: a change under the platform's backend directory, detected through rules:changes:, or a label, read from $CI_MERGE_REQUEST_LABELS. GitLab ANDs if: with changes: within one rule entry, so each rule set has two entries, one per trigger. Each rule set tests the whole label string on its own, so several labels select the union of their platforms. The *-tests labels are unscoped so that a merge request can carry several of them. GitLab makes scoped labels (backend::NEON) mutually exclusive, so it would allow only one.
The rule sets for arch/SVE and arch/SME also list Eigen/src/Core/util/ConfigureVectorization.h in changes:, because that header decides whether either backend is compiled at all. SVE runs one build per vector length. Eigen takes EIGEN_ARM64_SVE_VL from __ARM_FEATURE_SVE_BITS, and only -msve-vector-bits sets that macro. test/sve_vector_length reads RDVL, so a binary run at another width fails instead of computing wrong answers.
all-platforms leaves out three rows of the platform table in ci.md because their jobs ignore the selection (the list of affected tests that select:tests computes). The AVX512-FP16 pair and the SME build are compile-only, with no paired test job, and the GPU jobs build buildtests_gpu. SME gets compile coverage rather than the selection. Its test jobs, one per streaming vector length (SVL), already run a curated subset of targets through EIGEN_CI_CTEST_REGEX, and the selection would conflict with that subset. The GPU row adds jobs from outside the tier through the affected-tests entry in .rules:libeigen:gpu. gpu-tests already triggers those jobs on its own, so it combines with affected-tests without a second rule entry. When adding a runner for a backend that has none (ZVector, MSA, HVX, HIP, SYCL), add its trigger to these rule sets too.
select:tests writes affected/targets.txt and affected/ctest_regex.txt. The paired build and test jobs on Linux and Windows read them through EIGEN_CI_BUILD_TARGET_FILE and EIGEN_CI_CTEST_REGEX_FILE. On Windows, the readers are build.windows.script.ps1 and test.windows.script.ps1.
The selector follows the textual #include graph and ignores preprocessor guards, so it selects a strict superset of the tests that actually compile a changed file. Changes to CMake, ci/scripts/, ci/docker/, or the BLAS/LAPACK shims force the full suite, because they invalidate the mapping itself. The ci/*.gitlab-ci.yml files only orchestrate jobs and cannot change which test includes which header, so they select nothing. Neither do the clang-tidy and lint images under ci/tidy/ and ci/lint/. Git rename detection is disabled for the input diff, so the selector sees both the old and the new path of a moved file. An old path absent from the current graph safely forces the full suite.
The selector maps sources to targets by reading the test registrations in the CMake files. The registrations include executables built from several translation units, and the GPU tests. GPU test sources are .cu, because ei_add_test takes the extension from EIGEN_ADD_TEST_FILENAME_EXTENSION. The selector treats a changed test source without a registration as an error; it does not drop the source as an unconfigured target. The selector skips test/buildsystem/. The consumer projects there are separate CMake projects, and only test:linux:buildsystem configures them. An add_executable in them is not a test registration, and no test in the main project depends on their sources.
Some targets need an optional dependency such as CHOLMOD, CUDA or SYCL and are absent from configurations without it. The build script filters the selection against ninja -t targets after the CMake configure, because ninja aborts on an unknown target. A selection of only such targets builds nothing and is not a failure. A missing selection artifact, however, must fail the job rather than fall back to the default target, which would silently build everything. A job reads a NONE selection before the toolchain setup and the configure step, so a merge request that affects no test costs a checkout rather than a full configure. rules: cannot keep such a job out of the pipeline, because GitLab evaluates them when it creates the pipeline, before select:tests runs. Only a child pipeline generated from the selection could skip the job entirely.
The build script shuffles the remaining targets and builds them in batches. Before that, it expands each target through ninja‘s phony edges into the targets it aggregates. Most selected names are aggregates: buildtests, and the parent target of every split test, which builds all of that test’s parts. Batching limits memory pressure by spreading the targets it is given across batches. An unexpanded parent target would put a whole test family into one batch and defeat that limit.
Two kinds of registration need special handling. First, buildtests aggregates only the ei_add_test targets, so in full-suite mode the selector names a bare add_executable, such as the bug1213 link regression, explicitly next to buildtests. Second, the compile-failure suite under failtest/ is EXCLUDE_FROM_ALL and is compiled at test time, so the selector picks those tests by their CTest names, <name>_ok and <name>_ko, and never hands them to the build job. Both matter because a -R filter silently drops whatever it does not name. The unfiltered runs in the other tiers include them automatically.
The buildfailtests fixture compiles the failtests. Every _ok and _ko test requires that fixture, so CTest adds it to any run that selects one of them, even if -E excludes it. The fixture builds every failtest target in one cmake --build that keeps going after errors. Separate builds, one per test, would have to run serially: concurrent builds in one binary directory collide whenever a CMake regeneration is pending. On the hosted runners, that serial suite took 80-95% of an affected test job's wall time.
The tests themselves only check which executables exist. _ok passes when its own executable exists. _ko passes when its own executable is missing and its _ok twin's exists. A missing compiler therefore fails both tests instead of making _ko pass. When a job excludes the failtests by name from an “ALL” selection, it must exclude ^buildfailtests$ as well, because an “ALL” selection applies no -R.
The RISC-V affected tier runs the failtest label on an amd64 job with the original cross compiler. Its native runtime job excludes those compile tests and the nested buildsystem scenarios: the runtime image has neither Ninja nor a compiler, and the cached compiler paths name amd64 executables. The separate test:linux:buildsystem job covers the nested consumers when build-system files change.
For a source in the compilation database, the driver first narrows the database with tidy_compile_db.py. A split test has one database entry per EIGEN_TEST_PART, or per range of parts that cmake/EigenTestPartGroups.cmake compiles together. clang-tidy parses the file once for each entry that names it: 14 times for test/array_cwise.cpp, which alone uses up the job's timeout. The narrowed database keeps one entry per distinct compiler configuration. Within a configuration that is split into parts, it keeps the entries whose parts actually compile the added lines. A line inside a CALL_SUBTEST_<n>(...) call or an #if defined(EIGEN_TEST_PART_<n>) guard needs part <n>; any other line can be checked under any part. The driver prints what it left out next to the file name, so a run that hits the cap names the parts it did not check instead of reporting the file clean.