Testing Eigen Changes

Use this guide when adding or changing tests. The checked-out source is authoritative:

Configure And Build

Configure a dedicated build directory. Unit tests are excluded from CMake's default all target, although a bare build can still build enabled auxiliary libraries.

cmake -G Ninja -S . -B build
cmake --build build --target buildtests
ctest --test-dir build --parallel --output-on-failure --no-tests=error

Useful aggregate targets are BuildOfficial, BuildContrib, buildsmoketests, buildtests_gpu, check, and check_gpu. Build and run one test explicitly when possible:

cmake --build build --target bdcsvd_3
ctest --test-dir build -R '^bdcsvd_3$' --output-on-failure --no-tests=error

Run the generated wrappers from the build directory because they invoke the configured build tool relative to their working directory:

cd build
./buildtests.sh <regex>
./check.sh <regex>

They filter registered parent names such as bdcsvd, not generated part names such as bdcsvd_3. For one part, use the explicit target recipe above.

Use a separate build directory for each materially different configuration. Do not rewrite one cache and describe the result as a second test run.

cmake -G Ninja -S . -B build-row-major -DEIGEN_DEFAULT_TO_ROW_MAJOR=ON
cmake -G Ninja -S . -B build-no-vector -DEIGEN_TEST_NO_EXPLICIT_VECTORIZATION=ON

Consult the top-level CMakeLists.txt and nearby test CMake files for current options instead of copying an option inventory into documentation.

Current Test Framework

Eigen currently uses its own framework, not GoogleTest:

  1. Add test/<name>.cpp or contrib/test/<name>.cpp.
  2. Include main.h, then the public umbrella header for tests of public behavior. A focused test of a private utility may include its implementation header only when that matches an established nearby pattern. Never present such a path as a user include.
  3. Use VERIFY, VERIFY_IS_EQUAL, VERIFY_IS_APPROX, and the other helpers exposed through test/main.h.
  4. End with EIGEN_DECLARE_TEST(<name>) { ... }.
  5. Register the source with ei_add_test(<name>) in the matching CMakeLists.txt, then reconfigure.

Keep test/main.h limited to framework configuration, registration, shared-helper aggregation, and the test driver. Put reusable utilities in a narrowly named helper header. Include it from main.h only when most tests need it.

For compile-failure coverage, use the established failtest/ pattern. Its _ok target must compile and its _ko target must fail with EIGEN_SHOULD_FAIL_TO_BUILD defined. Register it with ei_add_failtest above the closing ei_add_failtest_fixture() call. The buildfailtests fixture builds the whole suite at once. _ko passes when its own target failed to build and _ok built. Because _ok built, the toolchain works, but the _ko failure can still be a different compile error from the intended one. Keep the failing construct narrow.

Split Tests

ei_add_test scans the source for CALL_SUBTEST_N, EIGEN_TEST_PART_N, and EIGEN_SUFFIXES;... markers.

  • With EIGEN_SPLIT_LARGE_TESTS=ON, every discovered suffix becomes an executable <name>_<N> compiled with EIGEN_TEST_PART_<N>=1; the parent <name> target builds all parts.
  • EIGEN_SUFFIXES;... supplies an explicit suffix list when ordinary source scanning cannot see macro-generated or conditional parts.
  • With splitting off, tests containing only CALL_SUBTEST_N or EIGEN_SUFFIXES fold into one <name> executable compiled with EIGEN_TEST_PART_ALL=1.
  • An explicit EIGEN_TEST_PART_N marker forces splitting even when the option is off. If any such marker is present, all suffixes discovered in that source are emitted.
  • cmake/EigenTestPartGroups.cmake lists ranges of parts that compile together as one executable. The executable takes the name of the range‘s first part, e.g. array_cwise_1 for 1-4, and ctest -R '<name>' still selects it. Every part in a range must differ from the others only in its CALL_SUBTEST_N calls, and the smoke list must not name it individually. The range’s compile must stay under the 4 GiB peak RSS recorded in the file header. After adding or renumbering subtests inside a listed range, re-measure that compile, because the change makes it bigger.

ctest -R '^<name>$' does not match split parts. Use ctest -R '<name>' to run every part, or anchor a single generated name such as '^<name>_3$'.

After changing subtest registration, reconfigure and read back the generated target list. Two failure modes are silent. A subtest function whose CALL_SUBTEST call was dropped still compiles and looks like coverage. And under EIGEN_SPLIT_LARGE_TESTS=ON, a part invoked only through a dispatch macro is not built unless an EIGEN_SUFFIXES marker lists it.

Coverage That Can Fail

A test that passes when the change is reverted is not coverage. Establish that the test fails at the parent commit. When that is impractical, show that the test runs the new code by construction.

  • Test a new fast path through the public entry point that selects it, with inputs that actually take it, not only through a direct call to the new method. Where a flag or trait selects the fast path, pin the selection with a STATIC_CHECK on it in both directions: for a type that must opt in and for one that must stay out.
  • Cover the branches the change adds, not just one convenient shape:
    • sizes that are not a multiple of the packet or block dimension;
    • complex scalars where conjugation is otherwise a no-op;
    • both storage orders;
    • the uncompressed or strided variants of an input type.
  • Verify the complete result against an independent reference, including coefficients the test setup did not write. Corruption in those coefficients goes unnoticed when the check skips them.
  • Exercise the documented customization points, such as custom scalars and functors without declared traits, not only the built-in specializations that happen to satisfy a new precondition.

Build-System Tests

test/buildsystem holds the coverage for Eigen‘s own CMake surface: what an install tree contains, what find_package(Eigen3) and the version ranges in cmake/Eigen3ConfigVersion.cmake.in accept, and how an embedding project opts out of Eigen’s install rules. doc/TopicCMakeGuide.dox makes these claims to users, and nothing else checks them: the blocking documentation job builds the docs but does not run what they describe.

Not every scenario checks a documented claim. Others cover CMake behavior that no other test exercises: a find module that has to survive a second configure of the same build tree, or the wiring that routes a compiler launcher into a test's compile command.

cmake -G Ninja -S . -B build -DEIGEN_BUILD_TESTING=ON
cmake -E chdir build ctest -L buildsystem --output-on-failure --no-tests=error

Pass --no-tests=error to every ctest invocation, as this guide does, because CTest otherwise exits 0 when nothing matched. Nothing matches in three common cases: an anchored -R '^name$' against a split test, a mistyped name, and --test-dir under CMake 3.17 to 3.19. Those versions predate --test-dir; they ignore it and inspect the source directory instead. The cmake -E chdir form above works on them too.

No target needs building first: each scenario runs its own nested configure, build, and install into the CTest binary directory. To add a claim, drop a scenario in scenarios/ and name it in the list in test/buildsystem/CMakeLists.txt. The driver run_scenario.cmake supplies the assertion helpers.

Two hazards are specific to these tests. First, Eigen calls export(PACKAGE Eigen3), so CMake's user package registry names every Eigen build tree on the machine. A find_package scenario must therefore disable both the user and the system package registries and assert that the package came from the prefix it installed; otherwise it passes without reading that prefix at all. Second, CMake code registers these tests. If a guard around that code stops matching, CTest finds no tests instead of reporting a failure, so the CI job runs ctest with --no-tests=error.

Configurations The Test Suite Cannot See

  • In the default host-test configuration, no test compiles an EIGEN_NO_DEBUG code path: test/main.h undefines NDEBUG, and Macros.h derives EIGEN_NO_DEBUG from it. (HIP/SYCL device compilation and an explicit -DEIGEN_NO_DEBUG define it independently.) For behavior that depends on the macro, add a dedicated -DEIGEN_NO_DEBUG test target or a standalone -DNDEBUG check. Conversely, the compiler type-checks the condition inside an eigen_assert only where assertions are enabled. A condition that calls a member its argument type lacks still compiles in every release build.
  • When layout is in play, run an EIGEN_DEFAULT_TO_ROW_MAJOR build. Where a test aliases one object's storage through a view whose default layout is fixed, pin the layout explicitly.
  • When the change reasons about packets, alignment, or index width, cover EIGEN_TEST_NO_EXPLICIT_VECTORIZATION, EIGEN_UNALIGNED_VECTORIZE=0, or a narrower EIGEN_DEFAULT_DENSE_INDEX_TYPE.
  • Tests build optimized (CMAKE_BUILD_TYPE defaults to Release), and no CI job builds Debug. In C++14 an odr-used static constexpr member of a class template needs a namespace-scope definition. Without one, every CI build links and only an -O0 build fails; see conventions.md. When adding such constants, build one Debug tree.
  • Compiler fast-math coverage comes only from the targets that test/CMakeLists.txt registers with fast-math flags. The smoke list includes packetmath_fastmath, packetmath_fastmath_generic_16 where vector extensions are available, bfloat16_classification_fastmath, and parts of fastmath, bdcsvd_fastmath, and stable_norm_fastmath; these compile with -ffast-math where supported. Ordinary packetmath uses Eigen's EIGEN_FAST_MATH=1 approximation switch, which does not enable the compiler flag. When a changed path falls outside the existing fast-math tests, add focused coverage. numerics.md records the special-value hazards.

Numerical Assertions

VERIFY_IS_APPROX is a convenient broad comparison, not a machine-epsilon guarantee. test_precision<T>() uses NumTraits<T>::dummy_precision() generically and currently specializes float to 1e-3 and double/long double to 1e-6. Do not use it alone to claim ULP accuracy, backward stability, or IEEE special-value conformance.

For numerical kernels, add explicit named bounds based on epsilon, dimension, conditioning, or a backward-error model as appropriate. When their distinction matters, check NaN, infinity, and signed zero explicitly. Follow numerics.md for solver, packet, and scalar-math coverage.

Write such a bound as factor * NumTraits<RealScalar>::epsilon() at the site, and explain factor by its error model. Do not introduce tolerance wrapper helpers: the raw form is the established idiom across test/ and contrib/test/, and it shows the computation itself, so a bound like 10 * n * eps * A.norm() reads as the formula it is. A bare decimal literal is worse than opaque: 1e-9 demands impossible accuracy from a float instantiation.

Two kinds of comparison silently accept everything, and both have shipped here. The first is a tolerance computed by the operation under test: a bound formed as (A.cwiseAbs() * B.cwiseAbs()) goes through the product code being tested, so accumulate it independently instead. The second is a comparison that admits non-finite values: error <= tolerance holds for two infinities, and if (error > bound) never fires for a NaN error. Test for failure as the negation of the passing condition, !(error <= bound). Reject a non-finite tolerance.

When a numerical check fails for some seeds, find out whether the computation or the check is at fault before changing the tolerance. Compare the results with a reference computed in higher precision (quad or MPFR). Measure the backward error, and the forward error relative to the first-order condition bound, across enough seeds to see the tail:

  • A result worse than its conditioning allows is an accuracy defect. Fix the algorithm; a wider tolerance would hide it.
  • If a result is within that accuracy and the check still fails, the check asks for more than the working precision can deliver. Derive the tolerance from the conditioning rather than a flat factor.
  • Solving the same inputs in the next wider type proves neither. The wider type resolves what the working precision cannot, such as a tight cluster of roots that the narrower type can only locate to within a wider set.

An accuracy defect and an over-strict check can both be present. In the polynomialsolver flake, the companion eigenvalues had backward errors of 1.8e4 eps. Once every root's backward error was below one eps, the flat 3.16% check still failed at the same rate. Land such a computation fix and test fix as independent merge requests. In the computation fix, state whether it removes the failure, with seed sweeps against the parent commit as evidence.

Run reproducible failures directly with a fixed seed and repeat count:

EIGEN_REPEAT=10 EIGEN_SEED=1 build/test/foo_3
build/test/foo_3 r10 s1

External BLAS And Shim Libraries

EIGEN_TEST_EXTERNAL_BLAS=ON finds a system BLAS, defines EIGEN_USE_BLAS, and links that BLAS into applicable official tests. With it off, ordinary tests exercise Eigen‘s normal implementation; they do not transparently use the in-tree eigen_blas library. EIGEN_BUILD_BLAS and EIGEN_BUILD_LAPACK separately build Eigen’s ABI shim libraries. Some optional sparse backends also link against those shims. There is currently no EIGEN_TEST_EXTERNAL_LAPACK option.

Report the exact targets, CTest regexes, configurations, compiler, and seeds run. Also report relevant hardware or optional backends that were unavailable locally.