Use this guide for performance-sensitive changes and benchmark reviews. Back a performance claim with a benchmark that ships in the same merge request. Correctness tests still ship separately from the benchmark. Run them before timing.
Performance-critical changes should start from an explicit hypothesis about what limits performance and how the proposed change reduces that cost. Ground the hypothesis in a cost model appropriate to the operation, considering the relevant computer architecture and compiler code generation. Where useful, use roofline or speed-of-light analysis to estimate the available improvement.
A lightweight model is often sufficient:
“This loop is bandwidth-bound; eliminating this temporary removes one write and one read per coefficient.”
For this example, assembly analysis can check whether those accesses disappear, while benchmarks test whether the reduction improves performance at the relevant sizes. Check the model's assumptions, including whether bandwidth limits the operation, whether the compiler already eliminates the temporary, and how the working set fits in cache. Scale the analysis to the change; a formal model is not required for every contribution.
The supported and contrib benchmark trees are separate, standalone CMake projects. They are not part of Eigen's main test build and both require Google Benchmark:
cmake -G Ninja -S benchmarks -B build-bench -DCMAKE_BUILD_TYPE=Release cmake --build build-bench --target <benchmark-target> cmake -G Ninja -S contrib/benchmarks -B build-contrib-bench -DCMAKE_BUILD_TYPE=Release cmake --build build-contrib-bench --target <benchmark-target>
When CMake finds CUDAToolkit, the contrib benchmark project automatically adds its GPU subdirectory. That subdirectory also needs a working CUDA compiler and a selected CUDA architecture. On a host with only a partial toolkit installation, configure CPU-only contrib benchmarks with -DCMAKE_DISABLE_FIND_PACKAGE_CUDAToolkit=TRUE or report the GPU configuration as unavailable.
Consult benchmarks/CMakeLists.txt and contrib/benchmarks/CMakeLists.txt for current targets and compile settings. CUDA benchmarks also have a standalone project and instructions in contrib/benchmarks/GPU/CMakeLists.txt. No CI job builds or runs benchmarks, so CI checks neither that a benchmark compiles nor that a performance claim holds. Build and run the benchmark locally. Report the measurement conditions this guide requires.
benchmarks/GPU/ times the kernels Eigen generates for GpuDevice (elementwise expressions, launch overhead, reductions, contractions and allocation) against hand-written kernels and vendor baselines. It is part of the supported benchmark project but builds only when the EIGEN_BENCH_CUDA option is on, so a default configuration builds only the CPU benchmarks:
cmake -G Ninja -S benchmarks -B build-bench-gpu -DCMAKE_BUILD_TYPE=Release -DEIGEN_BENCH_CUDA=ON \ -DEIGEN_BENCH_CPU=OFF -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc -DCMAKE_CUDA_ARCHITECTURES=native cmake --build build-bench-gpu ./build-bench-gpu/GPU/bench_gpu_elementwise --benchmark_repetitions=10 --benchmark_report_aggregates_only=true
The subtree holds bench_gpu_elementwise, bench_gpu_launch, bench_gpu_reduction (against CUB), bench_gpu_contraction (against cuBLAS) and bench_gpu_alloc.
CMAKE_CUDA_ARCHITECTURES=native needs CMake 3.24. With older CMake, name the architecture instead, for example 89.
The benchmarks measure device time with CUDA events around a batch of launches (eigen_bench::timeLaunches) and report the time per launch through UseManualTime(). Allocation benchmarks use UseRealTime() to measure the cost of the host API. The host_us_per_launch counter is the host-side cost of enqueueing a launch. The bytes_per_second counter counts every operand as read once and the result as written once. The benchmarks check their results against a reference outside the timed loop.
With every number you report, quote the GPU: line the binary prints (it is also in the JSON context). Say whether the clocks were locked; a laptop under WSL2 cannot lock them.
Put each benchmark family in its own translation unit, bench_<topic>.cpp in the module directory (benchmarks/LU/bench_lu.cpp). Register it in the CMakeLists.txt beside it with eigen_add_benchmark(<target> <source> [LIBRARIES ...] [DEFINITIONS ...]), which links benchmark_main, compiles at -O3 with NDEBUG, and takes the include path from the tree. Do not merge families into one file: combining them changes the code layout, and that alone has shifted timings of unchanged kernels by tens of percent.
For a multi-threaded benchmark, call UseRealTime() on the registration to measure elapsed time. The default CPU timer measures only the main thread and misses the worker threads. When total CPU consumption is also needed, add MeasureProcessCPUTime(), which includes those workers. See Google Benchmark's CPU timers.
bytes_per_second and items multipliers against the operation. A wrong count silently rescales every reported rate.DoNotOptimize and ClobberMemory where appropriate.Express static grids declaratively on the registration:
Args({a, b}) for individual points.Range, DenseRange, or Ranges for swept dimensions.ArgsProduct({{...}, {...}}) for Cartesian products.Do not use Apply(). Its callback takes a benchmark::internal::Benchmark*, a library-internal name that benchmark sources must not reference. A grid that seems to need Apply() can be written by listing its points with ArgsProduct or Args, or by registering several benchmarks.
uptime and stop or finish competing builds and compute-heavy work. Run only one benchmark process at a time; concurrent benchmarks invalidate both measurements.--benchmark_repetitions=10. Retain the raw results. Compare medians plus a dispersion measure such as MAD, IQR, or standard deviation; do not select the best run.A, B, A, B) to expose thermal or background-load drift. Use the same benchmark filter and arguments for each pair.When you cannot make the machine quiet enough to resolve the effect, use deterministic counters, which stay honest on a noisy machine: Callgrind instruction counts or allocation counts (e.g. -Wl,--wrap=malloc). Confirm that both variants produce identical result checksums. Report the counts as counter measurements that name the tool, not as timings. A counter measurement together with a statement that wall-clock timing was inconclusive is a complete performance claim; an unqualified ratio from a loaded host is not.
Never infer a general speedup from one convenient size or one warm run. State the tested domain. Include regressions as well as improvements. Keep numerical accuracy results separate from performance measurements.
A good merge request connects the hypothesis, the code change, and supporting evidence. Benchmark measurements establish the observed performance effect. Callgrind counts, assembly analysis, or both help test whether the change achieves the cost reduction the hypothesis predicted. For performance-critical changes, strongly prefer this complementary evidence even when timings are stable. Choose the tools that test the hypothesis; neither Callgrind nor assembly analysis is mandatory for every contribution.
Ir) for the affected operation using identical inputs, compiler flags, ISA, and a fixed number of iterations. Isolate the operation from startup, unrelated allocation, input generation, and benchmark calibration. Pausing the benchmark timer does not pause Callgrind collection. Report counts per operation, and name the region you measured. Do not compare whole-process totals from runs that did different amounts of work. Use optimized builds with debug information, so counts can be attributed to source. Retain the commands, tool version, and relevant callgrind_annotate output. See the Callgrind manual for collection controls. Label optional cache and branch simulation results as simulated events.provenance.md.Connect this evidence to the measured results in the contribution's performance summary. Instruction counts and assembly explain mechanisms. Fewer instructions alone do not establish a speedup on a particular CPU. Investigate results that contradict the hypothesis or disagree with timings. Report unresolved uncertainty. State the workloads and configurations over which the evidence supports the claim.
If the tools cannot analyze the relevant ISA or backend, say so and use the evidence that does apply. Do not silently switch to a different target and present its results as if they came from the original configuration.