blas-microbenchmark1.11.0-dev

Your first benchmarks

Run a routine, read what it prints, sweep sizes and threads, save the results.

Each BLAS routine has its own benchmark, bmb_<routine>: bmb_dgemm for the matrix product, bmb_ddot for the dot product, and so on, 113 in all. They all take the same options, but for the arguments a routine does not have (--trans on ddot), which it refuses.

bmb_dgemm -m 512:2048

This multiplies square matrices of 512, 1024 and 2048 rows, each size timed ten times, and prints one row per size as it goes:

# blas-microbenchmark 1.10.1
# backend: openblas (OpenBLAS 0.3.26 NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=64)
# kernels: Haswell
# verified: every point, against a reference, before it was timed
# layout: row-major
# cpu: Intel(R) Core(TM) Ultra 7 155U (14 logical CPUs, 1 NUMA node)
# caches: L1d 48K, L1i 64K, L2 2M, L3 12M
# os: Linux 6.6.87.1-microsoft-standard-WSL2 x86_64
# date: 2026-11-05T09:23:49Z
# affinity: CPUs 0-13
# command: bmb_dgemm -m 512:2048
# run: 0.2 s and 1 call of warm-up, 10 samples, batch size chosen per point, hot cache
# operands: aligned to 64 bytes, leading dimensions + 0
# arguments: trans NN, batch_count 1, batch_mode loop
# compiler: gcc 13.3.0, CFLAGS -g -O2 -fopenmp
# routine: dgemm
Thread count    Matrix dim1 (M=K)   Matrix dim2 (N)     time [s]        GFLOP/s       median [s]      IQR [%]
1               512                 512                 0.004127511     65.036        0.004204376     7.3
1               1024                1024                0.034874541     61.577        0.035437426     4.1
1               2048                2048                0.279438241     61.480        0.282298940     2.4
  • The lines in # say where the numbers come from: the library, its exact build and the kernels it chose, the CPU and its caches, the system, the date; then how the run was made: the CPUs it could use, the command, the samples, the operands, the routine's arguments (here GEMM's defaults: no transpose, one matrix), the compiler, the thread and placement variables set. They go with the results into every file, so that a result is never mistaken for another machine's, and can be measured again.
  • verified: each point was computed once and checked against a reference before it was timed. A library that gets it wrong stops the run, with exit status 2, instead of looking fast (Checking results).
  • time [s] is the time of one call, from the fastest of the ten samples (each one a batch of calls, averaged); GFLOP/s the rate it makes, from the routine's operation count. Level 1 and 2 routines, limited by memory, also give GB/s (How a number is measured).
  • median [s] and IQR [%]: the median of the samples, and how far apart the middle half of them were, as a percentage of it. A large spread says something else was running.

The Outputs page shows real runs of this version, in each format.

Level 1 routines work on vectors, sized with -v; level 2 and 3 on matrices, sized with -m, and -M for a second dimension. Each takes a sweep:

SweepMeasures
4096that one size
256:4096doubling: 256 512 1024 2048 4096
1000:4000:1000a step: 1000 2000 3000 4000
64,1000,4096exactly those
bmb_ddot -v 1024:16777216
bmb_dgemv -m 256:4096 -M 256:4096

Without -M, matrices are square; with it, every combination of the two is measured, which bmb_report draws as a heatmap. The overview says which dimension is which for each routine.

-t sets the number of threads the library uses, and sweeps like a size:

bmb_dgemm -m 2048 -t 1:8

It wins over the library's own variables (OMP_NUM_THREADS, OPENBLAS_NUM_THREADS, …), with a warning when they disagree; without -t, those apply. Where threads run, and on which memory, matters as soon as there are several: Threads and pinning them.

-o saves to a file as well, JSON or CSV by its extension. JSON is what bmb_report reads:

bmb_dgemm -m 512:4096 -t 1:8 -o results/dgemm.json

-l labels a run, to tell two conditions apart on the same machine, and -s adds the mean, the standard deviation and the slowest of the samples:

bmb_dgemm -m 512:4096 -s -l "turbo off" -o results/dgemm-turbo-off.json

  • The CPU frequency can change during the run: the frequency governor is not performance, or turbo is on, so timings vary from run to run. The warning gives the commands that pin the frequency; the run goes on, and the result records it.
  • The run uses up to N threads, but this process may only run on M CPUs: an affinity (taskset, a Slurm job) or a CPU quota (a container) leaves fewer CPUs than -t asks for, and the threads would take turns.
  • … is ignored! Set to N but option -t is set to M: a thread variable of the library disagrees with -t, which wins.

Next: compare libraries, or every option in Options.