blas-microbenchmark1.11.0-dev

Install

From a release, with Spack, in a container or from git; against the BLAS library of your choice.

You need a C compiler (GCC 12 or newer, or a recent Clang), make, and a BLAS library with its development files: OpenBLAS is the simplest, libopenblas-dev on Debian and Ubuntu, openblas-devel on Fedora. To compare several libraries, see The BLAS libraries.

Take blas-microbenchmark-<version>.tar.gz from the latest release (not GitHub's "Source code" archives, which have no configure), then:

tar xf blas-microbenchmark-<version>.tar.gz
cd blas-microbenchmark-<version>
./configure
make
make check
sudo make install

make check runs the tests, every benchmark among them, against the library you built with: it takes a minute or two. ./configure --prefix=$HOME/.local installs elsewhere than /usr/local, without sudo.

Each release tarball is attested by the workflow that built it, in this repository. With the GitHub CLI:

gh attestation verify blas-microbenchmark-<version>.tar.gz --repo MathieuCAISSA/blas-microbenchmark

Or, from 1.3.0, without it, with cosign and the .sigstore.json bundle attached to the release:

cosign verify-blob-attestation --type slsaprovenance1 \
  --bundle blas-microbenchmark-<version>.tar.gz.sigstore.json \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity-regexp '^https://github\.com/MathieuCAISSA/blas-microbenchmark/\.github/workflows/release\.yml@refs/(tags/v|heads/main$)' \
  blas-microbenchmark-<version>.tar.gz

sha256sum -c SHA256SUMS checks it against the release's checksums.

Without an option, configure looks for a library with a CBLAS interface, OpenBLAS first. --with-blas-backend names one:

./configure --with-blas-backend=blis

The choices are openblas, blis, aocl, mkl, nvpl, armpl, netlib and flexiblas; The BLAS libraries says where each is looked for, and how its threads are set. To measure several, build once per library, each in its own directory:

mkdir build-blis && cd build-blis
../configure --with-blas-backend=blis
make

A library outside the usual places is found through the variables a module sets (OPENBLAS_ROOT, BLIS_INCDIR, MKLROOT, AOCL_ROOT, …), or named on the command line, which wins over everything else:

./configure --with-blas-backend=openblas --with-blas-incpath=/opt/openblas/include --with-blas-libpath=/opt/openblas/lib

WhatWhere (with the default prefix)
The benchmarks, by BLAS level/usr/local/libexec/blas-microbenchmark/level{1,2,3}
bmb_suite, bmb_report and bmb_mcp/usr/local/bin
The manual pages/usr/local/share/man
The ReFrame tests/usr/local/share/blas-microbenchmark/reframe
The skills, for an AI agent/usr/local/share/blas-microbenchmark/skills

The benchmarks are not on the PATH, so as not to crowd it. To run them by name:

BMB=/usr/local/libexec/blas-microbenchmark
export PATH="$BMB/level1:$BMB/level2:$BMB/level3:$PATH"

sudo make uninstall, from the same directory, removes all of it.

The repository is a Spack package repository; Spack builds the BLAS library you name, or uses yours:

spack repo add https://github.com/MathieuCAISSA/blas-microbenchmark.git
spack install blas-microbenchmark ^openblas
spack load blas-microbenchmark

spack load puts the benchmarks on the PATH. The library can be ^openblas, ^blis, ^amdblis, ^intel-oneapi-mkl, ^nvpl-blas, ^armpl-gcc, ^netlib-lapack or ^flexiblas; spack test run blas-microbenchmark checks the install.

From 1.6.0, each release's image is on GitHub's registry, for x86_64, built against OpenBLAS (tags latest, <version> and <version>-openblas), BLIS (<version>-blis) or Netlib (<version>-netlib), and attested like the tarball. The benchmarks are on its PATH; it writes to /results, here the current directory:

IMAGE=ghcr.io/mathieucaissa/blas-microbenchmark:latest
gh attestation verify oci://$IMAGE --repo MathieuCAISSA/blas-microbenchmark
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/results" $IMAGE bmb_dgemm -m 64:1024 -o dgemm.json

Apptainer, which most HPC sites run, takes the same image, and writes where you are, as you:

apptainer exec docker://ghcr.io/mathieucaissa/blas-microbenchmark:latest bmb_dgemm -m 64:1024 -o dgemm.json

To build an image yourself, from a git clone, against OpenBLAS, BLIS, Netlib or Intel MKL (which is not published, since the image would redistribute Intel's libraries):

docker build -f packaging/docker/Dockerfile --build-arg BLAS_BACKEND=mkl -t blas-microbenchmark:mkl .

A container runs on the host's CPU, which is what the results record. docker run --cpus and --cpuset-cpus leave the benchmark fewer CPUs than it sees: -t beyond those gets a warning, since the threads would take turns. Docker Desktop runs containers in a virtual machine, whose own limits the numbers include.

A clone has no configure yet: ./autogen.sh makes it, with autoconf, automake and libtool. Then as from a release:

git clone https://github.com/MathieuCAISSA/blas-microbenchmark.git
cd blas-microbenchmark
./autogen.sh
./configure && make && make check

Next: your first benchmarks.