blas-microbenchmark1.11.0-dev
Menu

bmb_mcp

microbenchmark, for AI agents

bmb_mcp [--results-dir dir] [--time-cap seconds] [--memory-limit bytes]

bmb_mcp lets an AI agent use the benchmarks of blas-microbenchmark(1): list the routines, describe the machine, run a benchmark or the suite, read a result and make the report. It is a Model Context Protocol server: the agent's client starts it and talks to it on its standard input and output, a JSON-RPC 2.0 message a line, and it is not run by hand.

A tool's arguments are fields, never a command line: a routine, its sizes and threads, its arguments (trans, uplo, ...), the run's options. Every run is checked before it starts, and refused with a message that says what to do instead (see LIMITS); every result is verified against a reference, as the benchmarks do by default.

bmb_mcp finds the benchmarks as bmb_suite(1) does: those of the build tree it is in, or those installed with it, in /usr/local/libexec/blas-microbenchmark. It needs Python 3, and nothing else.

Where the results and the reports go, created if need be, and the only place a tool reads from or writes to. Default: ./bmb-results.
The longest a run may take: one that goes past is stopped, and the tool says so. Default: 900.
The most memory a run's operands may take. Default: a quarter of the memory the system says is available.
Show a summary of the usage.
Show the version.

The routines, filtered by level, precision or family: each one's level, precision, family, and what -v, -m, -M and -K are for it.
What --version says: the BLAS library and its version, the kernels it chose, the CPU, caches, frequency and OS; with the CPUs this process may use, the memory limit, the results directory and the time cap.
One routine's benchmark, with its sizes (vector_size, dim1, dim2, dim3), threads, the routine's arguments, and the run's options (layout, cache, iterations, min_time, label, peak_gflops, ...), each named as the long option it stands for. The result goes into the results directory as JSON, named routine-n.json or as name; the tool returns a row a point (threads, sizes, time, median, IQR, GFLOP/s, GB/s), whether it was verified, and the warnings.
bmb_suite(1) into a directory of the results directory: every routine, or some levels, precisions or routines, at a profile's sizes, for one library or several (blas, flexiblas). It returns a line a result, with its best rate, and the benchmarks that failed.
A result file of the results directory, summarised as run_benchmark does.
bmb_report(1) on a directory of the results directory, a peak given or not; it returns the page's path.

Each is checked before anything runs.

  • A run's arguments are those the benchmark takes: they are first given to it with BMB_COUNTS, which checks them and counts each point's operands without measuring or allocating anything.
  • The largest point's operands fit within the memory limit.
  • No thread count is above the number of CPUs the process may use.
  • Every path, and every name given, is inside the results directory, and a result already there is not overwritten.

run_suite's blas names suites by their paths, and runs their benchmarks: they are programs, which bmb_suite(1) reads the backend of by running them. Configure the client to ask before it calls run_suite, or give it only paths of installs you trust.

A run that goes past the time cap is stopped. A benchmark that fails, a wrong result included, is the tool's error, with the benchmark's message: the server goes on.

A client that reads its servers from JSON, such as Claude Code's .mcp.json, starts it with:

{"mcpServers": {"blas-microbenchmark":


    {"command": "bmb_mcp", "args": ["--results-dir", "bmb-results"]}}}

0 when the client closes its end; 2 on a usage error.

blas-microbenchmark(1), bmb_suite(1), bmb_report(1)