Getting Started

Name: poly-bench
Author: poly-bench

Get started with poly-bench in just a few steps.

Overview

poly-bench is a multi-language benchmarking framework that lets you define benchmarks once and run them across Go, TypeScript, Rust, and more with unified output and fair comparisons:

Define once, run everywhere — Write one .bench file; poly-bench compiles to native code in each target language.
Fair comparisons — Same iterations, same data, same warmup strategy across all languages ensures apples-to-apples results.
Rich output — Console tables, markdown reports, JSON for CI, and SVG charts from a single run.
Developer experience — Custom DSL with LSP support, auto-calibration, and portable fixtures.

You can learn more in Why poly-bench.

Installation

Install poly-bench using the native binary installer:

$curl -sSL https://polybench.evm-tooling.tools | bash

For custom install paths, unreleased commits, and security notes, see Installation.

Quick Start

Install the poly-bench VS Code extension (recommended)

Use VS Code. We recommend VS Code over Cursor for poly-bench development — the extension is not yet available on the Cursor marketplace.

You can find the extension on the VS Code Marketplace. It provides, out of the box:

Full syntax highlighting
LSP support
Hover documentation
Code completion
Linting
Formatting

1. Define your bench suite

Initialise a new Polybench project by running the init CLI command. you can do so by copying the command below into your terminal

$poly-bench init my-benchmarks --languages go,ts,rust

Alternatively, run poly-bench init with no flags — it will walk you through an interactive menu to select languages. If you don't have a given language installed, poly-bench can install it for you.

Declare a suite with the required header tokens: suiteType, runMode, and sameDataset. Use performance for timing, timeBased for auto-calibration, and set sameDataset: true when all benchmarks share the same fixture data.

1declare suite hashBench performance timeBased sameDataset: true {
2  description: "String Concatenation hash benchmark"
3  baseline: "go"
4  warmup: 100
5  targetTime: 1000ms
6  count: 1
7
8  # ... setup and benchmarks next
9}

2. Define your params

Add fixtures (shared test data) and setup blocks for each language. Fixtures ensure all implementations receive identical input. Setup blocks define imports and helpers.

1declare suite hashBench performance timeBased sameDataset: true {
2  description: "String Concatination hash benchmark"
3  baseline: "go"
4  warmup: 100
5  targetTime: 1000ms
6  count: 1
7
8   setup go {
9    import (
10          "encoding/binary"
11          "strings"
12      )
13    
14    helpers {
15        func concatGo(data []byte) []byte {
16          n := binary.LittleEndian.Uint32(data[0:4])
17          var b strings.Builder
18          b.Grow(int(n))  // pre-allocate
19          for i := uint32(0); i < n; i++ {
20              b.WriteByte('x')
21          }
22          return []byte(b.String())
23      }
24    }
25  }
26
27  setup ts {
28      helpers {
29          function concatTs(data: Uint8Array): Uint8Array {
30              const n = new DataView(data.buffer, data.byteOffset, 4).getUint32(0, true)
31              let s = ''
32              for (let i = 0; i < n; i++) s += 'x'
33              return new TextEncoder().encode(s)
34          }
35      }
36  }
37}

3. Add your bench tests and plot your chart

Add benchmark blocks that call your helpers across languages. Use after blocks to generate charts from the results.

1use std::charting
2
3declare suite hashBench performance timeBased sameDataset: true {
4  description: "String Concatination hash benchmark"
5  baseline: "go"
6  warmup: 100
7  targetTime: 1000ms
8  count: 1
9
10   setup go { 
11      import ("encoding/binary")
12      
13      helpers {
14        import (
15              "encoding/binary"
16              "strings"
17          )
18        
19        helpers {
20            func concatGo(data []byte) []byte {
21              n := binary.LittleEndian.Uint32(data[0:4])
22              var b strings.Builder
23              b.Grow(int(n))  // pre-allocate
24              for i := uint32(0); i < n; i++ {
25                  b.WriteByte('x')
26              }
27              return []byte(b.String())
28          }
29    }
30  }
31
32  setup ts {
33      helpers {
34          function concatTs(data: Uint8Array): Uint8Array {
35              const n = new DataView(data.buffer, data.byteOffset, 4).getUint32(0, true)
36              let s = ''
37              for (let i = 0; i < n; i++) s += 'x'
38              return new TextEncoder().encode(s)
39          }
40      }
41  }
42
43  fixture data {
44      hex: "64000000"
45  }
46
47  bench stringConcatBench {
48      go: concatGo(data)
49      ts: concatTs(data)
50  }
51
52  after {
53      charting.drawTable(
54          title: "String Conctenation Performance",
55          output: "str-concat-table.svg"
56      )
57      charting.drawSpeedupChart(
58          title: "String Conctenation Speedup",
59          output: "str-concat-speedup.svg"
60      )
61  }
62   }
63}

Run the benchmark:

$poly-bench run benchmarks/string-concat.bench

Charts are written to the output directory (e.g. out/hash-table.svg).

CLI at a glance

A quick tour of the main commands you'll use day to day.

plot

Generate charts from existing results without re-running benchmarks. Use plot from-file to apply chart directives from a .bench file's after block, or use subcommands like bar-chart, line-chart, speedup-chart, and table with a results file:

$poly-bench plot from-file benchmarks/hash.bench --results out/results.json

compile

Verify that your benchmark scripts will build across all languages without actually running them. Useful in CI or before committing:

$poly-bench compile benchmarks/hash.bench

install and build

poly-bench install — Install dependencies declared in polybench.toml into each language's runtime environment (e.g. go get, npm install, cargo add).
poly-bench build — Regenerate the .polybench/ directory, run codegen, and compile. Use this when you've changed dependencies or want to refresh the runtime environment.

$poly-bench install
$poly-bench build

add-runtime

Add a new language runtime to an existing project. Creates the runtime config in polybench.toml and scaffolds .polybench/runtime-env/<lang>/:

$poly-bench add-runtime python
$poly-bench add-runtime zig

Supported runtimes: go, ts, rust, python, c, csharp, zig.

add

Add language dependencies using the runtime flag (--go, --ts, --rust, etc.). Each dependency is appended to the corresponding section in polybench.toml and installed into .polybench/runtime-env/<lang>/:

$poly-bench add --go github.com/example/pkg@v1.0.0
$poly-bench add --ts lodash@4.17.21
$poly-bench add --rs serde@1.0

The `polybench.toml` manifest

Your project manifest lives at the root and configures defaults, languages, and dependencies. When you run poly-bench add, it writes entries into the [go.dependencies], [ts.dependencies], and similar sections. Key sections:

1[project]
2name = "my-benchmarks"
3version = "0.0.1"
4
5[defaults]
6iterations = 1000
7warmup = 100
8languages = ["go", "ts", "rust"]
9
10# Per-language config and dependencies
11[go]
12version = "1.21"
13
14[go.dependencies]
15"github.com/example/pkg" = "latest"
16
17[ts.dependencies]
18lodash = "4.17.21"
19
20[rust.dependencies]
21serde = "1.0"
22tokio = { version = "1.0", features = ["rt"] }
23
24[output]
25output_dir = "out"
26auto_save_results = true

Section	Purpose
`[project]`	Project name and version
`[defaults]`	Default iterations, warmup, and language list
`[go]`, `[ts]`, `[rust]`, etc.	Language-specific settings (version, module path, etc.)
`[go.dependencies]`, `[ts.dependencies]`, etc.	Dependencies for each runtime; `poly-bench add` updates these
`[output]`	Output directory, report formats, auto-save behavior

Next steps

Platform Compatibility — Default versions, toolchain paths, and dependencies
Requirements — Per-language setup and runtime details
Suites — Full suite configuration reference
Charting — Chart types and options
CLI Reference — All command options