Fixtures Guide
How to define, load, and use test data fixtures in poly-bench
Fixtures define shared test data that is passed to every language implementation in a benchmark. They guarantee that all languages operate on identical bytes — a prerequisite for fair cross-language comparisons.
When poly-bench generates benchmark code, each fixture is decoded once and injected as a native byte array into the generated code for each language:
| Language | Type |
|---|---|
| Go | []byte |
| TypeScript | Uint8Array |
| Rust | &[u8] (slice reference) |
| Python | bytes |
| C | uint8_t* / bytes |
| C# | byte[] |
| Zig | []const u8 |
Fixtures are defined inside a suite block and are available to all bench blocks in that suite by name.
The simplest fixture: a hex-encoded string inlined directly in the .bench file. Use this for small, fixed test vectors.
1fixture shortInput {2 hex: "68656c6c6f20776f726c64" # "hello world" in UTF-83}4
5fixture address {6 hex: "d8da6bf26964af9d7eed9e03e53415d37aa96045" # 20-byte Ethereum address7}8
9fixture emptyInput {10 hex: "" # zero-length byte slice11}Using fixtures in bench blocks:
1bench hashShort {2 go: keccak256Go(shortInput) # shortInput is []byte3}& prefix when passing a fixture to a function that takes &[u8]. Go and TypeScript receive the value directly.For larger payloads, store the hex data in an external file and reference it with @file(...). The path is relative to the .bench file's location.
1fixture largePayload {2 hex: @file("fixtures/keccak/input_1024.hex")3}4
5fixture sortData {6 hex: @file("fixtures/sort/sort_1000.hex")7}8
9fixture matrixData {10 hex: @file("fixtures/matmul/mat_64.hex")11}fixtures/ subdirectory next to your .bench files. Organize by benchmark type (e.g. fixtures/sort/, fixtures/keccak/) when you have many.Fixture files contain raw hex bytes — one long hex string, no newlines. You can generate them with any language:
$# Generate 1024 random bytes as hex$go run -e 'import "crypto/rand"; import "encoding/hex"; import "os"$func main() {$ b := make([]byte, 1024)$ rand.Read(b)$ os.WriteFile("fixtures/input_1024.hex", []byte(hex.EncodeToString(b)), 0644)$}'The most common fixture pattern is a series of fixtures with increasing sizes, paired with a bench block for each. poly-bench's speedup chart extracts the numeric suffix from benchmark names for the x-axis.
1use std::charting2
3declare suite sortN performance timeBased sameDataset: true {4 description: "O(n log n) sort — stdlib sort on int32 array"5 targetTime: 500ms6 compare: true7 baseline: "go"8
9 setup go {10 import (11 "encoding/binary"12 "sort"13 )14
15 helpers {16 func sortGo(data []byte) []byte {17 n := len(data) / 418 arr := make([]int32, n)19 for i := 0; i < n; i++ {20 arr[i] = int32(binary.BigEndian.Uint32(data[i*4 : (i+1)*4]))21 }22 sort.Slice(arr, func(i, j int) bool { return arr[i] < arr[j] })23 out := make([]byte, len(data))24 for i := 0; i < n; i++ {25 binary.BigEndian.PutUint32(out[i*4:(i+1)*4], uint32(arr[i]))26 }27 return out28 }29 }30 }31
32 setup ts {33 helpers {34 function sortTs(data: Uint8Array): Uint8Array {35 const n = data.length / 436 const arr = []37 for (let i = 0; i < n; i++) {38 arr.push(data[i*4]<<24 | data[i*4+1]<<16 | data[i*4+2]<<8 | data[i*4+3])39 }40 arr.sort((a, b) => a - b)41 const out = new Uint8Array(data.length)42 for (let i = 0; i < n; i++) {43 const v = arr[i] >>> 044 out[i*4] = (v>>24)&0xff; out[i*4+1] = (v>>16)&0xff45 out[i*4+2] = (v>>8)&0xff; out[i*4+3] = v&0xff46 }47 return out48 }49 }50 }51
52 setup rust {53 helpers {54 fn sort_rust(data: &[u8]) -> Vec<u8> {55 let n = data.len() / 4;56 let mut arr: Vec<i32> = (0..n).map(|i| {57 let j = i * 4;58 i32::from_be_bytes([data[j], data[j+1], data[j+2], data[j+3]])59 }).collect();60 arr.sort_unstable();61 let mut out = vec![0u8; data.len()];62 for (i, &v) in arr.iter().enumerate() {63 out[i*4..(i+1)*4].copy_from_slice(&v.to_be_bytes());64 }65 out66 }67 }68 }69
70 # Ten fixtures — n=100 through n=100071 fixture s100 { hex: @file("fixtures/sort/sort_100.hex") }72 fixture s200 { hex: @file("fixtures/sort/sort_200.hex") }73 fixture s300 { hex: @file("fixtures/sort/sort_300.hex") }74 fixture s400 { hex: @file("fixtures/sort/sort_400.hex") }75 fixture s500 { hex: @file("fixtures/sort/sort_500.hex") }76 fixture s600 { hex: @file("fixtures/sort/sort_600.hex") }77 fixture s700 { hex: @file("fixtures/sort/sort_700.hex") }78 fixture s800 { hex: @file("fixtures/sort/sort_800.hex") }79 fixture s900 { hex: @file("fixtures/sort/sort_900.hex") }80 fixture s1000 { hex: @file("fixtures/sort/sort_1000.hex") }81
82 # Benchmark names encode the size — drawSpeedupChart uses this for the x-axis83 bench n100 { go: sortGo(s100) ts: sortTs(s100) rust: sort_rust(&s100) }84 bench n200 { go: sortGo(s200) ts: sortTs(s200) rust: sort_rust(&s200) }85 bench n300 { go: sortGo(s300) ts: sortTs(s300) rust: sort_rust(&s300) }86 bench n400 { go: sortGo(s400) ts: sortTs(s400) rust: sort_rust(&s400) }87 bench n500 { go: sortGo(s500) ts: sortTs(s500) rust: sort_rust(&s500) }88 bench n600 { go: sortGo(s600) ts: sortTs(s600) rust: sort_rust(&s600) }89 bench n700 { go: sortGo(s700) ts: sortTs(s700) rust: sort_rust(&s700) }90 bench n800 { go: sortGo(s800) ts: sortTs(s800) rust: sort_rust(&s800) }91 bench n900 { go: sortGo(s900) ts: sortTs(s900) rust: sort_rust(&s900) }92 bench n1000 { go: sortGo(s1000) ts: sortTs(s1000) rust: sort_rust(&s1000) }93
94 after {95 charting.drawSpeedupChart(96 title: "Sort Performance — O(n log n)",97 output: "sort-line.svg",98 xlabel: "Array Size (n elements)"99 )100
101 charting.drawTable(102 title: "Sort Comparison",103 output: "sort-bar.svg",104 xlabel: "Array Size"105 )106 }107}description and shape AnnotationsBoth fields are optional and for documentation / tooling purposes only — they have no effect on benchmark execution.
1fixture keccakInput {2 description: "32-byte input for keccak256 — typical EVM calldata size"3 hex: "68656c6c6f20776f726c6468656c6c6f20776f726c6468656c6c6f20776f72"4}5
6fixture matrixData {7 description: "64×64 float64 matrix encoded as little-endian bytes"8 hex: @file("fixtures/matmul/mat_64.hex")9 shape: { rows: 64, cols: 64, dtype: "float64", encoding: "little-endian" }10}11
12fixture sortData {13 description: "1000 random int32 values in big-endian byte order"14 hex: @file("fixtures/sort/sort_1000.hex")15 shape: { count: 1000, dtype: "int32", encoding: "big-endian" }16}A fixture defined once in a suite is available to every bench block in that suite. This is the primary reason to use fixtures over hardcoding values in bench expressions.
1declare suite cryptoBench performance timeBased sameDataset: true {2 # ... setup ...3
4 fixture data32 { hex: "68656c6c6f20776f726c6468656c6c6f20776f726c6468656c6c6f20776f72" }5 fixture data256 { hex: @file("fixtures/keccak/input_256.hex") }6
7 # Both benchmarks use the same fixtures8 bench keccak256_32 {9 go: keccak256Go(data32)10 ts: keccak256Ts(data32)11 rust: keccak256_rust(&data32)12 }13
14 bench keccak256_256 {15 go: keccak256Go(data256)16 ts: keccak256Ts(data256)17 rust: keccak256_rust(&data256)18 }19
20 bench sha256_32 {21 go: sha256Go(data32)22 ts: sha256Ts(data32)23 rust: sha256_rust(&data32)24 }25
26 bench sha256_256 {27 go: sha256Go(data256)28 ts: sha256Ts(data256)29 rust: sha256_rust(&data256)30 }31}| Pattern | When to use |
|---|---|
data, input | Single-fixture benchmarks |
small, medium, large | Two or three size tiers |
s100, s500, s1000 | Scaling series (prefix + size number) |
n100, n200, ..., n1000 | Scaling series where benchmark names mirror fixture names |
mat8, mat16, ..., mat64 | Matrix benchmarks |
r100, r200, ..., r1000 | Random data series |
drawSpeedupChart, name your bench blocks (not the fixtures) with a numeric suffix — e.g. bench n100, bench n200. poly-bench extracts the number from the benchmark name for the x-axis, not from the fixture name.drawSpeedupChart