Docs/Foundational Primitives/Evaluation Harnesses
Evaluation Harness Benchmark Suite
The EvaluationHarness allows developers to write automated benchmark test suites to score model accuracy, tool calling precision, and latency across providers.
Evaluation Suite Setup
Define test cases with string matchers or custom assertion evaluators:
harness-test.ts
| 1 | import { EvaluationHarness } from "@bablusingh-dev/agentcore"; |
| 2 | |
| 3 | const harness = new EvaluationHarness() |
| 4 | .addTestCase({ |
| 5 | id: "test-echo", |
| 6 | description: "Verify agent echos exact response", |
| 7 | input: "Echo back 'hello world'", |
| 8 | expectedOutput: "hello world", |
| 9 | }) |
| 10 | .addTestCase({ |
| 11 | id: "test-regex", |
| 12 | description: "Verify agent version response", |
| 13 | input: "Print current version", |
| 14 | expectedOutput: /^v\d+\./, |
| 15 | }); |
| 16 | |
| 17 | const results = await harness.run(async (input) => { |
| 18 | return "hello world"; |
| 19 | }); |
| 20 | |
| 21 | console.log(results); |