harvey-labs
An open-source benchmark that measures how well AI agents handle realistic legal work. It provides tasks and evaluation for improving legal agents.
Share on XLicense: MIT
Overview
Harvey LAB, short for Legal Agent Benchmark, is an open-source project from Harvey AI for measuring how well LLM agents handle realistic legal work. It has two parts: a dataset of tasks with instructions, documents and rubrics, and an execution harness that runs agents on those tasks and scores them. A tutorial walks through one M&A data-room assignment from setup to comparison dashboards.
Key features
- Task dataset with instructions, documents and rubrics
- Execution harness for running and evaluating agents
- All-pass rubric scoring with an LLM judge
- Reports and comparison dashboards
- Adapters for adding models and tasks
Best for
Researchers and teams building or comparing AI agents for legal work who need a shared test set. The README says the task set and harness are still being extended.
- Upstream
- harveyai/harvey-labs
- Fork on GitHub
- Guo-astro/harvey-labs
- Upstream stars
- 1.4k
- Category
- AI agents and LLM tools
- Language
- Python
- License
- MIT
- Forked
- 2026-09-28
- Sync status
- In syncLast synced 2026-10-10
More in AI agents and LLM tools
An open-source curriculum for learning AI engineering by implementing model internals, retrieval pipelines, and agent runtimes. Study through lessons, interactive labs, or staged coding projects, and inspect the code and evaluation results.
Forked 2026-10-09Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub
A small, hackable character-level language model that learns patterns from line-based text files and generates similar items, such as name ideas.
Forked 2026-10-09Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub
uzu is an inference engine for running AI models in apps, helping developers keep inference on-device instead of relying on remote services. Use it to download a model and build chat features.
Forked 2026-10-09Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub
Anti Slop gives AI coding agents rules to avoid generic UI, filler text, and AI-shaped code. Use it as a filter alongside your own design direction.
Forked 2026-10-08Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub