# Changelog

## 0.3.2 (2026-04-07)

### Fixes

- **`cliwatch validate` resolves `file://` globs** — `file://tasks/*.yaml` references now expand correctly relative to the config file's directory, instead of failing with "referenced file not found".

## 0.3.1 (2026-04-05)

### Features

- **`cliwatch skills setup`** - step-by-step setup guide that works for any CLI (detect, scaffold, write tasks, validate, run)

## 0.3.0 (2026-04-05)

### Features

- **gh-style help output** - grouped commands by purpose (Getting Started, Benchmarking, Results, Reference) with examples
- **Subcommand help** - `cliwatch runs --help`, `cliwatch init --help`, etc. for all commands
- **Contextual hints** - every subcommand shows follow-up commands so agents can navigate the CLI
- **Clickable workspace URL** - `cliwatch whoami` shows a direct link to your workspace
- **Improved runs output** - ANSI-safe table alignment, model names, pagination with `--limit`/`--offset`
- **`cliwatch/` folder convention** - `cliwatch init` scaffolds into a `cliwatch/` subfolder with separate task files
- **Updated skills docs** - CI integration and troubleshooting updated for new folder paths

## 0.2.5 (2026-03-06)

### Fixes

- **Better error when run ID not found** - suggests `--cli` flag for seq-based lookups
- **Rewritten benchmark suite** - 20 tasks with realistic user intents across 4 tiers

## 0.2.4 (2026-03-06)

### Fixes

- **`--version` now reads from package.json** instead of returning hardcoded `0.1.0`

## 0.2.3 (2026-03-06)

### Features

- **Post-release smoke test** - CLI benchmarks now run automatically after npm publish via `workflow_call` chaining

## 0.2.0 (2026-02-11)

### Features

- **Interactive `cliwatch init`** — guided setup with CLI name auto-detection, GitHub Actions workflow generation, and PATH checking. Uses `--cli <name>` flag for non-interactive/agent use.
- **`system_prompt` in cli-bench.yaml** — customize the LLM system prompt with extra context about your CLI
- **Updated skills docs** — all documentation topics updated for AI Gateway, thresholds, and CI integration

### Fixes

- **API key references** — replaced per-provider keys (ANTHROPIC_API_KEY, etc.) with single `AI_GATEWAY_API_KEY` (Vercel AI Gateway) across all templates and skills
- **`cliwatch validate`** — shows "Config is valid! Install {cli} before running benchmarks." instead of misleading "All checks passed!" when CLI is not in PATH

## 0.1.0 (2026-02-08)

Initial release.

- `cliwatch init` — scaffold cli-bench.yaml
- `cliwatch validate` — validate config files
- `cliwatch runs` — list and view bench runs
- `cliwatch trend` — pass rate trends
- `cliwatch skills` — AI-agent-friendly documentation