--- goat-flow-reference-version: "1.12.1" --- # Page Capture Reference Use this when a task requires visiting a list of pages in a real browser, capturing a screenshot of each, and emitting a markdown record per page. Typical triggers: documenting new pages after a feature ships, capturing a UI baseline before a release, recording before/after evidence across multiple pages, or producing an audit-style page snapshot. Page Capture is the durable batch workflow: scripted, repeatable, evidence-grade. `browser-use` is the right tool for one-off mid-task observation. Playwright (any integration path) plus this reference is the right tool for batch capture across multiple pages with load verification, especially when authentication or framework-specific patterns matter. ## Availability Check Batch page capture requires Playwright. Playwright is a browser automation library, not a protocol - MCP is one way to use it, but running a Python or Node script is equally valid. Check these tiers in order and use the first that works: **Tier 1 - Playwright MCP** (best: native tool calls, no script needed) Check if these tools are callable: `browser_navigate`, `browser_screenshot`, `browser_snapshot`, `browser_evaluate`, `browser_resize`, `browser_wait_for`, `browser_console_messages`, `browser_network_requests`. If available, use them directly - the per-page loop in Step 2 maps to these tool calls. **Tier 2 - Project-local Playwright** (good: carries project config, fixtures, and browser version expectations) Check if the project has Playwright installed as a dependency: ```bash node -e "require.resolve('@playwright/test')" 2>/dev/null && echo "ok" # or: npm ls @playwright/test playwright --depth=0 npx playwright --version ``` For JS/TS projects, prefer this tier over the Python wrapper - project-local Playwright may carry storage-state conventions, browser version pins, or test fixtures that the generic Python path won't have. The agent writes a Node.js capture script and executes it. **Tier 3 - Python Playwright** (good: works for any coding agent that can run shell commands) ```bash browser-use-python -c "from playwright.sync_api import sync_playwright; print('ok')" # or the standard Playwright CLI if the venv wrapper is not present: python -m playwright --version ``` If a Python venv exposes Playwright through a `browser-use-python` wrapper (some environments place one on `PATH`, e.g. at `~/.local/bin/browser-use-python`), check it before declaring Playwright unavailable. If no Python Playwright path is available, offer to install it with `pip install playwright && python -m playwright install chromium`; never install it without approval. The agent writes a Python capture script using `playwright.sync_api`, executes it, and reads the output. See "Writing a capture script" below. **Tier 4 - browser-use CLI** (downgrade: less control, missing console/network capture) ```bash command -v browser-use && browser-use doctor ``` Only use browser-use for batch capture if no Playwright path (MCP, Node, or Python) is available AND the user explicitly accepts the downgrade. Ask: "No Playwright path found. Use browser-use CLI instead? Console error capture will be unavailable." Do not silently fall back. Use `browser-use open`, `browser-use screenshot`, `browser-use state`. This path cannot capture console errors - mark captures with `Console errors: not captured (browser-use CLI)` and note reduced evidence quality in the index. **Tier 5 - Manual fallback** (last resort: human provides evidence) See "Fallback When No Playwright Path Is Available" at the bottom. **If all automated tiers fail,** state which checks were run and their output. Do not silently fall back to a less capable tool. ## Intent A coding agent uses page capture to create durable, repeatable browser evidence across known pages. The output is not a screenshot gallery; it is an evidence bundle with one markdown record per page, a screenshot path that resolves on disk, load verification, console-error accounting, and an index that makes partial failures visible. ## Boundary | Job | Tool | Reference | |---|---|---| | Single observation, mid-skill, agent decides what to look at | `browser-use` CLI | `browser-use.md` | | Visit N known pages, screenshot each, emit structured MD | Playwright (any integration) | this file | | Test-suite-driven evidence (assertion -> screenshot -> record on fail) | Playwright Test + custom reporter | not in goat-flow; see Playwright reporter docs | | Crawl/discover pages an agent hasn't seen | `browser-use` with agent loop | `browser-use.md` | If the task fits row 1 or row 4, stop and load `browser-use.md` instead. If the task fits row 3, document it as a project-level test-runner concern, not a goat-flow workflow. ## Writing a Capture Script (Tier 2/3) When using Python or Node Playwright, the agent writes and executes a capture script. Minimal pattern (Python): ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.launch(headless=True) ctx = browser.new_context( viewport={"width": 1280, "height": 800}, storage_state="path/to/state.json" # omit if no auth ) page = ctx.new_page() errors = [] page.on("console", lambda m: errors.append(m.text) if m.type == "error" else None) page.goto(url) page.wait_for_selector(".expected-element", timeout=10000) page.screenshot(path="screenshots/slug.png", full_page=True) title = page.title() # write MD record with title, url, viewport, timestamp, len(errors) browser.close() ``` Adapt to the project's needs (SPA navigation, framework wait conditions, auth flows). The evidence requirements are the same regardless of implementation: consistent viewport, navigate each page, wait for a content anchor, screenshot, collect console error count, write one MD record per page, write an index, verify screenshots exist on disk. **Storage state by tier:** - Tier 1 (MCP): context-level configuration, tool-dependent - Tier 2 (Node): `browser.newContext({ storageState: "path/to/state.json" })` - Tier 3 (Python): `browser.new_context(storage_state="path/to/state.json")` - Tier 4 (browser-use): `browser-use --profile ` if supported by version ## Project Reference Pattern This file is generic by design. Project-specific knowledge does NOT live in `.goat-flow/skill-docs/` - that directory is reserved for goat-flow's own references. Project knowledge splits across two layers: | Layer | What it holds | Where it lives | |---|---|---| | Project reference | How-to playbook for driving the project's app - auth flow, framework gotchas, selectors, viewport conventions | `.goat-flow/learning-loop/patterns/-playwright.md` (vendor-neutral, readable by any coding agent) | | Project learning loop | Recurring traps with file evidence, dated lessons from incidents, repeatable patterns | `.goat-flow/learning-loop/footguns/.md`, `.goat-flow/learning-loop/lessons/.md`, `.goat-flow/learning-loop/patterns/.md` | A project reference typically covers: - Auth flow specifics - login URL, 2FA, storage state path - Framework gotchas - debug toolbars to hide, JS init waits, dropdown libraries that bypass native events, editor models that hide their value - Selector conventions - data attributes, class patterns, ID conventions - Viewport conventions if the project tests at non-default sizes - Pages or routes to exclude - admin, destructive, slow, flaky A goat-flow skill loading this `page-capture.md` reference should also: 1. Detect and load any project reference at the conventional path 2. Run the standard footgun grep against `.goat-flow/learning-loop/footguns/` for the target area before capture begins ## Capture Workflow ### Step 0 - Intake Confirm before proceeding. Ask only what's missing. 1. **Page list** - explicit URLs, manifest path, or a diff to derive from 2. **Auth** - required? If yes: storage state path, login flow, or none 3. **Viewport** - default `1280x800` desktop unless stated; `390x844` for mobile checks 4. **Output dir** - default: `.goat-flow/logs/sessions/-