--- name: evoke-analytics-auditor description: > Starts the Astro dev server, visits every page, clicks configurable CSS selectors, captures all browser console messages, and writes a deduplication report of unique log messages grouped by page. Use this skill whenever the user asks to audit, record, or report on analytics, metrics, or events. --- # Analytics Auditor Skill ## Purpose Start the dev server, click interactive elements, harvest every `console.*` message, and produce a timestamped report of **unique messages** (grouped by page) written to `analytics-audit-report-.md` in the project root. --- ## Instructions ### Step 0 — Load the Playwright MCP tool schemas Before doing anything else, call: ``` ToolSearch({ query: "select:mcp__plugin_playwright_playwright__browser_navigate,mcp__plugin_playwright_playwright__browser_click,mcp__plugin_playwright_playwright__browser_console_messages,mcp__plugin_playwright_playwright__browser_snapshot,mcp__plugin_playwright_playwright__browser_wait_for,mcp__plugin_playwright_playwright__browser_evaluate,mcp__plugin_playwright_playwright__browser_handle_dialog,mcp__plugin_playwright_playwright__browser_close" }) ``` You will need all of those tools to be loaded and callable. --- ### Step 1 — Start the dev server and detect its port Always start a fresh dev server. Use a timestamped log filename so the poll can never accidentally read the port from a previously running server's log: ```bash ASTRO_LOG="/tmp/astro-dev-$(date +%s).log" npm run dev > "$ASTRO_LOG" 2>&1 & echo "$ASTRO_LOG" ``` Save the log path printed by the `echo` — you will need it for polling and cleanup. Then poll the log file until Astro prints its ready URL. **Do not use `grep -P`** — it is not supported on Windows. Use this portable polling loop instead: ```bash port="" for i in $(seq 1 30); do port=$(grep -o 'localhost:[0-9]*' "$ASTRO_LOG" | head -1 | grep -o '[0-9]*$') if [ -n "$port" ]; then echo "$port"; break; fi sleep 1 done if [ -z "$port" ]; then echo "ERROR: dev server did not produce a port within 30 seconds" >&2 cat "$ASTRO_LOG" >&2 exit 1 fi ``` If the loop exits without a port, stop and report the error to the user — do not continue with subsequent steps. Extract the port from that output and use it as `DEV_PORT` for all subsequent URLs (e.g. `http://localhost:${DEV_PORT}/home/`). Once the port has been extracted, delete the log file: ```bash rm -f "$ASTRO_LOG" ``` --- ### Step 2 — Capture timestamp, discover pages, and define click selectors #### Report timestamp Before starting any browser work, generate the timestamp that will be used in the report filename. Use `browser_evaluate` once the browser is open, or compute it via any available JS execution context: ```js new Date().toISOString().replace(/[^\d]/g, '').substring(0, 14) ``` Store the result as **`TIMESTAMP`** (e.g. `20260616143022`). It is used once in Step 5 to name the output file. #### Page URLs Do **not** hardcode the page list. Instead, read the `src/pages` directory to discover all `.astro` files: ```bash find src/pages -name "*.astro" | sort ``` **Skip any file whose name contains `[` or `]`** — these are dynamic route templates that require parameter values and cannot be visited directly. Record each skipped file in a list for the report's header section. Convert each remaining file path to a URL route using these rules: - Strip the `src/pages/` prefix and `.astro` extension - A file named `index.astro` maps to `/` - All other files map to `//` (trailing slash) - Nested files (e.g. `src/pages/about/moa.astro`) map to `/about/moa/` Prepend `http://localhost:${DEV_PORT}` from Step 1 to form the full URL. Example — `src/pages/home.astro` → `http://localhost:${DEV_PORT}/home/` #### Click selectors **Before finalizing the selector list, read `src/lib/client/main.ts`** (and any other files it imports from `src/lib/client/`) and check whether `IVA.configure()` or `IVA.init()` is called with a config object that contains any of these properties: - **`selectors.clickable`** — if present, it is a CSS selector string that replaces the non-`[data-evotrk]` selectors in the default list. Replace those defaults with this value, but **keep `[data-evotrk]`** (or the `attrs.data`-derived selector, see below) in the final list. - **`attrs.data`** — if present, it is the attribute name used instead of `data-evotrk` to mark tracked elements. It may or may not include surrounding brackets. Normalize it: if it already starts with `[`, use it as-is; otherwise wrap it as `[]`. Replace `[data-evotrk]` in the selector list with this normalized value. - **`attrs.navigateDisabled`** — if present, it is the attribute name IVA checks to decide whether to skip its `navigate()` call. Store this value (without brackets) as **`NAVIGATE_DISABLED_ATTR`** for use in Step 3. Default if absent: `data-navigate-disabled`. - **`attrs.gotoslide`** — if present, it is the attribute name IVA uses to identify which `` elements it has bound its click handler to. Store this value (without brackets) as **`GOTO_SLIDE_ATTR`** for use in Step 3. Default if absent: `data-gotoslide`. If none of these properties are found, use this default list unchanged: ``` [data-evotrk] a button [role="button"] [role="link"] [role="tab"] input[type="checkbox"] input[type="radio"] [role="checkbox"] [role="radio"] ``` On every page, query for all elements matching the final selector list and click each one that is visible and not disabled. Use a single combined CSS selector string joined with commas when querying the page. The user may supply additional or replacement selectors in their request — merge them into this list rather than replacing it entirely unless the user says otherwise. --- ### Step 3 — Per-page procedure For **each page URL** discovered in Step 2, also initialize a **`pageMessages`** list (reset to empty for each new page) that accumulates all console messages seen on the current page. 1. Navigate to the page URL. 2. Wait for the page to load (use `browser_wait_for` with a known selector like `h1`). 3. **Disable navigation on all links** using `browser_evaluate`. Pass `NAVIGATE_DISABLED_ATTR` as the attribute name. This approach is reliable because it works at the DOM level before any click handler fires: - **IVA-bound links** — if the `` element has the `GOTO_SLIDE_ATTR` attribute, IVA has registered its click handler on it. Set `="true"` on the element so IVA skips its `navigate()` call. IVA already calls `e.preventDefault()` unconditionally, so browser default navigation is also suppressed. - **All other links** — if the `` element does not have `GOTO_SLIDE_ATTR` (external links, PDF links, plain anchors, etc.), remove the `href` attribute entirely (store the original value in `data-original-href`). Without an `href` the browser will not navigate; any IVA tracking handler still fires normally. Use this script (substitute the actual attribute names for the placeholders): ```js (gotoSlideAttr, navigateDisabledAttr) => { document.querySelectorAll('a').forEach(a => { if (a.hasAttribute(gotoSlideAttr)) { // IVA-bound link — use IVA's own escape hatch a.setAttribute(navigateDisabledAttr, 'true'); } else { // Not IVA-bound — strip href to prevent browser navigation const href = a.getAttribute('href'); if (href) { a.dataset.originalHref = href; a.removeAttribute('href'); } } }); } ``` Call this evaluate **after** the page loads (Step 3.2) and **again after any DOM mutation** (modal open, tab switch, dynamic content) that may have added new `` elements. 4. Take a `browser_snapshot` to get the current accessibility tree. Use the refs from this snapshot for all subsequent clicks on this page. 5. For **each matched element** (visible, not disabled): a. Click it using `browser_click`. b. Wait 500ms for async effects. c. **Call `browser_console_messages` with `level: "debug"`** to get all messages accumulated on this page so far. Identify the **newly appeared messages** by comparing the full returned list against `pageMessages` (new = any message whose de-timestamped text is not already present in `pageMessages`). Add all new messages to `pageMessages`. d. Take a `browser_snapshot`. If the tree is structurally unchanged from the previous snapshot, the existing refs remain valid — no action needed. If the tree changed, use the new snapshot's refs going forward. e. If the snapshot shows a modal is now open (`Modal state` present in the tree): i. Use `browser_evaluate` to collect identifying information for all clickable elements inside the modal: ```js () => { const modal = document.querySelector('[role="dialog"], [aria-modal="true"]'); if (!modal) return []; const sel = 'a,button,[role="button"],[role="link"],[role="tab"],input[type="checkbox"],input[type="radio"],[role="checkbox"],[role="radio"]'; return Array.from(modal.querySelectorAll(sel)) .filter(el => !el.disabled && el.offsetParent !== null) .map(el => el.getAttribute('aria-label') || el.textContent.trim() || el.tagName); } ``` Cross-reference the returned labels/text against the current snapshot to identify the corresponding accessibility-tree refs for clicking. ii. For each visible, non-disabled modal element: click it using its ref, wait 500ms, then apply the same **step 5c** above (call `browser_console_messages` and diff). Re-snapshot if the modal content changes. iii. After all modal elements have been clicked, dismiss the modal by clicking its close button — look for a button with `aria-label="Close"` or visible text "×", "Close", or "Dismiss" in the snapshot. Only use `browser_handle_dialog` if a **native** browser dialog (`alert`/`confirm`/ `prompt`) is present — it has no effect on custom DOM modals. iv. Take a fresh `browser_snapshot` after the modal closes before continuing to the next page-level element — refs from before the modal opened are stale. f. If no modal appeared but the DOM changed significantly, use the new snapshot's refs before proceeding. 6. After all clicks on the page, call `browser_console_messages` **once more** with `level: "debug"` to catch any async messages that fired after the last click. Diff against `pageMessages` and append any remaining new messages. 7. Store all messages in `pageMessages` with the page route (e.g. `/home/`) as their source. 8. **Flag ambiguous click data.** After collecting tracking events for the page, check each event against these criteria and mark it ambiguous if **any** apply: - **Missing or empty field** — `slide`, `type`, or `name` is absent, `null`, `""`, or the literal string `"undefined"`. - **Generic name** — `name` is a value like `"button"`, `"link"`, `"click"`, `"undefined"`, or matches the pattern `/^(button|link|click|item|element|tab)\d*$/i` with no further specificity. - **Name equals type** — `name` and `type` have the same value (e.g. `{ type: "button", name: "button" }`). - **Duplicate (slide, name) pair on the same page** — the same `(slide, name)` combination fires more than once from distinct elements on the same page, making it impossible to distinguish which element the user clicked. - **Slide mismatch** — the `slide` value in the event does not match the route of the page it was captured on (e.g. `slide: "home"` on `/about/`). For each ambiguous event store: ``` { page: string, // route where it was captured event: { slide, type, name }, reasons: string[] // one entry per applicable criterion above } ``` --- ### Step 4 — Parse, deduplicate, and structure the data #### Identify tracking events When an element is clicked the app emits a tracking log in this format: ``` [HH:MM:SS.mmm] DEBUG: track {slide: 'home', type: 'button', name: 'button_evo_flip_card_trigger'} ``` For every console message, check whether its text matches this pattern: ``` DEBUG: track {…} ``` If it matches, parse out the object payload `{ slide, type, name }` — these are the primary data points of interest. Store them as structured tracking events separate from general log messages. #### Collect all messages Gather every console message captured across all pages into a flat list: ``` { page: string, // route, e.g. "/home/" level: "log"|"warn"|"error"|"info"|"debug", text: string, // full raw message text tracking: { // only present when the message is a tracking event slide: string, type: string, name: string } | null } ``` #### Deduplicate Before comparing, **strip the leading timestamp prefix** from each message's `text` (the `[HH:MM:SS.mmm]` portion) — timestamps differ per run and would prevent identical messages from being recognized as duplicates. Compare only the de-timestamped text. Two messages are duplicates if their de-timestamped texts are equal. When the same message appears on multiple pages, record all pages it appeared on. #### Classify global vs. slide-specific tracking events After collecting all tracking events, split them into two groups. An event is **Global** if **either** condition is true: 1. Its `type` starts with `header-`, `footer-`, or `nav-`; **or** 2. The `(type, name)` pair appears on **≥ 80%** of the total audited pages. Everything else is a **Slide Event**. For global events, drop the `slide` dimension entirely — only `type` and `name` are needed. For slide events, retain `slide`, `type`, and `name`. #### Deduplicate ambiguous events Apply the same timestamp-stripping deduplication to the ambiguous events list. Two ambiguous events are duplicates if their `(page, slide, type, name)` tuples are equal — keep one entry but merge their `reasons` arrays (union, no duplicates). #### Tiers Organize into five tiers: 1. **Tracking Events** — messages matching the `DEBUG: track` pattern (split into Global and Slide sub-sections, see Step 5) 2. **Ambiguous Events** — tracking events that matched one or more ambiguity criteria in Step 3.8 3. **Errors** — `console.error` 4. **Warnings** — `console.warn` 5. **Info / Log** — `console.log`, `console.info`, `console.debug` (excluding tracking events) --- ### Step 5 — Write the report Before inserting any message text into a Markdown table cell, **escape every `|` character as `\|`** to prevent table corruption. Write a Markdown file to `analytics-audit-report-.md` in the project root with this structure: ```markdown # Browser Console Log Audit **Date:** **Pages audited:** /home/, /about/, /about/moa/ **Total unique messages:** tracking events ( global, slide-specific, ambiguous), errors, warnings, info/log > **Skipped (dynamic routes):** src/pages/[slug].astro, src/pages/[...path].astro --- ## Tracking Events () > Events are grouped by slide. Each slide's `type` and `name` are set by the IVA tracking library at click time; the `slide` value is defined by each page's own configuration. ### Global Events ( — type prefixed `header-`/`footer-`/`nav-`, or present on ≥80% of pages) These `(type, name)` pairs represent shared UI (navigation, ISI controls, header/footer links, etc.). The slide name is omitted since it is not meaningful here. | Type | Name | |------|------| | `nav-button` | `nav-button_about_pbc` | ### Slide Events ( — page-specific) These events fire on fewer than 80% of pages and are meaningful to a specific slide or feature. | Slide | Type | Name | |-------|------|------| | `home` | `button` | `button_important_safety_information` | ## Ambiguous Events () > These tracking events could not be unambiguously attributed to a specific element or > action. Review and correct the tracking configuration for each one. | Page | Slide | Type | Name | Reason(s) | |------|-------|------|------|-----------| | `/home/` | `home` | `button` | `button` | Generic name; Name equals type | ## Errors () | Message | Pages | |---------|-------| | `` | /home/, /about/ | ## Warnings () | Message | Pages | |---------|-------| | `` | /home/ | ## Info / Log () | Message | Pages | |---------|-------| | `` | /about/moa/ | --- *Generated by the analytics-auditor skill.* ``` Omit the "Skipped" blockquote if no dynamic routes were found. If a tier has zero messages, write `_None_` instead of a table. If there are no slide-specific events, write `_None_` under that sub-section. Omit the Global Events sub-section header note if there are no global events. If there are no ambiguous events, write `_None_` under the Ambiguous Events section. After writing the Markdown file, write a CSV file to the project root using the same base name with the extension changed to `.csv` (e.g. `analytics-audit-report-20260616143022.csv`). The CSV contains one row per unique tracking event (global and slide, excluding ambiguous). Use this header and column rules: ``` slide,type,name ``` - **Global events** — set `slide` to `GLOBAL`. Use the event's own `type` and `name`. - **Slide events** — set `slide` to the event's `slide` value. Use `type` and `name` as-is. Output global events first (sorted by `type`, then `name`), followed by slide events (sorted by `slide`, then `type`, then `name`). If a field value contains a comma, double-quote, or newline, wrap it in double quotes and escape any internal double quotes by doubling them (`"` → `""`). Example: ```csv slide,type,name GLOBAL,nav-button,nav-button_about_pbc GLOBAL,nav-button,nav-button_home home,button,button_important_safety_information about,button,button_request_info ``` After writing the file, tell the user both filenames and give a brief summary (e.g. "Found 8 tracking events (5 global, 3 slide-specific, 2 ambiguous), 2 errors, 1 warning, 5 info messages across 3 pages"). If there are any ambiguous events, call them out explicitly in the summary so the user knows action is needed. --- ### Step 6 — Shut down and clean up Call `browser_close` to end the Playwright session. Then stop the dev server and delete the Playwright working directory: ```bash kill $(lsof -ti tcp:$DEV_PORT) 2>/dev/null || true rm -rf .playwright-mcp ``` --- ## Notes - The Astro dev server defaults to port **4321** but increments on collision — always read the actual port from the server output as described in Step 1. - **`grep -P` is not supported on Windows** — use `grep -o` with basic regex patterns. - **Snapshot refs go stale** after DOM mutations (modal open/close, tab switches). Always re-snapshot after any click that visibly changes the page structure. - **`browser_console_messages` fails if a native dialog is showing** — dismiss with `browser_handle_dialog` before calling it. - **`browser_handle_dialog` only works for native browser dialogs** (`alert`, `confirm`, `prompt`) — it has no effect on custom DOM modals. Dismiss those by clicking their close button. - **Navigation prevention uses DOM attributes, not event listeners.** IVA's click handler calls `e.preventDefault()` unconditionally and then checks `="true"` before calling `navigate()`. Setting that attribute on IVA-bound links is therefore the most reliable way to suppress navigation without interfering with tracking. All other links have their `href` removed instead. - **Re-run the link-disabling evaluate after DOM mutations** (modal open, dynamic tab content, etc.) — newly inserted `` elements will not have the attribute yet. - **`browser_console_messages` is called after every click** so that tracking events are captured incrementally. The final per-page call catches any late async messages. - If the user provides their own list of selectors or pages, replace the defaults in Step 2 before proceeding. - If a selector does not match any element on a page, log a warning in the report's header section rather than failing.