# Build Your Own Connector

This doc tells the Forge Help AI how to author a custom connector for
the user. The user asks something like _"build me a connector for
JIRA"_ or _"add a connector that scrapes my internal wiki"_; you walk
them through requirements, generate a manifest, and install it locally.

## Mental model

A connector is a YAML manifest that exposes **tools** the chat LLM
can call. Each tool runs in one of three places:

| `protocol` | Where it runs | Used for |
|---|---|---|
| `browser` (default) | User's tab, via the Forge browser extension | DOM-scraping the user's logged-in UI (Mantis, GitLab UI, Teams) |
| `http` | Forge server | Clean REST APIs with PAT auth (GitHub, Stripe, …) |
| `shell` | Forge server | Local CLI tools (`git`, `gh`, `kubectl`, …) |

Choose **browser** when the user is already logged in to a web app and
you don't want to manage tokens. Choose **http** when there's a clean
REST API. Choose **shell** when wrapping a local CLI.

## The interview (ask the user)

Before writing anything, ask:

1. **What service / site?** (e.g. JIRA Server at jira.acme.com)
2. **What actions are needed?** Read-only? Read + write? List of verbs.
3. **Auth?** Logged-in browser session, PAT, OAuth, local CLI?
4. **What settings does the user need to fill in?** (base_url, PAT, default project, …)

Stop after this; show the user a one-paragraph plan; let them adjust.

## Manifest shape

Required fields: `id`, `name`, `version`, and either `tools` or
`connectors[]`. Everything else is optional.

```yaml
id: jira                       # lowercase, alphanumerics + - / _
name: JIRA
icon: "📋"
version: "0.1.0"
author: "<user-provided>"
description: |
  Multi-line description shown in the marketplace.

# Optional. Locks install to Forge versions that have the required runtime features.
min_forge_version: "0.8.0"

# Browser runner. Default 'main'. Use 'isolated' for strict-CSP
# sites (Teams, github.com) that block eval in the page world.
runner: main

# Per-user settings rendered as a form in Settings → Connectors.
settings:
  base_url:
    type: string
    label: JIRA base URL
    required: true
  token:
    type: secret              # encrypted at rest (AES-256-GCM)
    label: Personal access token
    description: Create at <base_url>/secure/ViewProfile.jspa

# Where the extension finds the authenticated tab. {settings.*} expanded server-side.
host_match: "{base_url}/*"

# Substring detected after navigation → "user not logged in".
login_redirect: "/login.jsp"

tools:
  list_my_issues:
    description: List JIRA issues assigned to me.
    parameters:
      project: { type: string, description: "Limit to one project key (optional)" }
      status:  { type: select, options: ["open","done","all"], default: "open" }
      limit:   { type: number, default: 25 }
    # protocol omitted → browser
    page:
      url: "{base_url}/issues/?jql=assignee=currentUser()"
      on_target: "/issues/"   # skip navigation if URL already matches
    script: |
      const rows = Array.from(document.querySelectorAll('.issuerow'));
      return rows.slice(0, args.limit).map(r => ({
        key:     r.dataset.issuekey,
        title:   r.querySelector('.summary')?.textContent?.trim(),
        status:  r.querySelector('.status')?.textContent?.trim(),
        link:    r.querySelector('a.issue-link')?.href,
      }));

  add_comment:
    destructive: true          # extension prompts before running
    description: Add a comment to a JIRA issue.
    parameters:
      issue_key: { type: string, required: true }
      text:      { type: string, required: true }
    page:
      url: "{base_url}/browse/{args.issue_key}"
    script: |
      // Same-origin fetch with user's session cookie auto-attached
      const r = await fetch(`/rest/api/2/issue/${args.issue_key}/comment`, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ body: args.text }),
      });
      if (!r.ok) return { _error: `HTTP ${r.status}: ${await r.text()}` };
      return { ok: true };
```

### Template tokens

- `{base_url}`, `{settings.<name>}` — expanded server-side at API
  response time from the user's saved settings.
- `{args.<name>}` — expanded at runtime from the LLM's tool input.
  In a `script` body, prefer `args.foo` (a JS identifier) over the
  literal `{args.foo}` string.

### `script` contract

- Receives `args` (the LLM's parsed parameters).
- Returns a JSON-serialisable value (no DOM nodes, no functions).
- Has access to `document`, `fetch`, `URL`, `Headers`, etc. (page context).
- Same-origin `fetch()` — the user's session cookies attach automatically.
- Must be **self-contained** — no closures over Forge / extension code,
  no `import`, no `require`.
- Throws are caught by the runner and surfaced as tool errors;
  prefer `return { _error: '...' }` for predictable failure messages.

### http protocol

For services with a clean REST API:

```yaml
tools:
  get_repo:
    protocol: http
    parameters:
      owner: { type: string, required: true }
      repo:  { type: string, required: true }
    request:
      method: GET
      url: "https://api.github.com/repos/{args.owner}/{args.repo}"
      headers:
        Accept: "application/vnd.github+json"
        Authorization: "Bearer {settings.token}"
    timeout_ms: 15000
```

#### Auth schemes

Manifests can declare one auth scheme at the top and Forge applies it to every `protocol: http` tool — no need to hand-craft an `Authorization` header per tool, no manifest-side base64.

```yaml
# Connector-level (applies to all http tools).
auth:
  type: basic                       # basic | bearer | header | query | none
  username: '{settings.username}'
  password: '{settings.api_token}'

# Variants:
auth:
  type: bearer
  token: '{settings.token}'

auth:
  type: header
  name: PRIVATE-TOKEN
  value: '{settings.token}'

auth:
  type: query
  name: access_token
  value: '{settings.token}'
```

A tool can override (or disable) the inherited scheme with its own `auth:` block. `{ type: none }` skips auth entirely (public endpoint).

##### `bearer-token-exchange` — auto-refreshing JWTs

For APIs that don't accept static tokens — you POST credentials to an exchange endpoint, get a short-lived JWT/bearer, then use that on every subsequent call. Two real-world flavours, both handled by the same auth type:

**Flavour A — header-mode (Black Duck style)**. You have a long-lived API token; you send it in the Authorization header to the exchange URL.

```yaml
auth:
  type: bearer-token-exchange
  api_token: '{settings.api_token}'
  exchange_url: '{settings.base_url}/api/tokens/authenticate'
  exchange_method: POST                         # default
  exchange_auth_header: 'token {settings.api_token}'   # default
  exchange_headers:                             # optional vendored Accept
    Accept: 'application/vnd.blackducksoftware.user-4+json'
  bearer_path: bearerToken                      # default — JSON path in response
  expires_path: expiresInMilliseconds           # default — JSON path in response
  # bearer_format: bearer                       # default — sends 'Authorization: Bearer <jwt>'
```

**Flavour B — body-mode (FortiNCM / NAC / most appliances)**. You have username + password; you POST them as a JSON body to a login endpoint and the JWT comes back in some nested field. The server expects the bare token (no `Bearer ` prefix).

```yaml
auth:
  type: bearer-token-exchange
  exchange_url: 'https://{args.host}/api/v3/auth/login'   # {args.*} works — host can be per-call
  exchange_method: POST
  exchange_body:                                # JSON body; values templated
    username: '{settings.username}'
    password: '{settings.password}'
  bearer_path: result.jwt                       # dotted path into response JSON
  bearer_format: bare                           # → 'Authorization: <jwt>', no Bearer prefix
  expires_ttl_sec: 540                          # fallback TTL when response doesn't include expiry
```

Forge handles the round-trip transparently: first tool call pays the exchange round-trip, the bearer is cached in-memory keyed by `<credential identity>|<resolved exchange_url>`, and refreshed when within 60s of expiry. Tools see only their own `args.*` — they never touch the token.

Cache key derivation:
- **Credential identity**: `api_token` (header-mode) OR the serialised expanded body (body-mode). Password rotation invalidates the cache.
- **Resolved exchange_url**: settings + args fully resolved, so multi-host installs (`{args.host}` in URL) cache per-host independently.

All fields are templated:

| Field | Required | Default | Notes |
|---|---|---|---|
| `exchange_url` | yes | — | `{settings.*}` and `{args.*}` both work |
| `exchange_method` | no | `POST` | `POST` or `GET` |
| `api_token` | no¹ | — | Header-mode credential |
| `exchange_auth_header` | no | `token {api_token}` | Ignored when `exchange_body` is set |
| `exchange_body` | no¹ | — | JSON body for body-mode login |
| `exchange_headers` | no | — | Extra request headers for the exchange |
| `bearer_path` | no | `bearerToken` | Dotted path into response JSON |
| `expires_path` | no | `expiresInMilliseconds` | If missing in response → use `expires_ttl_sec` |
| `expires_ttl_sec` | no | `300` | Fallback TTL (seconds) |
| `bearer_format` | no | `bearer` | `bearer` → `Authorization: Bearer <jwt>`; `bare` → `Authorization: <jwt>` |

¹ Provide ONE of `api_token` or `exchange_body`.

When picking between the two: if the API has its own "Generate API token" UI for users, prefer header-mode (token rotates rarely, user manages it). If the API only accepts username/password login, use body-mode (Forge stores the password encrypted in settings).

#### Per-parameter URL encoding

Default behaviour for an `{args.X}` in a URL path is `encodeURIComponent` — slashes become `%2F`. This is right for GitLab-style project paths but wrong for systems that expect literal slashes (Jenkins folder paths). Override per parameter:

```yaml
parameters:
  job_path:
    type: string
    url_encoding: none              # uri_component (default) | none | path_segments
    description: 'Pre-formatted Jenkins path with "job/" prefixes, e.g. "job/team-x/job/build".'
  artifact_path:
    type: string
    url_encoding: path_segments     # encode each / segment individually, preserve slashes
```

#### Form-urlencoded bodies (`body_form`)

For old-school form POST APIs (Jenkins `/buildWithParameters`, Slack `/chat.postMessage`, etc.). Point at a JSON-typed parameter and Forge serialises it with `URLSearchParams` (`application/x-www-form-urlencoded`):

```yaml
parameters:
  params:
    type: json
    description: 'Flat object of build params: { "BRANCH": "main", "ENV": "stg" }'
request:
  method: POST
  url: '{settings.base_url}/{args.job_path}/buildWithParameters'
  body_form: '{args.params}'
```

LLMs sometimes JSON-stringify nested objects even when the schema says `type: json` — Forge automatically `JSON.parse`'s string-form values, so both shapes work.

#### Server-side inject — credentials the LLM should never see

Two forms.

**1. `body_form_inject` (static keys, manifest-baked)** — useful when the manifest knows exactly which Jenkins / API param name the credential goes under.

```yaml
request:
  body_form: '{args.params}'
  body_form_inject:
    GITLAB_PAT: '{settings.gitlab_pat}'         # key + value both templated against settings
```

Forge expands both the key and the value against settings (NOT args, so the LLM can't shadow). Empty / unsubstituted entries are dropped — optional secrets don't post blank values.

**2. `body_form_inject_from` (dynamic, user-configured per instance)** — when each Jenkins job uses different param names, let the user supply a list of `{name, value}` rows in the settings UI; Forge injects every row.

```yaml
# settings declaration:
settings:
  inject_params:
    type: instances                 # repeating-row UI
    label: "Auto-inject build params"
    fields:
      name:  { type: string, required: true, label: "Param name" }
      value: { type: secret, required: true, label: "Value" }

# tool spec:
request:
  body_form: '{args.params}'
  body_form_inject_from: 'inject_params'   # name of the instances settings field
```

User adds rows in the UI (e.g. `TOKEN_PASSWORD` → `<the-real-pat>`); Forge merges every row into the form body server-side. LLM passes only build-specific params, never sees the credentials.

#### Multi-instance support (`settings.instances`)

To let one install hit multiple servers of the same kind (prod + staging Jenkins, multiple GitLab tenants, etc.), declare your config under a `type: instances` field:

```yaml
settings:
  instances:
    type: instances
    required: true
    fields:
      name:      { type: string, required: true }    # how the LLM picks one
      base_url:  { type: string, required: true }
      username:  { type: string, required: true }
      api_token: { type: secret, required: true }
```

Each tool gains an implicit `instance` parameter (string). When the LLM passes `instance: "prod"`, Forge looks up the matching row from `settings.instances` and **overlays its fields onto the settings namespace** before template expansion — so your tool spec keeps using `{settings.base_url}` / `{settings.api_token}` as if they were flat. Omitting the arg defaults to the first row.

Pair `instances` with `body_form_inject_from` for fully per-instance secret injection (each Jenkins instance has its own list of credentials to inject).

Secrets nested inside an instance row (e.g. `api_token: { type: secret }` as a sub-field) are encrypted at rest by the same AES-256-GCM pipeline as flat top-level secrets; the UI masks them with bullets and the Settings → Connectors panel renders a "Replace" button to change them.

#### Nested `instances` (rows of rows)

A sub-field of an `instances` schema can itself be `type: instances`, and the renderer handles the nesting. This is how Jenkins's `inject_params` lives inside each instance row:

```yaml
settings:
  instances:
    type: instances
    fields:
      name:     { type: string, required: true }
      base_url: { type: string, required: true }
      api_token: { type: secret, required: true }
      inject_params:               # nested instances inside a row
        type: instances
        fields:
          name:  { type: string, required: true }
          value: { type: secret, required: true }
```

UI: each top-level instance row expands to a form that includes a nested rows-list for its sub-collection. Encryption recurses, so per-row secrets in either level encrypt correctly.

### test block

A connector can ship a self-test so the Settings → Connectors UI's
**Test** button has something to call. Two probe kinds:

**`probe: http`** (default) — server issues a one-shot HTTP request
with `{settings.*}` expanded. Use for REST-API connectors with a
quick `/me`-style endpoint.

```yaml
test:
  description: "GET /api/v4/user — verifies token works"
  probe: http
  request:
    method: GET
    url: "{settings.base_url}/api/v4/user"
    headers:
      PRIVATE-TOKEN: "{settings.token}"
  ok_status: [200]                              # default [200]
  ok_template: "Authenticated as {{username}} ({{name}})"
  timeout_ms: 15000                              # default 15s
```

`ok_template` accepts `{{<json-path>}}` placeholders that resolve
against the parsed response body. Missing paths render `?`.

**`probe: browser`** — forwarded to the paired Forge browser
extension. The extension opens / acquires a tab matching
`host_match`, waits for navigation, and reports whether the final
URL contains `login_redirect`. Use for browser-side connectors
(Mantis, Teams, PMDB) where auth is the user's session cookie
rather than a token Forge can verify server-side.

```yaml
test:
  description: "Opens a Mantis tab and checks the session is alive."
  probe: browser
  timeout_ms: 30000                              # default 30s
```

No `request:` needed — the probe reuses the manifest's top-level
`host_match` + `login_redirect`.

**Extension wire contract** (for extension implementers):

```ts
// bridge method: 'connector.probe'
// params:
{
  pluginId: string;
  host_match: string;          // expanded with {settings.*}
  login_redirect?: string;     // expanded with {settings.*}
  runner: 'main' | 'isolated'; // inherits from manifest.runner
  timeout_ms: number;          // honour or cap, your call
}
// response:
{ ok: true,  url: '<final tab URL>' }
{ ok: false, error: 'login required' | '<other>', url?: string }
```

The extension's existing tab-acquisition logic already knows how
to handle `host_match` + `login_redirect`; the probe just runs the
acquire step and skips `executeScript`. If the extension isn't
connected, the Forge route surfaces a clear "install + sign in to
the Forge extension" message.

### shell protocol

For local CLI tools. **Every arg is templated independently** — no
shell injection.

```yaml
tools:
  git_log:
    protocol: shell
    parameters:
      repo: { type: string, required: true }
      n:    { type: number, default: 20 }
    command: ['git', '-C', '{args.repo}', 'log', '-n', '{args.n}', '--oneline']
    timeout_ms: 5000
```

## How you, the AI, install it

You have direct filesystem access to the running user's Forge data
directory. There are two paths:

### Path A — write the manifest directly (preferred for AI)

Find the data directory (usually `~/.forge/data/`, override via
`FORGE_DATA_DIR` env). Write:

```
<dataDir>/connectors/<id>/manifest.yaml
```

…then call the install-local API to register it:

```bash
curl -s -X POST http://localhost:8403/api/connectors/install-local \
  -H "X-Forge-Token: $TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -nc --arg yaml "$(cat <dataDir>/connectors/<id>/manifest.yaml)" '{yaml: $yaml}')"
```

(See `lib/help-docs/CLAUDE.md` for the auth pattern.)

### Path B — generate a zip for hand-installation

If the user wants to share the connector with a colleague or store
it in version control, package it as a zip:

```
my-connector.zip
├── manifest.yaml
├── README.md          (optional)
├── icon.svg           (optional)
└── tools/             (optional — separate files when scripts grow long)
    └── list_my_issues.js
```

`manifest.yaml` must be at the root. The user then uploads via the
"+ Upload" button in Settings → Connectors (or drags-and-drops onto
the panel).

## Checklist before declaring done

- [ ] `id` is lowercase + URL-safe
- [ ] At least one tool defined
- [ ] `description` reads like one a stranger would understand
- [ ] `version: "0.1.0"` (or whatever the user specifies)
- [ ] `host_match` + `login_redirect` set for browser connectors
- [ ] Settings include any secrets as `type: secret` (encrypted at rest)
- [ ] Run a 1-tool smoke test by triggering the chat agent to call it,
      and surface any error to the user
- [ ] Tell the user where the manifest was saved + that they can edit
      it under `<dataDir>/connectors/<id>/manifest.yaml`

## Iterating

When the user reports a bug ("the list_my_issues tool returned 0 rows"):

1. Open the page they were on (they can navigate, you can suggest URLs).
2. Use the browser extension's DOM inspector (or `mcp__chrome__` if
   available) to find a stable selector.
3. Edit the script body in the manifest. Bump `version` (patch bump).
4. Tell the user to either close + reopen the Settings panel
   ("Refresh" in Connectors), or just re-invoke the tool — the
   extension picks up the new manifest on next call.

## Limits and gotchas

- Browser scripts cannot use Chrome extension APIs (`chrome.*`) — only
  page-context globals.
- For sites that virtualise lists (react-window, ag-grid), scroll
  programmatically inside the script before scraping; otherwise you
  only see what's currently in the DOM.
- For strict-CSP sites (Teams, github.com), pure `eval` is blocked.
  Use `runner: isolated` (which the extension's MV3 sandbox enables)
  but you lose access to `window` globals (MSAL tokens, etc.) — stick
  to pure DOM extraction.
- If a connector breaks after a site redesign, only the YAML's
  `script` body needs to change. Bump version, save, retry — the
  registry-based update path is for connectors that came from a
  shared `forge-connectors` repo.

## Long-running tools — the `async` block (background watch)

A tool that kicks off work which finishes minutes later (a firmware
upgrade, a test run) should NOT hold the chat open. Declare an `async`
block: the tool runs and returns immediately (detach), and Forge
registers a background **watch** that polls until done, then reports
back into the originating chat session — no AI babysitting.

```yaml
upgrade:
  protocol: ssh
  async:
    poll: get_version              # another tool in THIS connector to poll
    poll_args:                     # built once from the trigger's args/result
      host: "{args.host}"          # {args.*}=trigger input, {result.*}=trigger return
    # completion test — one of:
    done_path: done                #   (a) poll-result path is truthy
    done_match:                    #   (b) value compare
      path: captured.build
      equals: "{result.captured.target_build}"
    fail_path: error               # optional: truthy = failed
    interval_sec: 60               # poll cadence (min 30)
    timeout_sec: 900               # overall deadline
    max_polls: 15                  # hard cap
    on_done: { mode: chat, message: "Done — build {poll.captured.build}." }
    on_fail: { mode: chat, message: "Not confirmed — check manually." }
    progress: { show: true, message: "Working… {poll_count}/{max_polls}" }
```

- `on_done`/`on_fail.mode`: `chat` (default — assistant replies in the
  session; a telegram-origin session replies on telegram), `tool`
  (chain another tool — `tool` + `args`, depth-limited), or `none`.
- `progress` shows an ambient status chip per poll — it does NOT enter
  the message thread or trigger the LLM.
- Templating: `{poll.*}` = latest poll result; `{poll_count}`/`{max_polls}`.
- Guards (not overridable to unbounded): max_polls, timeout,
  max_lifetime, consecutive-error cutoff, chain depth, global active cap.
- Watches persist in SQLite (survive restart) and are listed/cancelable
  in /chat's "Background watches" panel.
- Secrets: don't put a password in `poll_args` (it persists in the watch
  row). Rely on the connector's saved default credential instead.