# Privacy Redactor / 隐私脱敏器

Local-first sensitive information redaction for text, local documents, logs, support tickets, and app integrations.

在把文本、日志、订单、工单、邮件或文档内容发送给 AI、应用、同事或公开社区之前，先在本地移除敏感信息。

## What It Does

Privacy Redactor is a small JavaScript redaction engine that can run in the browser, Node.js, a CLI, a Web Component, or the included browser extension prototype.

It is designed around four rules:

- Local-first: redaction runs where the user is, before data crosses a network boundary.
- Explainable: every redacted entity reports its type, location, confidence score, and replacement.
- Safe metadata by default: raw matched values and surrounding context are opt-in.
- Mode-based: apps can choose a narrow mode for fewer false positives or a broad mode for stronger protection.

## Modes

| Mode | Purpose | Examples |
|---|---|---|
| `general` | Common personal data | email, phone, card number, date of birth, address, SSN, national ID, IPv4/IPv6 address |
| `travel` | Travel and booking data | PNR, booking reference, ticket number, passport number, passenger name, loyalty number, QR payload |
| `developer` | Developer secrets and logs | API keys, passwords, bearer tokens, JWTs, URLs with embedded credentials, OpenAI keys, Google keys, Stripe keys, GitHub tokens, Slack tokens, AWS access key IDs, private key blocks |
| `finance` | Banking and payment operations | IBAN, routing number, bank account, SWIFT/BIC, tax ID, invoice, transaction ID |
| `healthcare` | Patient and insurance workflows | MRN, patient ID, NPI, prescription number, claim number, insurance member ID |
| `legal` | Legal support workflows | case number, docket number, client matter number, court file number, contract number |
| `hr` | HR and employment workflows | employee ID, payroll ID, candidate ID, benefits ID |
| `education` | Student and admissions workflows | student ID, application ID, transcript ID, financial aid ID |
| `all` | Broad default protection | all currently available rule groups |

Default mode: `all`.

## Example

```text
Passenger: PENG/YANG
Booking reference: ABC123
api_key=sk_test_1234567890abcdefghijklmnop
Email: yang@example.com
Flight LH455 SFO -> FRA

Passenger: [PASSENGER_NAME]
Booking reference: [BOOKING_REFERENCE]
api_key=[API_KEY]
Email: [EMAIL]
Flight LH455 SFO -> FRA
```

## Install

```bash
npm install privacy-redactor
```

For local development:

```bash
npm run verify
npm test
npm run evaluate
npm run smoke:cli
npm run smoke:demo
npm run smoke:extension
npm run smoke:package
npm run check:privacy
npm run check:release
npm run demo
```

Open `http://127.0.0.1:5174/demo/`.

Public demo:

`https://uilpx.github.io/Privacy-Redactor/demo/`

Third-party integration demo:

`https://uilpx.github.io/Privacy-Redactor/demo/integration.html`

Framework demos:

`https://uilpx.github.io/Privacy-Redactor/demo/frameworks.html`

Extension test page:

`https://uilpx.github.io/Privacy-Redactor/demo/extension-test.html`

## Library Usage

```js
import { redactText } from "privacy-redactor";

const result = redactText("Email: yang@example.com api_key=sk_test_1234567890abcdefghijklmnop", {
  mode: "developer"
});

console.log(result.redactedText);
console.log(result.entities);
```

TypeScript declarations are included for the main entry point and subpaths such as `privacy-redactor/extract`, `privacy-redactor/component`, and `privacy-redactor/adapters`.
Use `getAvailableEntityTypes("developer")` when an integration needs to build a mode-specific entity picker for `includeTypes` or `excludeTypes`.
Use `validateRedactionOptions(options)` before saving user-selected modes, sensitivity presets, entity filters, or custom rules.

Use sensitivity presets when you need a stricter false-positive tradeoff:

```js
redactText(sourceText, { mode: "all", sensitivity: "strict" });
redactText(sourceText, { mode: "all", sensitivity: "balanced" });
redactText(sourceText, { mode: "all", sensitivity: "aggressive" });
```

## Supported Local Files

The CLI and Web Component can read local files and redact extracted text without sending data to a server.

Supported inputs:

- Plain text and text-like files: `.txt`, `.md`, `.csv`, `.tsv`, `.json`, `.xml`, `.html`, `.log`, `.env`, `.ini`, `.conf`, `.sql`, `.yaml`, `.yml`, `.ics`, `.eml`
- Rich text: `.rtf`
- Word documents: `.docx`
- PDF: best-effort text extraction from text-based `.pdf` files

Current limits:

- Legacy `.doc` files are not supported. Save as `.docx` or export text first.
- Scanned PDFs and image-only documents need a local OCR adapter before redaction.
- DOCX files can be written to a new redacted `.docx` file; the original file is never overwritten.
- PDF files currently return redacted extracted text; PDF file rewriting is not supported.

Extraction helpers are exported for app integrations:

```js
import { extractTextFromBuffer, redactDocxBuffer, redactStructuredData, redactText } from "privacy-redactor";

const text = await extractTextFromBuffer(bytes, "ticket.docx");
const result = redactText(text, { mode: "all" });

const docx = await redactDocxBuffer(bytes, { mode: "all" });
await saveFileSomewhere(docx.redactedBuffer);
```

Advanced integrations can pass `pdfTextExtractor` or `ocrTextExtractor` adapters for PDF.js, Tesseract.js, native OCR, or another local-only extraction layer.
See `privacy-redactor/adapters` for small adapter factories that accept host-provided PDF.js or Tesseract instances.

Entity reports do not include the original matched value by default:

```js
redactText("Email: yang@example.com").entities[0];
// {
//   type: "EMAIL",
//   label: "Email address",
//   start: 7,
//   end: 23,
//   score: 0.98,
//   replacement: "[EMAIL]"
// }
```

For local debugging only, you can opt in to raw matched values and context:

```js
redactText(sourceText, {
  includeRaw: true,
  includeContext: true
});
```

Use a specific mode:

```js
redactText(sourceText, { mode: "general" });
redactText(sourceText, { mode: "travel" });
redactText(sourceText, { mode: "developer" });
redactText(sourceText, { mode: "finance" });
redactText(sourceText, { mode: "healthcare" });
redactText(sourceText, { mode: "legal" });
redactText(sourceText, { mode: "hr" });
redactText(sourceText, { mode: "education" });
redactText(sourceText, { mode: "all" });
redactText(sourceText, { mode: ["general", "travel"] });
```

Customize replacements:

```js
redactText(sourceText, {
  mode: "all",
  numberedPlaceholders: true,
  replacementMap: {
    EMAIL: "[CONTACT]",
    API_KEY: "[SECRET]"
  }
});
```

Preserve repeat relationships without exposing raw values:

```js
redactText("one@example.com two@example.com one@example.com", {
  includeTypes: ["EMAIL"],
  numberedPlaceholders: true,
  consistentPlaceholders: true
}).redactedText;
// "[EMAIL_1] [EMAIL_2] [EMAIL_1]"
```

`includeTypes` and `excludeTypes` are case-insensitive across the library API, Web Component attributes, and CLI.

Add project-specific rules without forking the package:

```js
redactText("Workspace ID: WS-123456", {
  mode: "general",
  customRules: [
    {
      type: "WORKSPACE_ID",
      label: "Workspace identifier",
      score: 0.87,
      valueGroup: 1,
      pattern: /Workspace ID:\s*(WS-\d{6})/i
    }
  ]
});
```

Custom rules can use built-in validators, validator functions, or named `customValidators`.
The library automatically runs custom regular expressions globally, so a non-`g` pattern can still match multiple fields.

Redact structured JSON-like data while keeping object shape:

```js
const result = redactStructuredData({
  user: {
    email: "yang@example.com"
  },
  session: {
    token: "short-token"
  }
});

console.log(result.redactedValue);
// { user: { email: "[EMAIL]" }, session: { token: "[SENSITIVE_FIELD]" } }
console.log(result.entities[0].path);
// ["user", "email"]
```

Structured redaction also redacts values under sensitive keys such as `password`, `token`, `secret`, and `apiKey` by default.
Object keys are used as local detection context, so fields like `{ phone: "+1 415 555 0123" }` can be detected without embedding the label in the value.

Create safe local reports:

```js
import { createRedactionReport, formatRedactionMarkdown, redactText } from "privacy-redactor";

const result = redactText(sourceText, { mode: "all" });
const jsonReport = createRedactionReport(result);
const markdownReport = formatRedactionMarkdown(result);
```

Reports omit raw matched values and source context by default.

## Web Component

With a local module:

```html
<script type="module" src="./src/component.js"></script>

<privacy-redactor mode="all" sensitivity="balanced" numbered-placeholders consistent-placeholders></privacy-redactor>
```

With a CDN:

```html
<script type="module" src="https://cdn.jsdelivr.net/npm/privacy-redactor/src/component.js"></script>

<privacy-redactor></privacy-redactor>
```

For production pages, pin a tested version instead of relying on the latest CDN package.

Listen for local redaction results:

```js
document.querySelector("privacy-redactor").addEventListener("redacted", event => {
  console.log(event.detail.redactedText);
  console.log(event.detail.entities);
});
```

The component does not send data anywhere. It only processes strings in the browser tab.
It can open supported local files, extract readable text, and redact the extracted text locally.
The input pane shows source name, size, character count, and line count after typing, sample loading, or file import.
Detected entities can be reviewed one by one or in bulk, so a user can keep lower-risk values while redacting higher-risk fields.
The review list can be filtered by entity type and shows each entity type, replacement, range, confidence label, and confidence explanation.
The component can copy redacted text, a safe JSON report, or a Markdown report.
Host pages can set `mode`, `sensitivity`, `include-types`, `exclude-types`, `numbered-placeholders`, and `consistent-placeholders` attributes, or assign `element.customRules` before calling `element.redact()`.

## Browser Extension Prototype

The repository root can be loaded as an unpacked Manifest V3 extension for local selected-text redaction.

```text
chrome://extensions -> Developer mode -> Load unpacked -> select this repository root
```

The extension popup can read the current page selection or focused text field after a user action, redact it locally, write redacted text back to the focused field, and copy redacted text or a safe JSON report. It does not upload text or call remote services.

## CLI

```bash
node ./bin/privacy-redactor.js --help
node ./bin/privacy-redactor.js --version
node ./bin/privacy-redactor.js --list-types
node ./bin/privacy-redactor.js --mode=developer --list-types
node ./bin/privacy-redactor.js --check ./samples/support-ticket.txt
node ./bin/privacy-redactor.js --config=./samples/privacy-redactor.config.json ./samples/support-ticket.txt
node ./bin/privacy-redactor.js ./samples/flight-booking.txt
node ./bin/privacy-redactor.js ./document.docx
node ./bin/privacy-redactor.js ./ticket.pdf
node ./bin/privacy-redactor.js ./samples/industry-records.txt
node ./bin/privacy-redactor.js --mode=developer ./logs.txt
node ./bin/privacy-redactor.js --mode=all --sensitivity=strict ./support-ticket.txt
node ./bin/privacy-redactor.js --include-types=EMAIL,API_KEY ./support-ticket.txt
node ./bin/privacy-redactor.js --exclude-types=IP_ADDRESS ./support-ticket.txt
node ./bin/privacy-redactor.js --include-types=EMAIL --numbered-placeholders --consistent-placeholders ./ticket.txt
node ./bin/privacy-redactor.js ./document.docx --output=./document.redacted.docx
node ./bin/privacy-redactor.js --structured ./payload.json
node ./bin/privacy-redactor.js --jsonl ./events.jsonl
node ./bin/privacy-redactor.js --format=markdown ./support-ticket.txt
node ./bin/privacy-redactor.js --format=json ./support-ticket.txt
node ./bin/privacy-redactor.js --json --mode=all ./support-ticket.txt
node ./bin/privacy-redactor.js --json --include-raw ./support-ticket.txt
node ./bin/privacy-redactor.js -- -dash-prefixed-file.txt
cat ./samples/flight-booking.txt | node ./bin/privacy-redactor.js
```

`--format=json` and `--format=markdown` produce safe reports without raw matched values by default. `--json` is kept for compatibility and returns the raw result object.
`--help` prints available modes and options. `--version` prints the installed package version. `--list-types` prints entity types available for the selected mode.
`--check` prints a safe detection summary without raw values or redacted source text, then exits with status `1` if sensitive fields are found and `0` if none are found.
`--config=path/to/privacy-redactor.json` reads local JSON defaults for `mode`, `sensitivity`, `format`, `includeTypes`, `excludeTypes`, `numberedPlaceholders`, `consistentPlaceholders`, and local output flags. Explicit CLI option values override matching config values. See `samples/privacy-redactor.config.json` for a bundled starter config.
`--include-types` and `--exclude-types` accept comma-separated entity types and work across text, DOCX output, structured JSON, and JSONL modes. Type names are case-insensitive; unknown type names fail fast with a `--list-types` hint.
`--numbered-placeholders` and `--consistent-placeholders` expose the same repeat-preserving placeholders as the library API.
`--structured` parses input as JSON and prints the redacted JSON value by default; combine it with `--format=json` for a structured report with entity paths.
`--jsonl` parses newline-delimited JSON and prints one redacted JSON value per line; combine it with `--format=json` for per-record line numbers and entity paths.
Use `--` before an input file path that starts with a dash.
Unknown options, empty option values, and multiple input files fail fast with a usage hint.

## Rule Evaluation

Synthetic regression fixtures can be checked independently from the full test suite:

```bash
npm run evaluate
```

Use this before publishing rule changes or reviewing false-positive / false-negative contributions.
Use `npm run verify` before a release to run tests, fixture evaluation, CLI smoke tests, demo smoke tests, extension smoke tests, package install smoke tests, documentation checks, privacy boundary checks, release consistency checks, and package dry-run together.

## Project Goals / 项目目标

- Build a general-purpose local redaction engine that other apps can embed.
- Make pre-send privacy review normal for AI workflows.
- Provide useful built-in modes without locking the project to one industry.
- Keep the core dependency-free and easy to audit.
- Preserve non-sensitive utility where possible, such as flight numbers, route facts, log structure, timestamps, and error messages.

## Implementation Approach / 实现方法

Privacy Redactor currently uses deterministic local detectors:

- Lightweight local text extraction for text-like files, `.rtf`, `.docx`, and text-based `.pdf`.
- Regular expressions for context-bound identifiers.
- Luhn validation for payment cards.
- Plausibility validators for dates, SSNs, China resident identity card numbers, identifiers, and generic secrets.
- Rule groups mapped to modes.
- Industry-specific context-bound modes for finance, healthcare, legal, HR, and education.
- Sensitivity presets for stricter or broader detection thresholds.
- Overlap resolution that keeps the highest-confidence entity.
- Structured output for UI review, logs, and app integration.

Future adapters can add stronger PDF parsing, local OCR, locale packs, and optional ML/NLP detectors. Remote processing should remain opt-in and outside the core package.

## Explicit Non-Goals For v0.1

- It is not a legal compliance product.
- It does not guarantee complete sensitive information removal.
- It only redacts strings or locally extracted text.
- It does not modify binary PDFs, office documents, or images in place.
- It does not perform cloud OCR.
- It does not use remote AI services.

## Documentation

- [Project goals](./docs/project-goals.md)
- [Implementation](./docs/implementation.md)
- [API reference](./docs/api.md)
- [Calling modes](./docs/calling-modes.md)
- [Integration guide](./docs/integration.md)
- [Browser extension prototype](./docs/browser-extension.md)
- [Framework examples](./docs/framework-examples.md)
- [Manual verification](./docs/manual-verification.md)
- [Privacy boundaries](./docs/privacy-boundaries.md)
- [Rule authoring guide](./docs/rule-authoring.md)
- [Release checklist](./docs/release-checklist.md)
- [Security review checklist](./docs/security-review-checklist.md)
- [Roadmap](./docs/roadmap.md)
- [Threat model](./docs/threat-model.md)
- [Market scan](./docs/market-scan.md)
- [Launch plan](./docs/launch-plan.md)

## Repository Name

Recommended GitHub repository:

`privacy-redactor`

Recommended npm package:

`privacy-redactor`

## License

MIT