Test Framework
Scaffolder

1. Requirements

Install the tools for the language you want to generate.

Install k6

brew install k6                         # macOS
winget install k6 --source winget       # Windows

2. Create a project

Recommended

mkdir automation-projects
cd automation-projects
npx test-framework-scaffolder@latest

Optional global install

npm install --global test-framework-scaffolder@latest
test-framework-scaffolder

The project is created under the directory where the command is launched. Run pwd first if unsure.

3. Prompt choices

4. First run

cd "/path/to/project"
npm install
npx playwright install
npm test

cd "/path/to/project"
python3 -m pip install -r requirements.txt
python3 -m playwright install
python3 -m pytest

cd "/path/to/project"
mvn test

cd "/path/to/project"
docker compose up -d
npm run test:grafana

Open in VS Code

code "/absolute/path/to/project"

If code is unavailable, select VS Code → File → Open Folder.

5. Project structure

project/
├── config/run.config.json       environment and execution settings
├── test-data/
│   ├── smoke.json
│   └── uat.json
├── utils/CommonMethods.*        reusable helpers
├── pages/                       UI selection only
├── api/                         API selection only
├── features/ and steps/         BDD projects
├── tests/                       executable tests
├── reporters/ and scripts/
├── playwright.config.*          TypeScript / JavaScript
└── .github/workflows/           optional CI

Only selected capabilities are generated. API-only projects do not contain UI files.

Generated k6 structure

k6-project/
├── config/
│   ├── run.config.json          environment, profile, thresholds
│   └── load-profiles.js         smoke, load, stress, spike
├── test-data/
│   ├── smoke.json
│   └── uat.json
├── lib/performance-app.js       reusable requests and checks
├── tests/                       thin k6 entry point
├── features/ and steps/         BDD projects only
├── observability/
│   ├── prometheus.yml
│   └── grafana/                 provisioning and dashboard
├── scripts/run-k6.mjs           environment/profile runner
├── docker-compose.yml
└── .github/workflows/k6.yml     optional CI

6. Run configuration

Edit config/run.config.json. Test commands read it automatically.

{
  "environment": "smoke",
  "timeout": 30000,
  "actionTimeout": 10000,
  "navigationTimeout": 30000,
  "headless": true,
  "retries": 0,
  "ciRetries": 2,
  "fullyParallel": true,
  "workers": null
}

k6 run configuration

{
  "environment": "smoke",
  "profile": "smoke",
  "summaryTrendStats": ["avg", "min", "med", "max", "p(90)", "p(95)", "p(99)"],
  "thresholds": {
    "http_req_failed": ["rate<0.01"],
    "http_req_duration": ["p(95)<500", "p(99)<1000"],
    "checks": ["rate>0.99"]
  }
}

environment selects the matching JSON data file. profile selects a workload from config/load-profiles.js. A failed threshold makes the command and CI job fail.

7. Add any environment

1. Copy test-data/smoke.json.
2. Rename it, for example test-data/crm.json.
3. Set "environment": "crm" in run.config.json.
4. Run the normal command.

{
  "baseUrl": "https://crm.example.com",
  "apiBaseUrl": "https://api.crm.example.com",
  "apiExpectedStatus": 200,
  "validUser": {
    "username": "standard_user",
    "password": "use-a-secret-variable"
  }
}

k6 environment data

{
  "baseUrl": "https://test.k6.io",
  "endpoint": "/",
  "expectedStatus": 200,
  "thinkTimeSeconds": 1
}

Add any name such as test-data/crm.json, then set "environment": "crm". The normal npm test command discovers it automatically.

8. Run tests

TypeScript and JavaScript

Python

python3 -m pytest                       # all
python3 -m pytest tests/test_login.py   # one file
python3 -m pytest -m smoke              # marker
TEST_ENV=crm python3 -m pytest          # environment

Java

mvn test                                # all
mvn -Dtest=LoginTest test               # one class
mvn test -Dcucumber.filter.tags="@smoke" # BDD tag
mvn -Denv=crm test                      # environment

k6 performance

npm test                                # run config defaults
npm run test:smoke                      # smoke profile
npm run test:load                       # load profile
npm test -- --env uat --profile stress # explicit data + profile
npm run test:grafana                    # publish live metrics

Set environment and profile in config/run.config.json. Any matching test-data/<environment>.json is selected automatically. Grafana is available at http://localhost:3000 after Docker Compose starts.

9. k6 + Prometheus + Grafana

k6 executes JavaScript performance journeys. Prometheus receives the live metrics and Grafana displays the provisioned dashboard.

BDD and Non-BDD

Available load profiles

Start and use the dashboard

1. Run npm run dashboard:up.
2. Run npm run test:grafana.
3. Open http://localhost:3000.
4. Sign in with local credentials admin / admin.
5. Open Performance → k6 Performance Overview.
6. Stop the services with npm run dashboard:down.

The dashboard shows request rate, p95 duration, failed-request rate, and active virtual users. The runner uses k6's experimental-prometheus-rw output and sends metrics to http://localhost:9090/api/v1/write.

10. Reports and diagnostics

• reports/errors.log records failures.
• test-results/ retains failure screenshots, video, and traces where supported.
• Open a trace: npx playwright show-trace path/to/trace.zip.
• HTML report: npx playwright show-report.
• Allure: npm run allure, then npm run allure:open.
• k6 thresholds fail the command when service-level goals are not met; Grafana shows live request rate, latency, failures, and VUs.

Traces combine actions, DOM snapshots, console events, and network requests.

11. Add tests

Quality checks

npm run typecheck
npm run lint
npm test

12. GitHub Actions

The optional workflow installs dependencies, runs the chosen environment, and uploads reports. Push the project, open Actions, choose Run workflow, and enter smoke, uat, or another test-data filename. Store credentials under Settings → Secrets and variables → Actions.

A generated k6 workflow uses grafana/setup-k6-action and provides separate environment and profile inputs. Choose smoke, load, stress, or spike from the Actions screen. The CI command fails when a configured performance threshold is crossed.

13. Publish generator updates

Run publishing commands from the folder that contains this package's package.json. npm requires a new version for every release and requires account 2FA or an appropriate publishing token.

cd "/path/to/NPM Scaffolds pacakge"
npm whoami
npm run typecheck
npm run lint
npm test
npm pack --dry-run
npm version patch
npm publish --access public

Verify the registry with npm view test-framework-scaffolder version. The npm page and its README change only after the new version is successfully published.

14. Troubleshooting

15. Quick reference

npx test-framework-scaffolder@latest

code "/absolute/project/path"

config/run.config.json
test-data/<environment>.json

npm test
python3 -m pytest
mvn test
npm run test:grafana

npm run dashboard:up
npm run test:grafana
npm run dashboard:down

Recommended flow: generate → open the printed path → install a browser for UI tests → review configuration → run the sample → replace sample data and coverage.