Mechasmmechasm logoMechasmbeta
Back to Docs

Manage Tests with the CLI

Create, edit, list, inspect, and run hosted tests using explicit project IDs.

Every test command requires --project PROJECT_ID. Prompts can be supplied inline, from a file, or through standard input using --prompt-file -. Use Markdown for headings, lists, and clear steps; plain text also works.

Create and edit

mechasm tests create --project PROJECT_ID --name "Login flow" --prompt-file prompt.md --wait --json
mechasm tests edit --project PROJECT_ID TEST_ID --prompt "Also check password reset" --wait --json

Create requires a name and a prompt. It creates the test and starts generation and a run. Editing the prompt of an enabled test also starts generation and a run. Name, tags, and enabled-state changes do not start generation. Use --wait to wait when a command starts work.

Create and edit support --tags LIST, --name NAME, and --enabled true|false where applicable. Test generation and healing follow the same workflow as other Mechasm tests.

List and inspect

mechasm tests list --project PROJECT_ID --json
mechasm tests list --project PROJECT_ID --offset 50 --limit 50 --json
mechasm tests get TEST_ID --project PROJECT_ID --json

tests list returns 50 tests by default. --offset starts at 0; --limit accepts 1–100. tests get returns one test and its prompt.

Run an existing test

mechasm tests run TEST_ID --project PROJECT_ID --wait --json

Commands that start a run return its runId and workspace runUrl. Without --wait, they return while the run continues. Failed and stopped runs use exit code 5. See Runs, Logs & Artifacts to follow progress and retrieve evidence.

Safe retries and parallel work

Test create, edit, and run commands include an idempotency key. If a connection fails and the result is unknown, retry the operation with the same --idempotency-key KEY; do not generate a new key for that retry. When the API says the same operation is still in progress, the CLI retries that response while its 30-second retry window remains open. The delay starts at 250 ms and doubles up to 4 seconds. This time-based API retry has no fixed request count, does not automatically retry every network failure, and does not describe retries inside a hosted run.

Concurrent runs count against your workspace’s parallel-run capacity. Runs that use the same local app share its application state and data, so isolate test data when parallel runs could interfere. Concurrent edits to one test are separate updates; the last saved change wins.