Run destructive lifecycle checks only in an isolated Sandbox and preserve evidence without credential-bearing response dumps.
What you will learn
- Understand the executed contract coverage and its limits.
- Reproduce safe contract validation in your own development environment.
Prerequisites
- Read API conventions.
- Complete the Sandbox quickstart.
Current executed coverage
The project test suite sends105 real HTTP requests through the unchanged front controller, Router, controller, services, Signer and local MySQL. It validates all 11 successful operation shapes and all 22 licensing/router error identifiers with 954 assertions. It checks request IDs, JSON content types, stateless behavior, missing permissive CORS, rate headers and error-header omissions.
Eight real limiter buckets are primed to the boundary using fresh synthetic test IPs/keys through the actual RateLimiter service; the next HTTP request must fail429. No limiter history is cleared and no public hosting load test is run.
Actual MySQL row-lock races exercise missing-row errors after controller preflight. A second local PHP request process reproduces simultaneous same-key creation: two licenses can commit before one response fails500. An expired idempotency row also reproduces a duplicate-before 500. These are disclosed product findings, not fixes.
An intentionally missing public-key path in a loopback-only test router exercises the actual global500 boundary. No real signing key is deleted and no hosted outage is induced. Scheduled cron delivery is suppressed only by the local harness, so audit/API metadata/queued events are real but no external webhook/email/payment is sent.
Sample and schema checks
Five core transport programs each execute all 11 operations and four refusal paths. Ten additional REST languages each execute public-key/activate/validate/deactivate plus wrong-email and post-deactivation refusals, with real local redirect/untrusted-TLS guards. Native Windows/Linux execution is not claimed by macOS tests.
Four SDK error-UX programs execute50 synthetic policy inputs using actual result models, plus actual NOT_ACTIVATED checks from fresh SDK storage. This tests handling, not runtime emission of every unrelated installer/billing/operational error. Those namespaces are clearly separated in Errors.
OpenAPI JSON/YAML equality, all examples, source-hash/route coverage and25 schema definitions are checked. Negative response fixtures and an intentionally missing operationId prove that validators fail rather than pass every input.
Running the repository tests
The repository contains tests/docs-p6-contract.py, tests/docs-p6-spec.py, tests/docs-p6-samples.py, tests/docs-p6-rest-languages.py, tests/docs-p6-errors.py and tests/docs-p6-collections.py. These are local development tools, not scripts to upload or run against existing customer data. The exact guarded fixture commands and pinned dependencies are recorded in docs-program/tooling/p6/README.md in the project source.
- Create a new explicitly named local test database with the guarded fixture setup. Expected: setup refuses an existing database and does not print credentials.
- Start the loopback-only PHP test router against that fixture. Expected: no production URL or Live database is used.
- Build the generated OpenAPI and exact source examples with pinned local toolchains. Expected: compilation/lint failures stop the run.
- Run the suites against the same explicit local Sandbox fixture. Expected: sanitized assertion/status evidence, not raw token/key output.
- Review known defects separately from schema drift. Expected: no silent licensing/payment/SDK logic change in a documentation patch.
If you only consume the API, use the collections in your own Sandbox instead of reproducing server fault/locking tests.
Expected result
Contract drift and refusal regressions are detected, while test-only faults and actual production behavior remain clearly distinguished.
Common mistakes
- Running a mutation suite against a real customer workspace or Live key.
- Claiming mocked fault/UX cases are real provider outages or every OS execution.
- Fixing a product defect silently to make a documentation test green.
Security notes
Never upload test routers, fixture keys or private reports. Keep new local fixtures separate from prior data; do not reset the project database.
Troubleshooting
| Symptom | Cause | Action |
|---|---|---|
| Guard refuses a database | Not an explicitly isolated P6 fixture | Use a new local guarded fixture, not production |
| Schema/example failure | Contract drift or malformed sample | Inspect the sanitized path/code and source builder |
| Creation500 with extra local row | Known idempotency defect reproduced | Record the finding; do not hide it with retries or a new key |
Next steps
Review the endpoint reference and signature verification.