Tests, review and completion evidence
Connect failure reproduction, regression scope and actual observations instead of reporting only passed.
Guide verification basis: Holmes-Kit 0.40.0 · 2026-11-05 · Installation target: latest at execution time
Choose a CLI, OS and installation mode to see the matching commands. This page does not inspect or execute on your computer.
AGY CLI ordinary integration is supported. AGY run start is refused in 0.40.0; this orchestration lesson has no executable AGY start procedure. Keep your choice, select a different CLI explicitly, or return to Basics.
Prerequisites, evidence and installation tradeoffs
Before you start
- Your own workspace and connected MCP session after the basics. Define HOLMES_CLI or PowerShell HolmesCli from the installation step in this terminal and verify its file and version.
- An approved A-SPEC and a real test environment
Produce a review package separating implemented, tested and runtime-observed behavior.
Documented for 0.40.0 · release contracts reviewed 2026-11-04. Native evidence is the historical 0.39.0 run of 2026-11-01. Read-only binding matched for Codex and Claude in an independent macOS workspace. Claude hook events were observed but routing is unverified; Codex hooks were not observed. Native revalidation on 0.40.0 and RED/GREEN acceptance of a small governed change remain incomplete. Windows/Linux instructions have no native acceptance run.
- 1. Separate user folder
- Keeps project dependencies and global npm installation unchanged; separate versioned folder. Manage its path and updates separately; retain the folder while workspaces use it.
- 2. Project local
- Project/team versioning through a lockfile; recommended for the practice project. Changes package.json and lockfile; may introduce npm metadata to a non-Node project.
- 3. Global npm
- Convenient shared installation across projects. Check write permissions and update impact on all projects sharing this installation.
Package location and client settings are separate. Even project-local Codex wiring may change user-level settings.
Step 1 / 4
Reproduce a behavioral failure
Choose a failing situation for each success criterion.
Ask your agent
Verify the current approved A-SPEC and persist maintenance_analyze. Anchor the test first line with its actual ID, then record assertion RED with test_run.Expected result — inspect your actual output
- The failure directly identifies the missing required behavior.
If the result differs
A package cannot load or a fixture is invalid.
Fix the test environment and rerun the behavioral check; do not use an environment failure as RED evidence.
What the agent does / terms
After approved scope and maintenance analysis, record assertion RED with official test_run.
- MCP
- The connection through which your AI calls Holmes tools.
- REQ → H-SPEC → A-SPEC → T-SPEC
- Requirement → high-level design → implementation design → test specification. Use actual IDs returned by the tools.
- Hook / binding
- A pre/post execution check / matching this session to the intended project and server.
Step 2 / 4
Verify the improvement with the same test
Review the T-SPEC and implementation scope.
Ask your agent
Implement minimally under the approved T-SPEC and rerun the same test. If an assertion changes, explain why and how it still checks the original requirement.Expected result — inspect your actual output
- The same behavior test is GREEN and edits remain within approved scope.
If the result differs
The test passes but another input fails.
Add boundary and failure inputs, fix the implementation and record new results.
What the agent does / terms
After mirroring T-SPEC approval, implement the minimal change and rerun the RED test.
- MCP
- The connection through which your AI calls Holmes tools.
- REQ → H-SPEC → A-SPEC → T-SPEC
- Requirement → high-level design → implementation design → test specification. Use actual IDs returned by the tools.
- Hook / binding
- A pre/post execution check / matching this session to the intended project and server.
Step 3 / 4
Check regressions and usage paths
Choose real user paths and existing behavior to preserve.
Ask your agent
Find actual verification commands in project scripts or test configuration and run them. For UI changes, inspect the real screen and direct URLs, and state what each check does not establish.Expected result — inspect your actual output
- Per-command results and actual screen observations are separated.
If the result differs
Environment limits prevent browser or service execution.
Label the path unverified and record reproduction conditions; do not turn build success into operational success.
What the agent does / terms
Run relevant regressions, type checks and build; inspect UI changes in a browser.
- MCP
- The connection through which your AI calls Holmes tools.
- REQ → H-SPEC → A-SPEC → T-SPEC
- Requirement → high-level design → implementation design → test specification. Use actual IDs returned by the tools.
- Hook / binding
- A pre/post execution check / matching this session to the intended project and server.
Step 4 / 4
Close independent review and remaining findings
Review important open findings and the deployment decision.
Ask your agent
Assemble the final diff, RED/GREEN, regressions, build, runtime observations and unverified items for review. Separate independent review results and open critical/important findings; keep deployment separate.Expected result — inspect your actual output
- Every completion claim has actual evidence and open findings remain visible.
If the result differs
The diff changes after review and no longer matches the evidence.
Refresh affected verification and review, and state the scope of earlier results.
What the agent does / terms
Provide requirement, diff and logs to a reviewer; never record self-review as independent review.
- MCP
- The connection through which your AI calls Holmes tools.
- REQ → H-SPEC → A-SPEC → T-SPEC
- Requirement → high-level design → implementation design → test specification. Use actual IDs returned by the tools.
- Hook / binding
- A pre/post execution check / matching this session to the intended project and server.
Recorded Holmes Lab change
A-SPEC-101.21 restored three hidden primary links on home—Technical journey, Work and Practice—removed graph fallback JSON links and expanded Practice content. RED, regression and build records in docs/evidence/engineering-perspective are historical evidence. Running these instructions in your project is a reproduction, not that historical execution.
Related case ↗Continue with your project
Use Adoption for existing code without specs, or Requirements when specifications already exist. Other workflows require their stated prerequisites.