From 02148210a4e9d8590d6178b8b8b8943fe442b259 Mon Sep 17 00:00:00 2001 From: ScriptedAlchemy Date: Wed, 12 Aug 2026 04:11:14 +0000 Subject: [PATCH 001/110] docs: propose Rstack context engine RFC --- docs/rfcs/0001-rstack-context-engine.md | 1194 +++++++++++++++++++++++ scripts/dictionary.txt | 10 + 2 files changed, 1204 insertions(+) create mode 100644 docs/rfcs/0001-rstack-context-engine.md diff --git a/docs/rfcs/0001-rstack-context-engine.md b/docs/rfcs/0001-rstack-context-engine.md new file mode 100644 index 00000000..18abfc4b --- /dev/null +++ b/docs/rfcs/0001-rstack-context-engine.md @@ -0,0 +1,1194 @@ +# RFC 0001: Rstack context engine + +| Field | Value | +| ------- | ---------------------------------------------------------------------------------- | +| Status | Proposed | +| Created | 2026-08-12 | +| Target | `rstack`, Rspack 2, Rsbuild 2, Rslib 1, Rstest 0.11, Rslint 0.7, Rsdoctor 2 | +| Scope | Headless build, lint, test, reachability, and package-contract evidence for agents | + +## Summary + +This RFC proposes a headless Rstack Context Engine that turns facts from Rsbuild, Rspack, Rslib, +Rslint, Rstest, and Rsdoctor into versioned evidence snapshots. One workspace-bound MCP server +exposes compact queries over those snapshots. Codex and Claude plugin bundles add task-oriented +skills that teach models how to combine the evidence safely. + +The first product use case is unused and dead-code investigation. The engine does not equate +"not observed" with "dead." It keeps the following claims independent: + +1. Is the definition reachable from a configured production root? +2. Is it reachable only from tests, examples, benchmarks, or development tooling? +3. Is it part of a published or otherwise protected public contract? +4. Was it shipped in a specific build and runtime? +5. Was it executed by a specific test capture? +6. Did the optimizer retain it for side effects or because analysis was incomplete? + +Rsdoctor supplies the canonical build-analysis data and remains the optional rich report viewer. +Its GUI is not required by the context engine, MCP server, Codex plugin, Claude plugin, or CI. + +## Motivation + +Rstack already presents one CLI and one configuration file for the JavaScript toolchain, but the +underlying tools expose different kinds of useful information: + +- Rspack knows the exact resolved production graph, chunks, runtimes, used exports, optimization + bailouts, and emitted assets. +- Rsdoctor enriches that graph with bundle, package, loader, plugin, rule, and tree-shaking data. +- Rslib knows which library variants are shipped and how package exports and externalization define + a consumer-facing contract. +- Rslint has high-confidence lexical and type-aware diagnostics, including unused local symbols and + safe fix information. +- Rstest knows the test projects, related-test graph, results, coverage, retries, snapshots, and + development-only consumers. + +Today an agent must invoke those tools separately, parse incompatible output, infer freshness, and +reconstruct causal links. Raw output also encourages unsafe conclusions: an unused export in one +browser build might be a public library API, a server-only export, a dynamic entry, or a test-only +helper. + +The context engine makes the combined evidence queryable without placing large logs, graphs, or +source files into the model context. + +## Goals + +- Provide one local, workspace-bound MCP surface for Rstack project intelligence. +- Reuse Rsdoctor's data model and headless analyzers instead of rebuilding its GUI or collectors. +- Model production, non-production, public-contract, build, and execution evidence independently. +- Support Rstack applications, libraries, multi-environment builds, and workspaces. +- Preserve user configuration and plugin order while adding opt-in passive observers. +- Keep build, lint, and test states independently fresh during development. +- Return small, typed, paginated answers with provenance, confidence, completeness, and source + locations. +- Offer task-oriented Codex and Claude skills for unused code, build analysis, impact analysis, + diagnostics, and test selection. +- Fail open: a collector or context-engine failure must not fail the user's build, dev server, lint, + or test command. +- Establish explicit safety boundaries for repository trust, source access, command execution, and + mutation. + +## Non-goals + +- Replacing the Rsdoctor report UI. +- Exposing MCP through an Rsbuild development-server route. +- Proving arbitrary local-symbol elimination from source maps or minified output. +- Treating a single build, test run, or coverage capture as proof that code is globally dead. +- Starting builds, tests, watchers, or long-lived report servers automatically when an agent session + opens. +- Sending source, configuration, environment variables, or build reports to a remote service. +- Replacing Knip, dependency-cruiser, CodeQL, or architecture-policy tools. Their evidence may be + integrated later as additional producer facets. +- Editing user configuration files to install instrumentation. + +## Design principles + +### Facts first, decisions later + +The compilers and runners collect facts. A workspace-level analyzer makes conclusions only after +merging all relevant product and non-product captures. + +This follows the core architecture of Astral's Hawk: each compiler invocation emits a fragment, +then a separate graph analysis combines production and non-production fragments before deciding +whether public Rust APIs are dead or unnecessarily visible. Rstack applies the same separation to +JavaScript build, library, lint, and test evidence while accounting for dynamic imports, CommonJS, +package exports, side effects, multiple runtimes, and external consumers. + +### Headless first + +Structured artifacts and APIs are the system of record. Visual reports project the same evidence; +they are never required to answer a query. + +### Progressive disclosure + +The model initially receives a short status or finding summary. It requests evidence paths, +diagnostics, logs, modules, or source only when needed. The server never returns an unbounded raw +Rspack Stats object or complete Rsdoctor report. + +### Honest uncertainty + +The engine uses categorical confidence (`exact`, `derived`, `inferred`, or `unknown`) and explicit +analysis bounds. It does not invent a single dead-code probability. + +### Stable meaning over producer internals + +Rspack module IDs, Rsdoctor numeric keys, PIDs, ports, and watch-cycle counters are capture-local. +The engine normalizes them into stable semantic identities and retains the original producer IDs as +provenance only. + +## Terminology + +| Term | Definition | +| ------------- | ----------------------------------------------------------------------------------------- | +| Workspace | One trusted Rstack configuration root and its allowed filesystem roots. | +| Product | A shipped application entry, server entry, worker, CLI, or library contract. | +| Context | One normalized combination of config, target, mode, runtime, environment, and conditions. | +| Run | A producer execution such as a build, lint request, or test cycle. | +| Generation | A monotonically increasing source-change epoch used to correlate concurrent producers. | +| Snapshot | An immutable, queryable view assembled from one or more runs. | +| Facet | Producer-specific evidence attached to a normalized entity. | +| Evidence | An immutable observation supporting or weakening a claim. | +| Finding | A classified, actionable claim with explicit bounds and evidence. | +| Root | A definition or module from which reachability is computed. | +| Contract root | An entry that external consumers are allowed to import or invoke. | + +## Architecture + +### System overview + +```mermaid +flowchart LR + subgraph Commands["Existing Rstack commands"] + Build["rs dev / rs build"] + Lib["rs lib"] + Lint["rs lint"] + Test["rs test"] + end + + subgraph Producers["Passive evidence producers"] + Rspack["Rspack observer"] + Doctor["Rsdoctor collector"] + Rslib["Rslib contract adapter"] + Rslint["Resident Rslint worker"] + Rstest["Rstest observer"] + end + + Coordinator["Per-workspace context coordinator"] + Store[("Immutable snapshots + bounded event log")] + Query["Reachability, classification, diff, and query engine"] + Broker["rs mcp stdio broker"] + + subgraph Hosts["Agent hosts"] + Codex["Codex plugin + skills"] + Claude["Claude plugin + skills"] + end + + OptionalUI["Optional Rsdoctor report UI"] + + Build --> Rspack + Build --> Doctor + Lib --> Rslib + Lint --> Rslint + Test --> Rstest + + Rspack --> Coordinator + Doctor --> Coordinator + Rslib --> Coordinator + Rslint --> Coordinator + Rstest --> Coordinator + Coordinator --> Store + Store --> Query + Query --> Broker + Broker --> Codex + Broker --> Claude + Store -. "explicit open report" .-> OptionalUI +``` + +### Component responsibilities + +| Component | Responsibility | Must not do | +| ------------ | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | +| Tool adapter | Add one passive collector after the user's resolved configuration and correlate its lifecycle with a run. | Modify the user's config file, reorder user plugins, or fail the command. | +| Producer | Emit bounded, schema-versioned facts and completeness metadata. | Make cross-tool dead-code decisions. | +| Coordinator | Assign workspace, context, run, generation, and snapshot identities; merge producer facts; persist bounded state. | Execute project code merely because an MCP client connected. | +| Analyzer | Compute roots, reachability, contract requirements, findings, explanations, and diffs. | Hide unknown dynamic behavior or partial captures. | +| MCP broker | Expose one stdio server, enforce roots/capabilities, paginate output, and link resources. | Mount on a development server or expose a second Rsdoctor MCP endpoint. | +| Skills | Choose the correct queries, combine evidence, explain limits, and guide safe next actions. | Parse raw logs or represent candidates as proven dead. | + +### Upstream and downstream ownership + +```mermaid +flowchart TB + subgraph Upstream["Upstream compiler and analyzer ownership"] + RP["Rspack: optimizer and runtime facts"] + RD["Rsdoctor: report contract, build graph, rules, Agent CLI"] + RT["Rstest: supported observer/watch API"] + end + + subgraph Rstack["Rstack ownership"] + Inject["Safe adapter injection"] + Identity["Stable identity + generations"] + Merge["Cross-producer evidence merge"] + Policy["Product roots, contracts, confidence, privacy"] + MCP["One MCP + plugin skills"] + end + + RP --> RD + RD --> Merge + RT --> Merge + Inject --> Merge + Identity --> Merge + Merge --> Policy --> MCP +``` + +Rspack and Rsdoctor own facts that cannot be reconstructed reliably after compilation: provided +exports with zero active edges, optimizer usage state, side-effect decisions, runtime activity, +dependency locations, transformed declarations, and optimization bailouts. Rstack owns composition, +not duplicate compiler instrumentation. + +## Evidence producers + +### Rsbuild and Rspack + +Rstack adds one global Rsbuild observer after resolving the user's app configuration. The observer +uses documented Rsbuild lifecycle hooks for environment/config/build events and a final +`tools.rspack` composition to append one no-op Rspack observer per environment. + +The default capture includes: + +- command, environment, target, mode, runtime, tool versions, and config fingerprint; +- build start, completion, failure, restart, close, and watch change sets; +- minimal Stats for hashes, timings, assets, chunks, diagnostics, entrypoints, and module inventory; +- completeness and extraction-cost metadata. + +Deep build capture is opt-in and adds: + +- module reasons and issuer paths; +- provided and used exports; +- optimization bailouts; +- runtime-aware ModuleGraph and ChunkGraph edges; +- Rsdoctor export-usage and tree-shaking data; +- bounded source-map attribution. + +Collection occurs only at lifecycle points where Rspack data is complete. JS proxy objects are +serialized immediately and never retained between callbacks. + +### Rsdoctor + +Rsdoctor is the canonical build-analysis provider. The context engine consumes static +`rsdoctor-data.json` or a normal `.rsdoctor/manifest.json` and its shards. It integrates +`@rsdoctor/agent-cli` in-process and reuses `@rsdoctor/shared` graph and diff operations where a +stable public surface exists. + +The default agent path does not depend on: + +- `@rsdoctor/client`; +- an HTTP or Socket.IO report server; +- the removed legacy `@rsdoctor/mcp-server`; +- a browser session. + +The Rsdoctor GUI remains available through an explicit `report_link` result for investigations where +a treemap or large interactive graph is materially more useful than a bounded path or table. + +### Rslib + +Rstack adds one global Rsbuild-compatible observer after resolving the CLI-specific Rslib config. +The adapter does not modify `lib[]` entries or the shared resolver used by Rstest. + +Rslib contributes: + +- selected library variants and environment identities; +- entry files, output formats, targets, filenames, and bundleless mode; +- `autoExternal` and explicit externals intent; +- emitted outputs and Stats per environment; +- declaration-output intent and coarse completion state; +- package name, files, `main`, `module`, `types`, `exports`, and `bin` contract roots; +- validation of package-export targets against actual outputs. + +Published libraries default to an open-world public contract. Internal libraries may opt into +closed-world workspace analysis. + +### Rslint + +One resident `Rslint` engine per workspace provides structured `lintFiles` and `lintText` results. +Requests are serialized through the engine. Rstack uses the generated Rslint config file so project +plugins and configuration behave exactly like `rs lint`. + +Rslint contributes: + +- lexical unused locals, parameters, private members, and unreachable-code diagnostics; +- rule IDs, severity, locations, suggestions, and fix ranges; +- whole-file fixed output for previews; +- optional TypeScript compiler diagnostics through a captured CLI subprocess. + +The public JS API has no cancellation or type-check/timing surface. Hard cancellation therefore +requires a killable worker or one-shot subprocess and recreation of the resident engine. + +### Rstest + +Rstest contributes two distinct evidence families: + +- static module dependency: which tests are related to a changed source file; +- runtime execution: which source ranges were covered by a specific test capture. + +Those signals never collapse into one claim. A related test can import a module without executing a +symbol, and a coverage hit can come from setup, module initialization, hooks, or shared state. + +The observer records run, file, suite, case, diagnostic, console, retry, snapshot, coverage, and +completion events. Project configuration supplies environment/browser metadata that reporter events +do not carry directly. + +Rstest's current programmatic and reporter APIs are experimental and must be pinned to an exact patch. +A supported append-only observer and watch-session control API should be added upstream before Rstack +offers first-class MCP watch control. + +### Workspace and Git + +Every snapshot records the source revision, dirty-diff digest, config file and dependency digest, +selected products, platform, command, and producer versions. Absolute paths, raw environment +variables, and arbitrary config objects are not persisted. + +## Configuration and activation + +Passive collection is off until the repository is trusted. This RFC adds an optional `context` +section to Rstack configuration for product intent and limits: + +```ts +export default define({ + context: { + enabled: true, + products: [ + { config: 'app', kind: 'application' }, + { config: 'lib', kind: 'published-library' }, + ], + capture: 'metadata', + tests: 'attach', + retention: { + snapshots: 20, + maxBytes: 100 * 1024 * 1024, + }, + }, +}); +``` + +The normative behavior is: + +- `metadata` is the default capture tier; +- `capture` accepts `off`, `metadata`, or `deep`; +- `tests` accepts `off` or `attach`; test execution is never implied; +- `products` is required only when Rstack cannot infer a safe application or published-library + product from `define.app` or `define.lib`; +- deep graph/source capture requires an explicit setting or command; +- tests attach only to a user-started Rstest session; +- CI does not persist or listen unless explicitly configured to emit a redacted artifact; +- `RSTACK_CONTEXT=0` is an emergency opt-out; +- instrumentation changes the resolved in-memory config only and never writes the user's config file. + +## Safe configuration injection + +```mermaid +flowchart LR + User["User config object or function"] + Resolve["Existing Rstack resolver"] + Clone["Shallow immutable clone"] + Append["Append one Rstack-owned observer"] + Tool["Underlying tool CLI/API"] + + User --> Resolve --> Clone --> Append --> Tool + User -. "never edited" .-> Tool +``` + +The injection points are intentionally tool-specific: + +| Tool | Injection point | Reason | +| ------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------ | +| Rsbuild | After `resolveRsbuildConfig` in `rsbuildConfig.ts` | Covers app CLI runs without leaking instrumentation into Rstest's app extension. | +| Rspack | Final composed `tools.rspack` result | Observes the actual bundler config after user composition. | +| Rslib | After CLI-only `resolveRslibConfig` | One global plugin covers all generated library environments without duplicating callbacks. | +| Rstest | Append observer after configured reporters are constructed | Rstest reporter configuration is replace-not-concatenate. | +| Rslint | Programmatic `Rslint` instance using generated config | Preserves structured diagnostics; CLI is retained only for type-check/timing gaps. | + +Observers must be per-instance idempotent, never module-global. They must not mutate hook arguments, +return values, assets, graphs, or diagnostics. + +## Information model + +### Core entities + +```mermaid +erDiagram + WORKSPACE ||--o{ CONTEXT : contains + CONTEXT ||--o{ RUN : executes + RUN }o--o{ SNAPSHOT : contributes + SNAPSHOT ||--o{ ENTITY : records + ENTITY ||--o{ EDGE : originates + ENTITY ||--o{ EVIDENCE : supports + SNAPSHOT ||--o{ FINDING : classifies + FINDING }o--o{ EVIDENCE : cites + + WORKSPACE { + string id + string rootDigest + } + CONTEXT { + string id + string configDigest + string target + string mode + string runtime + } + RUN { + string id + int generation + string producer + string status + } + SNAPSHOT { + string id + string sourceDigest + string completeness + } + ENTITY { + string id + string kind + string canonicalKey + } + EDGE { + string type + string targetId + } + EVIDENCE { + string id + string claim + string method + } + FINDING { + string id + string code + string confidence + } +``` + +Entity kinds include workspace, product, environment, route entry, module, symbol, export, package, +test project, test file, test case, chunk, asset, diagnostic, and report. + +Normalized edge kinds include: + +- `imports`, `dynamic_imports`, `requires`, and `reexports`; +- `declares`, `exports`, and `contract_exposes`; +- `routes_to`, `included_in`, and `emits`; +- `exercises`, `covers`, and `related_to_test`; +- `retained_for_side_effect`, `retained_by_bailout`, and `diagnosed_by`. + +### Identity + +- Workspace IDs derive from canonical repository identity, not the absolute checkout path. +- Context IDs derive from normalized config, target, mode, runtime, conditions, and redacted + environment digest. +- Semantic module, symbol, export, package, test, chunk, and route IDs are deterministic within a + workspace and do not include snapshot IDs. +- Snapshot and run IDs are immutable time-sortable IDs. +- Evidence IDs are content-addressed. +- Producer-local numeric IDs remain in the provenance facet only. + +### Evidence envelope + +```json +{ + "schemaVersion": 1, + "id": "ev_…", + "producer": "rspack", + "producerVersion": "2.x", + "runId": "run_…", + "generation": 184, + "contextId": "ctx_…", + "observedAt": "2026-08-12T03:30:00Z", + "method": "export_usage_graph", + "claim": "export has no active incoming edge in runtime main", + "source": { + "uri": "rstack://workspace/ws_…/source/packages/app/src/feature.ts", + "range": { + "startLine": 12, + "startColumn": 1, + "endLine": 18, + "endColumn": 2 + }, + "digest": "sha256:…" + }, + "bounds": { + "products": ["browser-production"], + "runtimes": ["main"], + "dynamicAccess": "unknown" + } +} +``` + +### Freshness, completeness, and confidence + +These dimensions are independent: + +| Dimension | Values | Meaning | +| ------------ | -------------------------------------------------------------------- | ------------------------------------------------------------------- | +| Status | `queued`, `running`, `pass`, `fail`, `cancelled`, `error`, `skipped` | What happened during the run. | +| Freshness | `live`, `fresh`, `stale`, `partial`, `unknown` | Whether the result applies to the current generation/source digest. | +| Completeness | Per-producer section map | Which facts were collected, disabled, truncated, or unsupported. | +| Confidence | `exact`, `derived`, `inferred`, `unknown` | How directly the conclusion follows from the evidence. | + +A green result may be stale. An empty section may mean "nothing found," "collector disabled," or +"producer unsupported"; the schema must preserve that distinction. + +## Product and reachability model + +### Root classes + +| Root class | Examples | Default policy | +| ------------------------- | -------------------------------------------------------------------------------- | ---------------------------------------------------- | +| Production executable | Browser entry, server entry, worker, Node CLI | Seeds production reachability. | +| Published contract | `package.json#exports`, `main`, `module`, `types`, `bin` | Protected from closed-world deletion. | +| Internal library | Workspace-only library explicitly declared internal | Uses actual workspace consumers as roots. | +| Non-production executable | Test, example, benchmark, doctest-equivalent, setup file | Seeds non-production reachability only. | +| Conservative runtime root | Dynamic namespace, nonliteral CommonJS, reflection, registration, generated code | Preserves liveness and lowers confidence. | +| Side-effect root | Explicit or inferred effectful module | Preserves module execution, not necessarily exports. | + +### Independent state axes + +Each definition is classified along at least these axes: + +```text +productionReachability: live | unreachable | unknown +nonProductionReachability: live | unreachable | unknown +publicContract: required | not-required | unknown +shipped: yes | no | unknown (per build/runtime) +executed: yes | no | unknown (per capture) +optimizerRetention: used | side-effect | bailout | removed | unknown +``` + +### Finding classifier + +```mermaid +flowchart TD + Start["Definition or export candidate"] + Complete{"Required producer sections complete?"} + Dynamic{"Dynamic / CJS / reflection uncertainty?"} + Contract{"Protected public contract?"} + Prod{"Reachable from any production root?"} + NonProd{"Reachable from non-production roots?"} + Shipped{"Shipped or retained in any selected build?"} + + Start --> Complete + Complete -- "no" --> Partial["insufficient-evidence"] + Complete -- "yes" --> Dynamic + Dynamic -- "yes" --> Candidate["candidate with explicit uncertainty"] + Dynamic -- "no" --> Contract + Contract -- "yes" --> Protected["public-contract-unused or no finding"] + Contract -- "no" --> Prod + Prod -- "yes" --> Live["live; consider unnecessary visibility/export only"] + Prod -- "no" --> NonProd + NonProd -- "yes" --> TestOnly["test-only or development-only"] + NonProd -- "no" --> Shipped + Shipped -- "yes" --> Retained["retained unexpectedly; explain side effect/bailout"] + Shipped -- "no" --> Dead["dead-code candidate"] +``` + +### Finding codes + +| Code | Meaning | Default action | +| --------------------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- | +| `unused-local` | Rslint proves a non-exported local/private definition is unused. | Offer a lint fix preview when available. | +| `unused-export` | An export is provided but unused in all selected production runtimes. | Investigate contract and dynamic bounds; do not delete automatically. | +| `dead-export` | The export is unreachable from production and non-production roots and is not contract-required. | Offer a removal plan after verification. | +| `unnecessary-export` | The definition is live but no selected consumer requires it to be exported. | Offer visibility/export reduction. | +| `test-only-export` | Reachable only from tests or other non-production roots. | Explain test-only status; avoid production bundle claims. | +| `dead-module` | No selected root reaches the module and no required side effect preserves it. | Offer module removal after multi-context verification. | +| `retained-for-side-effects` | No exports are used, but the module is retained for effects. | Explain effect locations and package metadata. | +| `tree-shaking-bailout` | Rspack cannot optimize the module/export as expected. | Explain bailout and likely remediation. | +| `not-shipped` | Present in source but absent from a specific build. | Report as build-scoped evidence, not global dead code. | +| `not-executed` | Included in coverage scope but had no hits in a capture. | Report as test-scoped evidence, not reachability proof. | +| `insufficient-evidence` | Required producers were disabled, stale, truncated, or unsupported. | Recommend the smallest safe capture that fills the gap. | + +Arbitrary local-symbol DCE remains heuristic unless Rspack exposes its internal inner-graph facts. +Source-map absence is not proof because inlining, renaming, minification, concatenation, and constant +folding can remove names and ranges. + +## Development and watch mode + +Build, lint, and test must remain independent. Lint and test never block HMR or change the Rsbuild +success result. + +```mermaid +sequenceDiagram + participant FS as File system + participant RB as Rsbuild/Rspack watch + participant C as Context coordinator + participant L as Rslint worker + participant T as Rstest watch + participant M as MCP client + + FS->>RB: changed files + RB->>C: generation 184 + invalidation set + C->>L: lint changed files for generation 184 + C-->>T: attach generation 184 change set + RB->>C: build 184 finished + C-->>M: build fresh, lint running, tests stale + L->>C: lint 184 finished + C-->>M: build + lint fresh, tests stale + T->>C: affected tests 184 finished + C-->>M: snapshot 184 complete +``` + +Example status exposed to the model: + +```text +DEV source=9a73f2 + 2 uncommitted files generation=184 +Build PASS 412ms [FRESH] +Lint RUNNING [generation 184] +Tests PASS 31/31 [STALE: generation 183; 2 changed files] +Next wait for lint, or inspect the changed-file diagnostics already available +``` + +Rslint behavior during development: + +- one resident engine per workspace; +- debounce and coalesce changed paths; +- lint changed files immediately; +- schedule program-wide type checking on explicit request or idle policy; +- bind each result to generation and source digest; +- cancel by terminating and recreating the worker only when necessary. + +Rstest behavior during development: + +- attach only when the user already started `rs test --watch` or explicitly requested it; +- correlate each watch cycle with the latest observed generation; +- use related-test evidence to explain affected selection; +- preserve previous results as stale until the new cycle finishes; +- distinguish cancellation, infrastructure failure, and product test failure; +- never claim exact case-to-symbol execution without an appropriately scoped coverage capture. + +## Coordinator and transport + +### Process model + +One coordinator runs per trusted project. Rstack commands and watch processes publish over a private +Unix-domain socket or Windows named pipe. Agent hosts launch `rs mcp`, a stdio broker that discovers +or starts the coordinator and exposes the sole MCP surface. + +```mermaid +flowchart LR + Commands["rs command processes"] -->|"private socket / named pipe"| Daemon["Project coordinator"] + Daemon --> WAL[("bounded WAL + snapshots")] + Codex["Codex"] -->|stdio| Broker["rs mcp"] + Claude["Claude Code"] -->|stdio| Broker + Broker -->|"private socket / named pipe"| Daemon +``` + +The coordinator manifest is per-user and atomic. Discovery validates PID, daemon boot ID, workspace +identity, owner, schema version, and lease. Stale manifests and sockets are removed only after those +checks. Unfinished runs recovered from the event log are marked aborted and stale. + +Loopback Streamable HTTP may be added later for explicit multi-client use. It is not the default and +must require a random bearer capability, strict origin validation, session limits, and loopback-only +binding. + +### Storage + +- Append-only events are a crash-recovery mechanism, not the query API. +- Snapshots are immutable and content-addressed where practical. +- Only a bounded latest history is retained by default. +- Source, maps, logs, coverage, and deep graphs have independent caps and retention policies. +- The store is disposable cache, never the only copy of a user artifact. +- Raw Rsdoctor artifacts stay in their project-selected output location and are not copied unless a + snapshot explicitly requires it. + +## MCP contract + +### One server + +The Codex and Claude bundles register one local stdio server named `rstack`. Rsdoctor tools are +adapted behind it; the legacy live Rsdoctor MCP server is not started. + +### Resources + +```text +rstack://workspace/{workspaceId}/contexts +rstack://context/{contextId}/head +rstack://context/{contextId}/live/{kind}/{entityId} +rstack://snapshot/{snapshotId} +rstack://snapshot/{snapshotId}/{kind}/{entityId} +rstack://query/{queryHandle} +rstack://run/{runId}/events +``` + +Live resources resolve one head snapshot per read and may be subscribed to. Snapshot resources are +immutable. Large catalogs are discoverable through resource templates and tool-returned links, not +through an unbounded `resources/list`. + +### Read-only tools + +| Tool | Purpose | +| -------------------- | ---------------------------------------------------------------------------------------------- | +| `project_status` | Return active contexts, producer health, current generations, freshness, and evidence gaps. | +| `findings_list` | Filter and page findings by code, product, package, path, confidence, freshness, and severity. | +| `finding_explain` | Return the shortest causal/evidence paths, bounds, counter-evidence, and safe next actions. | +| `entity_get` | Get one normalized entity and selected producer facets. | +| `relationship_trace` | Traverse bounded dependencies, dependents, routes, tests, chunks, or export-use paths. | +| `snapshot_list` | List recent compatible snapshots. | +| `snapshot_diff` | Compare findings, sizes, diagnostics, tests, and graph edges between compatible snapshots. | +| `diagnostics_list` | Return deduplicated build, lint, type, test, and Rsdoctor diagnostics. | +| `tests_related` | Explain static related-test selection for files or modules. | +| `coverage_scope` | Return bounded execution evidence for files, symbols, or tests. | +| `rsdoctor_analyze` | Invoke the supported in-process Agent CLI catalog against an explicit artifact. | +| `report_link` | Return an explicit command/resource link for opening an existing Rsdoctor report. | + +Read-only tools use `readOnlyHint: true`, `destructiveHint: false`, and `openWorldHint: false`. + +### Mutating tools + +Mutation is a later phase and remains separate: + +- `refresh_context` may run configured collectors only after explicit approval; +- `run_build`, `run_lint`, and `run_test` execute repository code and require approval; +- `apply_fix_preview` applies only a prior hash-bound preview to explicit paths; +- snapshot pin/unpin affects context-engine cache only. + +Run tools are conservatively annotated as non-read-only, destructive, and open-world because project +plugins and tests may execute arbitrary code. + +### Query consistency and pagination + +Every query without an explicit snapshot captures the current head once. A TTL-bound query handle +pins that snapshot and authorization scope. Opaque cursors page the frozen result. Responses include +totals, truncation, completeness, and the snapshot ID. + +### Progressive model presentation + +```mermaid +flowchart LR + Hint["1. Small freshness/status hint"] + Skill["2. Task skill selects queries"] + Summary["3. Compact findings summary"] + Card["4. One evidence card"] + Path["5. Bounded path / table / source"] + Report["6. Optional Rsdoctor report"] + + Hint --> Skill --> Summary --> Card --> Path --> Report +``` + +The default response is decision-ready and short: + +```text +UNUSED CODE snapshot=snap_01… source=9a73f2 + 2 files [PARTIAL] +Candidates 7 exports · 2 modules · 4 high-confidence unused locals +Strongest packages/app/src/legacy.ts:18 `parseLegacyToken` +Evidence no production/test inbound path; absent from browser+node output +Boundary package is internal; dynamic CommonJS scan incomplete +Next inspect the only dynamic loader before proposing removal +``` + +An expanded finding is an evidence card, not a log dump: + +```json +{ + "id": "finding_…", + "code": "dead-export", + "subject": { + "name": "parseLegacyToken", + "location": "packages/app/src/legacy.ts:18" + }, + "state": { + "productionReachability": "unreachable", + "nonProductionReachability": "unreachable", + "publicContract": "not-required", + "shipped": "no", + "executed": "unknown" + }, + "confidence": "derived", + "freshness": "fresh", + "evidence": ["ev_static_graph", "ev_rspack_browser", "ev_rspack_node"], + "bounds": ["dynamic CommonJS scan incomplete"], + "actions": ["trace dynamic loaders", "preview removal", "copy verification command"] +} +``` + +## Plugin bundles + +### Codex + +```text +rstack-codex-plugin/ +├── .codex-plugin/plugin.json +├── .mcp.json +├── mcp/server.mjs +└── skills/ + ├── orient-rstack-project/ + ├── find-unused-code/ + ├── explain-dead-code/ + ├── assess-change-impact/ + ├── analyze-build/ + ├── debug-dev-cycle/ + ├── select-affected-tests/ + └── review-build-regression/ +``` + +Codex workflows are skills. Version 1 does not require hooks, an app, a bundled LSP, or a separate +subagent registry. The prebuilt MCP runtime and compatible Rsdoctor Agent CLI are pinned in the +published artifact. + +### Claude code + +```text +rstack-claude-plugin/ +├── .claude-plugin/plugin.json +├── .mcp.json +├── server/context.mjs +├── skills/ +│ ├── orient-rstack-project/ +│ ├── find-unused-code/ +│ ├── explain-dead-code/ +│ ├── assess-change-impact/ +│ ├── analyze-build/ +│ ├── debug-dev-cycle/ +│ └── select-affected-tests/ +├── agents/ +│ ├── code-explorer.md +│ ├── change-impact-reviewer.md +│ └── build-performance-analyst.md +└── workflows/ + ├── review-change.js + └── build-regression.js +``` + +Claude Code skills and agents share the same MCP schemas and evidence semantics. State is stored in +the host-provided plugin data directory, never the immutable plugin cache. A startup hook is omitted +until `project_status` is proven consistently fast and side-effect-free. + +## Skill design + +Skills are the primary user-facing interface. MCP tools provide facts; skills provide workflow, +selection policy, safety rules, and presentation. + +| Skill | Trigger examples | Evidence workflow | +| ------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | +| `orient-rstack-project` | "How is this project structured?", "What Rstack tools are active?" | Status → products/contexts → architecture summary → freshness gaps. | +| `find-unused-code` | "Find dead code", "What can I remove?", "Unused exports" | Establish products/contracts → list candidates → require independent signals → rank → inspect uncertainty → propose verification. | +| `explain-dead-code` | "Why is this considered dead?", "Why is this retained?" | Resolve subject → shortest root paths → optimizer/side-effect evidence → tests/contract evidence → bounds. | +| `assess-change-impact` | "What breaks if I change this?", "Who depends on X?" | Resolve entity → dependents across contexts → affected products/tests/chunks → stale/unknown edges. | +| `analyze-build` | "Why is the bundle large?", "Why isn't this tree-shaken?" | Select fresh Rsdoctor artifact → summary → narrow query → ranked evidence → optional report link. | +| `debug-dev-cycle` | "Why is dev stale?", "What failed after my edit?" | Correlate generation → show build/lint/test states → first actionable failure → exact rerun. | +| `select-affected-tests` | "What tests should I run?" | Static related tests → changed context → prior coverage → explicit selection rationale and gaps. | +| `review-build-regression` | "What changed in this build?" | Validate comparable snapshots → diff → regressions/fixes → causal module/package paths. | + +### Normative `find-unused-code` workflow + +```mermaid +flowchart TD + Status["Read project status"] + Products["Resolve products and contract roots"] + Candidates["Query unused/dead candidates"] + Evidence["Require independent evidence families"] + Dynamic["Inspect dynamic, generated, and side-effect bounds"] + Rank["Rank by actionability, not a probability score"] + Plan["Return verification or hash-bound removal preview"] + + Status --> Products --> Candidates --> Evidence --> Dynamic --> Rank --> Plan +``` + +The skill must: + +1. Refuse to analyze a published library as a closed world unless explicitly configured. +2. Prefer high-confidence Rslint local findings before cross-module candidates. +3. Distinguish unused export, unreachable source, not shipped, not executed, and optimizer bailout. +4. Require at least two independent evidence families before recommending deletion of an exported + definition. +5. Treat dynamic imports, nonliteral `require`, reflection, registration, generated code, and missing + contexts as uncertainty. +6. Show the shortest evidence path and the analysis bounds. +7. Never apply edits directly; produce a preview and verification plan. + +### Skill output contract + +Every investigative skill returns: + +- conclusion and finding code; +- status, freshness, completeness, and confidence; +- direct evidence with source locations; +- shortest causal paths; +- known bounds and counter-evidence; +- one safe recommended next action; +- snapshot/run provenance; +- resource links for deeper inspection. + +## Security and privacy + +### Trust boundary + +Rstack configuration, plugins, tests, reports, source comments, diagnostics, and paths are untrusted +project input. They may contain prompt injection or secrets. The server treats them as data, never as +instructions. + +### Capability tiers + +| Tier | Access | Default | +| ---- | ----------------------------------------------------------------- | ------------------------------- | +| 0 | Tool versions, contexts, run status, counts, freshness | Enabled after repository trust. | +| 1 | Sanitized diagnostics, package/module names, relative paths | Enabled after repository trust. | +| 2 | Source ranges, graph paths, maps, logs, coverage, report contents | Explicit workspace capability. | +| 3 | Build, test, lint, refresh, or mutation | Explicit per-action approval. | + +### Required controls + +- Host-facing transport is stdio; internal IPC uses owner-only sockets or named pipes. +- Never expose MCP or report queries on the dev-server host/port. +- Resolve and realpath every requested path; reject traversal, symlink escape, and paths outside MCP + roots. +- Persist allowlisted schema fields only. Never serialize raw config objects, functions, plugin + instances, environment variables, headers, cookies, URLs, loader options, or arbitrary argv. +- Relativize workspace paths and redact secrets at collection, persistence, logging, and response + boundaries. +- Cap files, bytes, entities, edges, logs, source maps, coverage, diagnostics, time, and concurrency. +- Bind every fix preview to workspace, path, source digest, and schema version. Reject stale or + changed inputs. +- Do not upload artifacts or enable telemetry by default. +- Do not start commands, watchers, indexing, or a daemon merely because a plugin was installed. + +## Performance budgets + +Passive metadata collection targets: + +| Metric | Budget | +| ------------------------------ | ------------------------------------------------------------- | +| One-shot build overhead | Less than 2% | +| Context-engine startup | Less than 100 ms after package load | +| Incremental/watch observer p95 | Less than 25 ms per generation | +| Coordinator resident memory | Less than 50 MiB excluding explicitly retained deep artifacts | +| Default MCP query | Less than 500 ms warm | +| Bounded graph query | One concurrent query, 2 s deadline, 1 MiB response cap | + +High-cardinality module/resolution hooks, module sources, full reasons, source maps, deep coverage, +and Rspack/Rsdoctor profiling are opt-in. Every snapshot records extraction time, heap delta where +available, normalized row counts, serialized bytes, truncation, and drop counts. + +When a producer exceeds its queue or time budget, it coalesces progress events before diagnostics, +emits a drop marker, degrades the relevant facet to partial, and never blocks the underlying command. + +## Error and lifecycle semantics + +- Producer failures become `collector-error` evidence and partial completeness; they do not change + the tool's exit status. +- A tool failure remains a tool failure even if collection succeeded. +- A successful Rspack compilation is not necessarily a successful Rslib build; declaration generation + may fail afterward. +- Cancelled and infrastructure-failed tests are not product test failures. +- Watch restart creates a new instance identity and preserves the prior snapshot as stale. +- A process exit without a completion marker closes the run as aborted/unknown. +- The coordinator rejects unsupported schema majors and records compatible minor capabilities. +- Source changes invalidate only affected facets; unaffected results may remain fresh when their + dependency digest proves applicability. + +## Upstream work + +Rstack should land small upstream changes before depending on unstable private APIs. + +### Rsdoctor + +1. **Preview packages:** add `pkg.pr.new` pull-request releases so Rstack can validate upstream + changes before npm publication. This is tracked by + [web-infra-dev/rsdoctor#1900](https://github.com/web-infra-dev/rsdoctor/pull/1900). +2. **Versioned artifact contract:** add schema version, producer version, output mode, compiler/build + identity, enabled features, collected sections, and capability flags to brief JSON and normal + manifests. Preserve the distinction between disabled and legitimately empty sections. +3. **Export-usage ingestion:** enable Rspack's existing `exportUsageGraph`; normalize its edges into + Rsdoctor's dormant export, variable, side-effect, statement, and module-graph model; persist + declaration/reference locations and runtime bounds where supplied. +4. **Headless parity:** expose high-value module, loader, plugin timing, package, rule, and full bundle + diff queries through the Agent CLI catalog using bounded filters and pagination. +5. **Stable semantic keys and diffs:** retain process-local numeric IDs for transport, but add stable + module/export/package keys and export/finding deltas for cross-build comparisons. + +### Rspack + +No new Rspack feature is required for the first export-usage graph: Rspack 2 already exposes an +experimental Rsdoctor export-usage payload. Later PRs may be required for: + +- complete provided-export inventory including zero-edge exports; +- runtime-specific usage and inactive conditional edges; +- authoritative side-effect/purity state; +- declaration/local-binding ranges and supported inner-graph relationships; +- module/chunk phase timings and cache status. + +Those facts must be exposed by Rspack rather than reconstructed from minified assets. + +### Rstest + +Add a supported API that: + +- appends an observer after user reporters are constructed; +- exposes a watch-session handle with ready, cycle, rerun, cancel, close, and completion semantics; +- includes project environment/browser identity in observer events; +- preserves `(project, testPath)` identity in aggregate results. + +Until then, Rstack pins the exact Rstest patch and limits supported integration to one-shot reads and +passive attachment where safe. + +## Alternatives considered + +### Use only the Rsdoctor GUI + +Rejected. It is valuable for humans but requires a browser/report server and cannot provide the +cross-tool product, lint, test, contract, freshness, or permission model. + +### Ship separate MCP servers for each tool + +Rejected. It duplicates lifecycle, trust, roots, transport, and discovery; gives the model conflicting +schemas; and prevents atomic cross-producer snapshots. + +### Mount MCP on the Rsbuild dev server + +Rejected. It couples agent access to application networking, exposes dangerous host/port/CORS +surfaces, and makes context disappear when the dev server stops. + +### Parse command output only + +Rejected. Human output is unstable, lossy, hard to cancel, and missing structured lifecycle, +completeness, and identity. CLI subprocess capture remains a narrow fallback for surfaces not exposed +programmatically. + +### Build a knip replacement from source alone + +Rejected. Source reachability is valuable but cannot replace the actual configured compilation, +runtime/chunk graph, optimizer decisions, loaders, plugins, or Rslib product contract. A future +Knip-compatible producer can complement build evidence. + +### Inject collectors into stored user config + +Rejected. It would affect every config consumer, leak instrumentation into unrelated commands, and +write surprising persistent changes. Injection belongs in CLI-specific resolved-config adapters. + +### Return raw graphs to the model + +Rejected. Large graph dumps waste context and obscure decisions. The query engine returns bounded +paths, trees, tables, and evidence cards; a full visual graph is optional investigation UI. + +## Delivery plan + +```mermaid +timeline + title Rstack Context Engine delivery + Phase 0 : Publish contracts and upstream preview packages + : Version Rsdoctor artifacts + Phase 1 : Passive build and Rsdoctor snapshots + : One read-only MCP server + Phase 2 : Product roots and unused-code findings + : Codex and Claude skills + Phase 3 : Development generations + : Rslint worker and passive Rstest attachment + Phase 4 : Snapshot diffs and CI artifacts + : Safe fix previews + Phase 5 : Optional remote transport and thin visual summaries +``` + +### Phase 0: contracts + +- Define normalized entity, edge, evidence, snapshot, finding, and compatibility schemas. +- Land Rsdoctor preview packages and artifact metadata. +- Contract-test the Rspack/Rsdoctor payload against pinned versions. + +### Phase 1: passive build context + +- Add trusted, metadata-only Rsbuild/Rspack and Rslib observers. +- Ingest static Rsdoctor artifacts through the in-process Agent CLI. +- Implement the per-project coordinator, immutable snapshots, `rs mcp`, status, diagnostics, basic + entity queries, and report links. + +### Phase 2: reachability and skills + +- Add product/contract roots, runtime-aware export usage, causal paths, completeness, and classifier. +- Ship `find-unused-code`, `explain-dead-code`, `assess-change-impact`, and `analyze-build` skills for + Codex and Claude. +- Report candidates only; no edit/apply tools. + +### Phase 3: development intelligence + +- Add source generations, bounded event subscriptions, resident Rslint worker, and passive Rstest + attachment. +- Ship `debug-dev-cycle` and `select-affected-tests` skills. + +### Phase 4: change and mutation workflows + +- Add compatible snapshot diffs, build regression skill, redacted CI artifacts, and budgets. +- Add hash-bound fix previews and explicit apply/verify flow. + +### Phase 5: optional presentation + +- Add thin MCP-app/status views only if headless workflows prove a concrete need. +- Continue linking the existing Rsdoctor report for rich build visualization rather than duplicating + it. + +## Validation strategy + +### Schema and graph correctness + +- Golden fixtures for ESM, reexports, star/default/namespace imports, type-only imports, dynamic + imports, literal and nonliteral CommonJS, barrels, cycles, side effects, concatenation, generated + code, and multiple runtimes. +- Byte-stable normalized snapshots across repeated equivalent builds. +- Property tests for canonicalization, traversal bounds, public roots, and finding invariants. +- Compatibility fixtures for each supported producer patch and schema version. + +### Product matrix + +- Rsbuild client/server and multi-environment applications. +- Rslib ESM/CJS, bundleless, declarations, externals, published exports, and internal libraries. +- Rstest Node, DOM, browser, projects, retries, watch cycles, snapshots, and Istanbul/V8 coverage. +- Rslint object/function config, plugins, lint text/files, suggestions, fixes, and type checking. +- Rsdoctor brief JSON, normal manifests, multi-compiler series, missing sections, and malformed/large + artifacts. + +### Dead-code safety invariants + +No high-confidence dead finding may include: + +- a reachable production definition; +- a protected published export; +- a side-effect-only module; +- a known dynamic-import target; +- a target reachable in another selected environment or runtime; +- a test/development-only definition mislabeled as globally unused; +- a finding derived from a stale, partial, or incompatible producer without that bound displayed. + +### Transport and lifecycle + +- Raw stdio JSON-RPC transcripts and MCP SDK clients. +- Initialization ordering, schema negotiation, invalid params, cancellation, progress, pagination, + subscriptions, and stdout purity. +- Concurrent readers, producer backpressure, coordinator restart, stale manifest, PID reuse, crash + recovery, version skew, and orphan cleanup. +- Watch tests wait for generation changes rather than sleeping. + +### Security + +- Malicious config/plugin/report text and prompt injection. +- Secret canaries, path traversal, symlink escape, malformed/huge artifacts, graph bombs, and log + flooding. +- Denied capabilities, fork/CI behavior, subprocess environment scrubbing, and network binding. +- Stale fix token, changed source, deleted path, and outside-root mutation attempts. + +### Performance + +Benchmark small, medium, and large workspaces across cold, warm, and incremental runs. Track build +overhead, incremental p95, extraction bytes, queue drops, coordinator RSS, query p95, and response +size. Pull requests fail only on statistically meaningful regressions beyond versioned budgets; +large stress cases run nightly. + +## Acceptance criteria + +The first stable release is complete when: + +1. Codex and Claude can answer "what is stale or failing?" from the same read-only MCP schema. +2. A user can ask "find unused code" and receive ranked candidates with production, + non-production, public-contract, shipped, optimizer, freshness, confidence, and evidence bounds. +3. A user can ask "why is this included?" and receive a bounded root-to-module/export path with + runtime and chunk evidence. +4. A user can ask "what tests should I run?" and receive related tests with an explicit selection + rationale, without claiming exact execution unless coverage supports it. +5. Rsbuild HMR remains independent from lint and test completion. +6. No GUI, network listener, build, test, or indexing job starts merely because the plugin is + installed or an MCP client connects. +7. Collector crashes and unsupported producer versions do not change the underlying command result. +8. Security, false-positive, watch, protocol, compatibility, and performance test suites meet the + budgets in this RFC. + +## References + +- [Astral Hawk architecture](https://github.com/astral-sh/hawk/blob/main/docs/architecture.md) +- [Model Context Protocol resources](https://modelcontextprotocol.io/specification/2025-11-25/server/resources) +- [Model Context Protocol tools](https://modelcontextprotocol.io/specification/2025-11-25/server/tools) +- [Rspack Stats JSON](https://rspack.rs/api/javascript-api/stats-json) +- [Rspack tree shaking](https://rspack.rs/guide/optimization/tree-shaking) +- [Rsbuild plugin hooks](https://rsbuild.rs/plugins/dev/hooks) +- [Rslib JavaScript API](https://lib.rsbuild.dev/api/javascript-api/instance) +- [Rslint JavaScript API](https://rslint.rs/guide/js-api) +- [Rstest reporter API](https://rstest.rs/api/javascript-api/reporter) +- [Rsdoctor AI integration](https://rsdoctor.rs/guide/start/ai) +- [Rsdoctor pull-request preview packages](https://github.com/web-infra-dev/rsdoctor/pull/1900) +- [Knip analysis model](https://knip.dev/explanations/how-knip-works) +- [Codex plugin packaging](https://developers.openai.com/plugins/build/plugins) +- [Claude Code plugin reference](https://code.claude.com/docs/en/plugins-reference) diff --git a/scripts/dictionary.txt b/scripts/dictionary.txt index 06b17167..6bf5ca40 100644 --- a/scripts/dictionary.txt +++ b/scripts/dictionary.txt @@ -1,9 +1,13 @@ # Custom Dictionary Words applypatch +backpressure +bundleless cdpath clippy dirents +doctest editmsg +effectful errexit esac extglob @@ -11,14 +15,18 @@ fnames huskyrc indentable jsonline +killable llms napi noformat +nonliteral noprettier nosystem oxfmt quasis +realpath rsbuild +rsdoctor rslib rslint rslog @@ -30,6 +38,8 @@ rstest shiki shikijs solidjs +streamable +treemap turborepo typicode worktank From f62903dc09e29ea1a65e56c3ad5ec44e6f8324d1 Mon Sep 17 00:00:00 2001 From: ScriptedAlchemy Date: Wed, 12 Aug 2026 04:47:35 +0000 Subject: [PATCH 002/110] docs: improve RFC architecture diagrams --- docs/rfcs/0001-rstack-context-engine.md | 136 +++++++++++------------- 1 file changed, 63 insertions(+), 73 deletions(-) diff --git a/docs/rfcs/0001-rstack-context-engine.md b/docs/rfcs/0001-rstack-context-engine.md index 18abfc4b..6d0af028 100644 --- a/docs/rfcs/0001-rstack-context-engine.md +++ b/docs/rfcs/0001-rstack-context-engine.md @@ -136,20 +136,14 @@ provenance only. ### System overview ```mermaid -flowchart LR - subgraph Commands["Existing Rstack commands"] - Build["rs dev / rs build"] - Lib["rs lib"] - Lint["rs lint"] - Test["rs test"] - end - - subgraph Producers["Passive evidence producers"] - Rspack["Rspack observer"] - Doctor["Rsdoctor collector"] - Rslib["Rslib contract adapter"] - Rslint["Resident Rslint worker"] - Rstest["Rstest observer"] +flowchart TB + subgraph Capture["Passive evidence capture"] + direction LR + Build["rs dev / rs build"] --> Rspack["Rspack observer"] + Build --> Doctor["Rsdoctor collector"] + Lib["rs lib"] --> Rslib["Rslib contract adapter"] + Lint["rs lint"] --> Rslint["Resident Rslint worker"] + Test["rs test"] --> Rstest["Rstest observer"] end Coordinator["Per-workspace context coordinator"] @@ -164,12 +158,6 @@ flowchart LR OptionalUI["Optional Rsdoctor report UI"] - Build --> Rspack - Build --> Doctor - Lib --> Rslib - Lint --> Rslint - Test --> Rstest - Rspack --> Coordinator Doctor --> Coordinator Rslib --> Coordinator @@ -180,7 +168,7 @@ flowchart LR Query --> Broker Broker --> Codex Broker --> Claude - Store -. "explicit open report" .-> OptionalUI + Doctor -. "explicit report artifact" .-> OptionalUI ``` ### Component responsibilities @@ -198,18 +186,18 @@ flowchart LR ```mermaid flowchart TB - subgraph Upstream["Upstream compiler and analyzer ownership"] - RP["Rspack: optimizer and runtime facts"] - RD["Rsdoctor: report contract, build graph, rules, Agent CLI"] - RT["Rstest: supported observer/watch API"] + subgraph Upstream["Upstream ownership"] + RP["Rspack
optimizer + runtime facts"] + RD["Rsdoctor
report contract + build graph"] + RT["Rstest
observer/watch API"] end subgraph Rstack["Rstack ownership"] - Inject["Safe adapter injection"] - Identity["Stable identity + generations"] - Merge["Cross-producer evidence merge"] - Policy["Product roots, contracts, confidence, privacy"] - MCP["One MCP + plugin skills"] + Inject["Safe adapter
injection"] + Identity["Stable identity
+ generations"] + Merge["Cross-producer
evidence merge"] + Policy["Product roots + contracts
confidence + privacy"] + MCP["One MCP
+ plugin skills"] end RP --> RD @@ -366,7 +354,7 @@ The normative behavior is: ## Safe configuration injection ```mermaid -flowchart LR +flowchart TB User["User config object or function"] Resolve["Existing Rstack resolver"] Clone["Shallow immutable clone"] @@ -398,12 +386,12 @@ return values, assets, graphs, or diagnostics. erDiagram WORKSPACE ||--o{ CONTEXT : contains CONTEXT ||--o{ RUN : executes - RUN }o--o{ SNAPSHOT : contributes - SNAPSHOT ||--o{ ENTITY : records + RUN }|--o| SNAPSHOT : contributes + SNAPSHOT }|--o{ ENTITY : records ENTITY ||--o{ EDGE : originates ENTITY ||--o{ EVIDENCE : supports SNAPSHOT ||--o{ FINDING : classifies - FINDING }o--o{ EVIDENCE : cites + FINDING }o--|{ EVIDENCE : cites WORKSPACE { string id @@ -545,28 +533,36 @@ optimizerRetention: used | side-effect | bailout | removed | unknown ### Finding classifier ```mermaid -flowchart TD +flowchart TB Start["Definition or export candidate"] - Complete{"Required producer sections complete?"} - Dynamic{"Dynamic / CJS / reflection uncertainty?"} - Contract{"Protected public contract?"} - Prod{"Reachable from any production root?"} - NonProd{"Reachable from non-production roots?"} - Shipped{"Shipped or retained in any selected build?"} + Complete["Complete evidence?"] + Dynamic["Dynamic access uncertain?"] + Contract["Protected contract?"] + Prod["Production-reachable?"] + NonProd["Non-production-reachable?"] + Shipped["Shipped or retained?"] Start --> Complete - Complete -- "no" --> Partial["insufficient-evidence"] + Complete -- "no" --> Partial Complete -- "yes" --> Dynamic - Dynamic -- "yes" --> Candidate["candidate with explicit uncertainty"] + Dynamic -- "yes" --> Candidate Dynamic -- "no" --> Contract - Contract -- "yes" --> Protected["public-contract-unused or no finding"] + Contract -- "yes" --> Protected Contract -- "no" --> Prod - Prod -- "yes" --> Live["live; consider unnecessary visibility/export only"] + Prod -- "yes" --> Live Prod -- "no" --> NonProd - NonProd -- "yes" --> TestOnly["test-only or development-only"] + NonProd -- "yes" --> TestOnly NonProd -- "no" --> Shipped - Shipped -- "yes" --> Retained["retained unexpectedly; explain side effect/bailout"] - Shipped -- "no" --> Dead["dead-code candidate"] + Shipped -- "yes" --> Retained + Shipped -- "no" --> Dead + + Partial["insufficient evidence"] + Candidate["uncertain candidate"] + Protected["protected contract"] + Live["live export"] + TestOnly["test/development only"] + Retained["retained unexpectedly"] + Dead["dead-code candidate"] ``` ### Finding codes @@ -596,22 +592,20 @@ success result. ```mermaid sequenceDiagram - participant FS as File system - participant RB as Rsbuild/Rspack watch + participant RB as Build watch participant C as Context coordinator participant L as Rslint worker participant T as Rstest watch - participant M as MCP client + participant M as Model - FS->>RB: changed files - RB->>C: generation 184 + invalidation set - C->>L: lint changed files for generation 184 - C-->>T: attach generation 184 change set - RB->>C: build 184 finished + RB->>C: generation 184 (changed files) + C->>L: lint changed files + C-->>T: attach change set + RB->>C: build finished C-->>M: build fresh, lint running, tests stale - L->>C: lint 184 finished + L->>C: lint finished C-->>M: build + lint fresh, tests stale - T->>C: affected tests 184 finished + T->>C: affected tests finished C-->>M: snapshot 184 complete ``` @@ -652,7 +646,7 @@ Unix-domain socket or Windows named pipe. Agent hosts launch `rs mcp`, a stdio b or starts the coordinator and exposes the sole MCP surface. ```mermaid -flowchart LR +flowchart TB Commands["rs command processes"] -->|"private socket / named pipe"| Daemon["Project coordinator"] Daemon --> WAL[("bounded WAL + snapshots")] Codex["Codex"] -->|stdio| Broker["rs mcp"] @@ -741,7 +735,7 @@ totals, truncation, completeness, and the snapshot ID. ### Progressive model presentation ```mermaid -flowchart LR +flowchart TB Hint["1. Small freshness/status hint"] Skill["2. Task skill selects queries"] Summary["3. Compact findings summary"] @@ -1050,19 +1044,15 @@ paths, trees, tables, and evidence cards; a full visual graph is optional invest ## Delivery plan ```mermaid -timeline - title Rstack Context Engine delivery - Phase 0 : Publish contracts and upstream preview packages - : Version Rsdoctor artifacts - Phase 1 : Passive build and Rsdoctor snapshots - : One read-only MCP server - Phase 2 : Product roots and unused-code findings - : Codex and Claude skills - Phase 3 : Development generations - : Rslint worker and passive Rstest attachment - Phase 4 : Snapshot diffs and CI artifacts - : Safe fix previews - Phase 5 : Optional remote transport and thin visual summaries +flowchart TB + P0["Phase 0: contracts
Preview packages + versioned Rsdoctor artifacts"] + P1["Phase 1: passive build context
Snapshots + one read-only MCP server"] + P2["Phase 2: reachability
Product roots + unused-code skills"] + P3["Phase 3: development intelligence
Generations + Rslint + Rstest"] + P4["Phase 4: change workflows
Snapshot diffs + CI + fix previews"] + P5["Phase 5: optional presentation
Remote transport + thin visual summaries"] + + P0 --> P1 --> P2 --> P3 --> P4 --> P5 ``` ### Phase 0: contracts From 70c4a90f957de553f559995d3c20daafc09d0aed Mon Sep 17 00:00:00 2001 From: ScriptedAlchemy Date: Wed, 12 Aug 2026 06:14:36 +0000 Subject: [PATCH 003/110] feat(rstack): scaffold context evidence store --- docs/rfcs/0001-rstack-context-engine.md | 237 ++++++++----- .../2026-08-12-context-store-foundation.md | 214 +++++++++++ packages/rstack/src/context/index.ts | 22 ++ packages/rstack/src/context/model.ts | 85 +++++ packages/rstack/src/context/store.ts | 335 ++++++++++++++++++ packages/rstack/src/context/workspace.ts | 95 +++++ packages/rstack/src/projectCache.ts | 2 +- packages/rstack/tests/context/store.test.ts | 226 ++++++++++++ .../rstack/tests/context/workspace.test.ts | 82 +++++ 9 files changed, 1204 insertions(+), 94 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-12-context-store-foundation.md create mode 100644 packages/rstack/src/context/index.ts create mode 100644 packages/rstack/src/context/model.ts create mode 100644 packages/rstack/src/context/store.ts create mode 100644 packages/rstack/src/context/workspace.ts create mode 100644 packages/rstack/tests/context/store.test.ts create mode 100644 packages/rstack/tests/context/workspace.test.ts diff --git a/docs/rfcs/0001-rstack-context-engine.md b/docs/rfcs/0001-rstack-context-engine.md index 6d0af028..0bc959cb 100644 --- a/docs/rfcs/0001-rstack-context-engine.md +++ b/docs/rfcs/0001-rstack-context-engine.md @@ -117,39 +117,43 @@ provenance only. ## Terminology -| Term | Definition | -| ------------- | ----------------------------------------------------------------------------------------- | -| Workspace | One trusted Rstack configuration root and its allowed filesystem roots. | -| Product | A shipped application entry, server entry, worker, CLI, or library contract. | -| Context | One normalized combination of config, target, mode, runtime, environment, and conditions. | -| Run | A producer execution such as a build, lint request, or test cycle. | -| Generation | A monotonically increasing source-change epoch used to correlate concurrent producers. | -| Snapshot | An immutable, queryable view assembled from one or more runs. | -| Facet | Producer-specific evidence attached to a normalized entity. | -| Evidence | An immutable observation supporting or weakening a claim. | -| Finding | A classified, actionable claim with explicit bounds and evidence. | -| Root | A definition or module from which reachability is computed. | -| Contract root | An entry that external consumers are allowed to import or invoke. | +| Term | Definition | +| ------------- | --------------------------------------------------------------------------------------------- | +| Repository | Stable source identity shared by related clones and working trees when it can be established. | +| Workspace | One authorized checkout or worktree and its allowed filesystem roots. | +| Product | A shipped application entry, server entry, worker, CLI, or library contract. | +| Context | One normalized combination of config, target, mode, runtime, environment, and conditions. | +| Run | A producer execution such as a build, lint request, or test cycle. | +| Generation | A producer-local monotonically increasing build, lint, or test cycle. | +| Snapshot | An immutable, queryable view assembled from one or more runs. | +| Facet | Producer-specific evidence attached to a normalized entity. | +| Evidence | An immutable observation supporting or weakening a claim. | +| Finding | A classified, actionable claim with explicit bounds and evidence. | +| Root | A definition or module from which reachability is computed. | +| Contract root | An entry that external consumers are allowed to import or invoke. | ## Architecture ### System overview ```mermaid -flowchart TB +flowchart LR subgraph Capture["Passive evidence capture"] - direction LR + direction TB Build["rs dev / rs build"] --> Rspack["Rspack observer"] Build --> Doctor["Rsdoctor collector"] Lib["rs lib"] --> Rslib["Rslib contract adapter"] Lint["rs lint"] --> Rslint["Resident Rslint worker"] Test["rs test"] --> Rstest["Rstest observer"] + Rspack --> Publish["Atomic record publisher"] + Doctor --> Publish + Rslib --> Publish + Rslint --> Publish + Rstest --> Publish end - Coordinator["Per-workspace context coordinator"] - Store[("Immutable snapshots + bounded event log")] - Query["Reachability, classification, diff, and query engine"] - Broker["rs mcp stdio broker"] + Store[("Workspace evidence store
.rstack/cache/context-v1")] + Broker["rs mcp
stdio reader + query engine"] subgraph Hosts["Agent hosts"] Codex["Codex plugin + skills"] @@ -158,29 +162,23 @@ flowchart TB OptionalUI["Optional Rsdoctor report UI"] - Rspack --> Coordinator - Doctor --> Coordinator - Rslib --> Coordinator - Rslint --> Coordinator - Rstest --> Coordinator - Coordinator --> Store - Store --> Query - Query --> Broker - Broker --> Codex - Broker --> Claude + Publish --> Store + Codex --> Broker + Claude --> Broker + Broker -->|"bounded reads"| Store Doctor -. "explicit report artifact" .-> OptionalUI ``` ### Component responsibilities -| Component | Responsibility | Must not do | -| ------------ | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | -| Tool adapter | Add one passive collector after the user's resolved configuration and correlate its lifecycle with a run. | Modify the user's config file, reorder user plugins, or fail the command. | -| Producer | Emit bounded, schema-versioned facts and completeness metadata. | Make cross-tool dead-code decisions. | -| Coordinator | Assign workspace, context, run, generation, and snapshot identities; merge producer facts; persist bounded state. | Execute project code merely because an MCP client connected. | -| Analyzer | Compute roots, reachability, contract requirements, findings, explanations, and diffs. | Hide unknown dynamic behavior or partial captures. | -| MCP broker | Expose one stdio server, enforce roots/capabilities, paginate output, and link resources. | Mount on a development server or expose a second Rsdoctor MCP endpoint. | -| Skills | Choose the correct queries, combine evidence, explain limits, and guide safe next actions. | Parse raw logs or represent candidates as proven dead. | +| Component | Responsibility | Must not do | +| ---------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | +| Tool adapter | Add one passive collector after the user's resolved configuration and correlate its lifecycle with a run. | Modify the user's config file, reorder user plugins, or fail the command. | +| Producer | Emit bounded, schema-versioned facts and completeness metadata into its own immutable run directory. | Make cross-tool dead-code decisions or mutate another producer's record. | +| Workspace store | Provide a disposable, task-runner-independent rendezvous of atomically published records for one checkout. | Execute project code, schedule tasks, or require a resident process. | +| Status reader/analyzer | Validate records and compute roots, reachability, contracts, findings, explanations, and diffs. | Hide malformed, unsupported, unknown, or partial evidence. | +| MCP broker | Expose one stdio server, enforce roots/capabilities, paginate output, and link resources. | Mount on a development server or expose a second Rsdoctor MCP endpoint. | +| Skills | Choose the correct queries, combine evidence, explain limits, and guide safe next actions. | Parse raw logs or represent candidates as proven dead. | ### Upstream and downstream ownership @@ -395,7 +393,8 @@ erDiagram WORKSPACE { string id - string rootDigest + string repositoryId + string checkoutDigest } CONTEXT { string id @@ -449,7 +448,10 @@ Normalized edge kinds include: ### Identity -- Workspace IDs derive from canonical repository identity, not the absolute checkout path. +- Repository IDs derive from canonical repository identity when available and remain stable across + related working trees. +- Workspace IDs are checkout/worktree-scoped. Their opaque value may incorporate a canonical root or + Git worktree identity, but absolute paths are never exposed through the MCP contract. - Context IDs derive from normalized config, target, mode, runtime, conditions, and redacted environment digest. - Semantic module, symbol, export, package, test, chunk, and route IDs are deterministic within a @@ -593,70 +595,109 @@ success result. ```mermaid sequenceDiagram participant RB as Build watch - participant C as Context coordinator + participant S as Workspace store participant L as Rslint worker participant T as Rstest watch + participant MCP as rs mcp participant M as Model - RB->>C: generation 184 (changed files) - C->>L: lint changed files - C-->>T: attach change set - RB->>C: build finished - C-->>M: build fresh, lint running, tests stale - L->>C: lint finished - C-->>M: build + lint fresh, tests stale - T->>C: affected tests finished - C-->>M: snapshot 184 complete + RB->>S: atomically publish build generation 184 + L->>S: publish explicit or requested lint result + T->>S: publish completed watch cycle + M->>MCP: project_status + MCP->>S: read completed records + MCP-->>M: build fresh, lint fresh, tests stale ``` Example status exposed to the model: ```text -DEV source=9a73f2 + 2 uncommitted files generation=184 -Build PASS 412ms [FRESH] -Lint RUNNING [generation 184] -Tests PASS 31/31 [STALE: generation 183; 2 changed files] +DEV source=9a73f2 + 2 uncommitted files +Build PASS 412ms [FRESH: build generation 184] +Lint RUNNING [lint generation 52] +Tests PASS 31/31 [STALE: test cycle 31; 2 changed files] Next wait for lint, or inspect the changed-file diagnostics already available ``` Rslint behavior during development: -- one resident engine per workspace; -- debounce and coalesce changed paths; -- lint changed files immediately; +- no background lint process is started merely because a dev server or MCP client exists; +- an explicit `rs lint` run publishes its result, while an approved MCP lint request may reuse one + resident engine within that MCP process; +- lint requested or explicitly changed files without blocking build or HMR; - schedule program-wide type checking on explicit request or idle policy; -- bind each result to generation and source digest; +- bind each result to its producer generation and source digest; - cancel by terminating and recreating the worker only when necessary. Rstest behavior during development: - attach only when the user already started `rs test --watch` or explicitly requested it; -- correlate each watch cycle with the latest observed generation; +- correlate each watch cycle with its source digest and the nearest observed build generation; - use related-test evidence to explain affected selection; - preserve previous results as stale until the new cycle finishes; - distinguish cancellation, infrastructure failure, and product test failure; - never claim exact case-to-symbol execution without an appropriately scoped coverage capture. -## Coordinator and transport +## Workspace store and transport ### Process model -One coordinator runs per trusted project. Rstack commands and watch processes publish over a private -Unix-domain socket or Windows named pipe. Agent hosts launch `rs mcp`, a stdio broker that discovers -or starts the coordinator and exposes the sole MCP surface. +Version 1 requires no coordinator process. Rstack commands and watch processes resolve identity from +their actual loaded config or package path and atomically publish into the checkout-local disposable +cache. Agent hosts may launch `rs mcp` from the repository root, a package, or another authorized MCP +root; the broker locates the workspace store without treating its CWD as package or build identity. ```mermaid flowchart TB - Commands["rs command processes"] -->|"private socket / named pipe"| Daemon["Project coordinator"] - Daemon --> WAL[("bounded WAL + snapshots")] - Codex["Codex"] -->|stdio| Broker["rs mcp"] - Claude["Claude Code"] -->|stdio| Broker - Broker -->|"private socket / named pipe"| Daemon + subgraph Shells["Commands may run in any package or shell"] + LibA["rs lib --watch
packages/a"] + LibB["rs lib --watch
packages/b"] + App["rs dev
apps/web"] + Tests["rs test --watch"] + end + + Store[(".rstack/cache/context-v1
immutable per-run records")] + Codex["Codex root session"] -->|stdio| Broker["rs mcp"] + Claude["Claude root session"] -->|stdio| Broker + LibA -->|"atomic publish"| Store + LibB -->|"atomic publish"| Store + App -->|"atomic publish"| Store + Tests -->|"atomic publish"| Store + Broker -->|"validate + query"| Store ``` -The coordinator manifest is per-user and atomic. Discovery validates PID, daemon boot ID, workspace -identity, owner, schema version, and lease. Stale manifests and sockets are removed only after those -checks. Unfinished runs recovered from the event log are marked aborted and stale. +Each producer owns `runs/`. It first publishes an immutable run manifest, then publishes each +completed context generation under that run. Publication writes a unique same-directory temporary +file and atomically links it into its final name; readers ignore temporary files and never observe a +partially written completed record. + +```text +.rstack/cache/context-v1/ +└── runs/ + └── / + ├── run.json + └── contexts/ + └── / + └── generations/ + └── -.json +``` + +The resolved hierarchy is checkout → package → tool/config → product → environment → run → +generation. A single Rslib process may therefore publish separate ESM, CJS, DTS, or bundleless +contexts, and a single Rsbuild process may publish client, server, or worker contexts. Concurrent +processes targeting the same context remain separate sessions; status reports ambiguity rather than +silently choosing one. + +Workspace discovery prefers the nearest package-manager workspace manifest, then a Git checkout +marker, then the nearest package root. It requires neither Turbo nor Nx and does not parse their task +graphs. Rstack CLI users receive adapters through resolved-config injection. Direct Rsbuild, Rspack, +Rslib, and Rstest users must add the corresponding explicit Rstack plugin or reporter; arbitrary +third-party processes cannot be instrumented safely by inference. + +Commands may start before the MCP process, and multiple MCP processes may read the same records. Each +broker keeps only a disposable in-memory query cache. A resident coordinator may be reconsidered if +measured multi-client cache duplication or event throughput proves it necessary; it is not part of +the version 1 contract. Loopback Streamable HTTP may be added later for explicit multi-client use. It is not the default and must require a random bearer capability, strict origin validation, session limits, and loopback-only @@ -664,7 +705,7 @@ binding. ### Storage -- Append-only events are a crash-recovery mechanism, not the query API. +- Completed records are immutable; incomplete run directories and temporary files are not queryable. - Snapshots are immutable and content-addressed where practical. - Only a bounded latest history is retained by default. - Source, maps, logs, coverage, and deep graphs have independent caps and retention policies. @@ -909,7 +950,8 @@ instructions. ### Required controls -- Host-facing transport is stdio; internal IPC uses owner-only sockets or named pipes. +- Host-facing transport is stdio; producers and brokers exchange evidence only through the + checkout-local project cache in version 1. - Never expose MCP or report queries on the dev-server host/port. - Resolve and realpath every requested path; reject traversal, symlink escape, and paths outside MCP roots. @@ -927,14 +969,14 @@ instructions. Passive metadata collection targets: -| Metric | Budget | -| ------------------------------ | ------------------------------------------------------------- | -| One-shot build overhead | Less than 2% | -| Context-engine startup | Less than 100 ms after package load | -| Incremental/watch observer p95 | Less than 25 ms per generation | -| Coordinator resident memory | Less than 50 MiB excluding explicitly retained deep artifacts | -| Default MCP query | Less than 500 ms warm | -| Bounded graph query | One concurrent query, 2 s deadline, 1 MiB response cap | +| Metric | Budget | +| ------------------------------- | ------------------------------------------------------------- | +| One-shot build overhead | Less than 2% | +| Context-engine startup | Less than 100 ms after package load | +| Incremental/watch observer p95 | Less than 25 ms per generation | +| MCP query-cache resident memory | Less than 50 MiB excluding explicitly retained deep artifacts | +| Default MCP query | Less than 500 ms warm | +| Bounded graph query | One concurrent query, 2 s deadline, 1 MiB response cap | High-cardinality module/resolution hooks, module sources, full reasons, source maps, deep coverage, and Rspack/Rsdoctor profiling are opt-in. Every snapshot records extraction time, heap delta where @@ -953,7 +995,7 @@ emits a drop marker, degrades the relevant facet to partial, and never blocks th - Cancelled and infrastructure-failed tests are not product test failures. - Watch restart creates a new instance identity and preserves the prior snapshot as stale. - A process exit without a completion marker closes the run as aborted/unknown. -- The coordinator rejects unsupported schema majors and records compatible minor capabilities. +- The status reader rejects unsupported schema majors and reports compatible minor capabilities. - Source changes invalidate only affected facets; unaffected results may remain fresh when their dependency digest proves applicability. @@ -1012,7 +1054,15 @@ cross-tool product, lint, test, contract, freshness, or permission model. ### Ship separate MCP servers for each tool Rejected. It duplicates lifecycle, trust, roots, transport, and discovery; gives the model conflicting -schemas; and prevents atomic cross-producer snapshots. +schemas; and prevents consistent cross-producer querying. + +### Require a workspace coordinator daemon + +Deferred unless measurements justify it. A daemon can centralize query caches and event delivery, but +it also introduces process discovery, sockets or named pipes, leases, restart recovery, version skew, +and cross-worktree isolation before those costs are necessary. Immutable per-run files already allow +commands and any number of root-launched MCP processes to rendezvous without a task runner or shared +process. A future daemon must consume the same store contract rather than replace it. ### Mount MCP on the Rsbuild dev server @@ -1045,7 +1095,7 @@ paths, trees, tables, and evidence cards; a full visual graph is optional invest ```mermaid flowchart TB - P0["Phase 0: contracts
Preview packages + versioned Rsdoctor artifacts"] + P0["Phase 0: foundation + contracts
Workspace discovery + immutable records"] P1["Phase 1: passive build context
Snapshots + one read-only MCP server"] P2["Phase 2: reachability
Product roots + unused-code skills"] P3["Phase 3: development intelligence
Generations + Rslint + Rstest"] @@ -1055,8 +1105,10 @@ flowchart TB P0 --> P1 --> P2 --> P3 --> P4 --> P5 ``` -### Phase 0: contracts +### Phase 0: foundation and contracts +- Implement config-path-based checkout/package discovery, the versioned workspace evidence store, + immutable publication, bounded validation, and the deterministic status reader. - Define normalized entity, edge, evidence, snapshot, finding, and compatibility schemas. - Land Rsdoctor preview packages and artifact metadata. - Contract-test the Rspack/Rsdoctor payload against pinned versions. @@ -1065,8 +1117,7 @@ flowchart TB - Add trusted, metadata-only Rsbuild/Rspack and Rslib observers. - Ingest static Rsdoctor artifacts through the in-process Agent CLI. -- Implement the per-project coordinator, immutable snapshots, `rs mcp`, status, diagnostics, basic - entity queries, and report links. +- Implement `rs mcp`, status, diagnostics, basic entity queries, record retention, and report links. ### Phase 2: reachability and skills @@ -1077,8 +1128,8 @@ flowchart TB ### Phase 3: development intelligence -- Add source generations, bounded event subscriptions, resident Rslint worker, and passive Rstest - attachment. +- Add producer-local source generations, bounded event subscriptions, MCP-process-scoped resident + Rslint workers, and passive Rstest attachment. - Ship `debug-dev-cycle` and `select-affected-tests` skills. ### Phase 4: change and mutation workflows @@ -1129,8 +1180,8 @@ No high-confidence dead finding may include: - Raw stdio JSON-RPC transcripts and MCP SDK clients. - Initialization ordering, schema negotiation, invalid params, cancellation, progress, pagination, subscriptions, and stdout purity. -- Concurrent readers, producer backpressure, coordinator restart, stale manifest, PID reuse, crash - recovery, version skew, and orphan cleanup. +- Concurrent readers and writers, immutable-name collisions, ignored temporary files, incomplete run + directories, crash recovery, bounded retention, schema skew, and orphan cleanup. - Watch tests wait for generation changes rather than sleeping. ### Security @@ -1144,9 +1195,9 @@ No high-confidence dead finding may include: ### Performance Benchmark small, medium, and large workspaces across cold, warm, and incremental runs. Track build -overhead, incremental p95, extraction bytes, queue drops, coordinator RSS, query p95, and response -size. Pull requests fail only on statistically meaningful regressions beyond versioned budgets; -large stress cases run nightly. +overhead, incremental p95, extraction bytes, queue drops, store bytes, MCP query-cache RSS, query p95, +and response size. Pull requests fail only on statistically meaningful regressions beyond versioned +budgets; large stress cases run nightly. ## Acceptance criteria diff --git a/docs/superpowers/plans/2026-08-12-context-store-foundation.md b/docs/superpowers/plans/2026-08-12-context-store-foundation.md new file mode 100644 index 00000000..514e208c --- /dev/null +++ b/docs/superpowers/plans/2026-08-12-context-store-foundation.md @@ -0,0 +1,214 @@ +# Context store foundation implementation plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development +> (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use +> checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Establish the smallest task-runner-independent foundation that lets Rstack producers running +anywhere in a checkout publish immutable context snapshots for a root-launched MCP process to read. + +**Architecture:** Producers resolve their checkout and package identity from their actual config path, +then write bounded, versioned records into `.rstack/cache/context-v1`. Every run owns a unique +directory, so concurrent Rslib, Rsbuild, Rstest, Rslint, Rspack, and Rsdoctor processes do not share a +mutable database. A read-only status API scans only completed records; no daemon, socket, task-runner +integration, or MCP transport is introduced in this foundation. + +**Tech Stack:** TypeScript, Node.js filesystem APIs, Rstack project cache, Rstest. + +## Global constraints + +- Work only on a non-main `codex/` branch. +- Treat MCP CWD as an authorization/discovery start, never as package or build identity. +- Do not depend on Turbo, Nx, pnpm recursive execution, or any other task runner. +- Store only workspace-relative package/config paths in records. +- Use immutable per-run records and atomic publication; readers must ignore temporary files. +- Bound individual records to 1 MiB and report malformed or unsupported records as store issues. +- Cache failures must be observable but must not force a future producer to fail its underlying tool. +- Do not add a CLI command, daemon, MCP server, collector injection, or public package export yet. + +--- + +### Task 1: resolve checkout and package identity + +**Files:** + +- Create: `packages/rstack/src/context/workspace.ts` +- Create: `packages/rstack/tests/context/workspace.test.ts` + +**Interfaces:** + +- Consumes: an existing config file or directory path supplied by a producer. +- Produces: `resolveContextWorkspace(startPath): Promise` where the result + contains canonical `workspaceRoot`, `packageRoot`, and optional `packageName`. + +- [ ] **Step 1: Write the failing workspace tests** + +```ts +test('resolves a package from its config path without using process cwd', async () => { + const result = await resolveContextWorkspace(configPath); + expect(result).toEqual({ workspaceRoot, packageRoot, packageName: '@repo/lib' }); +}); + +test('falls back to a standalone package root', async () => { + const result = await resolveContextWorkspace(configPath); + expect(result.workspaceRoot).toBe(packageRoot); +}); +``` + +- [ ] **Step 2: Run the focused test and verify RED** + +Run: `pnpm --filter rstack test -- tests/context/workspace.test.ts` + +Expected: FAIL because `src/context/workspace.ts` does not exist. + +- [ ] **Step 3: Implement the minimal resolver** + +Walk canonical ancestors once. Prefer the nearest `pnpm-workspace.yaml`, +`pnpm-workspace.yml`, or `package.json#workspaces`; otherwise use the nearest Git checkout marker, +then the nearest package root, then the start directory. Read the nearest `package.json#name` without +executing project code. + +- [ ] **Step 4: Run the focused test and verify GREEN** + +Run: `pnpm --filter rstack test -- tests/context/workspace.test.ts` + +Expected: PASS with both workspace cases green. + +### Task 2: publish and read immutable context records + +**Files:** + +- Create: `packages/rstack/src/context/model.ts` +- Create: `packages/rstack/src/context/store.ts` +- Create: `packages/rstack/src/context/index.ts` +- Create: `packages/rstack/tests/context/store.test.ts` + +**Interfaces:** + +- Consumes: the workspace root from Task 1, one `ContextRunManifest`, and immutable + `ContextSnapshot` records. +- Produces: `writeContextRunManifest`, `writeContextSnapshot`, and + `readContextWorkspaceStatus`; all schemas use `contextStoreSchemaVersion = 1`. + +- [ ] **Step 1: Write the failing store tests** + +```ts +test('publishes concurrent run snapshots and reads each latest context', async () => { + expect(await writeContextRunManifest(rootPath, run)).toMatchObject({ written: true }); + expect(await writeContextSnapshot(rootPath, first)).toMatchObject({ written: true }); + expect(await writeContextSnapshot(rootPath, second)).toMatchObject({ written: true }); + expect(await readContextWorkspaceStatus(rootPath)).toMatchObject({ + runs: [{ run, contexts: [{ context: run.contexts[0], latestSnapshot: second }] }], + }); +}); + +test('does not replace an immutable record', async () => { + expect(await writeContextSnapshot(rootPath, first)).toMatchObject({ written: true }); + expect(await writeContextSnapshot(rootPath, replacement)).toMatchObject({ written: false }); +}); + +test('reports malformed completed records without reading temporary files', async () => { + const status = await readContextWorkspaceStatus(rootPath); + expect(status.issues).toEqual([ + expect.objectContaining({ code: 'invalid-record', path: expect.any(String) }), + ]); +}); +``` + +- [ ] **Step 2: Run the focused test and verify RED** + +Run: `pnpm --filter rstack test -- tests/context/store.test.ts` + +Expected: FAIL because the context model and store do not exist. + +- [ ] **Step 3: Implement the minimal immutable store** + +Use the existing `ensureProjectCacheDir()` and the layout +`context-v1/runs//run.json` plus +`context-v1/runs//contexts//generations/-.json`. +Serialize JSON with a trailing newline, reject unsafe IDs and records over 1 MiB, write a unique +same-directory temporary file, and atomically hard-link it into its final immutable name. The reader +must validate schema version and required fields, return stable sorting, and report bounded relative +issue paths. + +- [ ] **Step 4: Run both context test files and verify GREEN** + +Run: `pnpm --filter rstack test -- tests/context/workspace.test.ts tests/context/store.test.ts` + +Expected: PASS with no warnings. + +### Task 3: make the lean architecture normative + +**Files:** + +- Modify: `docs/rfcs/0001-rstack-context-engine.md` + +**Interfaces:** + +- Consumes: the approved workspace-store architecture and the concrete Task 1/2 contract. +- Produces: an RFC whose diagrams, lifecycle, identity, storage, alternatives, budgets, and delivery + plan consistently describe a daemon-free version 1. + +- [ ] **Step 1: Replace the coordinator diagrams and lifecycle** + +Show independent package-local producers atomically publishing into the workspace evidence store and +root-launched Codex/Claude stdio MCP processes reading it. Explain that all MCP instances share the +same immutable cache without sharing process memory. + +- [ ] **Step 2: Specify discovery and identity** + +Distinguish stable repository identity from checkout/worktree identity. State that resolved config, +package root, tool, product, environment, run, and generation identify observations; CWD never does. + +- [ ] **Step 3: Update development mode, alternatives, and delivery phases** + +Keep build, Rslint, and Rstest independent producers. Explicitly defer a coordinator daemon until +measured multi-client caching or event throughput proves it necessary. Move the workspace store and +status reader into Phase 0. + +- [ ] **Step 4: Re-render every Mermaid diagram** + +Run the repository Mermaid validation command and render all RFC diagrams in light and dark themes. +Inspect every resulting image for clipped text, invalid edges, unreadable contrast, or misleading +process ownership. + +### Task 4: verify and commit the foundation + +**Files:** + +- Verify all files from Tasks 1-3. + +**Interfaces:** + +- Consumes: completed implementation and documentation. +- Produces: one reviewed commit on `codex/rstack-mcp-observability`. + +- [ ] **Step 1: Format and run focused tests** + +Run: `pnpm exec rs fmt packages/rstack/src/context packages/rstack/tests/context docs/rfcs/0001-rstack-context-engine.md docs/superpowers/plans/2026-08-12-context-store-foundation.md` + +Run: `pnpm --filter rstack test -- tests/context` + +- [ ] **Step 2: Build and run repository checks** + +Run: `pnpm --filter rstack build` + +Run: `pnpm check` + +Run: `pnpm check:spell` + +- [ ] **Step 3: Review the final diff and requirements** + +Confirm the branch is not `main` or `master`; confirm no daemon, socket, MCP server, task-runner +dependency, config mutation, or public export was added; confirm every stored path is relative and +every completed record is immutable. + +- [ ] **Step 4: Commit** + +```bash +git add docs/rfcs/0001-rstack-context-engine.md \ + docs/superpowers/plans/2026-08-12-context-store-foundation.md \ + packages/rstack/src/context \ + packages/rstack/tests/context +git commit -m "feat: scaffold context evidence store" +``` diff --git a/packages/rstack/src/context/index.ts b/packages/rstack/src/context/index.ts new file mode 100644 index 00000000..232f5d3c --- /dev/null +++ b/packages/rstack/src/context/index.ts @@ -0,0 +1,22 @@ +export { + contextStoreMaxRecordBytes, + contextStoreSchemaVersion, + type ContextCompleteness, + type ContextDescriptor, + type ContextProducer, + type ContextRunManifest, + type ContextRunStatus, + type ContextRunStatusEntry, + type ContextSnapshot, + type ContextStatus, + type ContextStoreIssue, + type ContextStoreWriteResult, + type ContextWorkspaceStatus, + type JsonValue, +} from './model.ts'; +export { + readContextWorkspaceStatus, + writeContextRunManifest, + writeContextSnapshot, +} from './store.ts'; +export { resolveContextWorkspace, type ResolvedContextWorkspace } from './workspace.ts'; diff --git a/packages/rstack/src/context/model.ts b/packages/rstack/src/context/model.ts new file mode 100644 index 00000000..8cc024a6 --- /dev/null +++ b/packages/rstack/src/context/model.ts @@ -0,0 +1,85 @@ +const contextStoreSchemaVersion = 1 as const; +const contextStoreMaxRecordBytes: number = 1024 * 1024; + +type JsonPrimitive = boolean | null | number | string; +type JsonValue = JsonPrimitive | JsonValue[] | { [key: string]: JsonValue }; + +type ContextProducer = 'rsbuild' | 'rspack' | 'rslib' | 'rstest' | 'rslint' | 'rsdoctor'; +type ContextRunStatus = 'queued' | 'running' | 'pass' | 'fail' | 'cancelled' | 'error'; +type ContextCompleteness = 'complete' | 'partial' | 'disabled' | 'unsupported'; + +type ContextDescriptor = { + contextId: string; + packageRoot: string; + product: string; + packageName?: string; + configPath?: string; + environment?: string; + target?: string; + mode?: string; +}; + +type ContextRunManifest = { + schemaVersion: typeof contextStoreSchemaVersion; + runId: string; + producer: ContextProducer; + command: string; + startedAt: string; + contexts: ContextDescriptor[]; +}; + +type ContextSnapshot = { + schemaVersion: typeof contextStoreSchemaVersion; + snapshotId: string; + runId: string; + contextId: string; + sequence: number; + observedAt: string; + status: ContextRunStatus; + completeness: Record; + facets: Record; + source?: { + revision?: string; + dirtyDigest?: string; + }; +}; + +type ContextStoreWriteResult = + { written: true; path: string } | { written: false; path: string; error: unknown }; + +type ContextStoreIssue = { + code: 'invalid-record' | 'oversized-record' | 'unsupported-schema'; + path: string; +}; + +type ContextStatus = { + context: ContextDescriptor; + latestSnapshot?: ContextSnapshot; +}; + +type ContextRunStatusEntry = { + run: ContextRunManifest; + contexts: ContextStatus[]; +}; + +type ContextWorkspaceStatus = { + schemaVersion: typeof contextStoreSchemaVersion; + runs: ContextRunStatusEntry[]; + issues: ContextStoreIssue[]; +}; + +export { contextStoreMaxRecordBytes, contextStoreSchemaVersion }; +export type { + ContextCompleteness, + ContextDescriptor, + ContextProducer, + ContextRunManifest, + ContextRunStatus, + ContextRunStatusEntry, + ContextSnapshot, + ContextStatus, + ContextStoreIssue, + ContextStoreWriteResult, + ContextWorkspaceStatus, + JsonValue, +}; diff --git a/packages/rstack/src/context/store.ts b/packages/rstack/src/context/store.ts new file mode 100644 index 00000000..ccfc5df6 --- /dev/null +++ b/packages/rstack/src/context/store.ts @@ -0,0 +1,335 @@ +import { randomUUID } from 'node:crypto'; +import { link, mkdir, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'; +import path from 'node:path'; +import { ensureProjectCacheDir, getProjectCacheDir } from '../projectCache.ts'; +import { + contextStoreMaxRecordBytes, + contextStoreSchemaVersion, + type ContextCompleteness, + type ContextDescriptor, + type ContextProducer, + type ContextRunManifest, + type ContextRunStatus, + type ContextSnapshot, + type ContextStoreIssue, + type ContextStoreWriteResult, + type ContextWorkspaceStatus, +} from './model.ts'; + +const contextStoreDirectoryName = 'context-v1'; +const safeIdentifierPattern = /^[a-zA-Z0-9][a-zA-Z0-9._-]{0,127}$/u; +const producers = new Set([ + 'rsbuild', + 'rspack', + 'rslib', + 'rstest', + 'rslint', + 'rsdoctor', +]); +const statuses = new Set([ + 'queued', + 'running', + 'pass', + 'fail', + 'cancelled', + 'error', +]); +const completenessValues = new Set([ + 'complete', + 'partial', + 'disabled', + 'unsupported', +]); + +const isObject = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value); + +const isSafeIdentifier = (value: unknown): value is string => + typeof value === 'string' && safeIdentifierPattern.test(value); + +const isRelativeRecordPath = (value: unknown): value is string => { + if (typeof value !== 'string' || value.length === 0 || value.includes('\\')) { + return false; + } + return ( + value === '.' || + (!path.posix.isAbsolute(value) && + !value.split('/').includes('..') && + path.posix.normalize(value) === value) + ); +}; + +const isContextDescriptor = (value: unknown): value is ContextDescriptor => + isObject(value) && + isSafeIdentifier(value.contextId) && + isRelativeRecordPath(value.packageRoot) && + typeof value.product === 'string' && + value.product.length > 0 && + (value.packageName === undefined || typeof value.packageName === 'string') && + (value.configPath === undefined || isRelativeRecordPath(value.configPath)) && + (value.environment === undefined || typeof value.environment === 'string') && + (value.target === undefined || typeof value.target === 'string') && + (value.mode === undefined || typeof value.mode === 'string'); + +const isContextRunManifest = (value: unknown): value is ContextRunManifest => + isObject(value) && + value.schemaVersion === contextStoreSchemaVersion && + isSafeIdentifier(value.runId) && + producers.has(value.producer as ContextProducer) && + typeof value.command === 'string' && + typeof value.startedAt === 'string' && + Array.isArray(value.contexts) && + value.contexts.length > 0 && + value.contexts.every(isContextDescriptor); + +const isCompleteness = (value: unknown): value is Record => + isObject(value) && Object.values(value).every((entry) => completenessValues.has(entry as never)); + +const isContextSnapshot = (value: unknown): value is ContextSnapshot => + isObject(value) && + value.schemaVersion === contextStoreSchemaVersion && + isSafeIdentifier(value.snapshotId) && + isSafeIdentifier(value.runId) && + isSafeIdentifier(value.contextId) && + Number.isSafeInteger(value.sequence) && + (value.sequence as number) >= 0 && + typeof value.observedAt === 'string' && + statuses.has(value.status as ContextRunStatus) && + isCompleteness(value.completeness) && + isObject(value.facets); + +const getContextStoreRoot = (workspaceRoot: string): string => + path.join(getProjectCacheDir(workspaceRoot), contextStoreDirectoryName); + +const getRunRoot = (storeRoot: string, runId: string): string => + path.join(storeRoot, 'runs', runId); + +const getRunManifestPath = (storeRoot: string, runId: string): string => + path.join(getRunRoot(storeRoot, runId), 'run.json'); + +const getSnapshotPath = (storeRoot: string, snapshot: ContextSnapshot): string => + path.join( + getRunRoot(storeRoot, snapshot.runId), + 'contexts', + snapshot.contextId, + 'generations', + `${snapshot.sequence.toString().padStart(10, '0')}-${snapshot.snapshotId}.json`, + ); + +const serializeRecord = (record: unknown): string => { + const content = `${JSON.stringify(record)}\n`; + if (Buffer.byteLength(content) > contextStoreMaxRecordBytes) { + throw new Error(`Context record exceeds ${contextStoreMaxRecordBytes} bytes.`); + } + return content; +}; + +const publishImmutableRecord = async ( + filePath: string, + content: string, +): Promise => { + const temporaryPath = path.join( + path.dirname(filePath), + `.${path.basename(filePath)}.${process.pid}.${randomUUID()}.tmp`, + ); + + try { + await mkdir(path.dirname(filePath), { recursive: true }); + await writeFile(temporaryPath, content, { flag: 'wx' }); + await link(temporaryPath, filePath); + return { written: true, path: filePath }; + } catch (error) { + return { written: false, path: filePath, error }; + } finally { + await rm(temporaryPath, { force: true }).catch(() => undefined); + } +}; + +const unavailableWrite = (workspaceRoot: string, error: unknown): ContextStoreWriteResult => ({ + written: false, + path: getContextStoreRoot(workspaceRoot), + error, +}); + +const writeContextRunManifest = async ( + workspaceRoot: string, + run: ContextRunManifest, +): Promise => { + if (!isContextRunManifest(run)) { + return unavailableWrite(workspaceRoot, new Error('Invalid context run manifest.')); + } + + try { + const cache = await ensureProjectCacheDir(workspaceRoot); + if (cache.status === 'unavailable') { + return unavailableWrite(workspaceRoot, cache.error); + } + return publishImmutableRecord( + getRunManifestPath(path.join(cache.path, contextStoreDirectoryName), run.runId), + serializeRecord(run), + ); + } catch (error) { + return unavailableWrite(workspaceRoot, error); + } +}; + +const writeContextSnapshot = async ( + workspaceRoot: string, + snapshot: ContextSnapshot, +): Promise => { + if (!isContextSnapshot(snapshot)) { + return unavailableWrite(workspaceRoot, new Error('Invalid context snapshot.')); + } + + try { + const cache = await ensureProjectCacheDir(workspaceRoot); + if (cache.status === 'unavailable') { + return unavailableWrite(workspaceRoot, cache.error); + } + return publishImmutableRecord( + getSnapshotPath(path.join(cache.path, contextStoreDirectoryName), snapshot), + serializeRecord(snapshot), + ); + } catch (error) { + return unavailableWrite(workspaceRoot, error); + } +}; + +type ReadRecordResult = + | { status: 'missing' } + | { status: 'issue'; issue: ContextStoreIssue } + | { status: 'value'; value: unknown }; + +const readRecord = async (filePath: string, relativePath: string): Promise => { + try { + if ((await stat(filePath)).size > contextStoreMaxRecordBytes) { + return { status: 'issue', issue: { code: 'oversized-record', path: relativePath } }; + } + return { status: 'value', value: JSON.parse(await readFile(filePath, 'utf8')) as unknown }; + } catch (error) { + if (error instanceof Error && 'code' in error && error.code === 'ENOENT') { + return { status: 'missing' }; + } + return { status: 'issue', issue: { code: 'invalid-record', path: relativePath } }; + } +}; + +const readDirectoryNames = async (directoryPath: string): Promise => { + try { + const entries = await readdir(directoryPath, { withFileTypes: true }); + return entries + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort(); + } catch { + return []; + } +}; + +const readLatestSnapshot = async ( + storeRoot: string, + run: ContextRunManifest, + context: ContextDescriptor, + issues: ContextStoreIssue[], +): Promise => { + const relativeGenerationRoot = path.posix.join( + 'runs', + run.runId, + 'contexts', + context.contextId, + 'generations', + ); + const generationRoot = path.join(storeRoot, ...relativeGenerationRoot.split('/')); + let fileNames: string[]; + try { + fileNames = (await readdir(generationRoot)) + .filter((fileName) => fileName.endsWith('.json')) + .sort(); + } catch { + return undefined; + } + + let latestSnapshot: ContextSnapshot | undefined; + for (const fileName of fileNames) { + const relativePath = path.posix.join(relativeGenerationRoot, fileName); + const record = await readRecord(path.join(generationRoot, fileName), relativePath); + if (record.status === 'issue') { + issues.push(record.issue); + continue; + } + if ( + record.status !== 'value' || + !isContextSnapshot(record.value) || + record.value.runId !== run.runId || + record.value.contextId !== context.contextId + ) { + if (record.status === 'value') { + issues.push({ code: 'invalid-record', path: relativePath }); + } + continue; + } + if ( + latestSnapshot === undefined || + record.value.sequence > latestSnapshot.sequence || + (record.value.sequence === latestSnapshot.sequence && + record.value.snapshotId > latestSnapshot.snapshotId) + ) { + latestSnapshot = record.value; + } + } + return latestSnapshot; +}; + +const readContextWorkspaceStatus = async ( + workspaceRoot: string, +): Promise => { + const storeRoot = getContextStoreRoot(workspaceRoot); + const issues: ContextStoreIssue[] = []; + const runs = []; + + for (const runId of await readDirectoryNames(path.join(storeRoot, 'runs'))) { + const relativePath = path.posix.join('runs', runId, 'run.json'); + const record = await readRecord(getRunManifestPath(storeRoot, runId), relativePath); + if (record.status === 'issue') { + issues.push(record.issue); + continue; + } + if (record.status === 'missing') { + continue; + } + if (!isObject(record.value) || record.value.schemaVersion !== contextStoreSchemaVersion) { + issues.push({ + code: isObject(record.value) ? 'unsupported-schema' : 'invalid-record', + path: relativePath, + }); + continue; + } + if (!isContextRunManifest(record.value) || record.value.runId !== runId) { + issues.push({ code: 'invalid-record', path: relativePath }); + continue; + } + + const run = record.value; + runs.push({ + run, + contexts: await Promise.all( + run.contexts.map(async (context) => { + const latestSnapshot = await readLatestSnapshot(storeRoot, run, context, issues); + return { + context, + ...(latestSnapshot === undefined ? {} : { latestSnapshot }), + }; + }), + ), + }); + } + + issues.sort((left, right) => + left.path === right.path + ? left.code.localeCompare(right.code) + : left.path.localeCompare(right.path), + ); + return { schemaVersion: contextStoreSchemaVersion, runs, issues }; +}; + +export { readContextWorkspaceStatus, writeContextRunManifest, writeContextSnapshot }; diff --git a/packages/rstack/src/context/workspace.ts b/packages/rstack/src/context/workspace.ts new file mode 100644 index 00000000..f2946de9 --- /dev/null +++ b/packages/rstack/src/context/workspace.ts @@ -0,0 +1,95 @@ +import { readFile, realpath, stat } from 'node:fs/promises'; +import path from 'node:path'; + +type ResolvedContextWorkspace = { + workspaceRoot: string; + packageRoot: string; + packageName?: string; +}; + +type PackageMetadata = { + exists: boolean; + isWorkspace: boolean; + name?: string; +}; + +const pathExists = async (filePath: string): Promise => { + try { + await stat(filePath); + return true; + } catch { + return false; + } +}; + +const readPackageMetadata = async (directoryPath: string): Promise => { + try { + const value = JSON.parse(await readFile(path.join(directoryPath, 'package.json'), 'utf8')) as { + name?: unknown; + workspaces?: unknown; + }; + + return { + exists: true, + isWorkspace: + Array.isArray(value.workspaces) || + (typeof value.workspaces === 'object' && value.workspaces !== null), + ...(typeof value.name === 'string' ? { name: value.name } : {}), + }; + } catch (error) { + if (error instanceof SyntaxError) { + return { exists: true, isWorkspace: false }; + } + return { exists: false, isWorkspace: false }; + } +}; + +const hasPnpmWorkspaceManifest = async (directoryPath: string): Promise => + (await pathExists(path.join(directoryPath, 'pnpm-workspace.yaml'))) || + (await pathExists(path.join(directoryPath, 'pnpm-workspace.yml'))); + +const resolveContextWorkspace = async (startPath: string): Promise => { + const canonicalStartPath = await realpath(startPath); + const startStats = await stat(canonicalStartPath); + const startDirectory = startStats.isDirectory() + ? canonicalStartPath + : path.dirname(canonicalStartPath); + let currentPath = startDirectory; + let packageRoot: string | undefined; + let packageName: string | undefined; + let workspaceRoot: string | undefined; + let checkoutRoot: string | undefined; + + while (true) { + const packageMetadata = await readPackageMetadata(currentPath); + if (packageRoot === undefined && packageMetadata.exists) { + packageRoot = currentPath; + packageName = packageMetadata.name; + } + if ( + workspaceRoot === undefined && + (packageMetadata.isWorkspace || (await hasPnpmWorkspaceManifest(currentPath))) + ) { + workspaceRoot = currentPath; + } + if (checkoutRoot === undefined && (await pathExists(path.join(currentPath, '.git')))) { + checkoutRoot = currentPath; + } + + const parentPath = path.dirname(currentPath); + if (parentPath === currentPath) { + break; + } + currentPath = parentPath; + } + + const resolvedWorkspaceRoot = workspaceRoot ?? checkoutRoot ?? packageRoot ?? startDirectory; + return { + workspaceRoot: resolvedWorkspaceRoot, + packageRoot: packageRoot ?? resolvedWorkspaceRoot, + ...(packageName === undefined ? {} : { packageName }), + }; +}; + +export { resolveContextWorkspace }; +export type { ResolvedContextWorkspace }; diff --git a/packages/rstack/src/projectCache.ts b/packages/rstack/src/projectCache.ts index 86d0ffcb..5959b1ce 100644 --- a/packages/rstack/src/projectCache.ts +++ b/packages/rstack/src/projectCache.ts @@ -31,5 +31,5 @@ const ensureProjectCacheDir = async (rootPath: string): Promise Promise, +): Promise => { + const workspaceRoot = await mkdtemp(path.join(os.tmpdir(), 'rstack-context-store-')); + + try { + await callback(workspaceRoot); + } finally { + await rm(workspaceRoot, { force: true, recursive: true }); + } +}; + +test('publishes immutable run snapshots and reads the latest context state', async () => { + await withTempWorkspace(async (workspaceRoot) => { + expect(await writeContextRunManifest(workspaceRoot, run)).toMatchObject({ written: true }); + expect(await writeContextSnapshot(workspaceRoot, firstSnapshot)).toMatchObject({ + written: true, + }); + expect(await writeContextSnapshot(workspaceRoot, secondSnapshot)).toMatchObject({ + written: true, + }); + + await expect(readContextWorkspaceStatus(workspaceRoot)).resolves.toEqual({ + schemaVersion: contextStoreSchemaVersion, + runs: [ + { + run, + contexts: [{ context, latestSnapshot: secondSnapshot }], + }, + ], + issues: [], + }); + + const cacheRoot = path.join(workspaceRoot, '.rstack', 'cache'); + expect(await readFile(path.join(cacheRoot, '.gitignore'), 'utf8')).toBe('*\n'); + expect( + (await readdir(path.join(cacheRoot, 'context-v1'), { recursive: true })).sort(), + ).not.toEqual(expect.arrayContaining([expect.stringMatching(/\.tmp$/u)])); + }); +}); + +test('does not replace an immutable snapshot record', async () => { + await withTempWorkspace(async (workspaceRoot) => { + await writeContextRunManifest(workspaceRoot, run); + expect(await writeContextSnapshot(workspaceRoot, firstSnapshot)).toMatchObject({ + written: true, + }); + + const replacement = { + ...firstSnapshot, + facets: { summary: { errors: 42 } }, + } satisfies ContextSnapshot; + expect(await writeContextSnapshot(workspaceRoot, replacement)).toMatchObject({ + written: false, + }); + + const status = await readContextWorkspaceStatus(workspaceRoot); + expect(status.runs[0]?.contexts[0]?.latestSnapshot).toEqual(firstSnapshot); + }); +}); + +test('reports malformed completed records and ignores temporary files', async () => { + await withTempWorkspace(async (workspaceRoot) => { + await writeContextRunManifest(workspaceRoot, run); + await writeContextSnapshot(workspaceRoot, firstSnapshot); + const generationRoot = path.join( + workspaceRoot, + '.rstack', + 'cache', + 'context-v1', + 'runs', + run.runId, + 'contexts', + context.contextId, + 'generations', + ); + await mkdir(generationRoot, { recursive: true }); + await writeFile(path.join(generationRoot, '0000000002-broken.json'), '{broken'); + await writeFile(path.join(generationRoot, '.pending.tmp'), '{broken'); + + const status = await readContextWorkspaceStatus(workspaceRoot); + expect(status.issues).toEqual([ + { + code: 'invalid-record', + path: path.posix.join( + 'runs', + run.runId, + 'contexts', + context.contextId, + 'generations', + '0000000002-broken.json', + ), + }, + ]); + expect(status.runs[0]?.contexts[0]?.latestSnapshot).toEqual(firstSnapshot); + }); +}); + +test('rejects records larger than the store limit', async () => { + await withTempWorkspace(async (workspaceRoot) => { + await writeContextRunManifest(workspaceRoot, run); + expect( + await writeContextSnapshot(workspaceRoot, { + ...firstSnapshot, + snapshotId: 'snap_oversized', + facets: { payload: 'x'.repeat(contextStoreMaxRecordBytes) }, + }), + ).toMatchObject({ written: false }); + }); +}); + +test('reports oversized records without parsing them', async () => { + await withTempWorkspace(async (workspaceRoot) => { + await writeContextRunManifest(workspaceRoot, run); + const generationRoot = path.join( + workspaceRoot, + '.rstack', + 'cache', + 'context-v1', + 'runs', + run.runId, + 'contexts', + context.contextId, + 'generations', + ); + const fileName = '0000000001-snap_oversized.json'; + await mkdir(generationRoot, { recursive: true }); + await writeFile( + path.join(generationRoot, fileName), + JSON.stringify({ + ...firstSnapshot, + snapshotId: 'snap_oversized', + facets: { payload: 'x'.repeat(contextStoreMaxRecordBytes) }, + }), + ); + + const status = await readContextWorkspaceStatus(workspaceRoot); + expect(status.issues).toEqual([ + { + code: 'oversized-record', + path: path.posix.join( + 'runs', + run.runId, + 'contexts', + context.contextId, + 'generations', + fileName, + ), + }, + ]); + }); +}); + +test('rejects unsafe identifiers and escaping record paths', async () => { + await withTempWorkspace(async (workspaceRoot) => { + expect( + await writeContextRunManifest(workspaceRoot, { + ...run, + runId: '../outside', + }), + ).toMatchObject({ written: false }); + expect( + await writeContextRunManifest(workspaceRoot, { + ...run, + contexts: [{ ...context, packageRoot: '../outside' }], + }), + ).toMatchObject({ written: false }); + await expect(readContextWorkspaceStatus(workspaceRoot)).resolves.toEqual({ + schemaVersion: contextStoreSchemaVersion, + runs: [], + issues: [], + }); + }); +}); diff --git a/packages/rstack/tests/context/workspace.test.ts b/packages/rstack/tests/context/workspace.test.ts new file mode 100644 index 00000000..09d705f6 --- /dev/null +++ b/packages/rstack/tests/context/workspace.test.ts @@ -0,0 +1,82 @@ +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import { expect, test } from 'rstack/test'; +import { resolveContextWorkspace } from '../../src/context/workspace.ts'; + +const withTempDirectory = async (callback: (rootPath: string) => Promise): Promise => { + const rootPath = await mkdtemp(path.join(os.tmpdir(), 'rstack-context-workspace-')); + + try { + await callback(rootPath); + } finally { + await rm(rootPath, { force: true, recursive: true }); + } +}; + +test('resolves a package from its config path without using process cwd', async () => { + await withTempDirectory(async (workspaceRoot) => { + const packageRoot = path.join(workspaceRoot, 'packages', 'library'); + const configPath = path.join(packageRoot, 'rslib.config.ts'); + await mkdir(path.join(workspaceRoot, '.git')); + await mkdir(packageRoot, { recursive: true }); + await writeFile( + path.join(workspaceRoot, 'pnpm-workspace.yaml'), + "packages:\n - 'packages/*'\n", + ); + await writeFile( + path.join(packageRoot, 'package.json'), + JSON.stringify({ name: '@repo/library' }), + ); + await writeFile(configPath, 'export default {};\n'); + + await expect(resolveContextWorkspace(configPath)).resolves.toEqual({ + workspaceRoot, + packageRoot, + packageName: '@repo/library', + }); + }); +}); + +test('falls back to a standalone package root', async () => { + await withTempDirectory(async (packageRoot) => { + const configPath = path.join(packageRoot, 'rsbuild.config.ts'); + await writeFile(path.join(packageRoot, 'package.json'), JSON.stringify({ name: 'standalone' })); + await writeFile(configPath, 'export default {};\n'); + + await expect(resolveContextWorkspace(configPath)).resolves.toEqual({ + workspaceRoot: packageRoot, + packageRoot, + packageName: 'standalone', + }); + }); +}); + +test('uses the checkout root when a nested package has no workspace manifest', async () => { + await withTempDirectory(async (workspaceRoot) => { + const packageRoot = path.join(workspaceRoot, 'packages', 'library'); + await mkdir(path.join(workspaceRoot, '.git')); + await mkdir(packageRoot, { recursive: true }); + await writeFile(path.join(packageRoot, 'package.json'), JSON.stringify({ name: 'library' })); + + await expect(resolveContextWorkspace(packageRoot)).resolves.toEqual({ + workspaceRoot, + packageRoot, + packageName: 'library', + }); + }); +}); + +test('uses the start directory when no workspace markers exist', async () => { + await withTempDirectory(async (rootPath) => { + const sourceDirectory = path.join(rootPath, 'nested'); + const configPath = path.join(sourceDirectory, 'rspack.config.js'); + await mkdir(sourceDirectory); + await writeFile(configPath, 'export default {};\n'); + + await expect(resolveContextWorkspace(configPath)).resolves.toEqual({ + workspaceRoot: sourceDirectory, + packageRoot: sourceDirectory, + }); + }); +}); From 0dfc187233c8b2f6f4fbcb32e0f28bd2fbe1ae9a Mon Sep 17 00:00:00 2001 From: ScriptedAlchemy Date: Wed, 12 Aug 2026 06:25:22 +0000 Subject: [PATCH 004/110] docs: design passive build context phase --- ...2026-08-12-passive-build-context-design.md | 233 ++++++++++++++++++ 1 file changed, 233 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-12-passive-build-context-design.md diff --git a/docs/superpowers/specs/2026-08-12-passive-build-context-design.md b/docs/superpowers/specs/2026-08-12-passive-build-context-design.md new file mode 100644 index 00000000..9c6b003a --- /dev/null +++ b/docs/superpowers/specs/2026-08-12-passive-build-context-design.md @@ -0,0 +1,233 @@ +# Passive build context design + + + +**Status:** Approved for implementation by the request to design and implement the next RFC phase. + +**Foundation:** `70c4a90 feat(rstack): scaffold context evidence store` + +## Purpose + +Deliver the smallest end-to-end Phase 1 slice that proves Rstack commands can publish useful build +metadata from any package in a standalone repository or monorepo, and that one repository-root MCP +process can read all of it without knowing which package launched each command. + +The slice covers trusted metadata capture for `rs dev`, `rs build`, and `rs lib`, plus one read-only +`rs mcp` status tool. It deliberately excludes deep compiler graphs and background coordination. + +## Chosen approach + +Use the existing CLI-specific Rsbuild and Rslib config loaders as the only automatic injection +points. After the user's config resolves, Rstack shallow-clones it and appends one global +Rstack-owned Rsbuild plugin. The plugin publishes immutable per-environment snapshots through the +existing `.rstack/cache/context-v1` store. + +`rs mcp` is a local stdio process. It resolves the checkout from its launch path, reads completed +records on demand, and exposes a compact `project_status` tool. It does not listen on a port, own +producer processes, index a task graph, or cache authoritative state. + +This is Phase 1A. Phase 1B will add static Rsdoctor ingestion and richer build diagnostics. Phase 1C +will add bounded retention and report links after real artifact sizes and access patterns are +measured. + +## Alternatives considered + +### Implement all of RFC phase 1 at once + +This would combine compiler observation, Rsdoctor schema adaptation, record retention, report +discovery, entity queries, and MCP transport. Those parts have different versioning and failure +modes, making a single review and rollback boundary too large. + +### Implement producers without MCP + +This is smaller, but it would leave the most important architectural claim untested: an agent host +launched once at the repository root can discover data from independently launched package builds. + +### Add a coordinator daemon + +A daemon does not solve an observed Phase 1A problem. Immutable producer-owned files already allow +many writers and readers, survive producer restarts, and require no port or process discovery. + +## Configuration and trust + +Add `define.context` as Rstack-owned configuration, separate from the configuration forwarded to +Rsbuild or Rslib: + +```ts +define.context({ + enabled: true, + capture: 'metadata', +}); +``` + +The supported shape is: + +```ts +type ContextConfig = { + enabled?: boolean; + capture?: 'off' | 'metadata' | 'deep'; +}; +``` + +Rules: + +- capture is disabled unless `enabled` is `true` or `RSTACK_CONTEXT=1` is set; +- `RSTACK_CONTEXT=0` always disables capture; +- `capture: 'off'` disables capture even when enabled; +- omitted `capture` means `metadata`; +- `deep` activates metadata capture but records the deep facet as `unsupported` in Phase 1A; +- config objects and config functions are never mutated; +- `define.context` is never forwarded into an underlying tool config. + +## Producer architecture + +### Injection + +`loadRsbuildConfig` and `loadRslibConfig` retain the existing pure resolvers. Their CLI-only loader +paths perform the following steps: + +1. load the Rstack config and obtain its actual `filePath`; +2. resolve the app or library config with the original `ConfigParams`; +3. evaluate the context activation policy; +4. resolve checkout and package identity from `filePath`, falling back to the actual launch + directory only when no config file exists; +5. shallow-clone the resolved config and append exactly one observer to top-level `plugins`. + +Rslib `lib[]` entries are never modified. A global plugin observes every Rslib-generated Rsbuild +environment without duplicate global callbacks. Rstest's use of the pure app/library resolvers is +not instrumented. + +### Run and context identity + +Each plugin instance owns one run and never uses module-global mutable state. + +- `runId` is a safe, unique `run_