diff --git a/.agents/skills/tool-registry-boundary/SKILL.md b/.agents/skills/tool-registry-boundary/SKILL.md index a1463f1c8fa..6e1caaf0a64 100644 --- a/.agents/skills/tool-registry-boundary/SKILL.md +++ b/.agents/skills/tool-registry-boundary/SKILL.md @@ -68,6 +68,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.agents/skills/v2-api-conventions/SKILL.md b/.agents/skills/v2-api-conventions/SKILL.md new file mode 100644 index 00000000000..3431b558056 --- /dev/null +++ b/.agents/skills/v2-api-conventions/SKILL.md @@ -0,0 +1,219 @@ +--- +name: v2-api-conventions +description: The response, error, pagination, and validation contract every `/api/v2` endpoint must satisfy. Use when adding or changing a route under `apps/sim/app/api/v2/`, or when auditing one for conformance. +argument-hint: +--- + +# v2 API Conventions + +The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** + +``` +success (single) { "data": {...} } +success (collection) { "data": [...], "nextCursor": "..." | null } +failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } +``` + +Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. + +That promise is worth stating as a rule because it has been broken five separate ways, each time by a route or a builder taking a shortcut that looked local: + +- `GET /workflows?limit=1.5` returned **500**. The contract was copied from a sibling and lost its `.int()`, so a fractional limit passed validation and reached Postgres as `LIMIT 2.5`. +- A malformed JSON body returned **`{"error":"Request body must be valid JSON"}`** — a bare string. The envelope was a per-route opt-in that only 8 of 77 routes remembered. +- `GET /api/v2/nonexistent` returned a **full HTML 404 document**, because no route file matched and the request fell through to the app's global not-found page. +- Four collections returned `nextCursor` while **silently discarding** any `limit` the caller sent, because Zod strips unknown keys unless the schema is `.strict()`. +- Handing back a `nextCursor` from any timestamp-sorted list and passing it straight in returned **500**. The value was validated and bound — but bound with no SQL type, into `date_trunc`, which is overloaded, so Postgres could resolve no overload. Validation was never the missing half; the type was. + +Each was one line. The rules below are the generalisations. + +## Where the machinery lives + +| Concern | File | +|---|---| +| Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | +| Rollout gate (`v2-api` flag) | `apps/sim/app/api/v2/lib/gate.ts` | +| Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | +| Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | +| Contracts | `apps/sim/lib/api/contracts/v2/**` | +| Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | + +## Rule 1 — the envelope is produced by helpers, never by hand + +`v2Data`, `v2CursorList`, and `v2Error` in `response.ts` are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. + +A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. + +**The envelope must hold for every failure mode, including the ones that happen before your handler runs.** That is what the four bugs above have in common. Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. + +## Rule 2 — status codes mean specific things + +| Status | `code` | Meaning | +|---|---|---| +| 200 / 201 | — | Success. 201 only for a created resource. | +| 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | +| 401 | `UNAUTHORIZED` | No/!valid API key. **Runs before the rollout gate.** | +| 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Carries a machine-readable `details.code` from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). | +| 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** the rollout gate, **and** an unknown path. | +| 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | +| 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes`. | +| 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | +| 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | + +Two of these carry real design weight: + +**404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The rollout gate answers the same way for the same reason (`gate.ts`: "an ungated caller cannot distinguish 'not in the rollout cohort' from 'no such endpoint'"), and so does the unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` — its body is byte-identical to the gate's on purpose. + +**500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. A caller-reachable 500 has shipped three times — a fractional `limit` reaching `LIMIT 2.5`, a plain `HEAD` tripping the builder's method guard, and a keyset cursor's timestamp reaching `date_trunc` as an untyped placeholder — so treat "a well-formed request produced a 500" as the highest-severity class of defect on this surface. + +**Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). + +And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. + +**Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: + +- An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. +- An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. + +**Every 403 names its cause in `error.details.code`.** One status covers four different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. + +The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from. Adding a member fails to compile until it is documented, so a code cannot reach the wire unpublished. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. + +The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. + +Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, and hand-assembled lists are how three knowledge reads and three upload operations quietly lost it. + +**HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. + +## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them + +Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/__tests__/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. + +Build the query slice from the shared helper, never by hand: + +```ts +...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) +``` + +That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Re-declaring `limit: z.coerce.number()...` inline is how the 500 happened; there is one schema so the family cannot drift again. + +Two cursor schemes exist, both opaque base64-JSON from `response.ts`. Which one you use is decided by what the read can express, not by taste: + +- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort is stamped into the cursor and re-checked on replay, so changing `sortBy` mid-pagination is a 400, not a silently skipped page. +- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. An offset cursor **must** be stamped with `offsetCursorScope(...)` covering every param that filters or orders the sequence (not `limit`, which only selects how much of it to return). A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results — the exact failure the keyset's sort stamp already prevents. + +**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. + +Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. + +**Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Fourteen lists take the pair. Two — `GET /logs` and `GET /workflows/{id}/runs` — have exactly one sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That split is documented in both contracts and is the *only* sanctioned deviation. A new list picks the pair. Do not "fix" the two by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency, and renaming `order` would break every shipped caller. + +**A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`. It coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`, so it is a strict widening of a `z.enum(['true','false'])` — which is what two v2 params used to be, purely by inheritance from the internal shapes they reused. Reusing an internal `.shape.x` inherits the internal spelling; re-declare instead when the internal one is not the v2 convention. + +## Rule 4 — reject what you do not implement + +Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. + +Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. + +Error messages name the field and, where there is one, the escape hatch: + +``` +limit must be a whole number +limit cannot exceed 100 +search cannot be empty +sortBy: expected one of "name" | "createdAt" | "updatedAt" +Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export +``` + +That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. + +## Rule 5 — contract first, then use case, then route + +Order matters because each layer is checked against the one before it. + +1. **Contract** in `lib/api/contracts/v2/.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. +2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives **only the use-case result**, so anything the presenter needs (e.g. the active `sortBy`/`sortOrder` to stamp a cursor) must be returned by the use case. +3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. +4. **OpenAPI description** in `lib/api/contracts/v2/openapi/.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. + +## Rule 6 — a transient failure says when to come back + +A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: + +| Status | Source of the value | Where | +|---|---|---| +| 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | +| 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | + +The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. + +Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. + +**Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. That value used to be dropped when the descriptor was mapped onto a preprocess error, so a concurrency denial arrived as a bare 429 with no `Retry-After` even though the policy had named the wait. It now travels `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. + +**A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. + +RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; Sim's 413 is a fixed byte ceiling, so it correctly sends none. + +## Deliberate non-adoptions + +Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. + +| Practice | Verdict | Why | +|---|---|---| +| **RFC 9457 `application/problem+json`** | No | 9457 §4 steers APIs with an existing format toward keeping it: "Problem details are intended to avoid the necessity of establishing new 'fault' or 'error' document formats, **not to replace existing domain-specific formats**." Nothing in it is a `MUST` to adopt, and none of Stripe, GitHub, or Google use it. Our envelope is load-bearing for every client. **The default error shape does not change.** | +| **`RateLimit`/`RateLimit-Policy` (IETF draft)** | No | Still an unpublished draft (`-11`, May 2026), returned "Not ready" at HTTPDIR review, and on its **third mutually incompatible wire format** — anything built against `-07` or earlier is already broken. None of the three surveyed APIs emit it; GitHub uses `x-ratelimit-*`, as we do. | +| **Renaming `X-RateLimit-*` per RFC 6648** | No | 6648 is a `SHOULD NOT` binding *creators of new* parameters, and §1 item 4 "**makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated**". Appendix B argues the migration is itself the interoperability harm. A rename is a client-visible break bought with nothing. | +| **`X-RateLimit-Reset` as delta-seconds** | No | It is an absolute ISO 8601 timestamp, so it is clock-skew sensitive — but the response where timing actually decides behaviour (429) also carries `Retry-After`, which is skew-free. The absolute value stays useful for scheduling. | +| **422 for semantic validation** | No | RFC 9110 §15.5.21 defines 422, but Appendix B.3 records that 9110 **deleted** RFC 4918's clause saying 400 was inappropriate. 400 covers "cannot or will not process… perceived to be a client error". The split is convention, not requirement — GitHub splits, Stripe and Google do not. Our machine-readable `error.code` already carries the distinction, and restatusing now breaks clients. | +| **`Location` on 201** | No | §9.3.3 makes this a `SHOULD` **for POST**; the status code itself (§15.3.2) requires nothing and defines the fallback — absent `Location`, the target URI identifies the resource. Declined knowingly: several 201 responses (signed upload sessions, table exports, knowledge folders) have no canonical single-resource GET, so a `Location` would 404, and adopting it on some of the 19 is worse for a client than on none. Every 201 returns the full representation including its `id`. Revisit per-route if one gains a canonical GET. | +| **ETag / `If-None-Match` / `If-Match`** | No | Every v2 response is `Cache-Control: private, no-store` per-caller data, so `If-None-Match` buys nothing. For writes, `If-Match` needs a **strong** validator: §8.8.3.2's strong comparison fails if *either* tag is weak, so a weak ETag silently makes every `If-Match` fail. None of the three surveyed APIs does HTTP optimistic concurrency — Google does the semantics via a resource `etag` **field** (AIP-154), deliberately not the header. If Sim needs optimistic concurrency, do it that way. | +| **`Deprecation` / `Sunset` on v1** | Not yet | RFC 9745 (Standards Track) and RFC 8594 (Informational) both apply, and GitHub emits both. But `Sunset` is a timestamp and 9745 §4 makes `Sunset >= Deprecation` a `MUST`, so emitting either commits Sim to a v1 retirement date — a product decision, not an engineering one. When that date exists: `Deprecation` is an RFC 9651 Structured Field **Date** (`@1688169599`); `Sunset` is an **HTTP-date** (`Sat, 31 Dec 2033 23:59:59 GMT`). Two encodings in one response — the most common implementation error here. | +| **`application/merge-patch+json`** | No | v2 PATCH bodies are merge-patch *shaped* — absent means unchanged, `null` clears — but they are `.strict()`, so unknown members are rejected where RFC 7396 §2 would merge them, and nested objects are replaced wholesale rather than merged. Advertising the media type would over-claim. Document the semantics per contract instead. | + +## Idempotency: at-most-once, not replay + +`POST /workflows/{id}/execute` accepts `X-Run-Id`, a caller-supplied run identifier claimed through the `idempotency_key` table (`execution-id-claim.ts`). It is a **uniqueness claim, not an idempotency key**, and the distinction is deliberate and already published in the operation description: + +- First use wins and runs. +- Any reuse returns **409** with `error.details.code: "RUN_ID_CONFLICT"`, the run id in `error.details.runId`, and an `X-Run-Id` response header. It never replays the earlier run's result — the client recovers it by polling the runs resource. +- Claims are durable tombstones, so deleting execution logs cannot make an id reusable. + +That makes the money path safe against double-execution **for callers that opt in**. What it is not: a Stripe-style `Idempotency-Key` that stores and replays the original status and body. Building that means a request fingerprint, a retention window, an in-flight-vs-completed distinction (the expired IETF draft would have these be 422 and 409 respectively), and somewhere to put a large synchronous execution body. It is a designed piece of work, not an increment — do not half-build it by aliasing the header name, which would invite clients written against Stripe semantics to treat our 409 as a hard failure. + +## Cursors are opaque, not trusted + +The base64-JSON cursor is **not signed**, and does not need to be. Tampering is bounded by construction, and that is a property to preserve: + +- Every key value is re-validated by its `KeysetKey.bind`, which returns `null` for a wrong-typed or unparseable value and becomes a 400. A forged cursor cannot reach SQL as `NaN` or an `Invalid Date`. +- The sort is stamped into the cursor and re-checked (`decodeSortedCursor`), so a cursor from a differently-sorted query is a 400, not a silently skipped page. +- The offset codec rejects anything that is not a non-negative integer. +- Authorization is **never** carried in the cursor. Every list re-derives its workspace scope from the authenticated principal, so a cursor lifted from another query — or another tenant — can only move the caller within their own authorized result set. + +The consequence to keep true: **never put a resource id, filter, or scope into a cursor and then trust it on the way back.** A cursor is a position hint, never an input to an access decision. + +## Checklist + +Run this against any new or changed v2 endpoint. + +- [ ] Success body is exactly `{data}` or `{data, nextCursor}`; failures are exactly `{error:{code,message,details?}}`. +- [ ] Route uses a shared builder; no hand-built `NextResponse.json`. +- [ ] Query and body schemas are `.strict()`. +- [ ] No caller-supplied value can produce a 500 — check every numeric param reaches SQL as a validated integer, and that any bound value passed as an argument to a SQL function carries an explicit type. +- [ ] `limit` comes from `v2PaginationFields`, not a hand-written `z.coerce.number()`. +- [ ] If the response carries `nextCursor`, the query accepts `limit` + `cursor` and the query actually applies them. +- [ ] Keyset sorts end in a unique `id` key. +- [ ] The list is classified in `list-pagination.test.ts`. +- [ ] Cross-tenant access answers 404, never 403 — and carries `Cache-Control: private, no-store`, because RFC 9110 §15.5.5 makes 404 heuristically cacheable and an authorization-dependent 404 must never be stored. `v2Error` sets this unconditionally; do not build a v2 response any other way. +- [ ] A retryable failure says when: 429 and 503 carry `Retry-After`. No other status invents one. +- [ ] 403s carry a machine-readable `details.code` from `FORBIDDEN_DETAIL_CODES`, thrown as `ForbiddenOperationError` in the domain rather than attached at the route. +- [ ] Nested objects inside a `.strict()` body are strict too — `.strict()` does not recurse. +- [ ] Ordering uses `sortBy` + `sortOrder`; boolean query params use `booleanQueryFlagSchema`. +- [ ] Validation messages name the field and echo the valid set. +- [ ] Response schema matches every field the route actually emits. +- [ ] OpenAPI description regenerated and truthful about pagination. +- [ ] `bun run type-check`, `bun run check:api-validation`, `bun run check:openapi` pass. + +## Known gap + +A 405 on a path that *does* have a route file but does not export that verb is generated by Next.js before any Sim code runs: zero-byte body, no `content-type`, and no `Allow` header, which RFC 9110 §15.5.6 requires. Fixing it means either exporting explicit rejecting handlers from every v2 route file or intercepting in `apps/sim/proxy.ts` with a static path→methods table. Neither is done. Unknown *paths* are handled — the catch-all covers those. diff --git a/.claude/commands/tool-registry-boundary.md b/.claude/commands/tool-registry-boundary.md index 676fa256cbc..da6758efe3e 100644 --- a/.claude/commands/tool-registry-boundary.md +++ b/.claude/commands/tool-registry-boundary.md @@ -67,6 +67,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.claude/commands/v2-api-conventions.md b/.claude/commands/v2-api-conventions.md new file mode 100644 index 00000000000..89095067c66 --- /dev/null +++ b/.claude/commands/v2-api-conventions.md @@ -0,0 +1,218 @@ +--- +description: The response, error, pagination, and validation contract every `/api/v2` endpoint must satisfy. Use when adding or changing a route under `apps/sim/app/api/v2/`, or when auditing one for conformance. +argument-hint: +--- + +# v2 API Conventions + +The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** + +``` +success (single) { "data": {...} } +success (collection) { "data": [...], "nextCursor": "..." | null } +failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } +``` + +Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. + +That promise is worth stating as a rule because it has been broken five separate ways, each time by a route or a builder taking a shortcut that looked local: + +- `GET /workflows?limit=1.5` returned **500**. The contract was copied from a sibling and lost its `.int()`, so a fractional limit passed validation and reached Postgres as `LIMIT 2.5`. +- A malformed JSON body returned **`{"error":"Request body must be valid JSON"}`** — a bare string. The envelope was a per-route opt-in that only 8 of 77 routes remembered. +- `GET /api/v2/nonexistent` returned a **full HTML 404 document**, because no route file matched and the request fell through to the app's global not-found page. +- Four collections returned `nextCursor` while **silently discarding** any `limit` the caller sent, because Zod strips unknown keys unless the schema is `.strict()`. +- Handing back a `nextCursor` from any timestamp-sorted list and passing it straight in returned **500**. The value was validated and bound — but bound with no SQL type, into `date_trunc`, which is overloaded, so Postgres could resolve no overload. Validation was never the missing half; the type was. + +Each was one line. The rules below are the generalisations. + +## Where the machinery lives + +| Concern | File | +|---|---| +| Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | +| Rollout gate (`v2-api` flag) | `apps/sim/app/api/v2/lib/gate.ts` | +| Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | +| Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | +| Contracts | `apps/sim/lib/api/contracts/v2/**` | +| Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | + +## Rule 1 — the envelope is produced by helpers, never by hand + +`v2Data`, `v2CursorList`, and `v2Error` in `response.ts` are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. + +A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. + +**The envelope must hold for every failure mode, including the ones that happen before your handler runs.** That is what the four bugs above have in common. Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. + +## Rule 2 — status codes mean specific things + +| Status | `code` | Meaning | +|---|---|---| +| 200 / 201 | — | Success. 201 only for a created resource. | +| 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | +| 401 | `UNAUTHORIZED` | No/!valid API key. **Runs before the rollout gate.** | +| 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Carries a machine-readable `details.code` from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). | +| 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** the rollout gate, **and** an unknown path. | +| 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | +| 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes`. | +| 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | +| 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | + +Two of these carry real design weight: + +**404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The rollout gate answers the same way for the same reason (`gate.ts`: "an ungated caller cannot distinguish 'not in the rollout cohort' from 'no such endpoint'"), and so does the unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` — its body is byte-identical to the gate's on purpose. + +**500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. A caller-reachable 500 has shipped three times — a fractional `limit` reaching `LIMIT 2.5`, a plain `HEAD` tripping the builder's method guard, and a keyset cursor's timestamp reaching `date_trunc` as an untyped placeholder — so treat "a well-formed request produced a 500" as the highest-severity class of defect on this surface. + +**Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). + +And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. + +**Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: + +- An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. +- An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. + +**Every 403 names its cause in `error.details.code`.** One status covers four different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. + +The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from. Adding a member fails to compile until it is documented, so a code cannot reach the wire unpublished. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. + +The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. + +Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, and hand-assembled lists are how three knowledge reads and three upload operations quietly lost it. + +**HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. + +## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them + +Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/__tests__/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. + +Build the query slice from the shared helper, never by hand: + +```ts +...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) +``` + +That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Re-declaring `limit: z.coerce.number()...` inline is how the 500 happened; there is one schema so the family cannot drift again. + +Two cursor schemes exist, both opaque base64-JSON from `response.ts`. Which one you use is decided by what the read can express, not by taste: + +- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort is stamped into the cursor and re-checked on replay, so changing `sortBy` mid-pagination is a 400, not a silently skipped page. +- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. An offset cursor **must** be stamped with `offsetCursorScope(...)` covering every param that filters or orders the sequence (not `limit`, which only selects how much of it to return). A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results — the exact failure the keyset's sort stamp already prevents. + +**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. + +Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. + +**Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Fourteen lists take the pair. Two — `GET /logs` and `GET /workflows/{id}/runs` — have exactly one sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That split is documented in both contracts and is the *only* sanctioned deviation. A new list picks the pair. Do not "fix" the two by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency, and renaming `order` would break every shipped caller. + +**A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`. It coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`, so it is a strict widening of a `z.enum(['true','false'])` — which is what two v2 params used to be, purely by inheritance from the internal shapes they reused. Reusing an internal `.shape.x` inherits the internal spelling; re-declare instead when the internal one is not the v2 convention. + +## Rule 4 — reject what you do not implement + +Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. + +Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. + +Error messages name the field and, where there is one, the escape hatch: + +``` +limit must be a whole number +limit cannot exceed 100 +search cannot be empty +sortBy: expected one of "name" | "createdAt" | "updatedAt" +Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export +``` + +That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. + +## Rule 5 — contract first, then use case, then route + +Order matters because each layer is checked against the one before it. + +1. **Contract** in `lib/api/contracts/v2/.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. +2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives **only the use-case result**, so anything the presenter needs (e.g. the active `sortBy`/`sortOrder` to stamp a cursor) must be returned by the use case. +3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. +4. **OpenAPI description** in `lib/api/contracts/v2/openapi/.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. + +## Rule 6 — a transient failure says when to come back + +A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: + +| Status | Source of the value | Where | +|---|---|---| +| 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | +| 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | + +The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. + +Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. + +**Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. That value used to be dropped when the descriptor was mapped onto a preprocess error, so a concurrency denial arrived as a bare 429 with no `Retry-After` even though the policy had named the wait. It now travels `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. + +**A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. + +RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; Sim's 413 is a fixed byte ceiling, so it correctly sends none. + +## Deliberate non-adoptions + +Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. + +| Practice | Verdict | Why | +|---|---|---| +| **RFC 9457 `application/problem+json`** | No | 9457 §4 steers APIs with an existing format toward keeping it: "Problem details are intended to avoid the necessity of establishing new 'fault' or 'error' document formats, **not to replace existing domain-specific formats**." Nothing in it is a `MUST` to adopt, and none of Stripe, GitHub, or Google use it. Our envelope is load-bearing for every client. **The default error shape does not change.** | +| **`RateLimit`/`RateLimit-Policy` (IETF draft)** | No | Still an unpublished draft (`-11`, May 2026), returned "Not ready" at HTTPDIR review, and on its **third mutually incompatible wire format** — anything built against `-07` or earlier is already broken. None of the three surveyed APIs emit it; GitHub uses `x-ratelimit-*`, as we do. | +| **Renaming `X-RateLimit-*` per RFC 6648** | No | 6648 is a `SHOULD NOT` binding *creators of new* parameters, and §1 item 4 "**makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated**". Appendix B argues the migration is itself the interoperability harm. A rename is a client-visible break bought with nothing. | +| **`X-RateLimit-Reset` as delta-seconds** | No | It is an absolute ISO 8601 timestamp, so it is clock-skew sensitive — but the response where timing actually decides behaviour (429) also carries `Retry-After`, which is skew-free. The absolute value stays useful for scheduling. | +| **422 for semantic validation** | No | RFC 9110 §15.5.21 defines 422, but Appendix B.3 records that 9110 **deleted** RFC 4918's clause saying 400 was inappropriate. 400 covers "cannot or will not process… perceived to be a client error". The split is convention, not requirement — GitHub splits, Stripe and Google do not. Our machine-readable `error.code` already carries the distinction, and restatusing now breaks clients. | +| **`Location` on 201** | No | §9.3.3 makes this a `SHOULD` **for POST**; the status code itself (§15.3.2) requires nothing and defines the fallback — absent `Location`, the target URI identifies the resource. Declined knowingly: several 201 responses (signed upload sessions, table exports, knowledge folders) have no canonical single-resource GET, so a `Location` would 404, and adopting it on some of the 19 is worse for a client than on none. Every 201 returns the full representation including its `id`. Revisit per-route if one gains a canonical GET. | +| **ETag / `If-None-Match` / `If-Match`** | No | Every v2 response is `Cache-Control: private, no-store` per-caller data, so `If-None-Match` buys nothing. For writes, `If-Match` needs a **strong** validator: §8.8.3.2's strong comparison fails if *either* tag is weak, so a weak ETag silently makes every `If-Match` fail. None of the three surveyed APIs does HTTP optimistic concurrency — Google does the semantics via a resource `etag` **field** (AIP-154), deliberately not the header. If Sim needs optimistic concurrency, do it that way. | +| **`Deprecation` / `Sunset` on v1** | Not yet | RFC 9745 (Standards Track) and RFC 8594 (Informational) both apply, and GitHub emits both. But `Sunset` is a timestamp and 9745 §4 makes `Sunset >= Deprecation` a `MUST`, so emitting either commits Sim to a v1 retirement date — a product decision, not an engineering one. When that date exists: `Deprecation` is an RFC 9651 Structured Field **Date** (`@1688169599`); `Sunset` is an **HTTP-date** (`Sat, 31 Dec 2033 23:59:59 GMT`). Two encodings in one response — the most common implementation error here. | +| **`application/merge-patch+json`** | No | v2 PATCH bodies are merge-patch *shaped* — absent means unchanged, `null` clears — but they are `.strict()`, so unknown members are rejected where RFC 7396 §2 would merge them, and nested objects are replaced wholesale rather than merged. Advertising the media type would over-claim. Document the semantics per contract instead. | + +## Idempotency: at-most-once, not replay + +`POST /workflows/{id}/execute` accepts `X-Run-Id`, a caller-supplied run identifier claimed through the `idempotency_key` table (`execution-id-claim.ts`). It is a **uniqueness claim, not an idempotency key**, and the distinction is deliberate and already published in the operation description: + +- First use wins and runs. +- Any reuse returns **409** with `error.details.code: "RUN_ID_CONFLICT"`, the run id in `error.details.runId`, and an `X-Run-Id` response header. It never replays the earlier run's result — the client recovers it by polling the runs resource. +- Claims are durable tombstones, so deleting execution logs cannot make an id reusable. + +That makes the money path safe against double-execution **for callers that opt in**. What it is not: a Stripe-style `Idempotency-Key` that stores and replays the original status and body. Building that means a request fingerprint, a retention window, an in-flight-vs-completed distinction (the expired IETF draft would have these be 422 and 409 respectively), and somewhere to put a large synchronous execution body. It is a designed piece of work, not an increment — do not half-build it by aliasing the header name, which would invite clients written against Stripe semantics to treat our 409 as a hard failure. + +## Cursors are opaque, not trusted + +The base64-JSON cursor is **not signed**, and does not need to be. Tampering is bounded by construction, and that is a property to preserve: + +- Every key value is re-validated by its `KeysetKey.bind`, which returns `null` for a wrong-typed or unparseable value and becomes a 400. A forged cursor cannot reach SQL as `NaN` or an `Invalid Date`. +- The sort is stamped into the cursor and re-checked (`decodeSortedCursor`), so a cursor from a differently-sorted query is a 400, not a silently skipped page. +- The offset codec rejects anything that is not a non-negative integer. +- Authorization is **never** carried in the cursor. Every list re-derives its workspace scope from the authenticated principal, so a cursor lifted from another query — or another tenant — can only move the caller within their own authorized result set. + +The consequence to keep true: **never put a resource id, filter, or scope into a cursor and then trust it on the way back.** A cursor is a position hint, never an input to an access decision. + +## Checklist + +Run this against any new or changed v2 endpoint. + +- [ ] Success body is exactly `{data}` or `{data, nextCursor}`; failures are exactly `{error:{code,message,details?}}`. +- [ ] Route uses a shared builder; no hand-built `NextResponse.json`. +- [ ] Query and body schemas are `.strict()`. +- [ ] No caller-supplied value can produce a 500 — check every numeric param reaches SQL as a validated integer, and that any bound value passed as an argument to a SQL function carries an explicit type. +- [ ] `limit` comes from `v2PaginationFields`, not a hand-written `z.coerce.number()`. +- [ ] If the response carries `nextCursor`, the query accepts `limit` + `cursor` and the query actually applies them. +- [ ] Keyset sorts end in a unique `id` key. +- [ ] The list is classified in `list-pagination.test.ts`. +- [ ] Cross-tenant access answers 404, never 403 — and carries `Cache-Control: private, no-store`, because RFC 9110 §15.5.5 makes 404 heuristically cacheable and an authorization-dependent 404 must never be stored. `v2Error` sets this unconditionally; do not build a v2 response any other way. +- [ ] A retryable failure says when: 429 and 503 carry `Retry-After`. No other status invents one. +- [ ] 403s carry a machine-readable `details.code` from `FORBIDDEN_DETAIL_CODES`, thrown as `ForbiddenOperationError` in the domain rather than attached at the route. +- [ ] Nested objects inside a `.strict()` body are strict too — `.strict()` does not recurse. +- [ ] Ordering uses `sortBy` + `sortOrder`; boolean query params use `booleanQueryFlagSchema`. +- [ ] Validation messages name the field and echo the valid set. +- [ ] Response schema matches every field the route actually emits. +- [ ] OpenAPI description regenerated and truthful about pagination. +- [ ] `bun run type-check`, `bun run check:api-validation`, `bun run check:openapi` pass. + +## Known gap + +A 405 on a path that *does* have a route file but does not export that verb is generated by Next.js before any Sim code runs: zero-byte body, no `content-type`, and no `Allow` header, which RFC 9110 §15.5.6 requires. Fixing it means either exporting explicit rejecting handlers from every v2 route file or intercepting in `apps/sim/proxy.ts` with a static path→methods table. Neither is done. Unknown *paths* are handled — the catch-all covers those. diff --git a/.claude/rules/sim-queries.md b/.claude/rules/sim-queries.md index 14acca8f2cb..a71ee448dde 100644 --- a/.claude/rules/sim-queries.md +++ b/.claude/rules/sim-queries.md @@ -143,6 +143,22 @@ const handler = useCallback(() => { }, [data]) ``` +## Server prefetching + +A server prefetch fills the *same* cache key a client hook fills, so it must be indistinguishable from a client fetch. Five rules: + +1. **Read the data layer, never our own API over HTTP.** A server-to-server call to `/api/...` costs a round trip and a second authentication for data the process can already read. Where the route runs an application use case, call that same use case with a principal from the same auth policy the route declares — not a manager underneath it. +2. **Match the wire shape the hook caches.** The hook's data is whatever `requestJson(contract, …)` produced, so the seed must equal it. Two traps: a contract field declared `z.coerce.date()` means the hook holds a `Date` where raw route JSON holds a string; a passthrough response schema (`z.custom`) means the hook caches route JSON *verbatim*, so seeding raw rows leaks `Date`s and server-only fields. When the route projects before responding, share that projection — have the route and the prefetch call one function. +3. **Prove the viewer.** Data-layer reads carry no authorization; the route used to provide it. Resolve the viewer (`getWorkspaceHostContextForViewer`, already `cache`d by the layout so it costs nothing) and return early on failure, caching nothing — the client fetch then reaches the route for the real 403. Never widen what a viewer can see. +4. **Always `await`.** Only a settled query is dehydrated, so an unawaited prefetch is silently dropped from the payload and the pane waterfalls anyway. +5. **Don't repeat what the layout already seeded.** `getQueryClient()` builds a new client per server call, so a page re-seeding a layout key is a genuine second read — and `HydrationBoundary` defers an already-seen query to an effect, which SSR never runs, so it never reaches the server render either. + +Reuse the hook's exported `staleTime` constant and its key factory; `dehydrate` carries neither options nor `staleTime`, and freshness is per-observer. + +Seed with `setQueryData` only when the prefetch must be able to *decline* to create an entry (an empty list that has to fall through to a route's creation path). `prefetchQuery` and `ensureQueryData` always create one. + +Keep prefetch imports light. A page prefetch's imports land in that route's server graph, so pulling a barrel to reach one function can drag thousands of modules behind it — `bun run check:tool-registry-boundary` gates this per page. + ## Boundary Types - Hooks import named type aliases from `@/lib/api/contracts/**` (e.g., `import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'`). Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code. diff --git a/.cursor/commands/tool-registry-boundary.md b/.cursor/commands/tool-registry-boundary.md index d560d4f40e5..62fb47f53fd 100644 --- a/.cursor/commands/tool-registry-boundary.md +++ b/.cursor/commands/tool-registry-boundary.md @@ -63,6 +63,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.cursor/commands/v2-api-conventions.md b/.cursor/commands/v2-api-conventions.md new file mode 100644 index 00000000000..7fa3e1a18ae --- /dev/null +++ b/.cursor/commands/v2-api-conventions.md @@ -0,0 +1,213 @@ +# v2 API Conventions + +The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** + +``` +success (single) { "data": {...} } +success (collection) { "data": [...], "nextCursor": "..." | null } +failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } +``` + +Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. + +That promise is worth stating as a rule because it has been broken five separate ways, each time by a route or a builder taking a shortcut that looked local: + +- `GET /workflows?limit=1.5` returned **500**. The contract was copied from a sibling and lost its `.int()`, so a fractional limit passed validation and reached Postgres as `LIMIT 2.5`. +- A malformed JSON body returned **`{"error":"Request body must be valid JSON"}`** — a bare string. The envelope was a per-route opt-in that only 8 of 77 routes remembered. +- `GET /api/v2/nonexistent` returned a **full HTML 404 document**, because no route file matched and the request fell through to the app's global not-found page. +- Four collections returned `nextCursor` while **silently discarding** any `limit` the caller sent, because Zod strips unknown keys unless the schema is `.strict()`. +- Handing back a `nextCursor` from any timestamp-sorted list and passing it straight in returned **500**. The value was validated and bound — but bound with no SQL type, into `date_trunc`, which is overloaded, so Postgres could resolve no overload. Validation was never the missing half; the type was. + +Each was one line. The rules below are the generalisations. + +## Where the machinery lives + +| Concern | File | +|---|---| +| Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | +| Rollout gate (`v2-api` flag) | `apps/sim/app/api/v2/lib/gate.ts` | +| Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | +| Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | +| Contracts | `apps/sim/lib/api/contracts/v2/**` | +| Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | + +## Rule 1 — the envelope is produced by helpers, never by hand + +`v2Data`, `v2CursorList`, and `v2Error` in `response.ts` are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. + +A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. + +**The envelope must hold for every failure mode, including the ones that happen before your handler runs.** That is what the four bugs above have in common. Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. + +## Rule 2 — status codes mean specific things + +| Status | `code` | Meaning | +|---|---|---| +| 200 / 201 | — | Success. 201 only for a created resource. | +| 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | +| 401 | `UNAUTHORIZED` | No/!valid API key. **Runs before the rollout gate.** | +| 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Carries a machine-readable `details.code` from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). | +| 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** the rollout gate, **and** an unknown path. | +| 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | +| 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes`. | +| 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | +| 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | + +Two of these carry real design weight: + +**404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The rollout gate answers the same way for the same reason (`gate.ts`: "an ungated caller cannot distinguish 'not in the rollout cohort' from 'no such endpoint'"), and so does the unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` — its body is byte-identical to the gate's on purpose. + +**500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. A caller-reachable 500 has shipped three times — a fractional `limit` reaching `LIMIT 2.5`, a plain `HEAD` tripping the builder's method guard, and a keyset cursor's timestamp reaching `date_trunc` as an untyped placeholder — so treat "a well-formed request produced a 500" as the highest-severity class of defect on this surface. + +**Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). + +And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. + +**Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: + +- An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. +- An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. + +**Every 403 names its cause in `error.details.code`.** One status covers four different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. + +The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from. Adding a member fails to compile until it is documented, so a code cannot reach the wire unpublished. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. + +The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. + +Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, and hand-assembled lists are how three knowledge reads and three upload operations quietly lost it. + +**HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. + +## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them + +Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/__tests__/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. + +Build the query slice from the shared helper, never by hand: + +```ts +...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) +``` + +That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Re-declaring `limit: z.coerce.number()...` inline is how the 500 happened; there is one schema so the family cannot drift again. + +Two cursor schemes exist, both opaque base64-JSON from `response.ts`. Which one you use is decided by what the read can express, not by taste: + +- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort is stamped into the cursor and re-checked on replay, so changing `sortBy` mid-pagination is a 400, not a silently skipped page. +- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. An offset cursor **must** be stamped with `offsetCursorScope(...)` covering every param that filters or orders the sequence (not `limit`, which only selects how much of it to return). A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results — the exact failure the keyset's sort stamp already prevents. + +**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. + +Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. + +**Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Fourteen lists take the pair. Two — `GET /logs` and `GET /workflows/{id}/runs` — have exactly one sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That split is documented in both contracts and is the *only* sanctioned deviation. A new list picks the pair. Do not "fix" the two by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency, and renaming `order` would break every shipped caller. + +**A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`. It coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`, so it is a strict widening of a `z.enum(['true','false'])` — which is what two v2 params used to be, purely by inheritance from the internal shapes they reused. Reusing an internal `.shape.x` inherits the internal spelling; re-declare instead when the internal one is not the v2 convention. + +## Rule 4 — reject what you do not implement + +Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. + +Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. + +Error messages name the field and, where there is one, the escape hatch: + +``` +limit must be a whole number +limit cannot exceed 100 +search cannot be empty +sortBy: expected one of "name" | "createdAt" | "updatedAt" +Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export +``` + +That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. + +## Rule 5 — contract first, then use case, then route + +Order matters because each layer is checked against the one before it. + +1. **Contract** in `lib/api/contracts/v2/.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. +2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives **only the use-case result**, so anything the presenter needs (e.g. the active `sortBy`/`sortOrder` to stamp a cursor) must be returned by the use case. +3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. +4. **OpenAPI description** in `lib/api/contracts/v2/openapi/.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. + +## Rule 6 — a transient failure says when to come back + +A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: + +| Status | Source of the value | Where | +|---|---|---| +| 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | +| 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | + +The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. + +Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. + +**Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. That value used to be dropped when the descriptor was mapped onto a preprocess error, so a concurrency denial arrived as a bare 429 with no `Retry-After` even though the policy had named the wait. It now travels `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. + +**A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. + +RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; Sim's 413 is a fixed byte ceiling, so it correctly sends none. + +## Deliberate non-adoptions + +Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. + +| Practice | Verdict | Why | +|---|---|---| +| **RFC 9457 `application/problem+json`** | No | 9457 §4 steers APIs with an existing format toward keeping it: "Problem details are intended to avoid the necessity of establishing new 'fault' or 'error' document formats, **not to replace existing domain-specific formats**." Nothing in it is a `MUST` to adopt, and none of Stripe, GitHub, or Google use it. Our envelope is load-bearing for every client. **The default error shape does not change.** | +| **`RateLimit`/`RateLimit-Policy` (IETF draft)** | No | Still an unpublished draft (`-11`, May 2026), returned "Not ready" at HTTPDIR review, and on its **third mutually incompatible wire format** — anything built against `-07` or earlier is already broken. None of the three surveyed APIs emit it; GitHub uses `x-ratelimit-*`, as we do. | +| **Renaming `X-RateLimit-*` per RFC 6648** | No | 6648 is a `SHOULD NOT` binding *creators of new* parameters, and §1 item 4 "**makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated**". Appendix B argues the migration is itself the interoperability harm. A rename is a client-visible break bought with nothing. | +| **`X-RateLimit-Reset` as delta-seconds** | No | It is an absolute ISO 8601 timestamp, so it is clock-skew sensitive — but the response where timing actually decides behaviour (429) also carries `Retry-After`, which is skew-free. The absolute value stays useful for scheduling. | +| **422 for semantic validation** | No | RFC 9110 §15.5.21 defines 422, but Appendix B.3 records that 9110 **deleted** RFC 4918's clause saying 400 was inappropriate. 400 covers "cannot or will not process… perceived to be a client error". The split is convention, not requirement — GitHub splits, Stripe and Google do not. Our machine-readable `error.code` already carries the distinction, and restatusing now breaks clients. | +| **`Location` on 201** | No | §9.3.3 makes this a `SHOULD` **for POST**; the status code itself (§15.3.2) requires nothing and defines the fallback — absent `Location`, the target URI identifies the resource. Declined knowingly: several 201 responses (signed upload sessions, table exports, knowledge folders) have no canonical single-resource GET, so a `Location` would 404, and adopting it on some of the 19 is worse for a client than on none. Every 201 returns the full representation including its `id`. Revisit per-route if one gains a canonical GET. | +| **ETag / `If-None-Match` / `If-Match`** | No | Every v2 response is `Cache-Control: private, no-store` per-caller data, so `If-None-Match` buys nothing. For writes, `If-Match` needs a **strong** validator: §8.8.3.2's strong comparison fails if *either* tag is weak, so a weak ETag silently makes every `If-Match` fail. None of the three surveyed APIs does HTTP optimistic concurrency — Google does the semantics via a resource `etag` **field** (AIP-154), deliberately not the header. If Sim needs optimistic concurrency, do it that way. | +| **`Deprecation` / `Sunset` on v1** | Not yet | RFC 9745 (Standards Track) and RFC 8594 (Informational) both apply, and GitHub emits both. But `Sunset` is a timestamp and 9745 §4 makes `Sunset >= Deprecation` a `MUST`, so emitting either commits Sim to a v1 retirement date — a product decision, not an engineering one. When that date exists: `Deprecation` is an RFC 9651 Structured Field **Date** (`@1688169599`); `Sunset` is an **HTTP-date** (`Sat, 31 Dec 2033 23:59:59 GMT`). Two encodings in one response — the most common implementation error here. | +| **`application/merge-patch+json`** | No | v2 PATCH bodies are merge-patch *shaped* — absent means unchanged, `null` clears — but they are `.strict()`, so unknown members are rejected where RFC 7396 §2 would merge them, and nested objects are replaced wholesale rather than merged. Advertising the media type would over-claim. Document the semantics per contract instead. | + +## Idempotency: at-most-once, not replay + +`POST /workflows/{id}/execute` accepts `X-Run-Id`, a caller-supplied run identifier claimed through the `idempotency_key` table (`execution-id-claim.ts`). It is a **uniqueness claim, not an idempotency key**, and the distinction is deliberate and already published in the operation description: + +- First use wins and runs. +- Any reuse returns **409** with `error.details.code: "RUN_ID_CONFLICT"`, the run id in `error.details.runId`, and an `X-Run-Id` response header. It never replays the earlier run's result — the client recovers it by polling the runs resource. +- Claims are durable tombstones, so deleting execution logs cannot make an id reusable. + +That makes the money path safe against double-execution **for callers that opt in**. What it is not: a Stripe-style `Idempotency-Key` that stores and replays the original status and body. Building that means a request fingerprint, a retention window, an in-flight-vs-completed distinction (the expired IETF draft would have these be 422 and 409 respectively), and somewhere to put a large synchronous execution body. It is a designed piece of work, not an increment — do not half-build it by aliasing the header name, which would invite clients written against Stripe semantics to treat our 409 as a hard failure. + +## Cursors are opaque, not trusted + +The base64-JSON cursor is **not signed**, and does not need to be. Tampering is bounded by construction, and that is a property to preserve: + +- Every key value is re-validated by its `KeysetKey.bind`, which returns `null` for a wrong-typed or unparseable value and becomes a 400. A forged cursor cannot reach SQL as `NaN` or an `Invalid Date`. +- The sort is stamped into the cursor and re-checked (`decodeSortedCursor`), so a cursor from a differently-sorted query is a 400, not a silently skipped page. +- The offset codec rejects anything that is not a non-negative integer. +- Authorization is **never** carried in the cursor. Every list re-derives its workspace scope from the authenticated principal, so a cursor lifted from another query — or another tenant — can only move the caller within their own authorized result set. + +The consequence to keep true: **never put a resource id, filter, or scope into a cursor and then trust it on the way back.** A cursor is a position hint, never an input to an access decision. + +## Checklist + +Run this against any new or changed v2 endpoint. + +- [ ] Success body is exactly `{data}` or `{data, nextCursor}`; failures are exactly `{error:{code,message,details?}}`. +- [ ] Route uses a shared builder; no hand-built `NextResponse.json`. +- [ ] Query and body schemas are `.strict()`. +- [ ] No caller-supplied value can produce a 500 — check every numeric param reaches SQL as a validated integer, and that any bound value passed as an argument to a SQL function carries an explicit type. +- [ ] `limit` comes from `v2PaginationFields`, not a hand-written `z.coerce.number()`. +- [ ] If the response carries `nextCursor`, the query accepts `limit` + `cursor` and the query actually applies them. +- [ ] Keyset sorts end in a unique `id` key. +- [ ] The list is classified in `list-pagination.test.ts`. +- [ ] Cross-tenant access answers 404, never 403 — and carries `Cache-Control: private, no-store`, because RFC 9110 §15.5.5 makes 404 heuristically cacheable and an authorization-dependent 404 must never be stored. `v2Error` sets this unconditionally; do not build a v2 response any other way. +- [ ] A retryable failure says when: 429 and 503 carry `Retry-After`. No other status invents one. +- [ ] 403s carry a machine-readable `details.code` from `FORBIDDEN_DETAIL_CODES`, thrown as `ForbiddenOperationError` in the domain rather than attached at the route. +- [ ] Nested objects inside a `.strict()` body are strict too — `.strict()` does not recurse. +- [ ] Ordering uses `sortBy` + `sortOrder`; boolean query params use `booleanQueryFlagSchema`. +- [ ] Validation messages name the field and echo the valid set. +- [ ] Response schema matches every field the route actually emits. +- [ ] OpenAPI description regenerated and truthful about pagination. +- [ ] `bun run type-check`, `bun run check:api-validation`, `bun run check:openapi` pass. + +## Known gap + +A 405 on a path that *does* have a route file but does not export that verb is generated by Next.js before any Sim code runs: zero-byte body, no `content-type`, and no `Allow` header, which RFC 9110 §15.5.6 requires. Fixing it means either exporting explicit rejecting handlers from every v2 route file or intercepting in `apps/sim/proxy.ts` with a static path→methods table. Neither is done. Unknown *paths* are handled — the catch-all covers those. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cf2f27d7836..6030c49dba0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -697,7 +697,15 @@ jobs: desktop-release: name: Desktop Release needs: [create-release, check-desktop-signing, detect-version] - if: needs.check-desktop-signing.outputs.configured == 'true' + # Suppress the implicit success() check: check-desktop-signing has an + # intentionally skipped transitive dependency on main, which would + # otherwise cascade-skip this job even when every direct need succeeded. + if: >- + !cancelled() && + needs.create-release.result == 'success' && + needs.check-desktop-signing.result == 'success' && + needs.detect-version.result == 'success' && + needs.check-desktop-signing.outputs.configured == 'true' permissions: contents: write uses: ./.github/workflows/desktop-release.yml diff --git a/.github/workflows/desktop-e2e.yml b/.github/workflows/desktop-e2e.yml index 7632b0bea66..7b87a84bc8d 100644 --- a/.github/workflows/desktop-e2e.yml +++ b/.github/workflows/desktop-e2e.yml @@ -10,6 +10,9 @@ name: Desktop E2E on: workflow_dispatch: +permissions: + contents: read + concurrency: group: desktop-e2e-${{ github.ref }} cancel-in-progress: true diff --git a/apps/docs/app/[lang]/[[...slug]]/page.tsx b/apps/docs/app/[lang]/[[...slug]]/page.tsx index 36c31389949..a4ffafdad87 100644 --- a/apps/docs/app/[lang]/[[...slug]]/page.tsx +++ b/apps/docs/app/[lang]/[[...slug]]/page.tsx @@ -16,7 +16,7 @@ import { CodeBlock } from '@/components/ui/code-block' import { Heading } from '@/components/ui/heading' import { ResponseSection } from '@/components/ui/response-section' import { i18n } from '@/lib/i18n' -import { getApiSpecContent, openapi } from '@/lib/openapi' +import { getApiSpecContent, getAuthenticatedCodeSamples, openapi } from '@/lib/openapi' import { type PageData, source } from '@/lib/source' import { DOCS_BASE_URL } from '@/lib/urls' @@ -71,6 +71,7 @@ function stripLocalePrefix(url: string, lang: string): string { const APIPage = createAPIPage(openapi, { playground: { enabled: false }, + generateCodeSamples: getAuthenticatedCodeSamples, client: { operation: { APIExampleSelector }, }, diff --git a/apps/docs/app/global.css b/apps/docs/app/global.css index 6373ec36d75..6414f5797e3 100644 --- a/apps/docs/app/global.css +++ b/apps/docs/app/global.css @@ -42,6 +42,13 @@ body { --text-small: 13px; --text-base: 15px; --text-md: 16px; + + /* Code-token size for the API reference — a deliberate sixth step, between + --text-caption and --text-small, because the mono face reads small at 12px. */ + --text-code: 0.78125rem; + + --font-mono-stack: var(--font-geist-mono), ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, + "Liberation Mono", "Courier New", monospace; } /* Pure white light mode background */ @@ -134,7 +141,6 @@ body { --selection-dark: #264f78; --highlight-search-active: #f6ad55; --scrollbar-thumb-color: #c0c0c0; - --scrollbar-thumb-hover-color: #a8a8a8; --shadow-subtle: 0 2px 4px 0 rgba(0, 0, 0, 0.08); --shadow-medium: 0 4px 12px rgba(0, 0, 0, 0.1); --shadow-overlay: 0 10px 30px rgba(0, 0, 0, 0.11); @@ -216,34 +222,18 @@ body { --code-line-number: #a8a8a8; --selection-bg: #264f78; --scrollbar-thumb-color: #5a5a5a; - --scrollbar-thumb-hover-color: #6a6a6a; --shadow-overlay: 0 10px 30px rgba(0, 0, 0, 0.3); } -/* Scrollbars — platform thumb tokens, transparent track */ +/* Scrollbars — platform thumb tokens, transparent track. A non-auto + `scrollbar-width`/`scrollbar-color` makes Chromium ignore every + `::-webkit-scrollbar*` rule on the element, so no webkit block here. Hover + shading is not expressible through the standard properties. */ * { scrollbar-width: thin; scrollbar-color: var(--scrollbar-thumb-color) transparent; } -*::-webkit-scrollbar { - width: 8px; - height: 8px; -} - -*::-webkit-scrollbar-track { - background: transparent; -} - -*::-webkit-scrollbar-thumb { - background-color: var(--scrollbar-thumb-color); - border-radius: 9999px; -} - -*::-webkit-scrollbar-thumb:hover { - background-color: var(--scrollbar-thumb-hover-color); -} - /* Font family utilities */ .font-sans { font-family: var(--font-geist-sans), ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, @@ -251,8 +241,7 @@ body { } .font-mono { - font-family: var(--font-geist-mono), ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, - "Liberation Mono", "Courier New", monospace; + font-family: var(--font-mono-stack); } /* Platform UI font — Season Sans, used by the chip chrome to match the main app */ @@ -672,8 +661,7 @@ aside[data-sidebar], code, pre, pre code { - font-family: var(--font-geist-mono), ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, - "Liberation Mono", "Courier New", monospace; + font-family: var(--font-mono-stack); } /* Inline code — neutral colors aligned with sim design system */ @@ -912,16 +900,18 @@ video { display: none !important; } -/* Ensure API reference pages use the same font as the rest of the docs */ +/* Ensure API reference pages use the same font as the rest of the docs. + `.font-mono` is excluded: this selector (id + element) outranks the + `.font-mono` class rule, so without it every code identifier renders sans. */ #nd-page:has(.api-page-header), #nd-page:has(.api-page-header) h2, #nd-page:has(.api-page-header) h3, #nd-page:has(.api-page-header) h4, -#nd-page:has(.api-page-header) p, -#nd-page:has(.api-page-header) span, -#nd-page:has(.api-page-header) div, -#nd-page:has(.api-page-header) label, -#nd-page:has(.api-page-header) button { +#nd-page:has(.api-page-header) p:not(.font-mono), +#nd-page:has(.api-page-header) span:not(.font-mono), +#nd-page:has(.api-page-header) div:not(.font-mono), +#nd-page:has(.api-page-header) label:not(.font-mono), +#nd-page:has(.api-page-header) button:not(.font-mono) { font-family: var(--font-geist-sans), ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; } @@ -1162,23 +1152,45 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { position: relative; } +/* API-reference metadata face — the status trigger, the content-type label, the + `required` / `header` markers, and the status-code tabs. Defined once; each + consumer below adds only its own colour, content, and order. The `code.text-xs` + label further down needs `!important` to beat fumadocs and stays separate. */ +#nd-page:has(.api-page-header) button.response-section-dropdown-trigger, +.response-section-dropdown-trigger, +#nd-page:has(.api-page-header) span.response-section-content-type, +.response-section-content-type, +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose:has(span.text-red-400)::after, +#nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::before, +#nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::after, +#nd-page:has(.api-page-header) .flex.gap-3\.5.overflow-x-auto.not-prose > button { + font-size: var(--text-code); + line-height: 1.25rem; + font-weight: 400; + font-family: var(--font-mono-stack); +} + +/* Status-code trigger — matches the content-type label beside it. */ +#nd-page:has(.api-page-header) button.response-section-dropdown-trigger, .response-section-dropdown-trigger { display: flex; align-items: center; gap: 0.25rem; - padding: 0.125rem 0.25rem; - font-size: 0.875rem; - font-weight: 500; - color: var(--color-fd-muted-foreground); + height: 1.25rem; + padding: 0 0.25rem; + color: var(--text-secondary); background: none; border: none; cursor: pointer; border-radius: 0.375rem; transition: color 0.15s; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; } +/* Carries the same id-qualified prefix as the base rule above; without it the + base rule outranks this one and the trigger never changes colour on hover. */ +#nd-page:has(.api-page-header) button.response-section-dropdown-trigger:hover, .response-section-dropdown-trigger:hover { - color: var(--color-fd-foreground); + color: var(--text-primary); } .response-section-chevron { @@ -1226,7 +1238,7 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { color: var(--text-primary); } .response-section-dropdown-item-selected { - color: var(--color-fd-foreground); + color: var(--text-primary); } .response-section-check { @@ -1234,10 +1246,15 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { height: 0.875rem; } +/* Content-type label. The Response header renders this class; the Request Body + header renders a fumadocs `code.text-xs`. Keep the two in sync — the same + string at different weights reads as one being lighter than the other. */ +#nd-page:has(.api-page-header) span.response-section-content-type, .response-section-content-type { - font-size: 0.875rem; - color: var(--color-fd-muted-foreground); - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; + color: var(--text-secondary); + background: none; + border: none; + padding: 0; } /* Response schema container — remove border to match Path Parameters style */ @@ -1262,25 +1279,80 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { order: 1; } -/* Type badge — order 2, grey pill */ +/* Type token — order 2. Covers every shape the slot takes: scalar span, union + wrapper, schema-reference button, and the auth row's `::after` label. Reuses + the docs inline-code recipe, so a type reads as code wherever it appears; the + explicit 20px height keeps a union level with a scalar, which its nested + links would otherwise push to 26px. */ #nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose - > span.text-sm.font-mono.text-fd-muted-foreground { + > span.text-sm.font-mono.text-fd-muted-foreground, +#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > button, +#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > span:has(> button), +#nd-page:has(.api-page-header) + div.my-4 + > .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground::after { order: 2; - background-color: var(--surface-5); - color: var(--text-secondary); - padding: 0.1875rem 0.5rem; + display: inline-flex; + align-items: center; + height: 1.25rem; + /* No gap: an `array` slot holds its brackets as bare text nodes, which + become anonymous flex items, so any gap here would prise `array<` and `>` + away from the type they wrap. The union separator spaces itself instead. */ + gap: 0; + background-color: var(--surface-4); + border: 1px solid var(--border-1); + color: var(--text-body); + padding: 0 0.3125rem; border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; + font-size: var(--text-code); + line-height: 1; + font-weight: 400; + font-family: var(--font-mono-stack); +} + +/* Everything inside a type token inherits the token's own face, size, and ink. + Applied to every descendant, not just the links: a union's `|` separator is a + classless `span`, so the page-wide `span:not(.font-mono)` rule assigned it the + body sans face and one chip rendered in two faces. Anything fumadocs nests in + here later is covered by the same reset. + Underline is deferred to hover so links don't read heavier than a plain scalar + in the same box. The button that *is* the slot needs its own rule below: it + cannot `inherit`, which would pull the row's 14px sans back in. */ +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground + * { + text-decoration: none; + color: inherit; + font-size: inherit; + font-family: inherit; } -html.dark - #nd-page:has(.api-page-header) +#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose - > span.text-sm.font-mono.text-fd-muted-foreground { - background-color: var(--surface-4); + > button.text-sm.font-mono.text-fd-muted-foreground { + text-decoration: none; +} +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground + :is(a, button):hover, +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > button.text-sm.font-mono.text-fd-muted-foreground:hover { + text-decoration: underline; + text-underline-offset: 2px; +} + +/* Union separator — dimmed one step, no further: `string | null` started + reading as `string null` on the chip fill. Own margin; the slot has no gap. */ +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground + > span { + margin: 0 0.375rem; + color: var(--text-muted); } /* Hide the "*" inside the name span — we'll add "required" as a ::after on the flex row */ @@ -1288,21 +1360,15 @@ html.dark display: none; } -/* Required badge — order 3, red pill */ +/* Required marker — order 3. Error text colour but no fill: eight required + params on one page should not read as eight alarms. */ #nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose:has(span.text-red-400)::after { content: "required"; order: 3; display: inline-flex; align-items: center; - background-color: var(--badge-error-bg); color: var(--badge-error-text); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; } /* Optional "?" indicator — hide it */ #nd-page:has(.api-page-header) @@ -1326,79 +1392,52 @@ html.dark > span.font-medium.font-mono.text-fd-primary { order: 1; } +/* Auth rows collapse the real `` text to zero and draw the chip in the + `::after` below, so this span is a bare wrapper: it must drop the type-token + box it matches, or the chip renders inside a second, empty bordered box. */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose > span.text-sm.font-mono.text-fd-muted-foreground { order: 2; font-size: 0; - padding: 0 !important; - background: none !important; + padding: 0; + background: none; + border: none; + height: auto; line-height: 0; } +/* Only the label — the box comes from the shared type-token rule above, which + this pseudo-element is a member of. */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose > span.text-sm.font-mono.text-fd-muted-foreground::after { content: "string"; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; - background-color: var(--surface-5); - color: var(--text-secondary); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - display: inline-flex; - align-items: center; -} -html.dark - #nd-page:has(.api-page-header) - div.my-4 - > .flex.flex-wrap.items-center.gap-3.not-prose - > span.text-sm.font-mono.text-fd-muted-foreground::after { - background-color: var(--surface-4); } -/* "header" badge via ::before on the auth flex row */ +/* "header" location via ::before on the auth flex row — uncontained metadata, + matching the `required` marker rather than the type token. */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::before { content: "header"; order: 3; display: inline-flex; align-items: center; - background-color: var(--surface-5); color: var(--text-secondary); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; -} -html.dark - #nd-page:has(.api-page-header) - div.my-4 - > .flex.flex-wrap.items-center.gap-3.not-prose::before { - background-color: var(--surface-4); } -/* "required" badge via ::after on the auth flex row — red pill */ +/* "required" marker via ::after on the auth flex row */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::after { content: "required"; order: 4; display: inline-flex; align-items: center; - background-color: var(--badge-error-bg); color: var(--badge-error-text); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; } -/* Hide "In: header" text below auth property — redundant with the header badge */ -#nd-page:has(.api-page-header) div.my-4 .prose-no-margin p:has(> code) { +/* Hide the trailing "In: header" line — redundant with the header marker. + Matched by position, not shape: descriptions contain a `code` too (status + codes), so a bare `p:has(> code)` also hid the API-key description. */ +#nd-page:has(.api-page-header) div.my-4 .prose-no-margin > p:last-child:has(> code) { display: none !important; } @@ -1425,36 +1464,18 @@ html.dark border-color: var(--surface-active); } -/* Body/Callback section "application/json" label — remove inline code styling */ +/* Body/Callback "application/json" label — strip inline-code chrome and keep in + sync with `.response-section-content-type`; same string, two headers. */ #nd-page:has(.api-page-header) .flex.gap-2.items-center.justify-between p.not-prose code.text-xs, #nd-page:has(.api-page-header) .flex.justify-between.gap-2.items-end p.not-prose code.text-xs { background: none !important; border: none !important; padding: 0 !important; - color: var(--color-fd-muted-foreground) !important; - font-size: 0.875rem !important; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif !important; -} - -/* Object/array type triggers in property rows — order 2 + badge chip styling */ -#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > button, -#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > span:has(> button) { - order: 2; - background-color: var(--surface-5); - color: var(--text-secondary); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; -} -html.dark #nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > button, -html.dark - #nd-page:has(.api-page-header) - .flex.flex-wrap.items-center.gap-3.not-prose - > span:has(> button) { - background-color: var(--surface-4); + color: var(--text-secondary) !important; + font-size: var(--text-code) !important; + line-height: 1.25rem !important; + font-weight: 400 !important; + font-family: var(--font-mono-stack) !important; } /* Section headings (Authorization, Path Parameters, etc.) — consistent top spacing */ @@ -1463,15 +1484,15 @@ html.dark margin-bottom: 0.25rem !important; } -/* Code examples in right column — wrap long lines instead of horizontal scroll */ -#nd-page:has(.api-page-header) pre { - white-space: pre-wrap !important; - word-break: break-all !important; -} -#nd-page:has(.api-page-header) pre code { - width: 100% !important; - word-break: break-all !important; - overflow-wrap: break-word !important; +/* Example-panel code overflows rather than wraps: a wrapped line restarts at + column zero and misreports the JSON nesting depth. */ + +/* fumadocs' own lucide glyphs (heading anchor, copy button) ship at stroke-width + 2 while emcn strokes at 1.55, so they read heavier than everything near them. + Layout-wide on purpose: one icon weight across the docs. Retired once + createAPIPage is given renderHeading/renderCodeBlock. */ +#nd-docs-layout svg[class*="lucide"] { + stroke-width: 1.55; } /* Callout/alert — transparent background, no shadow, hide colored bar, add padding */ @@ -1497,7 +1518,7 @@ div.not-prose.rounded-md.border.bg-fd-card.p-2 { div.rounded-xl.border.bg-fd-card.shadow-md:has(> [role="none"]) > svg { fill: none !important; color: var(--color-fd-foreground) !important; - stroke-width: 1.75 !important; + stroke-width: 1.55 !important; flex-shrink: 0; width: 1rem !important; height: 1rem !important; diff --git a/apps/docs/components/icons.tsx b/apps/docs/components/icons.tsx index 447d38bb28f..66b13bcc673 100644 --- a/apps/docs/components/icons.tsx +++ b/apps/docs/components/icons.tsx @@ -2494,6 +2494,32 @@ export function DocumentIcon(props: SVGProps) { ) } +export function WindchillIcon(props: SVGProps) { + return ( + + + + + + + + + + + + + ) +} + export function MintlifyIcon(props: SVGProps) { return ( diff --git a/apps/docs/components/ui/icon-mapping.ts b/apps/docs/components/ui/icon-mapping.ts index efc90688641..3f5b071fca6 100644 --- a/apps/docs/components/ui/icon-mapping.ts +++ b/apps/docs/components/ui/icon-mapping.ts @@ -245,6 +245,7 @@ import { WebhookIcon, WhatsAppIcon, WikipediaIcon, + WindchillIcon, WizaIcon, WordpressIcon, WorkdayIcon, @@ -538,6 +539,7 @@ export const blockTypeToIconMap: Record = { webflow: WebflowIcon, whatsapp: WhatsAppIcon, wikipedia: WikipediaIcon, + windchill: WindchillIcon, wiza: WizaIcon, wordpress: WordpressIcon, workday: WorkdayIcon, diff --git a/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json b/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json index fb32a86e414..1017e55e280 100644 --- a/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json +++ b/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json @@ -9,6 +9,7 @@ "getWorkflowVersionV2", "exportWorkflow", "importWorkflow", + "getWorkflowDeployment", "deployWorkflow", "undeployWorkflow", "rollbackWorkflow", diff --git a/apps/docs/content/docs/en/integrations/meta.json b/apps/docs/content/docs/en/integrations/meta.json index da4fbf044e5..902c4cd44b1 100644 --- a/apps/docs/content/docs/en/integrations/meta.json +++ b/apps/docs/content/docs/en/integrations/meta.json @@ -259,6 +259,7 @@ "webflow-service-account", "whatsapp", "wikipedia", + "windchill", "wiza", "wordpress", "workday", diff --git a/apps/docs/content/docs/en/integrations/windchill.mdx b/apps/docs/content/docs/en/integrations/windchill.mdx new file mode 100644 index 00000000000..7f2b4dbfab1 --- /dev/null +++ b/apps/docs/content/docs/en/integrations/windchill.mdx @@ -0,0 +1,937 @@ +--- +title: Windchill +description: Manage documents, revisions, and content in PTC Windchill +--- + +import { BlockInfoCard } from "@/components/ui/block-info-card" + + + +{/* MANUAL-CONTENT-START:intro */} +[PTC Windchill](https://www.ptc.com/en/products/windchill) is the product lifecycle management system manufacturers use as the system of record for engineering data. Documents in Windchill are controlled objects: each one carries a number, a revision and iteration, a lifecycle state, folder placement, security labels, and a checkout status that decides who is allowed to change it right now. + +This integration talks to Windchill REST Services (WRS) 2.7 over OData — the query protocol Windchill exposes its data through — using a Basic-authenticated service account. Point it at a complete versioned service root — `https://your-host/Windchill/servlet/odata/v6` — and your agents can: + +- **Find and read documents**: list documents with an OData filter, sort order, field selection, page size, and a latest-version-only switch; fetch a single document by its object identifier (OID); and walk a document's structure through its usage links to see child documents with their versions and states. +- **Create and update**: create one document or a batch of them in a container and optional folder, and patch editable attributes on one or many documents. Name, Number, and Organization are rejected here and have their own operation, because Windchill changes those through a separate action and refuses it while a document is checked out. +- **Run the version and lifecycle cycle**: check documents out and back in with notes, undo a checkout, revise to the next revision, read the lifecycle states a document is actually allowed to move to, and transition it to one of them. +- **Move files**: download a document's primary content, or a specific attachment by its OID, into a Sim file — and upload files as primary content or attachments. Sim handles Windchill's CSRF token and its multi-step upload handshake for you. + +Bulk actions are atomic on Windchill's side: PTC documents that if the action fails for any object in the collection, the entire action is rolled back and nothing changes. + +Two limits are worth knowing before you build. Windchill identifies everything by OID (`OR:wt.doc.WTDocument:48796581`), so most operations need an OID you got from a list or get call rather than a document number. And this integration supports Basic authentication only — Windchill deployments fronted by OAuth are not currently supported. +{/* MANUAL-CONTENT-END */} + + +## Usage Instructions + +Integrate PTC Windchill REST Services 2.7 document management into your workflow using Basic authentication. Read and update document metadata, perform version and lifecycle actions, and transfer primary content and attachments. Windchill OAuth deployments are not currently supported. + + + +## Actions + +### Windchill List Documents + +List documents with an OData query, sorting, and pagination + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `select` | string | No | Comma-separated normalized document properties to return | +| `filter` | string | No | OData $filter expression | +| `orderBy` | string | No | OData $orderby expression | +| `top` | number | No | Maximum documents in the OData result set \($top\), from 1 to 2000 | +| `skip` | number | No | Documents to skip | +| `count` | boolean | No | Ask Windchill to include the total matching count | +| `latestVersion` | boolean | No | Return only the latest version of matching documents | +| `nextLink` | string | No | Verified @odata.nextLink from a previous list response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `documents` | array | Windchill documents | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | +| `pageInfo` | object | OData pagination information | +| ↳ `count` | number | Number of items returned in this page | +| ↳ `totalCount` | number | Total matching items | +| ↳ `nextLink` | string | URL returned by Windchill for the next page | + +### Windchill Get Document + +Get a WT.Document by OID + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `select` | string | No | Comma-separated normalized document properties to return | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `document` | object | Windchill document | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Get Document Structure + +Retrieve recursive document usage links and their parent and child documents + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `structureDepth` | number | No | Document structure expansion depth, from 1 to 3 | +| `nextLink` | string | No | Verified @odata.nextLink from a previous structure response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `structure` | array | Document usage links, including recursively expanded child links | +| ↳ `id` | string | Document usage link OID | +| ↳ `parent` | object | Parent document | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | +| ↳ `child` | object | Child document | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | +| ↳ `children` | array | Nested child usage links with the same recursive shape | +| `pageInfo` | object | OData pagination information | +| ↳ `count` | number | Number of items returned in this page | +| ↳ `totalCount` | number | Total matching items | +| ↳ `nextLink` | string | URL returned by Windchill for the next page | + +### Windchill Get Valid State Transitions + +Get lifecycle states a document can transition to from its current state + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `states` | array | Valid lifecycle transitions | +| ↳ `value` | string | Internal state value | +| ↳ `display` | string | Displayed state value | + +### Windchill Get Primary Content + +Get primary-content metadata for a document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `content` | object | Primary-content metadata | +| ↳ `id` | string | Content object identifier | +| ↳ `fileName` | string | Content file name | +| ↳ `description` | string | Content description | +| ↳ `format` | string | Windchill content format | +| ↳ `mimeType` | string | Content MIME type | +| ↳ `fileSize` | number | Content size in bytes | +| ↳ `contentType` | string | Windchill OData content entity type | +| ↳ `displayName` | string | Displayed content name | +| ↳ `urlLocation` | string | URL-data location | +| ↳ `externalLocation` | string | External-storage location | + +### Windchill List Attachments + +List attachment metadata for a document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `nextLink` | string | No | Verified @odata.nextLink from a previous attachment response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `attachments` | array | Document attachments | +| ↳ `id` | string | Content object identifier | +| ↳ `fileName` | string | Content file name | +| ↳ `description` | string | Content description | +| ↳ `format` | string | Windchill content format | +| ↳ `mimeType` | string | Content MIME type | +| ↳ `fileSize` | number | Content size in bytes | +| ↳ `contentType` | string | Windchill OData content entity type | +| ↳ `displayName` | string | Displayed content name | +| ↳ `urlLocation` | string | URL-data location | +| ↳ `externalLocation` | string | External-storage location | +| `pageInfo` | object | OData pagination information | +| ↳ `count` | number | Number of items returned in this page | +| ↳ `totalCount` | number | Total matching items | +| ↳ `nextLink` | string | URL returned by Windchill for the next page | + +### Windchill Create Document + +Create one WT.Document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `name` | string | Yes | Document name | +| `containerOid` | string | Yes | Container OID in which to create the document | +| `number` | string | No | Optional document number when manual numbering is enabled | +| `title` | string | No | Document title | +| `description` | string | No | Document description | +| `folderOid` | string | No | Optional folder OID for the new document | +| `attributes` | json | No | Optional installed Windchill document attributes as a JSON object | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Create Documents + +Create several documents in one atomic Windchill request + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documents` | array | Yes | Document inputs as a JSON array; each item requires name and containerOid | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Document + +Update one document's editable attributes + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `attributes` | json | Yes | Editable attributes as a JSON object. Name, Number, and Organization require the Update Common Properties operation and are not supported here. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Common Properties + +Update a document's Name, Number, and other common properties + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581. The document must not be checked out. | +| `commonProperties` | json | Yes | Common properties as a JSON object, for example \{"Name":"New name","Number":"NEW-001"\}. Enumerated properties take a value/display pair. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Documents + +Update several documents' editable attributes in one atomic request + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documents` | array | Yes | Document updates as a JSON array; each item requires id and the editable attributes to set. Name, Number, and Organization are not supported. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Delete Document + +Delete one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | + +### Windchill Delete Documents + +Delete multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | + +### Windchill Check Out Document + +Check out one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Check Out Documents + +Check out multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Check In Document + +Check in one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `checkInNote` | string | No | Check-in note | +| `keepCheckedOut` | boolean | No | Keep the document checked out after checking it in | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Check In Documents + +Check in multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | +| `checkInNote` | string | No | Check-in note | +| `keepCheckedOut` | boolean | No | Keep the document checked out after checking it in | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Undo Check Out Document + +Undo checkout for one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Undo Check Out Documents + +Undo checkout for multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Revise Document + +Create a new revision of one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `versionId` | string | No | Optional target revision identifier when override-on-revise is enabled | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Revise Documents + +Create new revisions of multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Set Lifecycle State + +Transition a document to a valid lifecycle state + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `stateValue` | string | Yes | Internal value of the target lifecycle state | +| `stateDisplay` | string | Yes | Display value of the target lifecycle state | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Document Security Labels + +Update installed security-label attributes for one or more documents + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `securityLabelUpdates` | array | Yes | Array of document IDs and installed security-label values | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Download Primary Content + +Download primary content into a canonical UserFile + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `fileName` | string | No | Optional downloaded file name override | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `file` | file | Downloaded content stored as a canonical UserFile | +| `fileName` | string | Downloaded file name | +| `mimeType` | string | Downloaded content MIME type | + +### Windchill Upload Primary Content + +Upload a primary-content file to a document that has none + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `primaryFile` | file | Yes | Primary content file to upload | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the upload | +| `uploadedFileNames` | array | Names of files accepted by Windchill | + +### Windchill Download Attachment + +Download a document attachment into a canonical UserFile + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `attachmentOid` | string | Yes | Windchill attachment content OID | +| `fileName` | string | No | Optional downloaded file name override | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `file` | file | Downloaded content stored as a canonical UserFile | +| `fileName` | string | Downloaded file name | +| `mimeType` | string | Downloaded content MIME type | + +### Windchill Upload Attachments + +Upload one or more files as document attachments + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `attachmentFiles` | file[] | Yes | Attachment files to upload | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the upload | +| `uploadedFileNames` | array | Names of files accepted by Windchill | + + diff --git a/apps/docs/content/docs/en/workflows/blocks/agent.mdx b/apps/docs/content/docs/en/workflows/blocks/agent.mdx index d4ce50bcacc..1a6fb899444 100644 --- a/apps/docs/content/docs/en/workflows/blocks/agent.mdx +++ b/apps/docs/content/docs/en/workflows/blocks/agent.mdx @@ -115,6 +115,7 @@ Live tool-call chips stream for **OpenAI, Anthropic, Azure Anthropic, Google, Ve | Google | Summaries only | `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` | | Vertex AI | Summaries only | `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview`, `vertex/gemini-2.5-pro`, `vertex/gemini-2.5-flash`, `vertex/gemini-2.5-flash-lite` | | DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-reasoner` | +| xAI | Full thinking deltas | `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-0309` | | Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.6-27b` | | Meta | Not streamed | `muse-spark-1.1` | | Kimi | Full thinking deltas | `kimi-k2.6` | @@ -142,7 +143,7 @@ The Agent reads the message from Start with `` and returns a result { question: "What are the memory options for the Agent block?", answer: "Four modes: None (no memory, each run is independent), Conversation (full history keyed by a conversation ID), Sliding window by messages (the N most recent messages), and Sliding window by tokens (messages up to a token budget). Memory needs a conversation ID to persist across runs." }, { question: "What is the difference between the tool usage controls (Auto, Force, None)?", answer: "In Auto, the model decides when to call a tool based on context. In Force, the model must call the tool on every run. In None, the tool is hidden from the model and never sent, which disables it without removing it from the block." }, { question: "How does the Response Format work?", answer: "It enforces structured output by providing a JSON Schema. When set, the model's response is constrained to match the schema exactly, and each field is read directly by downstream blocks using . Without a response format, the agent returns its standard outputs: content, model, tokens, and toolCalls." }, - { question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI o-series and GPT-5 models) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." }, + { question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI, Azure OpenAI, xAI Grok, DeepSeek, Groq, Meta, and Z.ai models that accept an effort level) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." }, { question: "When should I turn on Prompt Caching?", answer: "Turn it on when the same agent runs repeatedly with a large, stable system prompt or tool set — cached input bills at a tenth of the normal input rate. Leave it off for one-off runs, because writing the cache costs 1.25x and nothing reads it back. The setting appears only for Anthropic Claude models; OpenAI and Gemini cache automatically with no setting and no write fee. Anthropic only caches a prefix of at least 1,024 tokens (2,048 on Haiku), and entries expire after five minutes of no use." }, { question: "How does max output tokens work with Anthropic models?", answer: "The Agent block uses each Anthropic model's full max output token limit by default (for example, 64,000 tokens). You can override this with the Max Output Tokens setting. For non-streaming requests that exceed the SDK's internal threshold, the provider automatically uses internal streaming to avoid timeouts." }, { question: "Can I use the Agent block with a custom or self-hosted model?", answer: "Yes. Use any Ollama or VLLM-compatible model by typing the model name directly into the model combobox, as long as it exposes a compatible API endpoint." }, diff --git a/apps/docs/lib/openapi-code-samples-client.ts b/apps/docs/lib/openapi-code-samples-client.ts new file mode 100644 index 00000000000..f983da91e43 --- /dev/null +++ b/apps/docs/lib/openapi-code-samples-client.ts @@ -0,0 +1,37 @@ +'use client' + +import type { CodeUsageGeneratorFn } from 'fumadocs-openapi/requests/generators' +import { createCodeUsageGeneratorRegistry } from 'fumadocs-openapi/requests/generators' +import { registerDefault } from 'fumadocs-openapi/requests/generators/all' + +/** + * Context handed to {@link generateWithAuth} by the server: which built-in + * generator to delegate to, and the auth headers the sample must send. + */ +export interface AuthCodeSampleContext { + generatorId: string + headers: Record +} + +const generators = createCodeUsageGeneratorRegistry() +registerDefault(generators) + +/** + * Wraps a built-in code-usage generator so the sample carries the operation's + * security headers. Fumadocs builds request data from declared parameters only, + * so an operation's security requirement never reaches the generated snippet. + */ +export const generateWithAuth: CodeUsageGeneratorFn = (url, data, context) => { + const { generatorId, headers } = context.server as AuthCodeSampleContext + const generator = generators.get(generatorId) + if (!generator) { + throw new Error(`[docs] Unknown code usage generator: ${generatorId}`) + } + + const authHeaders: Record = {} + for (const [name, value] of Object.entries(headers)) { + authHeaders[name] = { value } + } + + return generator.generate(url, { ...data, header: { ...authHeaders, ...data.header } }, context) +} diff --git a/apps/docs/lib/openapi-code-samples.ts b/apps/docs/lib/openapi-code-samples.ts new file mode 100644 index 00000000000..ebfbf764bee --- /dev/null +++ b/apps/docs/lib/openapi-code-samples.ts @@ -0,0 +1,21 @@ +import type { InlineCodeUsageGenerator } from 'fumadocs-openapi/requests/generators' +import { createCodeUsageGeneratorRegistry } from 'fumadocs-openapi/requests/generators' +import { registerDefault } from 'fumadocs-openapi/requests/generators/all' +import { generateWithAuth } from '@/lib/openapi-code-samples-client' + +const generators = createCodeUsageGeneratorRegistry() +registerDefault(generators) + +/** + * Replace every built-in language sample with one that prepends `headers`, + * preserving the built-in tab order, language, and label. + */ +export function buildAuthCodeSamples(headers: Record): InlineCodeUsageGenerator[] { + return Array.from(generators.map().entries()).map(([id, generator]) => ({ + id, + lang: generator.lang, + label: generator.label, + source: generateWithAuth, + serverContext: { generatorId: id, headers }, + })) +} diff --git a/apps/docs/lib/openapi.ts b/apps/docs/lib/openapi.ts index f67d60db719..7841fd863ad 100644 --- a/apps/docs/lib/openapi.ts +++ b/apps/docs/lib/openapi.ts @@ -1,6 +1,9 @@ import { readFileSync } from 'node:fs' import { join } from 'node:path' +import type { MethodInformation } from 'fumadocs-openapi' +import type { InlineCodeUsageGenerator } from 'fumadocs-openapi/requests/generators' import { createOpenAPI } from 'fumadocs-openapi/server' +import { buildAuthCodeSamples } from '@/lib/openapi-code-samples' import { OPENAPI_SPEC_FILES } from '@/lib/openapi-specs' export const openapi = createOpenAPI({ @@ -75,6 +78,109 @@ function getSpecs(): Record[] { return cachedSpecs } +type SecurityRequirement = Record + +interface SecurityScheme { + type?: string + in?: string + name?: string + scheme?: string +} + +interface SharedSecurity { + security: SecurityRequirement[] + schemes: Record +} + +const AUTH_SAMPLE_VALUE = 'YOUR_API_KEY' + +let cachedSharedSecurity: SharedSecurity | null = null + +/** + * Document-level security shared by every rendered spec. Code samples are + * generated from an operation alone, with no handle on the document that owns + * it, so the specs must agree on their default security — a spec that diverges + * would silently get another document's auth in its samples. + */ +function getSharedSecurity(): SharedSecurity { + if (cachedSharedSecurity) return cachedSharedSecurity + + let shared: SharedSecurity | undefined + let sharedFile: string | undefined + + getSpecs().forEach((spec, index) => { + const file = OPENAPI_SPEC_FILES[index] + const current: SharedSecurity = { + security: (spec.security as SecurityRequirement[] | undefined) ?? [], + schemes: + ((spec.components as Record | undefined)?.securitySchemes as + | Record + | undefined) ?? {}, + } + + if (!shared) { + shared = current + sharedFile = file + return + } + + if (JSON.stringify(current) !== JSON.stringify(shared)) { + throw new Error( + `[docs] ${file} declares different default security than ${sharedFile}. Every OpenAPI spec must share one security scheme so generated code samples stay correct.` + ) + } + }) + + cachedSharedSecurity = shared ?? { security: [], schemes: {} } + return cachedSharedSecurity +} + +/** + * Resolve a security requirement to the request headers a sample must send. + * The first non-empty alternative wins — an empty one means the operation also + * accepts anonymous callers, which is not what a reference example should show. + */ +function resolveAuthHeaders( + security: SecurityRequirement[], + schemes: Record +): Record { + const requirement = security.find((item) => Object.keys(item).length > 0) + if (!requirement) return {} + + const headers: Record = {} + for (const name of Object.keys(requirement)) { + const scheme = schemes[name] + if (!scheme) { + throw new Error(`[docs] Operation references undefined security scheme "${name}"`) + } + if (scheme.type === 'apiKey' && scheme.in === 'header' && scheme.name) { + headers[scheme.name] = AUTH_SAMPLE_VALUE + continue + } + if (scheme.type === 'http' && scheme.scheme === 'bearer') { + headers.Authorization = `Bearer ${AUTH_SAMPLE_VALUE}` + continue + } + throw new Error( + `[docs] Security scheme "${name}" (type ${scheme.type}) cannot be rendered as a request header in code samples` + ) + } + return headers +} + +/** + * Code samples for an operation, with its authentication header included. + * Fumadocs derives sample requests from declared parameters only, so without + * this every endpoint documents an unauthenticated call that returns `401`. + */ +export function getAuthenticatedCodeSamples(method: MethodInformation): InlineCodeUsageGenerator[] { + const shared = getSharedSecurity() + const security = (method.security as SecurityRequirement[] | undefined) ?? shared.security + const headers = resolveAuthHeaders(security, shared.schemes) + if (Object.keys(headers).length === 0) return [] + return buildAuthCodeSamples(headers) +} + /** * Locate an operation by path + method across every rendered spec, returning the * operation together with the spec that owns it so `$ref`s resolve within the diff --git a/apps/docs/openapi-v2-billing.json b/apps/docs/openapi-v2-billing.json index 05e85b78dd7..f611771391b 100644 --- a/apps/docs/openapi-v2-billing.json +++ b/apps/docs/openapi-v2-billing.json @@ -174,10 +174,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum usage events per page, from 1 to 100.", + "description": "Maximum usage events per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum usage events per page, from 1 to 100.", + "description": "Maximum usage events per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -248,7 +248,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -283,13 +283,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -334,7 +334,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -454,7 +454,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { diff --git a/apps/docs/openapi-v2-files-audit.json b/apps/docs/openapi-v2-files-audit.json index 53404f27332..7e981126eeb 100644 --- a/apps/docs/openapi-v2-files-audit.json +++ b/apps/docs/openapi-v2-files-audit.json @@ -40,7 +40,7 @@ "get": { "operationId": "listFiles", "summary": "List Files", - "description": "List workspace files with search, sorting, folder filtering, and opaque cursor pagination.", + "description": "List workspace files with search, sorting, folder filtering, and opaque cursor pagination. Defaults to active files; pass `scope=archived` to page over soft-deleted files, whose `deletedAt` is non-null and which `POST /files/{fileId}/restore` can bring back.", "tags": ["Files"], "parameters": [ { @@ -64,6 +64,18 @@ "type": "string" } }, + { + "name": "scope", + "in": "query", + "required": false, + "description": "Which lifecycle set to list: `active` (default) for live files, `archived` for files a `DELETE` soft-deleted and that `POST /files/{fileId}/restore` can bring back. `folderPath` resolves against active folders only, so combining it with `scope=archived` returns 404 when the containing folder was archived too.", + "schema": { + "default": "active", + "description": "Which lifecycle set to list: `active` (default) for live files, `archived` for files a `DELETE` soft-deleted and that `POST /files/{fileId}/restore` can bring back. `folderPath` resolves against active folders only, so combining it with `scope=archived` returns 404 when the containing folder was archived too.", + "type": "string", + "enum": ["active", "archived"] + } + }, { "name": "search", "in": "query", @@ -104,11 +116,13 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum files per page, clamped to 1–1000.", + "description": "Maximum files per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "schema": { - "description": "Maximum files per page, clamped to 1–1000.", - "default": 100, - "type": "number" + "description": "Maximum files per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", + "type": "integer", + "minimum": 1, + "maximum": 1000, + "default": 100 } }, { @@ -368,6 +382,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -466,6 +483,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -553,6 +573,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -667,7 +690,7 @@ "delete": { "operationId": "deleteFile", "summary": "Delete File", - "description": "Archive a workspace file. This is a soft delete: the row is retained with a deletion timestamp, the file stops appearing in listings and is no longer readable through the API, and its stored bytes are never removed. An archived file can be restored from the workspace Recently Deleted settings; the v2 API exposes no restore operation.", + "description": "Archive a workspace file. This is a soft delete: the row is retained with a deletion timestamp, the file stops appearing in the default listing and is no longer readable through the API, and its stored bytes are never removed. List archived files with `GET /files?scope=archived` and reverse the delete with `POST /files/{fileId}/restore`.", "tags": ["Files"], "parameters": [ { @@ -823,6 +846,87 @@ } } }, + "/api/v2/files/{fileId}/restore": { + "post": { + "operationId": "restoreFile", + "summary": "Restore File", + "description": "Reverse a soft delete and return the file to the workspace. Restore is not a pure undo: the file comes back at the workspace root regardless of the folder it was deleted from, and it gains a `_restored` suffix when another file at the root already holds its name — so read `folderPath` and `name` off the response rather than assuming the pre-delete values. Restoring a file that is already active is a no-op that returns that file, so a retry is safe. Returns 400 when the workspace itself has been archived, and 409 when no free restore name could be found.", + "tags": ["Files"], + "parameters": [ + { + "name": "fileId", + "in": "path", + "required": true, + "description": "File identifier.", + "schema": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "pattern": "^[A-Za-z0-9_-]+$", + "description": "File identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Workspace scope for the archived file.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RestoreFileRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The file as it exists after the restore.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2RestoreFileResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "409": { + "$ref": "#/components/responses/Conflict" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/files/{fileId}/metadata": { "get": { "operationId": "getFile", @@ -975,20 +1079,18 @@ "description": "Include actions by users who have left the organization.", "schema": { "description": "Include actions by users who have left the organization.", - "default": "false", - "type": "string", - "enum": ["true", "false"] + "type": "boolean" } }, { "name": "limit", "in": "query", "required": false, - "description": "Maximum entries per page, from 1 to 100.", + "description": "Maximum audit entries to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum entries per page, from 1 to 100.", - "type": "number", + "description": "Maximum audit entries to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", "minimum": 1, "maximum": 100 } @@ -1000,7 +1102,8 @@ "description": "Opaque cursor returned by the previous page.", "schema": { "description": "Opaque cursor returned by the previous page.", - "type": "string" + "type": "string", + "minLength": 1 } }, { @@ -1849,7 +1952,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -1909,13 +2012,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -1960,7 +2063,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -2080,7 +2183,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -2178,6 +2286,19 @@ "description": "ISO 8601 timestamp of the last content or metadata write.", "format": "date-time", "examples": ["2026-01-15T10:30:00Z"] + }, + "deletedAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp when the file was archived by `DELETE /files/{fileId}`, or null while the file is active. Only `GET /files?scope=archived` returns files with a non-null value.", + "format": "date-time", + "examples": ["2026-01-16T09:00:00Z"] } }, "required": [ @@ -2189,7 +2310,8 @@ "folderPath", "uploadedByEmail", "uploadedAt", - "updatedAt" + "updatedAt", + "deletedAt" ], "additionalProperties": false, "title": "Workspace file", @@ -2233,7 +2355,8 @@ "folderPath": "/Engineering", "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", - "updatedAt": "2026-01-15T10:30:00Z" + "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null } ], "nextCursor": null @@ -2263,7 +2386,8 @@ "folderPath": "/Engineering", "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", - "updatedAt": "2026-01-15T10:30:00Z" + "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null } } ] @@ -2679,6 +2803,54 @@ } ] }, + "V2RestoreFileResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/V2File" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Restore file response", + "description": "The restored workspace file, at the root and under its post-restore name.", + "examples": [ + { + "data": { + "id": "wf_V1StGXR8z5jdHi6BmyT91", + "name": "data_restored.csv", + "size": 1024, + "type": "text/csv", + "key": "workspace/example/data.csv", + "folderPath": "/", + "uploadedByEmail": "jane@example.com", + "uploadedAt": "2026-01-15T10:30:00Z", + "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null + } + } + ] + }, + "RestoreFileRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the archived file." + } + }, + "required": ["workspaceId"], + "additionalProperties": false, + "title": "Restore file request", + "description": "Workspace scope for the archived file.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64" + } + ] + }, "V2FileShare": { "type": "object", "properties": { @@ -2795,6 +2967,19 @@ "format": "date-time", "examples": ["2026-01-15T10:30:00Z"] }, + "deletedAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp when the file was archived by `DELETE /files/{fileId}`, or null while the file is active. Only `GET /files?scope=archived` returns files with a non-null value.", + "format": "date-time", + "examples": ["2026-01-16T09:00:00Z"] + }, "share": { "anyOf": [ { @@ -2817,6 +3002,7 @@ "uploadedByEmail", "uploadedAt", "updatedAt", + "deletedAt", "share" ], "additionalProperties": false, @@ -2847,6 +3033,7 @@ "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null, "share": null } }, @@ -2861,6 +3048,7 @@ "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null, "share": { "id": "shr_8Hf3kL9wQ2mNpXr6Tz1Vb", "token": "share-token-example", diff --git a/apps/docs/openapi-v2-knowledge.json b/apps/docs/openapi-v2-knowledge.json index 37a2238c614..01ae9282ec0 100644 --- a/apps/docs/openapi-v2-knowledge.json +++ b/apps/docs/openapi-v2-knowledge.json @@ -36,7 +36,7 @@ "get": { "operationId": "listKnowledgeBases", "summary": "List Knowledge Bases", - "description": "List knowledge bases in a workspace with folder filtering, search, sorting, and the canonical cursor envelope. The bounded workspace set is returned in one page with `nextCursor` always null; there is no second page to fetch. An unknown `folderPath` is a 404. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "List knowledge bases in a workspace with folder filtering, search, and sorting. Paginate with `limit` and `cursor`, stopping when `nextCursor` is null. An unknown `folderPath` is a 404. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -95,6 +95,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum knowledge bases to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum knowledge bases to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor returned by the previous page.", + "schema": { + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -271,6 +295,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -447,7 +474,7 @@ "post": { "operationId": "searchKnowledge", "summary": "Search Knowledge", - "description": "Search one or more knowledge bases with semantic vector retrieval, optional hybrid full-text retrieval, and structured tag filters. The request body is capped at 2 MiB; a larger body is a 413.", + "description": "Search one or more knowledge bases with semantic vector retrieval, optional hybrid full-text retrieval, and structured tag filters. Set `rerankerEnabled` with a `rerankerModel` to re-order the retrieved chunks with a reranking model before truncating to `topK`; reranked results carry a `rerankerScore` and are ordered by it, and reranking is billed as an additional search unit. Every result names the `knowledgeBaseId` it came from. The request body is capped at 2 MiB; a larger body is a 413.", "tags": ["Knowledge Bases"], "requestBody": { "required": true, @@ -512,11 +539,87 @@ } } }, + "/api/v2/knowledge/{id}/tags": { + "get": { + "operationId": "listKnowledgeTags", + "summary": "List Tags", + "description": "List the knowledge base's tag vocabulary: each tag's display name, the slot it is stored in, and its field type. Display names are what tag filters and the tag values on document reads use; slots are what document writes set. Every slot listed here is writable, in its declared type: `tag1`..`tag7` take a string, `number1`..`number5` a number, `date1`..`date2` a `YYYY-MM-DD` string, and `boolean1`..`boolean3` a boolean. The vocabulary is bounded by the fixed slot table. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch.", + "tags": ["Knowledge Bases"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique knowledge base identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique knowledge base identifier." + } + }, + { + "name": "workspaceId", + "in": "query", + "required": true, + "description": "Workspace that owns the knowledge base.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the knowledge base." + } + } + ], + "responses": { + "200": { + "description": "The knowledge base tag vocabulary.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2KnowledgeTagListResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/knowledge/{id}/documents": { "get": { "operationId": "listKnowledgeDocuments", "summary": "List Documents", - "description": "List documents in a knowledge base with filename search, state filtering, sorting, and opaque cursor pagination.", + "description": "List documents in a knowledge base with filename search, state filtering, tag filtering, sorting, and opaque cursor pagination. Each document carries its tag values keyed by tag display name; resolve those names to write slots with `GET /api/v2/knowledge/{id}/tags`.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -618,6 +721,19 @@ "type": "string", "minLength": 1 } + }, + { + "name": "tagFilters", + "in": "query", + "required": false, + "description": "A JSON-encoded array of at most 10 tag filters, using the same display-name shape as knowledge search: `[{\"tagName\":\"category\",\"operator\":\"eq\",\"value\":\"billing\"}]`. A name that is not defined in this knowledge base is rejected, never ignored.", + "schema": { + "description": "A JSON-encoded array of at most 10 tag filters, using the same display-name shape as knowledge search: `[{\"tagName\":\"category\",\"operator\":\"eq\",\"value\":\"billing\"}]`. A name that is not defined in this knowledge base is rejected, never ignored.", + "examples": [ + "[{\"tagName\":\"category\",\"operator\":\"eq\",\"value\":\"billing\"}]" + ], + "type": "string" + } } ], "responses": { @@ -648,6 +764,83 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + }, + "patch": { + "operationId": "bulkUpdateKnowledgeDocuments", + "summary": "Bulk Enable or Disable Documents", + "description": "Enable or disable many documents in one request, either by identifier (up to 100) or, with `selectAll`, every document in the knowledge base optionally narrowed by `enabledFilter`. Disabling keeps a document indexed but excludes it from search. Bulk delete is deliberately not offered: the bulk path records no audit entries, so deletions go through `DELETE /api/v2/knowledge/{id}/documents/{documentId}`, which audits each one. An identifier request echoes the documents it changed in `documentIds`; a `selectAll` request omits that field because the selection is unbounded, and reports `updatedCount` alone. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "tags": ["Knowledge Bases"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique knowledge base identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique knowledge base identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Operation and the documents it applies to.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BulkUpdateKnowledgeDocumentsRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The number and identifiers of the documents that changed.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2BulkKnowledgeDocumentsResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -1228,6 +1421,94 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + }, + "patch": { + "operationId": "updateKnowledgeDocument", + "summary": "Update Document", + "description": "Rename a document, enable or disable it for search, set any of its 17 tag slots, or requeue it for processing. A tag slot takes its declared type — a string for `tag1`..`tag7`, a number for `number1`..`number5`, a `YYYY-MM-DD` string for `date1`..`date2`, a boolean for `boolean1`..`boolean3` — and a value that is not valid for the slot is a `400` rather than a silently cleared tag. Resolve a display name to its slot with `GET /api/v2/knowledge/{id}/tags`. Absent fields are unchanged. Only caller-owned fields are accepted: derived indexing state (`chunkCount`, `tokenCount`, `characterCount`, `processingStatus`, `processingError`) is written by the processing pipeline and cannot be asserted here. `retryProcessing: true` re-queues a failed or stuck document and must be sent on its own — it runs instead of, not alongside, the field updates — and it answers with a queue acknowledgement rather than the document. Otherwise the updated document is returned; it omits the connector provenance the detail read carries, so re-read with GET when that is needed. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "tags": ["Knowledge Bases"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique knowledge base identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique knowledge base identifier." + } + }, + { + "name": "documentId", + "in": "path", + "required": true, + "description": "Unique knowledge document identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique knowledge document identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Filename, search state, tag slot values, or a processing retry.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/UpdateKnowledgeDocumentRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The updated document, or the requeue acknowledgement.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2UpdateKnowledgeDocumentResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -1673,7 +1954,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -1708,13 +1989,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -1759,7 +2040,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -1879,10 +2160,15 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", - "content": { - "application/json": { - "schema": { + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, + "content": { + "application/json": { + "schema": { "$ref": "#/components/schemas/V2Error" } } @@ -2264,6 +2550,11 @@ "V2KnowledgeSearchResult": { "type": "object", "properties": { + "knowledgeBaseId": { + "type": "string", + "description": "Knowledge base the matching chunk came from; a search may span up to 20.", + "examples": ["7c9e6679-7425-40de-944b-e07fc1f90ae7"] + }, "documentId": { "type": "string", "description": "Identifier of the document containing the matching chunk.", @@ -2324,9 +2615,15 @@ "type": "number", "description": "Similarity score for vector search; tag-only matches use 1.", "examples": [0.8423] + }, + "rerankerScore": { + "description": "Relevance score assigned by the reranker, present only on results a reranker ordered. Results are ordered by this score when it is present, which is why it can disagree with `similarity`.", + "examples": [0.9312], + "type": "number" } }, "required": [ + "knowledgeBaseId", "documentId", "documentName", "sourceUrl", @@ -2485,7 +2782,7 @@ "maximum": 100 }, "tagFilters": { - "description": "Structured tag filters. Supported across multiple knowledge bases, but each filtered tag must resolve to the same slot and field type in every knowledge base selected; a tag missing from one of them, or defined inconsistently across them, is rejected and those knowledge bases must be searched separately. With a single knowledge base, an unknown tag name is simply ignored.", + "description": "Structured tag filters. Supported across multiple knowledge bases, but each filtered tag must resolve to the same slot and field type in every knowledge base selected; a tag missing from one of them, or defined inconsistently across them, is rejected and those knowledge bases must be searched separately. A tag name defined in none of the selected knowledge bases is rejected, never ignored; list the available names with GET /api/v2/knowledge/{id}/tags.", "type": "array", "items": { "$ref": "#/components/schemas/V2KnowledgeSearchTagFilter" @@ -2503,13 +2800,79 @@ "type": "null" } ] + }, + "rerankerEnabled": { + "description": "Re-order retrieved chunks with a reranking model before truncating to `topK`. Ignored for a tag-only search, which has no query to rank against. Reranking is billed as an additional search unit.", + "type": "boolean" + }, + "rerankerModel": { + "description": "Reranking model to use; required for reranking to run.", + "type": "string", + "enum": ["rerank-v4.0-pro", "rerank-v4.0-fast", "rerank-v3.5"] + }, + "rerankerInputCount": { + "description": "How many candidate chunks to retrieve before reranking. Defaults to four times `topK`, capped at 100. A larger pool costs more retrieval work but gives the reranker more to choose from.", + "type": "integer", + "minimum": 1, + "maximum": 100 } }, "required": ["workspaceId", "knowledgeBaseIds"], "title": "Search knowledge request", "description": "Knowledge bases, query, result limit, retrieval mode, and optional tag filters." }, - "V2KnowledgeDocumentSummary": { + "V2KnowledgeTag": { + "type": "object", + "properties": { + "displayName": { + "type": "string", + "description": "Display name used by tag filters and by tag values on document reads.", + "examples": ["category"] + }, + "tagSlot": { + "type": "string", + "description": "Storage slot the tag occupies. Document writes set tag values by slot (`tag1`..`tag7`).", + "examples": ["tag1"] + }, + "fieldType": { + "type": "string", + "description": "Value type stored in the slot; it determines the valid filter operators.", + "examples": ["text"] + } + }, + "required": ["displayName", "tagSlot", "fieldType"], + "additionalProperties": false, + "title": "Knowledge tag", + "description": "A tag defined on a knowledge base, and the slot it is stored in." + }, + "V2KnowledgeTagListResponse": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "$ref": "#/components/schemas/V2KnowledgeTag" + }, + "description": "Items in the current page." + }, + "nextCursor": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + } + }, + "required": ["data", "nextCursor"], + "additionalProperties": false, + "title": "Knowledge tag list response", + "description": "The full tag vocabulary of one knowledge base." + }, + "V2KnowledgeTaggedDocument": { "type": "object", "properties": { "id": { @@ -2575,6 +2938,36 @@ "description": "ISO 8601 timestamp when the document was uploaded, or null.", "format": "date-time", "examples": ["2025-06-18T16:45:00Z"] + }, + "tags": { + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "description": "Tag value; dates are ISO 8601 strings and an unset tag is null." + }, + "description": "Document tag values keyed by tag display name. Writes address the same tags by slot (`tag1`..`tag7`); resolve names to slots with GET /api/v2/knowledge/{id}/tags.", + "examples": [ + { + "category": "billing", + "priority": 2 + } + ] } }, "required": [ @@ -2588,11 +2981,12 @@ "tokenCount", "characterCount", "enabled", - "createdAt" + "createdAt", + "tags" ], "additionalProperties": false, - "title": "Knowledge document summary", - "description": "Summary returned by document lists and upload acknowledgements." + "title": "Knowledge document list item", + "description": "Document summary with the document tag values keyed by display name." }, "V2KnowledgeDocumentListResponse": { "type": "object", @@ -2600,7 +2994,7 @@ "data": { "type": "array", "items": { - "$ref": "#/components/schemas/V2KnowledgeDocumentSummary" + "$ref": "#/components/schemas/V2KnowledgeTaggedDocument" }, "description": "Items in the current page." }, @@ -2621,6 +3015,178 @@ "title": "Knowledge document list response", "description": "A cursor-paginated page of knowledge documents." }, + "V2BulkKnowledgeDocumentsData": { + "type": "object", + "properties": { + "operation": { + "type": "string", + "enum": ["enable", "disable"], + "description": "Operation that was applied." + }, + "updatedCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991, + "description": "Number of documents the operation changed.", + "examples": [42] + }, + "documentIds": { + "description": "Identifiers of the documents the operation changed. Present only for an explicit `documentIds` request, which is bounded to 100 documents; a `selectAll` request omits it because the selection is unbounded, and reports `updatedCount` instead.", + "type": "array", + "items": { + "type": "string" + } + } + }, + "required": ["operation", "updatedCount"], + "additionalProperties": false, + "title": "Bulk knowledge document update data", + "description": "Outcome of a bulk enable or disable across knowledge documents." + }, + "V2BulkKnowledgeDocumentsResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/V2BulkKnowledgeDocumentsData" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Bulk knowledge document response", + "description": "Outcome of a bulk enable or disable." + }, + "BulkUpdateKnowledgeDocumentsRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the knowledge base." + }, + "operation": { + "type": "string", + "enum": ["enable", "disable"], + "description": "Whether the selected documents become enabled or disabled for search." + }, + "documentIds": { + "description": "Documents to update, by identifier.", + "minItems": 1, + "maxItems": 100, + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "selectAll": { + "description": "Update every document in the knowledge base instead of an explicit list, narrowed by `enabledFilter`.", + "type": "boolean", + "const": true + }, + "enabledFilter": { + "description": "With `selectAll`, restrict the update to documents in this state.", + "type": "string", + "enum": ["all", "enabled", "disabled"] + } + }, + "required": ["workspaceId", "operation"], + "additionalProperties": false, + "title": "Bulk knowledge document request", + "description": "Operation and the documents it applies to.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64", + "operation": "disable", + "documentIds": ["b2d4f8a0-1c3e-4a5b-9d7c-2e6f0a8b4c12"] + } + ] + }, + "V2KnowledgeDocumentSummary": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Unique document identifier.", + "examples": ["b2d4f8a0-1c3e-4a5b-9d7c-2e6f0a8b4c12"] + }, + "knowledgeBaseId": { + "type": "string", + "description": "Knowledge base to which the document belongs.", + "examples": ["7c9e6679-7425-40de-944b-e07fc1f90ae7"] + }, + "filename": { + "type": "string", + "description": "Original filename of the uploaded document.", + "examples": ["getting-started.pdf"] + }, + "fileSize": { + "type": "number", + "description": "File size in bytes.", + "examples": [248913] + }, + "mimeType": { + "type": "string", + "description": "MIME type of the document file.", + "examples": ["application/pdf"] + }, + "processingStatus": { + "type": "string", + "enum": ["pending", "processing", "completed", "failed"], + "description": "Current document processing state.", + "examples": ["completed"] + }, + "chunkCount": { + "type": "number", + "description": "Number of indexed chunks; zero until processing completes.", + "examples": [24] + }, + "tokenCount": { + "type": "number", + "description": "Total tokens extracted from the document.", + "examples": [8123] + }, + "characterCount": { + "type": "number", + "description": "Total characters extracted from the document.", + "examples": [41205] + }, + "enabled": { + "type": "boolean", + "description": "Whether the document is enabled for search.", + "examples": [true] + }, + "createdAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp when the document was uploaded, or null.", + "format": "date-time", + "examples": ["2025-06-18T16:45:00Z"] + } + }, + "required": [ + "id", + "knowledgeBaseId", + "filename", + "fileSize", + "mimeType", + "processingStatus", + "chunkCount", + "tokenCount", + "characterCount", + "enabled", + "createdAt" + ], + "additionalProperties": false, + "title": "Knowledge document summary", + "description": "Summary returned by document lists and upload acknowledgements." + }, "V2KnowledgeDocumentSummaryResponse": { "type": "object", "properties": { @@ -3107,6 +3673,36 @@ "format": "date-time", "examples": ["2025-06-18T16:45:00Z"] }, + "tags": { + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "description": "Tag value; dates are ISO 8601 strings and an unset tag is null." + }, + "description": "Document tag values keyed by tag display name. Writes address the same tags by slot (`tag1`..`tag7`); resolve names to slots with GET /api/v2/knowledge/{id}/tags.", + "examples": [ + { + "category": "billing", + "priority": 2 + } + ] + }, "processingError": { "anyOf": [ { @@ -3190,6 +3786,7 @@ "characterCount", "enabled", "createdAt", + "tags", "processingError", "processingStartedAt", "processingCompletedAt", @@ -3214,6 +3811,167 @@ "title": "Knowledge document response", "description": "Full knowledge document detail." }, + "V2KnowledgeDocumentProcessing": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Identifier of the requeued document." + }, + "queued": { + "type": "boolean", + "const": true, + "description": "Confirms that processing was requeued." + }, + "processingStatus": { + "type": "string", + "description": "Processing state the document was moved to.", + "examples": ["pending"] + }, + "message": { + "type": "string", + "description": "Human-readable outcome of the requeue." + } + }, + "required": ["id", "queued", "processingStatus", "message"], + "additionalProperties": false, + "title": "Knowledge document processing acknowledgement", + "description": "Acknowledgement returned when a document is requeued for processing." + }, + "V2UpdateKnowledgeDocumentResponse": { + "type": "object", + "properties": { + "data": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2KnowledgeTaggedDocument" + }, + { + "$ref": "#/components/schemas/V2KnowledgeDocumentProcessing" + } + ], + "description": "Response data." + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Update knowledge document response", + "description": "The updated document, or the processing requeue acknowledgement." + }, + "UpdateKnowledgeDocumentRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the knowledge base." + }, + "filename": { + "description": "New filename for the document.", + "examples": ["getting-started-v2.pdf"], + "type": "string", + "minLength": 1, + "maxLength": 255 + }, + "enabled": { + "description": "Whether the document participates in search. Disabling keeps it indexed.", + "type": "boolean" + }, + "tag1": { + "description": "New value for tag slot 1.", + "type": "string", + "maxLength": 1000 + }, + "tag2": { + "description": "New value for tag slot 2.", + "type": "string", + "maxLength": 1000 + }, + "tag3": { + "description": "New value for tag slot 3.", + "type": "string", + "maxLength": 1000 + }, + "tag4": { + "description": "New value for tag slot 4.", + "type": "string", + "maxLength": 1000 + }, + "tag5": { + "description": "New value for tag slot 5.", + "type": "string", + "maxLength": 1000 + }, + "tag6": { + "description": "New value for tag slot 6.", + "type": "string", + "maxLength": 1000 + }, + "tag7": { + "description": "New value for tag slot 7.", + "type": "string", + "maxLength": 1000 + }, + "number1": { + "description": "New value for number tag slot 1.", + "type": "number" + }, + "number2": { + "description": "New value for number tag slot 2.", + "type": "number" + }, + "number3": { + "description": "New value for number tag slot 3.", + "type": "number" + }, + "number4": { + "description": "New value for number tag slot 4.", + "type": "number" + }, + "number5": { + "description": "New value for number tag slot 5.", + "type": "number" + }, + "date1": { + "description": "New value for date tag slot 1, formatted YYYY-MM-DD.", + "type": "string", + "pattern": "^\\d{4}-\\d{2}-\\d{2}$" + }, + "date2": { + "description": "New value for date tag slot 2, formatted YYYY-MM-DD.", + "type": "string", + "pattern": "^\\d{4}-\\d{2}-\\d{2}$" + }, + "boolean1": { + "description": "New value for boolean tag slot 1.", + "type": "boolean" + }, + "boolean2": { + "description": "New value for boolean tag slot 2.", + "type": "boolean" + }, + "boolean3": { + "description": "New value for boolean tag slot 3.", + "type": "boolean" + }, + "retryProcessing": { + "description": "Requeue the document for processing. Send it alone: no other field may accompany it.", + "type": "boolean", + "const": true + } + }, + "required": ["workspaceId"], + "additionalProperties": false, + "title": "Update knowledge document request", + "description": "Filename, search state, tag slot values, or a processing retry.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64", + "enabled": false, + "tag1": "billing" + } + ] + }, "V2Folder": { "type": "object", "properties": { diff --git a/apps/docs/openapi-v2-logs.json b/apps/docs/openapi-v2-logs.json index d4df4c2b0df..9ca4c9cb168 100644 --- a/apps/docs/openapi-v2-logs.json +++ b/apps/docs/openapi-v2-logs.json @@ -85,20 +85,24 @@ "name": "startDate", "in": "query", "required": false, - "description": "Only include runs started at or after this ISO 8601 timestamp.", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected.", "schema": { "type": "string", - "description": "Only include runs started at or after this ISO 8601 timestamp." + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected." } }, { "name": "endDate", "in": "query", "required": false, - "description": "Only include runs started at or before this ISO 8601 timestamp.", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected.", "schema": { "type": "string", - "description": "Only include runs started at or before this ISO 8601 timestamp." + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected." } }, { @@ -187,33 +191,36 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum log entries per page, clamped to 1–1000.", + "description": "Maximum log entries per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "schema": { - "description": "Maximum log entries per page, clamped to 1–1000.", - "default": 100, - "type": "number" + "description": "Maximum log entries per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", + "type": "integer", + "minimum": 1, + "maximum": 1000, + "default": 100 } }, { "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by a previous page.", + "description": "Opaque cursor returned by the previous page.", "schema": { + "description": "Opaque cursor returned by the previous page.", "type": "string", - "description": "Opaque cursor returned by a previous page." + "minLength": 1 } }, { "name": "order", "in": "query", "required": false, - "description": "Sort order by execution start time.", + "description": "Sort direction by execution start time. This operation deviates from the v2 `sortBy` + `sortOrder` convention: logs are sortable only by start time, so the direction is carried by this single `order` param and `sortBy`/`sortOrder` are not accepted.", "schema": { "default": "desc", "type": "string", "enum": ["desc", "asc"], - "description": "Sort order by execution start time." + "description": "Sort direction by execution start time. This operation deviates from the v2 `sortBy` + `sortOrder` convention: logs are sortable only by start time, so the direction is carried by this single `order` param and `sortBy`/`sortOrder` are not accepted." } }, { @@ -356,7 +363,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -391,13 +398,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -442,7 +449,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -562,7 +569,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -649,7 +661,7 @@ "failed", "cancelled" ], - "description": "Current execution status. `redacting` is transient while run output is scrubbed. `paused` is reported when a resume attempt did not run to completion and the run is waiting to be resumed again." + "description": "Current execution status, reported as persisted. `redacting` is transient while run output is scrubbed. `paused` is reported only when a resume attempt did not run to completion and the run is waiting to be resumed again. **This differs from the run resources for the same run:** `GET /api/v2/workflows/{id}/runs` and `GET /api/v2/workflows/{id}/runs/{runId}` additionally report `paused` for a run held at a human-in-the-loop pause point, which this field reports as `pending`. Use the run resources when the pause state matters." }, "level": { "type": "string", @@ -1045,7 +1057,7 @@ "failed", "cancelled" ], - "description": "Current execution status. `redacting` is transient while run output is scrubbed. `paused` is reported when a resume attempt did not run to completion and the run is waiting to be resumed again." + "description": "Current execution status, reported as persisted. `redacting` is transient while run output is scrubbed. `paused` is reported only when a resume attempt did not run to completion and the run is waiting to be resumed again. **This differs from the run resources for the same run:** `GET /api/v2/workflows/{id}/runs` and `GET /api/v2/workflows/{id}/runs/{runId}` additionally report `paused` for a run held at a human-in-the-loop pause point, which this field reports as `pending`. Use the run resources when the pause state matters." }, "level": { "type": "string", diff --git a/apps/docs/openapi-v2-resources.json b/apps/docs/openapi-v2-resources.json index 920147e91ce..2f03ec3e899 100644 --- a/apps/docs/openapi-v2-resources.json +++ b/apps/docs/openapi-v2-resources.json @@ -139,10 +139,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum members to return. Defaults to 50 and cannot exceed 100.", + "description": "Maximum members to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum members to return. Defaults to 50 and cannot exceed 100.", + "description": "Maximum members to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -152,9 +152,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the preceding page.", + "description": "Opaque cursor returned by the previous page.", "schema": { - "description": "Opaque cursor returned by the preceding page.", + "description": "Opaque cursor returned by the previous page.", "type": "string", "minLength": 1 } @@ -210,7 +210,7 @@ "get": { "operationId": "listMcpServers", "summary": "List MCP Servers", - "description": "List MCP servers registered in a workspace. Request-header values and OAuth client secrets are never returned. The bounded workspace set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch.", + "description": "List MCP servers registered in a workspace. Request-header values and OAuth client secrets are never returned. Nothing caps how many servers a workspace registers, so this list is paginated: paginate with `limit` and `cursor`, stopping when `nextCursor` is null. `connectionStatus`, `toolCount`, `lastError`, and `lastToolsRefresh` describe the most recent tool discovery and stay at their registration defaults until one runs — call `GET /api/v2/mcp-servers/{id}/tools` to run it.", "tags": ["MCP Servers"], "parameters": [ { @@ -259,6 +259,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum MCP servers to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum MCP servers to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor returned by the previous page.", + "schema": { + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -382,11 +406,11 @@ "name": "id", "in": "path", "required": true, - "description": "MCP server to retrieve, update, or delete.", + "description": "MCP server the operation acts on.", "schema": { "type": "string", "minLength": 1, - "description": "MCP server to retrieve, update, or delete." + "description": "MCP server the operation acts on." } }, { @@ -456,11 +480,11 @@ "name": "id", "in": "path", "required": true, - "description": "MCP server to retrieve, update, or delete.", + "description": "MCP server the operation acts on.", "schema": { "type": "string", "minLength": 1, - "description": "MCP server to retrieve, update, or delete." + "description": "MCP server the operation acts on." } } ], @@ -530,11 +554,11 @@ "name": "id", "in": "path", "required": true, - "description": "MCP server to retrieve, update, or delete.", + "description": "MCP server the operation acts on.", "schema": { "type": "string", "minLength": 1, - "description": "MCP server to retrieve, update, or delete." + "description": "MCP server the operation acts on." } }, { @@ -595,11 +619,100 @@ } } }, + "/api/v2/mcp-servers/{id}/tools": { + "get": { + "operationId": "listMcpServerTools", + "summary": "List MCP Server Tools", + "description": "Connect to a registered MCP server and return the tools it exposes. Unlike most reads this one has side effects: it opens a live connection to the third-party server and writes `connectionStatus`, `toolCount`, `lastError`, and `lastToolsRefresh` on the server resource, so registering a server and then calling this completes onboarding without opening the Sim UI. Because the pass is not a safe read, a `HEAD` request is answered with an empty `200` without connecting or writing, so it reports only that the endpoint exists and the caller is authorized. Results are served from a short-lived per-workspace cache, so an uncached call reflects whichever workspace member last ran discovery; pass `refresh=true` to reconnect under your own credentials and pick up tools added since the last pass, at the cost of a live round trip to the server. The set is bounded by discovery itself — at most 1,000 tools and 5 MB of tool payload per server. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch. An unreachable, slow, or cooling-down server is a `503`; a server whose stored OAuth grant no longer works is a `409` with `error.details.code` `MCP_SERVER_REAUTHORIZATION_REQUIRED`, meaning the registration is intact but a human must reauthorize it in Sim — your API key is fine and re-issuing it changes nothing. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key. Discovery resolves the calling user's own OAuth credentials for the server, which a workspace key cannot supply — so a workspace key that can register a server cannot list its tools.", + "tags": ["MCP Servers"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "MCP server the operation acts on.", + "schema": { + "type": "string", + "minLength": 1, + "description": "MCP server the operation acts on." + } + }, + { + "name": "workspaceId", + "in": "query", + "required": true, + "description": "Workspace that owns the MCP server.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the MCP server." + } + }, + { + "name": "refresh", + "in": "query", + "required": false, + "description": "Bypass the cached tool list and reconnect to the server. Slower, and the only way to pick up a tool added since the last refresh.", + "schema": { + "description": "Bypass the cached tool list and reconnect to the server. Slower, and the only way to pick up a tool added since the last refresh.", + "type": "boolean" + } + } + ], + "responses": { + "200": { + "description": "Tools exposed by the MCP server.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListMcpServerToolsResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "409": { + "$ref": "#/components/responses/Conflict" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/skills": { "get": { "operationId": "listSkills", "summary": "List Skills", - "description": "List workspace and built-in skills. Built-ins are marked read-only. The list omits skill bodies and uses the standard cursor envelope with `nextCursor` always null, so there is no second page to fetch; fetch one skill to read its content.", + "description": "List workspace and built-in skills. Built-ins are marked read-only. The list omits skill bodies; fetch one skill to read its content. Paginate with `limit` and `cursor`, stopping when `nextCursor` is null.", "tags": ["Skills"], "parameters": [ { @@ -648,6 +761,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum skills to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum skills to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor returned by the previous page.", + "schema": { + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -991,7 +1128,7 @@ "get": { "operationId": "listCustomTools", "summary": "List Custom Tools", - "description": "List code-backed custom tools defined in a workspace. Legacy personal tools are excluded. The bounded workspace set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch.", + "description": "List code-backed custom tools defined in a workspace. Legacy personal tools are excluded. Paginate with `limit` and `cursor`, stopping when `nextCursor` is null.", "tags": ["Custom Tools"], "parameters": [ { @@ -1040,6 +1177,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum custom tools to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum custom tools to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor returned by the previous page.", + "schema": { + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -1383,7 +1544,7 @@ "get": { "operationId": "listCredentials", "summary": "List Credentials", - "description": "List OAuth and service-account connections visible to the caller. Secret material is never returned. Credential mutations and single-resource reads are intentionally not exposed. The bounded set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch.", + "description": "List OAuth and service-account connections visible to the caller. Secret material is never returned. Credential mutations and single-resource reads are intentionally not exposed. Paginate with `limit` and `cursor`, stopping when `nextCursor` is null.", "tags": ["Credentials"], "parameters": [ { @@ -1454,6 +1615,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum credentials to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum credentials to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor returned by the previous page.", + "schema": { + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -1506,7 +1691,7 @@ "get": { "operationId": "listSecrets", "summary": "List Secrets", - "description": "List workspace and caller-owned personal secret metadata. Only names, scope, role, and timestamps are returned; secret values are never read or returned. The bounded set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "List workspace and caller-owned personal secret metadata. Only names, scope, role, and timestamps are returned; secret values are never read or returned. Paginate with `limit` and `cursor`, stopping when `nextCursor` is null. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", "tags": ["Secrets"], "parameters": [ { @@ -1566,6 +1751,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum secrets to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum secrets to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor returned by the previous page.", + "schema": { + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -1807,7 +2016,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -1842,13 +2051,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -1893,7 +2102,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -2013,7 +2222,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -2769,6 +2983,118 @@ } ] }, + "V2McpTool": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Tool name, as the MCP server reports it." + }, + "description": { + "description": "Tool description reported by the server.", + "type": "string" + }, + "inputSchema": { + "type": "object", + "properties": { + "type": { + "type": "string", + "const": "object", + "description": "JSON Schema type of the argument object. MCP requires `object`." + }, + "properties": { + "description": "Argument schemas keyed by argument name.", + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "description": "Server-defined JSON Schema for one tool argument." + } + }, + "required": { + "description": "Names of the arguments the tool requires.", + "type": "array", + "items": { + "type": "string", + "description": "Name of a required argument." + } + }, + "description": { + "description": "Description of the argument object.", + "type": "string" + } + }, + "required": ["type"], + "additionalProperties": { + "description": "Additional JSON Schema keyword reported by the server." + }, + "description": "JSON Schema for the tool's arguments, as reported by the server." + }, + "serverId": { + "type": "string", + "description": "Identifier of the MCP server exposing the tool." + }, + "serverName": { + "type": "string", + "description": "Display name of the MCP server exposing the tool." + } + }, + "required": ["name", "inputSchema", "serverId", "serverName"], + "additionalProperties": false, + "title": "MCP tool", + "description": "A tool exposed by a registered MCP server." + }, + "ListMcpServerToolsResponse": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "$ref": "#/components/schemas/V2McpTool" + }, + "description": "Items in the current page." + }, + "nextCursor": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + } + }, + "required": ["data", "nextCursor"], + "additionalProperties": false, + "title": "List MCP server tools response", + "description": "Tools exposed by the MCP server.", + "examples": [ + { + "data": [ + { + "name": "search_docs", + "description": "Search the internal documentation", + "inputSchema": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "Search terms" + } + }, + "required": ["query"] + }, + "serverId": "mcp-3f7a9c21", + "serverName": "Docs server" + } + ], + "nextCursor": null + } + ] + }, "V2SkillSummary": { "type": "object", "properties": { diff --git a/apps/docs/openapi-v2-tables.json b/apps/docs/openapi-v2-tables.json index 20c7ec7bb62..64093eeed92 100644 --- a/apps/docs/openapi-v2-tables.json +++ b/apps/docs/openapi-v2-tables.json @@ -100,9 +100,9 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum tables to return (1-1000). Fractional or out-of-range values are truncated and clamped into that range rather than rejected.", + "description": "Maximum tables to return per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "schema": { - "description": "Maximum tables to return (1-1000). Fractional or out-of-range values are truncated and clamped into that range rather than rejected.", + "description": "Maximum tables to return per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "type": "integer", "minimum": 1, "maximum": 1000, @@ -113,9 +113,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor from the previous page.", + "description": "Opaque cursor returned by the previous page.", "schema": { - "description": "Opaque cursor from the previous page.", + "description": "Opaque cursor returned by the previous page.", "type": "string", "minLength": 1 } @@ -1455,6 +1455,85 @@ } } }, + "/api/v2/tables/{tableId}/query/count": { + "post": { + "operationId": "countTableRows", + "summary": "Count Rows", + "description": "Count the rows matching a typed predicate across the entire table. The paged reads carry no total, and rowCount on the table resource counts every row rather than the predicate matches. Omit the predicate to count the whole table. A predicate larger than the request-body ceiling is a `413`.", + "tags": ["Tables"], + "parameters": [ + { + "name": "tableId", + "in": "path", + "required": true, + "description": "Unique table identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique table identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Workspace scope and the optional predicate whose matches are counted.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/CountTableRowsRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The number of matching table rows.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2CountTableRowsResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/tables/{tableId}/views": { "get": { "operationId": "listTableViews", @@ -3587,7 +3666,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -3622,13 +3701,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -3673,7 +3752,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -3793,7 +3872,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -4167,6 +4251,10 @@ "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], "description": "Column data type." }, + "required": { + "description": "Whether inserts must supply a value for this column.", + "type": "boolean" + }, "unique": { "description": "Whether values in the column must be unique.", "type": "boolean" @@ -4445,6 +4533,10 @@ "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], "description": "Column data type." }, + "required": { + "description": "Whether inserts must supply a value for this column.", + "type": "boolean" + }, "unique": { "description": "Whether values in the column must be unique.", "type": "boolean" @@ -4536,6 +4628,10 @@ "type": "string", "enum": ["string", "number", "currency", "boolean", "date", "json", "select"] }, + "required": { + "description": "Whether inserts must supply a value for this column.", + "type": "boolean" + }, "unique": { "description": "Whether values in the column must be unique.", "type": "boolean" @@ -4607,6 +4703,7 @@ } }, "required": ["workspaceId", "columnName"], + "additionalProperties": false, "title": "Delete table column request", "description": "Workspace scope and column name to delete.", "examples": [ @@ -4768,7 +4865,8 @@ "description": "Rows to insert, with cells keyed by column name." } }, - "required": ["workspaceId", "rows"] + "required": ["workspaceId", "rows"], + "additionalProperties": false }, { "type": "object", @@ -4793,7 +4891,8 @@ "minLength": 1 } }, - "required": ["workspaceId", "data"] + "required": ["workspaceId", "data"], + "additionalProperties": false } ], "title": "Create table rows request", @@ -4864,6 +4963,7 @@ } }, "required": ["workspaceId", "filter", "data"], + "additionalProperties": false, "title": "Update table rows request", "description": "Workspace scope, typed predicate, and row-data patch." }, @@ -4940,6 +5040,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Delete table rows request", "description": "Workspace scope and exactly one of a predicate or row identifier list.", "examples": [ @@ -4976,6 +5077,7 @@ } }, "required": ["workspaceId", "data"], + "additionalProperties": false, "title": "Update table row request", "description": "Workspace scope and row-data patch keyed by column name.", "examples": [ @@ -5068,6 +5170,7 @@ } }, "required": ["workspaceId", "data"], + "additionalProperties": false, "title": "Upsert table row request", "description": "Workspace scope, row data, and optional unique-column conflict target.", "examples": [ @@ -5138,7 +5241,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, "limit": { @@ -5154,6 +5258,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Query table rows request", "description": "Workspace scope, optional predicate and sort, and cursor pagination controls.", "examples": [ @@ -5178,6 +5283,65 @@ } ] }, + "V2QueryRowsCountData": { + "type": "object", + "properties": { + "totalCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991, + "description": "Number of rows matching the predicate across the entire table." + } + }, + "required": ["totalCount"], + "additionalProperties": false, + "title": "Query rows count data", + "description": "Total number of table rows matching a predicate." + }, + "V2CountTableRowsResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/V2QueryRowsCountData" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Count table rows response", + "description": "The total number of table rows matching the predicate." + }, + "CountTableRowsRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Unique workspace identifier." + }, + "predicate": { + "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + } + }, + "required": ["workspaceId"], + "additionalProperties": false, + "title": "Count table rows request", + "description": "Workspace scope and the optional predicate whose matches are counted.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64", + "predicate": { + "all": [ + { + "field": "status", + "op": "eq", + "value": "active" + } + ] + } + } + ] + }, "V2ApiTableView": { "type": "object", "properties": { @@ -5442,7 +5606,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, { @@ -5451,10 +5616,12 @@ ] } }, + "additionalProperties": false, "description": "Saved filter, sort, and column-layout configuration." } }, "required": ["workspaceId", "name", "config"], + "additionalProperties": false, "title": "Create table view request", "description": "Workspace scope, name, and saved filter, sort, and layout configuration." }, @@ -5552,7 +5719,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, { @@ -5560,7 +5728,8 @@ } ] } - } + }, + "additionalProperties": false }, "configPatch": { "description": "Saved-view configuration fields to shallow-merge.", @@ -5630,7 +5799,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, { @@ -5638,7 +5808,8 @@ } ] } - } + }, + "additionalProperties": false }, "isDefault": { "description": "Whether to promote this view to the table default.", @@ -5646,6 +5817,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Update table view request", "description": "Workspace scope and one or more saved-view changes." }, @@ -6483,6 +6655,7 @@ } }, "required": ["workspaceId", "groupIds"], + "additionalProperties": false, "title": "Run table columns request", "description": "Workspace scope, producer groups, execution mode, and optional row scope.", "examples": [ @@ -6515,6 +6688,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Run row enrichment request", "description": "Workspace scope for the row enrichment.", "examples": [ @@ -6612,11 +6786,13 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } } }, "required": ["workspaceId", "q"], + "additionalProperties": false, "title": "Find table rows request", "description": "Workspace scope, substring query, and optional predicate and sort.", "examples": [ @@ -7619,6 +7795,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Create table export request", "description": "Workspace scope and export format.", "examples": [ @@ -7749,6 +7926,7 @@ } }, "required": ["workspaceId", "scope"], + "additionalProperties": false, "title": "Cancel table runs request", "description": "Workspace scope, cancellation scope, and optional predicate or producer groups.", "examples": [ diff --git a/apps/docs/openapi-v2-workflows.json b/apps/docs/openapi-v2-workflows.json index 722d9688c06..1d9df284ce8 100644 --- a/apps/docs/openapi-v2-workflows.json +++ b/apps/docs/openapi-v2-workflows.json @@ -78,11 +78,11 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum workflows to return per page.", + "description": "Maximum workflows to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum workflows to return per page.", - "type": "number", + "description": "Maximum workflows to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", "minimum": 1, "maximum": 100 } @@ -91,10 +91,11 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor returned by the previous page.", "schema": { - "description": "Opaque pagination cursor returned by a previous request.", - "type": "string" + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 } }, { @@ -496,10 +497,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum deployment versions to return per page.", + "description": "Maximum deployment versions to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum deployment versions to return per page.", + "description": "Maximum deployment versions to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -509,10 +510,11 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor returned by the previous page.", "schema": { - "description": "Opaque pagination cursor returned by a previous request.", - "type": "string" + "description": "Opaque cursor returned by the previous page.", + "type": "string", + "minLength": 1 } } ], @@ -640,6 +642,72 @@ } } }, + "/api/v2/workflows/{id}/deployment": { + "get": { + "operationId": "getWorkflowDeployment", + "summary": "Get Workflow Deployment", + "description": "Read the current deployment state of a workflow: whether a version is live, when it went live, the most recent deployment attempt with its readiness and failure payload, and whether the editable draft has since diverged from the live version. This is the only place `needsRedeployment` is published — the deploy, undeploy, and rollback responses cannot carry it, because they answer at the moment the draft and the live version are equal.", + "tags": ["Workflows"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique workflow identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique workflow identifier.", + "examples": ["3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36"] + } + } + ], + "responses": { + "200": { + "description": "The current deployment state.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/WorkflowDeploymentResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/workflows/{id}/deploy": { "post": { "operationId": "deployWorkflow", @@ -1213,34 +1281,34 @@ "name": "startDate", "in": "query", "required": false, - "description": "Include runs started at or after this ISO 8601 timestamp.", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected.", "schema": { - "description": "Include runs started at or after this ISO 8601 timestamp.", "type": "string", "format": "date-time", - "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$" + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected." } }, { "name": "endDate", "in": "query", "required": false, - "description": "Include runs started at or before this ISO 8601 timestamp.", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected.", "schema": { - "description": "Include runs started at or before this ISO 8601 timestamp.", "type": "string", "format": "date-time", - "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$" + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected." } }, { "name": "limit", "in": "query", "required": false, - "description": "Maximum workflow runs to return per page.", + "description": "Maximum workflow runs to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum workflow runs to return per page.", + "description": "Maximum workflow runs to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -1250,9 +1318,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor returned by the previous page.", "schema": { - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor returned by the previous page.", "type": "string", "minLength": 1 } @@ -1355,8 +1423,7 @@ "description": "Include final and block outputs when true.", "schema": { "description": "Include final and block outputs when true.", - "type": "string", - "enum": ["true", "false"] + "type": "boolean" } }, { @@ -1981,7 +2048,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings > API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -2016,13 +2083,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying. Sent on `429` (derived from the caller rate-limit window) and on `503` (a fixed transient-failure floor). Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -2067,7 +2134,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. Where the cause is one a caller can act on, `error.details.code` names it, drawn from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address.\nA resource in a workspace the caller cannot reach at all answers `404`, not `403`, so absence and denial are indistinguishable to a caller who was never entitled to tell them apart.", "content": { "application/json": { "schema": { @@ -2187,7 +2254,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. The condition is transient, so the response normally carries `Retry-After` with the number of seconds to wait; treat that value as a floor and add jitter before retrying. One case deliberately omits the header: when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, the run may already have started, so retrying could start and bill a second run. Reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -3061,6 +3133,123 @@ "title": "Deployment operation error", "description": "Failure details for a deployment lifecycle operation." }, + "WorkflowDeployment": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Unique workflow identifier.", + "examples": ["3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36"] + }, + "isDeployed": { + "type": "boolean", + "description": "Whether a workflow version is currently live and available for API execution." + }, + "deployedAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp associated with the deployment, or null when unavailable.", + "format": "date-time", + "examples": ["2026-06-12T10:30:00.000Z"] + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Non-fatal synchronization warnings. Empty when there is nothing to report." + }, + "activeDeployment": { + "anyOf": [ + { + "$ref": "#/components/schemas/ActiveDeploymentSummary" + }, + { + "type": "null" + } + ], + "description": "Currently live deployment version, or null while no version is active." + }, + "latestDeploymentAttempt": { + "anyOf": [ + { + "$ref": "#/components/schemas/DeploymentOperationSummary" + }, + { + "type": "null" + } + ], + "description": "Most recent deployment lifecycle attempt, or null when none is available." + }, + "needsRedeployment": { + "type": "boolean", + "description": "Whether the editable draft has diverged from the live deployment version. False while a deployment attempt is still preparing or activating, and false when nothing is deployed." + } + }, + "required": [ + "id", + "isDeployed", + "deployedAt", + "warnings", + "activeDeployment", + "latestDeploymentAttempt", + "needsRedeployment" + ], + "additionalProperties": false, + "title": "Workflow deployment", + "description": "Current deployment state of a workflow, including draft-versus-live drift and the most recent deployment attempt." + }, + "WorkflowDeploymentResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/WorkflowDeployment" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Workflow deployment response", + "description": "Current deployment state, including draft-versus-live drift.", + "examples": [ + { + "data": { + "id": "3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36", + "isDeployed": true, + "needsRedeployment": true, + "deployedAt": "2026-06-12T10:30:00.000Z", + "warnings": [], + "activeDeployment": { + "deploymentVersionId": "depver_01J8ZK3QW4M6X2R9T7B5C0V2", + "version": 3, + "deployedAt": "2026-06-12T10:30:00.000Z" + }, + "latestDeploymentAttempt": { + "id": "depop_01J8ZK3QW4M6X2R9T7B5C0V1", + "deploymentVersionId": "depver_01J8ZK3QW4M6X2R9T7B5C0V2", + "version": 3, + "action": "deploy", + "status": "active", + "isCurrent": true, + "readiness": { + "webhooks": "ready", + "schedules": "ready", + "mcp": "not_applicable" + }, + "requestedAt": "2026-06-12T10:29:58.000Z", + "activatedAt": "2026-06-12T10:30:00.000Z", + "error": null + } + } + } + ] + }, "DeployResult": { "type": "object", "properties": { diff --git a/apps/realtime/src/handlers/file-doc-store.ts b/apps/realtime/src/handlers/file-doc-store.ts index f7b8fc5180a..c806da5e6be 100644 --- a/apps/realtime/src/handlers/file-doc-store.ts +++ b/apps/realtime/src/handlers/file-doc-store.ts @@ -18,10 +18,12 @@ * client receives each update exactly once, from its own task's local broadcast — no adapter * amplification, and every task's doc stays converged. (Awareness/presence stay on the adapter: they * are ephemeral and need no convergence or replay.) - * - {@link attachRoom} does a synchronous catch-up read from the head of the stream when a task first - * opens a file, so a late-joining task (the normal case under autoscaling) loads the current shared - * state before its first client syncs. Catch-up + tail are seamless: the tailer resumes from the - * exact id catch-up stopped at. + * - {@link attachRoom} reads the stream from the head when a task first opens a file, and the relay + * AWAITS it before attaching a client, so a late-joining task (the normal case under autoscaling) + * holds the current shared state before its first client syncs — a client must never watch the + * catch-up land entry by entry, which is the document's edit history replaying on screen. Catch-up + + * tail are seamless: the tailer resumes from the exact id catch-up stopped at, and {@link catchUp} + * can re-run at any time for a caller that must converge without waiting on the tailer. * - The one-time seed is written via the atomic {@link seedIfEmpty} (append-iff-empty in one Redis * step), so exactly one task ever writes the seed cluster-wide (the fix for split-brain) — even if two * tasks race. {@link shouldSeed} is a Redis lock + empty-stream check layered on top ONLY as an @@ -185,6 +187,18 @@ function applyEntryToDoc( } } +/** + * Whether stream id `id` sorts after `than`. A Redis stream id is `-`, so a lexicographic + * compare is wrong the moment the millisecond part changes digit length (`'9999-0' > '10000-0'`); + * compare the two parts numerically instead. The initial `'0'` (nothing applied) has no `-seq` part, + * which reads as sequence 0 — before every real entry. + */ +function isAfterStreamId(id: string, than: string): boolean { + const [ms, seq = '0'] = id.split('-') + const [thanMs, thanSeq = '0'] = than.split('-') + return Number(ms) === Number(thanMs) ? Number(seq) > Number(thanSeq) : Number(ms) > Number(thanMs) +} + /** Whether a doc carries the seed flag (mirrors the relay's `isDocSeeded`), so the store can tell the * one-time seed transition from a real post-seed edit without re-implementing the check divergently. */ function isDocSeeded(doc: Y.Doc): boolean { @@ -260,10 +274,9 @@ export class FileDocStore { } /** - * Register a locally-opened room and load the shared state into its doc: read the whole stream from - * the head, apply every entry (origin {@link REDIS_ORIGIN}), and remember the last id so the tailer - * resumes exactly after it. A brand-new file has an empty stream and loads nothing (it is seeded - * shortly after, via {@link shouldSeed}). No-op when disabled. + * Register a locally-opened room and load the shared state into its doc ({@link catchUp}). A + * brand-new file has an empty stream and loads nothing (it is seeded shortly after, via + * {@link shouldSeed}). No-op when disabled. */ async attachRoom(name: string, doc: Y.Doc): Promise { if (!this.enabled || !this.write) return @@ -277,12 +290,31 @@ export class FileDocStore { realEdited: false, } this.rooms.set(name, room) + await this.catchUp(name) + } + + /** + * PULL the shared state into a registered room: read the stream and apply every entry the doc has + * not integrated yet (origin {@link REDIS_ORIGIN}), advancing `lastId` so the tailer resumes exactly + * after it. This is the ONLY way a room loads shared state, so a caller that must not depend on the + * tailer's asynchronous push — the join, which may not serve a client a half-assembled document — + * can converge on demand. Idempotent and safe to call repeatedly; no-op when disabled or the room is + * not registered (a fast open→close detached it). Never throws. + */ + async catchUp(name: string): Promise { + if (!this.enabled || !this.write) return + const room = this.rooms.get(name) + if (!room) return try { const entries = await this.write.xRange(streamKey(name), '-', '+') for (const entry of entries) { - // The room can be detached + its doc destroyed while catch-up is in flight (a fast open→close); - // stop touching it the moment that happens. + // The room can be detached + its doc destroyed while the read is in flight (a fast + // open→close); stop touching it the moment that happens. if (this.rooms.get(name) !== room) return + // Applying a Yjs update twice is a no-op, but `applyEntry`'s bookkeeping is not: re-applying + // the SEED after `seededObserved` latched would count it as a post-seed edit and let a + // compaction snapshot claim content no user ever typed. Skip what this room already holds. + if (!isAfterStreamId(entry.id, room.lastId)) continue this.applyEntry(room, entry.id, entry.message) } await this.write.expire(streamKey(name), STREAM_TTL_SEC) diff --git a/apps/realtime/src/handlers/file-doc.join-readiness.test.ts b/apps/realtime/src/handlers/file-doc.join-readiness.test.ts new file mode 100644 index 00000000000..9b7b6a1c7ec --- /dev/null +++ b/apps/realtime/src/handlers/file-doc.join-readiness.test.ts @@ -0,0 +1,305 @@ +/** + * @vitest-environment node + * + * The join's readiness contract, with the shared store ENABLED (`file-doc.test.ts` runs it disabled). + * + * A room loads its document from the file's Redis stream one entry at a time, into the same `Y.Doc` + * that fans every update out to the room. So a client attached while that is happening is not sent the + * document — it is sent the document's history, and it watches the history replay on screen (reload + * right after moving a block and the block moves again in front of you). These tests pin the fix: the + * join waits for the room to hold the whole document, so the client's first sync is authoritative. + */ +import { + FILE_DOC_EVENTS, + FILE_DOC_MESSAGE_TYPE, + FILE_DOC_SEED, +} from '@sim/realtime-protocol/file-doc' +import * as decoding from 'lib0/decoding' +import * as encoding from 'lib0/encoding' +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest' +import * as syncProtocol from 'y-protocols/sync' +import * as Y from 'yjs' +import type { IRoomManager } from '@/rooms' + +const { mockAuthorizeRoom, mockFetchFileDocSeed } = vi.hoisted(() => ({ + mockAuthorizeRoom: vi.fn(), + mockFetchFileDocSeed: vi.fn(), +})) + +vi.mock('@sim/platform-authz/rooms', () => ({ authorizeRoom: mockAuthorizeRoom })) + +vi.mock('@/handlers/file-doc-app', () => ({ + fetchFileDocSeed: mockFetchFileDocSeed, + fetchFileDocMerge: vi.fn(), + fetchFileDocPersist: vi.fn().mockResolvedValue({ status: 'persisted', version: 1 }), +})) + +/** One in-memory Redis backing per test — only the stream/lock ops the store actually uses. */ +const backing = vi.hoisted(() => ({ + streams: new Map }[]>(), + kv: new Map(), + seq: 0, + /** Ticks of event-loop delay each xRange takes, modelling a remote (cross-region) Redis. */ + readDelayTicks: 0, +})) + +const seqOf = (id: string) => Number(id.split('-')[0]) + +vi.mock('redis', () => { + const makeClient = (): Record => { + const client: Record = { + connect: async () => {}, + quit: async () => {}, + on: () => client, + duplicate: () => makeClient(), + xAdd: async (key: string, _star: string, fields: Record) => { + const id = `${++backing.seq}-0` + const arr = backing.streams.get(key) ?? [] + arr.push({ id, message: { ...fields } }) + backing.streams.set(key, arr) + return id + }, + xRange: async (key: string) => { + for (let i = 0; i < backing.readDelayTicks; i++) await Promise.resolve() + return (backing.streams.get(key) ?? []).map((e) => ({ ...e })) + }, + xLen: async (key: string) => (backing.streams.get(key) ?? []).length, + xRead: async (streams: { key: string; id: string }[]) => { + const res: { name: string; messages: { id: string; message: Record }[] }[] = + [] + for (const { key, id } of streams) { + const after = (backing.streams.get(key) ?? []).filter((e) => seqOf(e.id) > seqOf(id)) + if (after.length) res.push({ name: key, messages: after.map((e) => ({ ...e })) }) + } + if (res.length) return res + await new Promise((r) => setTimeout(r, 5)) + return null + }, + set: async (key: string, val: string, opts?: { NX?: boolean }) => { + if (opts?.NX && backing.kv.has(key)) return null + backing.kv.set(key, val) + return 'OK' + }, + eval: async (script: string, opts: { keys: string[]; arguments: string[] }) => { + const [key] = opts.keys + if (script.includes('xlen')) { + const [field, value] = opts.arguments + const arr = backing.streams.get(key) ?? [] + if (arr.length > 0) return 0 + arr.push({ id: `${++backing.seq}-0`, message: { [field]: value } }) + backing.streams.set(key, arr) + return 1 + } + const [token] = opts.arguments + if (backing.kv.get(key) === token) { + backing.kv.delete(key) + return 1 + } + return 0 + }, + expire: async () => 1, + get: async (key: string) => backing.kv.get(key) ?? null, + exists: async (key: string) => (backing.kv.has(key) ? 1 : 0), + } + return client + } + return { createClient: () => makeClient() } +}) + +import { cleanupFileDocForSocket, setupWorkspaceFileDocHandlers } from '@/handlers/file-doc' +import { getFileDocStore, initFileDocStore } from '@/handlers/file-doc-store' + +const FILE_ID = 'file-1' +const ROOM_NAME = `workspace-file-doc:${FILE_ID}` +const STREAM_KEY = `filedoc:stream:${ROOM_NAME}` +const FIELD = 'default' + +type Handler = (payload?: unknown) => Promise | void + +interface FakeSocket { + id: string + emit: (event: string, payload: unknown) => void + rooms: Set +} + +/** + * An `io` that actually DELIVERS: a room emit reaches every socket that joined that room, so a frame + * the relay fans out mid-assembly lands on the joiner's `emit` exactly as it would in the browser. + * Recording the emits without routing them would hide the very thing these tests are about. + */ +function createIo(sockets: FakeSocket[]) { + const emitTo = (target: string, except: string | null, event: string, payload: unknown) => { + for (const socket of sockets) { + if (socket.id === except || !socket.rooms.has(target)) continue + socket.emit(event, payload) + } + } + const to = vi.fn((target: string) => ({ + except: (exclude: string) => ({ + emit: (event: string, payload: unknown) => emitTo(target, exclude, event, payload), + }), + emit: (event: string, payload: unknown) => emitTo(target, null, event, payload), + })) + return { + to, + in: vi.fn(() => ({ socketsLeave: () => {} })), + local: { to }, + } as unknown as IRoomManager['io'] +} + +function setup(id: string, sockets: FakeSocket[]) { + const handlers: Record = {} + const rooms = new Set() + const socket = { + id, + userId: 'user-1', + userName: 'Test User', + userImage: 'avatar.png', + disconnected: false, + rooms, + on: vi.fn((event: string, handler: Handler) => { + handlers[event] = handler + }), + emit: vi.fn(), + join: vi.fn((name: string) => rooms.add(name)), + leave: vi.fn((name: string) => rooms.delete(name)), + } + sockets.push(socket as unknown as FakeSocket) + setupWorkspaceFileDocHandlers( + socket as unknown as Parameters[0], + { isReady: () => true, io: createIo(sockets) } as unknown as IRoomManager + ) + return { socket, handlers } +} + +/** Append a Yjs update to the file's stream, exactly as `publish`/`seedIfEmpty` would. */ +function appendToStream(update: Uint8Array): void { + const arr = backing.streams.get(STREAM_KEY) ?? [] + arr.push({ id: `${++backing.seq}-0`, message: { u: Buffer.from(update).toString('base64') } }) + backing.streams.set(STREAM_KEY, arr) +} + +/** + * A warm room's history: the seed, then a later edit — the "I moved a block, then reloaded" case. + * Returns the markdown-equivalent text of each state. + */ +function seedWarmStreamHistory(): { intermediate: string; final: string } { + const doc = new Y.Doc() + doc.getText(FIELD).insert(0, 'AAA') + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.flag, true) + appendToStream(Y.encodeStateAsUpdate(doc)) + const afterSeed = Y.encodeStateVector(doc) + doc.getText(FIELD).insert(0, 'BBB') + appendToStream(Y.encodeStateAsUpdate(doc, afterSeed)) + doc.destroy() + return { intermediate: 'AAA', final: 'BBBAAA' } +} + +/** Every document state this socket was ever shown, in order. */ +function statesDeliveredTo(socket: { emit: ReturnType }): string[] { + const clientDoc = new Y.Doc() + const states: string[] = [] + for (const [event, payload] of socket.emit.mock.calls) { + if (event !== FILE_DOC_EVENTS.MESSAGE || !(payload instanceof Uint8Array)) continue + const decoder = decoding.createDecoder(payload) + if (decoding.readVarUint(decoder) !== FILE_DOC_MESSAGE_TYPE.SYNC) continue + syncProtocol.readSyncMessage(decoder, encoding.createEncoder(), clientDoc, null) + const text = clientDoc.getText(FIELD).toString() + if (text !== (states.at(-1) ?? '')) states.push(text) + } + clientDoc.destroy() + return states +} + +/** Let anything the join left running (a catch-up, a seed) settle, so a frame it fans out afterwards + * is counted — that late delivery IS the replay these tests exist to rule out. */ +async function flushPendingWork(): Promise { + for (let i = 0; i < 20; i++) await Promise.resolve() +} + +/** Ask the server for its state the way a client does after the join ack. */ +function requestSyncStep2(handlers: Record): void { + const encoder = encoding.createEncoder() + encoding.writeVarUint(encoder, FILE_DOC_MESSAGE_TYPE.SYNC) + syncProtocol.writeSyncStep1(encoder, new Y.Doc()) + handlers[FILE_DOC_EVENTS.MESSAGE](encoding.toUint8Array(encoder)) +} + +describe('file-doc join readiness (shared store enabled)', () => { + /** Every socket the test created, so a room emit can be routed to its members. */ + const sockets: FakeSocket[] = [] + + // One store for the whole file: `initFileDocStore` is idempotent once enabled, so a per-test + // shutdown would leave every later test running against a store with closed clients. + beforeAll(async () => { + await initFileDocStore('redis://fake') + }) + + afterAll(async () => { + await getFileDocStore().shutdown() + }) + + beforeEach(() => { + vi.clearAllMocks() + backing.streams.clear() + backing.kv.clear() + backing.seq = 0 + backing.readDelayTicks = 0 + mockAuthorizeRoom.mockResolvedValue({ + allowed: true, + status: 200, + workspaceId: 'ws-1', + workspacePermission: 'write', + }) + mockFetchFileDocSeed.mockResolvedValue(null) + }) + + afterEach(() => { + cleanupFileDocForSocket('socket-1', createIo(sockets), true) + sockets.length = 0 + }) + + it('hands a joiner the final document, never the room history it was rebuilt from', async () => { + const { intermediate, final } = seedWarmStreamHistory() + // The catch-up read is not instantaneous — the case that made this visible is a cross-region Redis. + backing.readDelayTicks = 6 + const { socket, handlers } = setup('socket-1', sockets) + + await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: FILE_ID, clientId: 1 }) + requestSyncStep2(handlers) + await flushPendingWork() + + // One state, and it is the final one: the client never saw the pre-move document. + expect(statesDeliveredTo(socket)).toEqual([final]) + expect(statesDeliveredTo(socket)).not.toContain(intermediate) + }) + + it('does not fetch a seed for a room the stream can already reconstruct', async () => { + seedWarmStreamHistory() + const { handlers } = setup('socket-1', sockets) + + await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: FILE_ID, clientId: 1 }) + + expect(mockFetchFileDocSeed).not.toHaveBeenCalled() + }) + + it('pulls a seed another writer put in the stream instead of waiting for the tailer to push it', async () => { + // The seed lock is held by a writer whose room has since been dropped (a fast open→close on a + // freshly created file), and its seed lands in the stream. Waiting to be told about it is what + // left a new file un-editable until the client's readiness deadline lapsed; the join reads it. + backing.kv.set(`filedoc:seedlock:${ROOM_NAME}`, 'held-by-a-writer-that-is-gone') + const doc = new Y.Doc() + doc.getText(FIELD).insert(0, 'seeded by the writer that held the lock') + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.flag, true) + appendToStream(Y.encodeStateAsUpdate(doc)) + doc.destroy() + + const { socket, handlers } = setup('socket-1', sockets) + await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: FILE_ID, clientId: 1 }) + requestSyncStep2(handlers) + await flushPendingWork() + + expect(mockFetchFileDocSeed).not.toHaveBeenCalled() + expect(statesDeliveredTo(socket)).toEqual(['seeded by the writer that held the lock']) + }) +}) diff --git a/apps/realtime/src/handlers/file-doc.test.ts b/apps/realtime/src/handlers/file-doc.test.ts index 938092d4484..4ba34c83de1 100644 --- a/apps/realtime/src/handlers/file-doc.test.ts +++ b/apps/realtime/src/handlers/file-doc.test.ts @@ -570,22 +570,42 @@ describe('setupWorkspaceFileDocHandlers', () => { expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# From server') }) - it('seeds the document only once from the server across concurrent joiners of the same file', async () => { + it('seeds once across concurrent joiners, and every one of them waits for that seed', async () => { // Keep the first seed fetch IN FLIGHT so the doc is still unseeded when the second socket joins: - // that forces the dedup onto `serverSeedStarted` (the in-flight guard) rather than `isDocSeeded`. + // that forces the dedup onto the in-flight seed rather than `isDocSeeded`. Both joins must WAIT + // for it — a joiner answered before the seed would be handed an empty document and would then + // watch the content arrive as a live update. let resolveSeed: (v: { update: Uint8Array; version: number } | null) => void = () => {} mockFetchFileDocSeed.mockReturnValueOnce(new Promise((resolve) => (resolveSeed = resolve))) const { io } = createIo() const a = setup('socket-a', io) const b = setup('socket-b', io) - await a.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) - await b.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 2 }) - // Second join happened with the fetch still pending; only after this does the seed land. + const joinA = a.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + const joinB = b.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 2 }) + await flushMicrotasks() + + // The second join found the seed already in flight, so it does not start another one — and + // neither join has been answered yet. expect(mockFetchFileDocSeed).toHaveBeenCalledTimes(1) + expect(joinSuccessFileId(a.socket)).toBeUndefined() + expect(joinSuccessFileId(b.socket)).toBeUndefined() + resolveSeed(seedResult('# From server')) - await flushMicrotasks() + await Promise.all([joinA, joinB]) expect(mockFetchFileDocSeed).toHaveBeenCalledTimes(1) + + // The joiner that never triggered the fetch is served the seeded document all the same. + b.socket.emit.mockClear() + b.handlers[FILE_DOC_EVENTS.MESSAGE]( + frame(FILE_DOC_MESSAGE_TYPE.SYNC, (e) => syncProtocol.writeSyncStep1(e, new Y.Doc())) + ) + const reply = b.socket.emit.mock.calls.find( + ([event, payload]) => event === FILE_DOC_EVENTS.MESSAGE && payload instanceof Uint8Array + ) + const clientDoc = new Y.Doc() + applySyncReply(reply?.[1] as Uint8Array, clientDoc) + expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# From server') }) it('marks an empty/absent-file doc seeded so clients still reach readiness', async () => { @@ -640,43 +660,58 @@ describe('setupWorkspaceFileDocHandlers', () => { expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# Recovered') }) - it('does not seed a room that was dropped while the seed fetch was in flight', async () => { + it('does not seed a room the joiner abandoned while the seed fetch was in flight', async () => { let resolveSeed: (v: { update: Uint8Array; version: number } | null) => void = () => {} mockFetchFileDocSeed.mockReturnValueOnce(new Promise((resolve) => (resolveSeed = resolve))) const { io } = createIo() - const { handlers } = setup('socket-1', io) + const { socket, handlers } = setup('socket-1', io) - await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) - // The only owner leaves → the room (and its doc) is destroyed while the fetch is still pending. - cleanupFileDocForSocket('socket-1', io, true) + const joining = handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + await flushMicrotasks() + // The client leaves before the room finished assembling → the join aborts and drops the room it + // was preparing (nothing else owns it). + handlers[FILE_DOC_EVENTS.LEAVE]({ fileId: 'file-1' }) // Resolving now must not touch the destroyed doc or throw (liveness re-check after the await). resolveSeed(seedResult('# Too late')) - await expect(flushMicrotasks()).resolves.toBeUndefined() + await expect(joining).resolves.toBeUndefined() + expect(joinSuccessFileId(socket)).toBeUndefined() + expect(socket.join).not.toHaveBeenCalled() }) - it('still seeds when content was synced into the doc before the seed returned', async () => { - // Defensive: the guard is `isDocSeeded`, NOT doc-emptiness. In practice a fresh client never - // writes ahead of the seed (@tiptap/y-tiptap suppresses the empty-paragraph placeholder and real - // edits are readiness-gated), but even if some update landed content in the doc before the seed - // fetch resolved, the seed must still apply and set the flag — or the client's - // `synced && initialContentLoaded` gate would never open. + it('attaches a client only once the document is whole — no empty sync, no frames before it', async () => { + // The room assembles itself into the same doc that fans updates out to its room, so a socket + // attached mid-assembly receives the document's history rather than the document. Nothing about + // the client exists in the room until the seed has landed: no membership, no sync, and any frame + // it sends meanwhile is not applied. let resolveSeed: (v: { update: Uint8Array; version: number } | null) => void = () => {} mockFetchFileDocSeed.mockReturnValueOnce(new Promise((resolve) => (resolveSeed = resolve))) const { io } = createIo() const { socket, handlers } = setup('socket-1', io) - await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + const joining = handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + await flushMicrotasks() - // The client syncs a placeholder update — content in the doc, but no seed flag. - const placeholder = new Y.Doc() - placeholder.getText(FILE_DOC_FIELD).insert(0, 'x') + expect(socket.join).not.toHaveBeenCalled() + expect(joinSuccessFileId(socket)).toBeUndefined() + expect( + socket.emit.mock.calls.some( + ([event, payload]) => event === FILE_DOC_EVENTS.MESSAGE && payload instanceof Uint8Array + ) + ).toBe(false) + + // A document frame sent before the join was answered reaches an unbound socket and is dropped. + const early = new Y.Doc() + early.getText(FILE_DOC_FIELD).insert(0, 'too early') handlers[FILE_DOC_EVENTS.MESSAGE]( frame(FILE_DOC_MESSAGE_TYPE.SYNC, (e) => - syncProtocol.writeUpdate(e, Y.encodeStateAsUpdate(placeholder)) + syncProtocol.writeUpdate(e, Y.encodeStateAsUpdate(early)) ) ) + resolveSeed(seedResult('# Seeded')) - await flushMicrotasks() + await joining + expect(joinSuccessFileId(socket)).toBe('file-1') + // The first thing the client is served is the finished document — content and seed flag together. socket.emit.mockClear() handlers[FILE_DOC_EVENTS.MESSAGE]( frame(FILE_DOC_MESSAGE_TYPE.SYNC, (e) => syncProtocol.writeSyncStep1(e, new Y.Doc())) @@ -687,7 +722,7 @@ describe('setupWorkspaceFileDocHandlers', () => { const clientDoc = new Y.Doc() applySyncReply(reply?.[1] as Uint8Array, clientDoc) expect(clientDoc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.flag)).toBe(true) - expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toContain('# Seeded') + expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# Seeded') }) it('merges a copilot edit into a seeded live room and relays it to editors', async () => { diff --git a/apps/realtime/src/handlers/file-doc.ts b/apps/realtime/src/handlers/file-doc.ts index a0152fd85f6..1e29f476de9 100644 --- a/apps/realtime/src/handlers/file-doc.ts +++ b/apps/realtime/src/handlers/file-doc.ts @@ -131,9 +131,13 @@ interface FileDocRoom { /** socketId → (clientId → its presence ownership). A socket owns one entry per collaborative provider * it mounted for this file (see {@link FileDocOwner}); an empty inner map is never kept. */ owners: Map> - /** True once the server-side seed fetch has started, so concurrent joins don't each fetch. - * Reset on a fetch FAILURE so a later join can retry (a genuinely empty file stays empty). */ - serverSeedStarted: boolean + /** + * The in-flight server seed for this room, or `null`. Concurrent joins await THIS promise rather + * than each starting a fetch — and, unlike a "started" boolean, awaiting it is what lets a second + * joiner be served a document that is already seeded instead of an empty one. Cleared when it + * settles, so a failed seed is re-attempted by a later join (a genuinely empty file stays empty). + */ + seeding: Promise | null /** The workspace this file belongs to, captured at join — needed to persist back to markdown. */ workspaceId: string | null /** The last collaborator to edit here, for persist attribution (blob metadata) only. */ @@ -170,6 +174,17 @@ interface FileDocRoom { * {@link FileDocStore.isAgentStreaming} flag. `0` when no agent stream is active. */ agentStreamingUntil: number + /** + * Resolves once this room's doc reflects the file's shared stream (see {@link FileDocStore.catchUp}). + * Never rejects — the catch-up logs and gives up — so awaiting it can never fail a join. + */ + hydrated: Promise + /** + * How many joins are currently preparing this room. A room is created by the first join and has no + * owner until that join commits, so without this a concurrent last-leave would tear down the very + * document being assembled. A room with a join in flight is not idle. + */ + pendingJoins: number } /** Live documents keyed by Socket.IO room name. Module-global: one Y.Doc per file. */ @@ -360,7 +375,12 @@ async function flushPersist(name: string, room: FileDocRoom, final: boolean): Pr } if (result.status === 'persisted') { room.syncedVersion = Math.max(room.syncedVersion ?? 0, result.version) - void store.setSyncedVersion(name, result.version) + // AWAITED, unlike every other version write: the room's own copy dies with the room, so this + // cluster key is the only record that survives a teardown or a process restart. Fire-and-forget + // here means a task that exits in the moments after a write comes back holding a version older + // than the file's, and — since a conflict neither writes nor advances the token — never persists + // that document again. One round trip after a blob write is not a cost worth that. + await store.setSyncedVersion(name, result.version) return } // status === 'conflict': the durable file advanced out-of-band since our If-Match token. We do NOT @@ -412,6 +432,12 @@ function isDocSeeded(doc: Y.Doc): boolean { return doc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.flag) === true } +/** The identity of the document this doc holds ({@link FILE_DOC_SEED.docIdKey}), if it carries one. */ +function docIdOf(doc: Y.Doc): string | undefined { + const docId = doc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.docIdKey) + return typeof docId === 'string' ? docId : undefined +} + /** * Decode the client IDs an awareness update carries, without applying it, to * check a frame only touches its sender's own presence. Mirrors the wire format @@ -435,10 +461,13 @@ function awarenessUpdateClientIds(update: Uint8Array): number[] { * memory. Before dropping, flush the converged doc back to durable markdown (the last collaborator on * this task leaving) and detach from the shared stream. A later joiner re-creates it — catching up * from the stream if the doc is still live on another task, or re-seeding from markdown otherwise. + * + * A room being PREPARED for a join is not idle even though it has no owners yet: tearing it down there + * would drop the hydration/seed that join is waiting on, and the join would have to start over. */ function destroyRoomIfIdle(name: string) { const room = fileDocRooms.get(name) - if (!room || room.owners.size > 0) return + if (!room || room.owners.size > 0 || room.pendingJoins > 0) return room.persistDeadline = null if (room.persistTimer) { clearTimeout(room.persistTimer) @@ -469,40 +498,83 @@ export async function flushAllFileDocRooms(): Promise { } /** - * Seed a room's document server-side, once, on the first join: ask the app to build the seed (the - * file's current markdown → Yjs, through the exact editor engine) and apply it, which relays the - * content to every connected client via `doc.on('update')`. No client is elected to import content. + * Bring a room's document to its AUTHORITATIVE state — reflecting the file's shared stream and + * carrying its seed — so the join can attach a client to a document that is already whole. Never + * rejects: a room that cannot be seeded is served unseeded, which the client's readiness deadline + * turns into its read-only fallback, exactly as an unreachable relay does. + */ +async function ensureRoomReady( + name: string, + room: FileDocRoom, + workspaceId: string | null +): Promise { + await room.hydrated + // The room can be dropped and re-created while the catch-up is in flight (a fast open→close); the + // join re-checks identity after this and abandons a stale room rather than serving from it. + if (fileDocRooms.get(name) !== room || !workspaceId) return + await ensureServerSeed(name, room, workspaceId) +} + +/** + * Seed a room's document server-side, once: ask the app to build the seed (the file's current markdown + * → Yjs, through the exact editor engine) and apply it. No client is elected to import content. + * + * MEMOIZED on the room, so concurrent joins await the same seed instead of the second one being served + * an empty document while the first one's fetch is still in flight. Cleared when it settles: a failed + * seed is re-attempted by the next join (a genuinely empty file stays empty and needs no retry). * * `isDocSeeded` is the sufficient guard: content only ever reaches the doc alongside the seed flag * (this seed, or a client's offline fallback), so an unseeded doc is genuinely empty and safe to seed. * A genuinely empty/missing file returns `null` (a read error throws instead), so still set the flag — - * an empty doc must reach readiness, not wait forever. After the fetch, re-check the room is still - * live and unseeded (an owner may have left, or a client seeded it, while the fetch was in flight). - * - * Recovery on failure is deliberately simple — no in-room retry loop: a single attempt bounded by a - * timeout shorter than the client's readiness deadline, then release the guard. A transient failure - * is re-attempted by the next join/reconnect; a persistent one lets the connected client's readiness - * deadline lapse into its read-only fallback. (An in-room backoff retry can outlast that client - * deadline, so it would keep trying a doc the client has already given up on — worse, not better.) + * an empty doc must reach readiness, not wait forever. */ -async function ensureServerSeed( +function ensureServerSeed(name: string, room: FileDocRoom, workspaceId: string): Promise { + if (isDocSeeded(room.doc)) return Promise.resolve() + room.seeding ??= runServerSeed(name, room, workspaceId).finally(() => { + room.seeding = null + }) + return room.seeding +} + +/** + * Whichever task wins the seed lock writes the seed; the others must end up holding the SAME seed + * before they serve anyone. They pull it, on this cadence, rather than waiting for the tailer to push + * it: a join's readiness may not depend on an asynchronous subscriber, because when that delivery is + * late or lost the client sits on an empty document until its readiness deadline lapses and the file + * opens read-only. Bounded by the longest a legitimate seed can take (the winner's own fetch bound), + * which stays inside the client's readiness deadline — see {@link FILE_DOC_TIMEOUTS}. + */ +const SEED_WAIT_RETRY_MS = 150 + +async function runServerSeed(name: string, room: FileDocRoom, workspaceId: string): Promise { + const store = getFileDocStore() + const deadline = Date.now() + FILE_DOC_TIMEOUTS.seedRequestMs + while (fileDocRooms.get(name) === room && !isDocSeeded(room.doc)) { + // Exactly one task across the cluster builds the seed; the others receive it via the stream (the + // fix for split-brain seeding). Returns a lock token here (single-pod: a sentinel token). + const token = await store.shouldSeed(name) + if (token) { + await seedUnderLock(name, room, workspaceId, token) + return + } + // No token: a peer holds the lock with its fetch in flight, or the stream is already seeded (which + // includes a PRIOR room for this same file whose seed landed after we read the stream). Either way + // the seed can only appear in the stream, so read it rather than wait to be told. + await store.catchUp(name) + if (isDocSeeded(room.doc) || Date.now() >= deadline) return + await sleep(SEED_WAIT_RETRY_MS) + } +} + +/** Fetch, publish, and apply the seed while holding the cluster's seed lock for this file. */ +async function seedUnderLock( name: string, room: FileDocRoom, - workspaceId: string + workspaceId: string, + token: string ): Promise { - if (room.serverSeedStarted || isDocSeeded(room.doc)) return - room.serverSeedStarted = true const store = getFileDocStore() - // Exactly one task across the cluster builds the seed; the others receive it via the stream (the fix - // for split-brain seeding). Returns a lock token here (single-pod: a sentinel token). - const token = await store.shouldSeed(name) - if (!token) { - // A peer is seeding (or already did). Release our guard so a later join can retry if the seed never - // arrives (e.g. the seeder died); the stream / this doc being seeded makes a retry safe. - room.serverSeedStarted = false - return - } - // We hold the seed lock — release it on EVERY exit from here (one `finally`, impossible to leak). + // Release the lock on EVERY exit from here (one `finally`, impossible to leak). try { const seed = await fetchFileDocSeed(workspaceId, room.fileId) if (fileDocRooms.get(name) !== room || isDocSeeded(room.doc)) return @@ -533,15 +605,13 @@ async function ensureServerSeed( if (didSeed) { Y.applyUpdate(room.doc, seedUpdate, SEED_ORIGIN) } else { - // A peer seeded first: its seed arrives via the tailer, so we must NOT apply our own — a second, - // different-client-id seed IS the split-brain. Clear the guard so a later join can retry if that - // peer seed somehow never lands (e.g. a fail-closed `xLen` error made `shouldSeed` skip a genuinely - // empty stream); a real peer-seed makes the retry a no-op. - room.serverSeedStarted = false + // A peer won the atomic append: we must NOT apply our own — a second, different-client-id seed IS + // the split-brain. Read THEIRS out of the stream instead of waiting for the tailer to deliver it, + // so this room is seeded by the time the caller is told it is ready. + await store.catchUp(name) } } catch (error) { logger.warn(`Server seed failed for file ${room.fileId} (workspace ${workspaceId})`, error) - room.serverSeedStarted = false } finally { await store.releaseSeedLock(name, token) } @@ -725,12 +795,14 @@ function getOrCreateRoom(io: Server, ref: RoomRef): FileDocRoom { // The server holds no cursor of its own; it only relays clients' awareness. awareness.setLocalState(null) + // Started BEFORE the room is registered so no join can observe a room without its hydration handle. + const hydrated = getFileDocStore().attachRoom(name, doc) const room: FileDocRoom = { fileId: ref.id, doc, awareness, owners: new Map(), - serverSeedStarted: false, + seeding: null, workspaceId: null, lastEditorUserId: null, edited: false, @@ -739,6 +811,8 @@ function getOrCreateRoom(io: Server, ref: RoomRef): FileDocRoom { persistDeadline: null, syncedVersion: null, agentStreamingUntil: 0, + hydrated, + pendingJoins: 0, } // Register synchronously BEFORE the async catch-up so a concurrent join sees this room, not a second. fileDocRooms.set(name, room) @@ -818,10 +892,6 @@ function getOrCreateRoom(io: Server, ref: RoomRef): FileDocRoom { broadcast(io, name, encoding.toUint8Array(encoder), originSocketId(origin)) }) - // Load the shared state into the doc and start tailing the stream (fire-and-forget: content streams - // in via `doc.on('update')` as it lands, mirroring the fire-and-forget seed below). Disabled → no-op. - void getFileDocStore().attachRoom(name, doc) - return room } @@ -1091,117 +1161,143 @@ export function setupWorkspaceFileDocHandlers( // awareness). Resolved here so the generation guard below also covers this await. const avatarUrl = await resolveAvatarUrl(socket, userId) - // Re-check access immediately before registering, mirroring the workflow join: the - // access re-validation sweep records a revocation BEFORE it evicts, so a join that - // authorized just before the revocation must not complete afterwards and re-bind - // the socket to the document. This RE-RESOLVES rather than peeking the cache — a - // peek treats an expired entry as unknown and fails open, which a join stalled - // longer than the cache TTL would slip straight through. Normally a cache hit (this - // join's own authorize just warmed it), so it costs no extra query. - const currentPermission = await resolveCurrentRoomPermission(userId, room, FILE_DOC_ACTION) - if (!satisfiesRoomMembership(currentPermission, ROOM_TYPES.WORKSPACE_FILE_DOC)) { - logger.warn(`User ${userId} lost write access to file ${fileId} before the join completed`) - emitJoinError(socket, fileId, 'Access denied to file', 'ACCESS_DENIED', false) - return - } - - // Abort a JOIN superseded during authorization/identity resolution: the socket - // disconnected, or a newer JOIN (a document switch) bumped the generation. Registering - // here would leak a dead socket's room or bind the socket to the wrong document. - // Last await before the commit, so nothing can interleave between the access - // re-check above and the registration below. - if (socket.disconnected || joinGeneration.get(socket.id) !== generation) return - const entry = getOrCreateRoom(io, room) + // The workspace the server-side persist writes back to — and what the seed is built from, so it + // must be captured BEFORE the room is prepared below. + if (authorized.workspaceId) entry.workspaceId = authorized.workspaceId - // A client id must be owned by at most one user, or a peer could bind an active - // collaborator's id and pass the per-frame ownership check to spoof/clear its caret. - // Distinguish a reconnect from a spoof by the owning user: the same user reclaiming its - // own client id (a dropped socket reconnecting reuses the Yjs client id, and its prior - // socket may not be cleaned up yet) takes over the stale binding; a DIFFERENT user is - // rejected. This runs BEFORE any teardown of the socket's current binding below, so a - // rejected rebind — even during a document switch — leaves the socket's existing document - // and caret untouched. - for (const [otherSid, clientMap] of entry.owners) { - if (otherSid === socket.id) continue - const owner = clientMap.get(clientId) - if (owner === undefined) continue - if (owner.userId !== userId) { - emitJoinError(socket, fileId, 'Client id already in use', 'CLIENT_ID_IN_USE', false) + // Hold the room open across the awaits below: it has no owner until this join commits, so a + // concurrent last-leave would otherwise tear down the very document being prepared. + entry.pendingJoins += 1 + try { + // A client is attached to a WHOLE document or to nothing. A room assembles itself from the + // shared stream and the server seed, and both land in the same Y.Doc that fans every update out + // to its room — so a socket attached mid-assembly is not sent the document, it is sent the + // document's history, and it watches that replay on screen (reload right after moving a block + // and the block moves again in front of you). Waiting here is what makes the handshake below + // authoritative: the client's first sync IS the finished document, in one message. + await ensureRoomReady(name, entry, entry.workspaceId) + + // Re-check access immediately before registering, mirroring the workflow join: the + // access re-validation sweep records a revocation BEFORE it evicts, so a join that + // authorized just before the revocation must not complete afterwards and re-bind + // the socket to the document. This RE-RESOLVES rather than peeking the cache — a + // peek treats an expired entry as unknown and fails open, which a join stalled + // longer than the cache TTL would slip straight through. Normally a cache hit (this + // join's own authorize just warmed it), so it costs no extra query. + const currentPermission = await resolveCurrentRoomPermission(userId, room, FILE_DOC_ACTION) + if (!satisfiesRoomMembership(currentPermission, ROOM_TYPES.WORKSPACE_FILE_DOC)) { + logger.warn( + `User ${userId} lost write access to file ${fileId} before the join completed` + ) + emitJoinError(socket, fileId, 'Access denied to file', 'ACCESS_DENIED', false) return } - // Same user reclaiming its client id on a stale prior socket: evict just THAT clientID's binding - // + caret from the old socket. If that leaves the old socket with no providers, also drop its - // room mapping + Socket.IO membership so it can no longer send document (sync) frames - // (handleMessage's SYNC path gates on socketToRoomName, not owners); an old socket that still - // hosts OTHER providers keeps them. Done inline rather than via cleanupFileDocForSocket, which - // could destroyRoomIfIdle the room we're joining. - clientMap.delete(clientId) - awarenessProtocol.removeAwarenessStates(entry.awareness, [clientId], null) - if (clientMap.size === 0) { - entry.owners.delete(otherSid) - socketToRoomName.delete(otherSid) - io.in(otherSid).socketsLeave(name) - } - } - // Only now that the rebind is guaranteed to succeed, leave a previously-joined document if - // switching (a socket edits at most one). A duplicate join of the SAME room falls through - // and simply re-runs the sync handshake, idempotently. - const currentName = socketToRoomName.get(socket.id) - if (currentName && currentName !== name) { - socket.leave(currentName) - cleanupFileDocForSocket(socket.id, io) - } + // Abort a JOIN superseded while the room was being prepared: the socket disconnected, a newer + // JOIN (a document switch) bumped the generation, or the room was dropped and re-created. + // Registering here would leak a dead socket's room, bind the socket to the wrong document, or + // attach it to a doc no longer registered. Last await before the commit, so nothing can + // interleave between the access re-check above and the registration below. + if ( + socket.disconnected || + joinGeneration.get(socket.id) !== generation || + fileDocRooms.get(name) !== entry + ) + return - // ADD this provider's clientID to the socket's ownership set (do NOT overwrite a sibling provider - // on the same socket — that lone-owner overwrite is exactly what dropped the chat preview's - // awareness when the Files editor co-mounted). A re-JOIN of the same clientID is idempotent. A - // single provider that later unmounts clears its own caret via its awareness removal; the whole - // set is dropped on the socket's LEAVE/disconnect (client emits LEAVE only after its LAST provider - // for the file tears down). - let clientMap = entry.owners.get(socket.id) - if (clientMap === undefined) { - clientMap = new Map() - entry.owners.set(socket.id, clientMap) - } - clientMap.set(clientId, { clientId, userId, userName, avatarUrl }) - socketToRoomName.set(socket.id, name) - socket.join(name) + // A client id must be owned by at most one user, or a peer could bind an active + // collaborator's id and pass the per-frame ownership check to spoof/clear its caret. + // Distinguish a reconnect from a spoof by the owning user: the same user reclaiming its + // own client id (a dropped socket reconnecting reuses the Yjs client id, and its prior + // socket may not be cleaned up yet) takes over the stale binding; a DIFFERENT user is + // rejected. This runs BEFORE any teardown of the socket's current binding below, so a + // rejected rebind — even during a document switch — leaves the socket's existing document + // and caret untouched. + for (const [otherSid, clientMap] of entry.owners) { + if (otherSid === socket.id) continue + const owner = clientMap.get(clientId) + if (owner === undefined) continue + if (owner.userId !== userId) { + emitJoinError(socket, fileId, 'Client id already in use', 'CLIENT_ID_IN_USE', false) + return + } + // Same user reclaiming its client id on a stale prior socket: evict just THAT clientID's + // binding + caret from the old socket. If that leaves the old socket with no providers, also + // drop its room mapping + Socket.IO membership so it can no longer send document (sync) frames + // (handleMessage's SYNC path gates on socketToRoomName, not owners); an old socket that still + // hosts OTHER providers keeps them. Done inline rather than via cleanupFileDocForSocket, which + // could destroyRoomIfIdle the room we're joining. + clientMap.delete(clientId) + awarenessProtocol.removeAwarenessStates(entry.awareness, [clientId], null) + if (clientMap.size === 0) { + entry.owners.delete(otherSid) + socketToRoomName.delete(otherSid) + io.in(otherSid).socketsLeave(name) + } + } - // Capture what the server-side persist needs: the workspace to write back to, and the current - // user for attribution (refreshed to the actual editor on each edit in `handleMessage`). - if (authorized.workspaceId) entry.workspaceId = authorized.workspaceId - entry.lastEditorUserId = userId - - socket.emit(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId }) - // Server-authenticated roster → everyone in the room, including this joiner. - broadcastFileDocPresence(io, name, entry) - - // Begin the sync handshake: send the server's state (sync step 1). The - // client replies with its updates and requests the server's in return. - const syncEncoder = encoding.createEncoder() - encoding.writeVarUint(syncEncoder, FILE_DOC_MESSAGE_TYPE.SYNC) - syncProtocol.writeSyncStep1(syncEncoder, entry.doc) - socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(syncEncoder)) - - // Send existing awareness so the new client immediately sees others' carets. - const states = entry.awareness.getStates() - if (states.size > 0) { - const awarenessEncoder = encoding.createEncoder() - encoding.writeVarUint(awarenessEncoder, FILE_DOC_MESSAGE_TYPE.AWARENESS) - encoding.writeVarUint8Array( - awarenessEncoder, - awarenessProtocol.encodeAwarenessUpdate(entry.awareness, Array.from(states.keys())) - ) - socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(awarenessEncoder)) - } + // Only now that the rebind is guaranteed to succeed, leave a previously-joined document if + // switching (a socket edits at most one). A duplicate join of the SAME room falls through + // and simply re-runs the sync handshake, idempotently. + const currentName = socketToRoomName.get(socket.id) + if (currentName && currentName !== name) { + socket.leave(currentName) + cleanupFileDocForSocket(socket.id, io) + } - // Seed the document server-side (once). Fire-and-forget: the join completes immediately and - // the seed relays to this socket via `doc.on('update')` the moment it lands. - if (authorized.workspaceId) void ensureServerSeed(name, entry, authorized.workspaceId) + // ADD this provider's clientID to the socket's ownership set (do NOT overwrite a sibling + // provider on the same socket — that lone-owner overwrite is exactly what dropped the chat + // preview's awareness when the Files editor co-mounted). A re-JOIN of the same clientID is + // idempotent. A single provider that later unmounts clears its own caret via its awareness + // removal; the whole set is dropped on the socket's LEAVE/disconnect (the client emits LEAVE + // only after its LAST provider for the file tears down). + let clientMap = entry.owners.get(socket.id) + if (clientMap === undefined) { + clientMap = new Map() + entry.owners.set(socket.id, clientMap) + } + clientMap.set(clientId, { clientId, userId, userName, avatarUrl }) + socketToRoomName.set(socket.id, name) + socket.join(name) + + // Attribution for the server-side persist, refreshed to the actual editor on each edit in + // `handleMessage`. + entry.lastEditorUserId = userId + + // Name the document this room holds, so a client that still carries a DIFFERENT one (its room + // outlived by a document rebuilt in its place) can refuse to merge instead of unioning two + // documents into the file twice over. Read after readiness — before it, the room has no doc yet. + socket.emit(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId, docId: docIdOf(entry.doc) }) + // Server-authenticated roster → everyone in the room, including this joiner. + broadcastFileDocPresence(io, name, entry) + + // Begin the sync handshake: send the server's state (sync step 1). The + // client replies with its updates and requests the server's in return. + const syncEncoder = encoding.createEncoder() + encoding.writeVarUint(syncEncoder, FILE_DOC_MESSAGE_TYPE.SYNC) + syncProtocol.writeSyncStep1(syncEncoder, entry.doc) + socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(syncEncoder)) + + // Send existing awareness so the new client immediately sees others' carets. + const states = entry.awareness.getStates() + if (states.size > 0) { + const awarenessEncoder = encoding.createEncoder() + encoding.writeVarUint(awarenessEncoder, FILE_DOC_MESSAGE_TYPE.AWARENESS) + encoding.writeVarUint8Array( + awarenessEncoder, + awarenessProtocol.encodeAwarenessUpdate(entry.awareness, Array.from(states.keys())) + ) + socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(awarenessEncoder)) + } - logger.info(`User ${userId} joined file-doc room ${fileId}`) + logger.info(`User ${userId} joined file-doc room ${fileId}`) + } finally { + entry.pendingJoins -= 1 + // A join that returned without registering may have left behind the room it created; drop it + // if nothing else claimed it. A no-op once this join committed (the room then has an owner). + destroyRoomIfIdle(name) + } } catch (error) { logger.error('Error joining file-doc room:', error) try { diff --git a/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx b/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx index 0043641a3e0..37c883552dd 100644 --- a/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx +++ b/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx @@ -1,7 +1,11 @@ +import { WORDMARK_PATHS, WORDMARK_VIEW_BOX } from '@/lib/branding/wordmark' + /** * Inline "sim" brand logotype (wordmark, no separate icon mark) - the paths * from the v1.0 brand guide's `simLogotype--dark.svg`, inlined so the logo - * ships as zero-request server-rendered HTML. + * ships as zero-request server-rendered HTML. They live in + * `@/lib/branding/wordmark` because the email header rasterizes the same + * outlines. * * Filled with a single solid `var(--text-body)` - the navbar's own text color * (the same token its nav-link chips use) - so the wordmark reads as one solid @@ -14,7 +18,7 @@ export function SimWordmark() { return ( - - - - + {WORDMARK_PATHS.map((d) => ( + + ))} ) diff --git a/apps/sim/app/_shell/providers/get-query-client.ts b/apps/sim/app/_shell/providers/get-query-client.ts index 681fd4ca84f..7fe869b9212 100644 --- a/apps/sim/app/_shell/providers/get-query-client.ts +++ b/apps/sim/app/_shell/providers/get-query-client.ts @@ -1,4 +1,4 @@ -import { defaultShouldDehydrateQuery, isServer, QueryClient } from '@tanstack/react-query' +import { isServer, QueryClient } from '@tanstack/react-query' import { isDesktopApp } from '@/lib/desktop' export function makeQueryClient() { @@ -6,7 +6,6 @@ export function makeQueryClient() { defaultOptions: { queries: { staleTime: 30 * 1000, - gcTime: 5 * 60 * 1000, // The desktop app window lives for days, so cross-session changes — // an admin upgrading your org/workspace role, a workspace you were // auto-added to, seat/entitlement changes — would otherwise stay @@ -18,16 +17,19 @@ export function makeQueryClient() { // frequent and noisy. Per-query overrides (e.g. useWorkspaceSchedules // pins this off) always win over this default. refetchOnWindowFocus: isDesktopApp(), - retry: 1, + /** + * Query core already defaults retries to 0 on the server and 3 in the browser; + * only the browser number is ours to change. Stating one value for both would + * silently opt server prefetches into a retry, and because the layout awaits + * them that spends a retry backoff of document latency on a read whose failure + * the client recovers from on its own. + */ + retry: isServer ? 0 : 1, retryOnMount: false, }, mutations: { retry: false, }, - dehydrate: { - shouldDehydrateQuery: (query) => - defaultShouldDehydrateQuery(query) || query.state.status === 'pending', - }, }, }) } diff --git a/apps/sim/app/_shell/providers/posthog-provider.tsx b/apps/sim/app/_shell/providers/posthog-provider.tsx index 368bb3fc913..17e3f8c040b 100644 --- a/apps/sim/app/_shell/providers/posthog-provider.tsx +++ b/apps/sim/app/_shell/providers/posthog-provider.tsx @@ -42,6 +42,28 @@ export function PostHogProvider({ children }: { children: React.ReactNode }) { password: true, email: false, }, + /** + * None of these nodes are painted, so replay fidelity is + * unchanged, while each full snapshot serializes fewer nodes on + * the main thread and ships a smaller payload. + * + * Enumerated rather than `true`/`'all'` on purpose — those + * presets also enable `headTitleMutations`, which would drop + * `document.title` changes and lose the page identity a replay + * viewer reads while scrubbing. + */ + slimDOMOptions: { + script: true, + comment: true, + headFavicon: true, + headWhitespace: true, + headMetaDescKeywords: true, + headMetaSocial: true, + headMetaRobots: true, + headMetaHttpEquiv: true, + headMetaAuthorship: true, + headMetaVerification: true, + }, recordCrossOriginIframes: false, recordHeaders: false, recordBody: false, diff --git a/apps/sim/app/_styles/fonts/season/season.ts b/apps/sim/app/_styles/fonts/season/season.ts index b778b47e985..eff2a3cec31 100644 --- a/apps/sim/app/_styles/fonts/season/season.ts +++ b/apps/sim/app/_styles/fonts/season/season.ts @@ -3,13 +3,26 @@ import localFont from 'next/font/local' /** * Season Sans variable font configuration * Uses variable font file to support any weight from 300-800 + * + * `display: 'block'`, not `swap`: this is the document font, so a swap is not a cosmetic change of + * typeface — the fallback's glyph advances differ, so paragraphs re-wrap and everything below them + * moves. In long-form prose (the Files editor) that reads as the line and paragraph spacing visibly + * correcting itself a beat after the text appears, on every hard refresh (a normal reload serves the + * font from cache and never swaps). `swap` is the setting that says "painting the wrong font first is + * fine"; for a brand face it is not. + * + * The block period costs nothing here because delivery is already optimal: `preload` emits a + * `Link: rel=preload` RESPONSE header, so the fetch starts before the HTML is parsed, and the file is + * one same-origin, immutably-cached 87KB woff2. The metric-adjusted Arial below stays as the safety + * net for the >3s tail, where the browser gives up blocking and swaps — i.e. the worst case is + * today's behavior, not a regression. */ export const season = localFont({ src: [ // Variable font - supports all weights from 300 to 800 { path: './SeasonSansUprightsVF.woff2', weight: '300 800', style: 'normal' }, ], - display: 'swap', + display: 'block', preload: true, variable: '--font-season', fallback: ['system-ui', 'Segoe UI', 'Roboto', 'Helvetica Neue', 'Arial', 'Noto Sans'], diff --git a/apps/sim/app/api/credentials/route.ts b/apps/sim/app/api/credentials/route.ts index 20b4a4bcac0..69ec1fb54e2 100644 --- a/apps/sim/app/api/credentials/route.ts +++ b/apps/sim/app/api/credentials/route.ts @@ -222,7 +222,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { types: type ? [type] : undefined, providerId, }) - const credentials = visible.map(({ hasServiceAccountKey: _hasKey, ...rest }) => rest) + const credentials = visible.data.map(({ hasServiceAccountKey: _hasKey, ...rest }) => rest) return NextResponse.json({ credentials }) } catch (error) { diff --git a/apps/sim/app/api/files/serve/[...path]/route.test.ts b/apps/sim/app/api/files/serve/[...path]/route.test.ts index 495f4ad4913..799fb19b97c 100644 --- a/apps/sim/app/api/files/serve/[...path]/route.test.ts +++ b/apps/sim/app/api/files/serve/[...path]/route.test.ts @@ -20,6 +20,10 @@ const { mockIsUsingCloudStorage, mockDownloadCopilotFile, mockInferContextFromKey, + mockParseWorkspaceFileKey, + mockAuthenticateWorkspaceFile, + mockReadWorkspaceFileContentByKey, + mockResolveServableDocBytes, mockGetContentType, mockFindLocalFile, mockCreateFileResponse, @@ -40,6 +44,10 @@ const { mockIsUsingCloudStorage: vi.fn(), mockDownloadCopilotFile: vi.fn(), mockInferContextFromKey: vi.fn(), + mockParseWorkspaceFileKey: vi.fn(), + mockAuthenticateWorkspaceFile: vi.fn(), + mockReadWorkspaceFileContentByKey: vi.fn(), + mockResolveServableDocBytes: vi.fn(), mockGetContentType: vi.fn(), mockFindLocalFile: vi.fn(), mockCreateFileResponse: vi.fn(), @@ -82,7 +90,19 @@ vi.mock('@/lib/execution/sandbox/run-task', () => ({ })) vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => ({ - parseWorkspaceFileKey: vi.fn().mockReturnValue(undefined), + parseWorkspaceFileKey: mockParseWorkspaceFileKey, +})) + +vi.mock('@/lib/workspace-files/api', () => ({ + internalWorkspaceFileServeAuth: { authenticate: mockAuthenticateWorkspaceFile }, +})) + +vi.mock('@/lib/workspace-files/application/read-workspace-file-content-by-key', () => ({ + readWorkspaceFileContentByKey: { execute: mockReadWorkspaceFileContentByKey }, +})) + +vi.mock('@/lib/copilot/tools/server/files/doc-compile', () => ({ + resolveServableDocBytes: mockResolveServableDocBytes, })) vi.mock('@/app/api/files/utils', () => ({ @@ -109,7 +129,27 @@ describe('File Serve API Route', () => { mockReadFile.mockResolvedValue(Buffer.from('test content')) mockIsUsingCloudStorage.mockReturnValue(false) storageServiceMockFns.mockHasCloudStorage.mockReturnValue(true) - mockInferContextFromKey.mockReturnValue('workspace') + mockInferContextFromKey.mockReturnValue('mothership') + mockParseWorkspaceFileKey.mockReturnValue(undefined) + mockAuthenticateWorkspaceFile.mockResolvedValue({ + kind: 'session', + userId: 'test-user-id', + sessionId: 'session-1', + }) + mockReadWorkspaceFileContentByKey.mockResolvedValue({ + file: { + id: 'file-1', + workspaceId: 'test-workspace-id', + name: 'report.pdf', + }, + content: Buffer.from('generated source'), + }) + mockResolveServableDocBytes.mockImplementation( + async ({ rawBuffer, fileName }: { rawBuffer: Buffer; fileName: string }) => ({ + buffer: rawBuffer, + contentType: mockGetContentType(fileName), + }) + ) mockGetContentType.mockReturnValue('text/plain') mockFindLocalFile.mockReturnValue('/test/uploads/test-file.txt') mockCreateFileResponse.mockImplementation( @@ -181,8 +221,59 @@ describe('File Serve API Route', () => { expect(storageServiceMockFns.mockDownloadFile).toHaveBeenCalledWith({ key: 'workspace/test-workspace-id/1234567890-image.png', - context: 'workspace', + context: 'mothership', + }) + }) + + it('serves a workspace document through the authorized use case and preserves the Principal', async () => { + const principal = { + kind: 'delegated' as const, + serviceId: 'executor' as const, + subjectUserId: 'test-user-id', + workspaceId: 'test-workspace-id', + delegationId: 'delegation-1', + audience: 'sim:workspace-files', + issuedAt: new Date('2026-08-01T00:00:00Z'), + expiresAt: new Date('2026-08-01T01:00:00Z'), + delegationContext: { + kind: 'workflow_execution' as const, + workflowId: 'workflow-1', + }, + } + mockInferContextFromKey.mockReturnValue('workspace') + mockParseWorkspaceFileKey.mockReturnValue('test-workspace-id') + mockAuthenticateWorkspaceFile.mockResolvedValue(principal) + mockResolveServableDocBytes.mockResolvedValue({ + buffer: Buffer.from('%PDF-compiled'), + contentType: 'application/pdf', }) + + const req = new NextRequest( + 'http://localhost:3000/api/files/serve/workspace/test-workspace-id/report.pdf' + ) + const response = await GET(req, { + params: Promise.resolve({ + path: ['workspace', 'test-workspace-id', 'report.pdf'], + }), + }) + + expect(response.status).toBe(200) + expect(mockReadWorkspaceFileContentByKey).toHaveBeenCalledWith({ + principal, + input: { + key: 'workspace/test-workspace-id/report.pdf', + assertedWorkspaceId: 'test-workspace-id', + }, + request: req, + }) + expect(mockResolveServableDocBytes).toHaveBeenCalledWith( + expect.objectContaining({ + workspaceId: 'test-workspace-id', + filePrincipal: principal, + }) + ) + expect(hybridAuthMockFns.mockCheckSessionOrInternalAuth).not.toHaveBeenCalled() + expect(mockVerifyFileAccess).not.toHaveBeenCalled() }) it('should return 404 when file not found', async () => { diff --git a/apps/sim/app/api/files/serve/[...path]/route.ts b/apps/sim/app/api/files/serve/[...path]/route.ts index 7ba54de8e06..0899cdd0dfa 100644 --- a/apps/sim/app/api/files/serve/[...path]/route.ts +++ b/apps/sim/app/api/files/serve/[...path]/route.ts @@ -1,11 +1,17 @@ import { readFile } from 'fs/promises' +import { type Principal, requirePrincipalSubjectUserId } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { NextResponse } from 'next/server' import { fileServeParamsSchema, fileServeQuerySchema } from '@/lib/api/contracts/storage-transfer' +import { + concealCrossTenantResourceError, + InternalUnauthenticatedError, +} from '@/lib/api/server/routes' import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' import { resolveServableDocBytes } from '@/lib/copilot/tools/server/files/doc-compile' import { DocCompileUserError } from '@/lib/copilot/tools/server/files/doc-compile-error' +import { asOrchestrationError } from '@/lib/core/orchestration/types' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { CopilotFiles, isUsingCloudStorage } from '@/lib/uploads' import type { StorageContext } from '@/lib/uploads/config' @@ -13,6 +19,8 @@ import { parseWorkspaceFileKey } from '@/lib/uploads/contexts/workspace/workspac import { downloadFile } from '@/lib/uploads/core/storage-service' import { resolveServableImageBytes } from '@/lib/uploads/server/image-derivative' import { inferContextFromKey } from '@/lib/uploads/utils/file-utils' +import { internalWorkspaceFileServeAuth } from '@/lib/workspace-files/api' +import { readWorkspaceFileContentByKey } from '@/lib/workspace-files/application/read-workspace-file-content-by-key' import { verifyFileAccess } from '@/app/api/files/authorization' import { createErrorResponse, @@ -66,9 +74,11 @@ async function resolveServableBytes(params: { workspaceId: string | undefined options: ServeOptions ownerKey: string | undefined + filePrincipal?: Principal signal: AbortSignal | undefined }): Promise<{ buffer: Buffer; contentType: string }> { - const { buffer, filename, storageKey, workspaceId, options, ownerKey, signal } = params + const { buffer, filename, storageKey, workspaceId, options, ownerKey, filePrincipal, signal } = + params if (options.raw) return { buffer, contentType: getContentType(filename) } if (options.preview) { @@ -82,6 +92,7 @@ async function resolveServableBytes(params: { rawBuffer: buffer, fileName: filename, workspaceId, + filePrincipal, ownerKey, signal, }) @@ -154,6 +165,23 @@ export const GET = withRouteHandler( return await handleLocalFilePublic(fullPath) } + const storageContext = inferContextFromKey(cloudKey) + const workspacePrincipal = + storageContext === 'workspace' + ? await internalWorkspaceFileServeAuth.authenticate(request, { path }) + : undefined + const legacyAuthResult = workspacePrincipal + ? undefined + : await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) + + if (legacyAuthResult && (!legacyAuthResult.success || !legacyAuthResult.userId)) { + logger.warn('Unauthorized file access attempt', { + path, + error: legacyAuthResult.error || 'Missing userId', + }) + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + } + const query = fileServeQuerySchema.parse({ raw: request.nextUrl.searchParams.get('raw'), preview: request.nextUrl.searchParams.get('preview'), @@ -165,17 +193,12 @@ export const GET = withRouteHandler( versioned: query.v != null, } - const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) - - if (!authResult.success || !authResult.userId) { - logger.warn('Unauthorized file access attempt', { - path, - error: authResult.error || 'Missing userId', - }) - return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + if (workspacePrincipal) { + return await handleWorkspaceFile(cloudKey, workspacePrincipal, options, request) } - const userId = authResult.userId + const userId = legacyAuthResult?.userId + if (!userId) throw new Error('Authenticated file serve request is missing a user ID') if (isUsingCloudStorage()) { return await handleCloudProxy(cloudKey, userId, options, request.signal) @@ -183,6 +206,11 @@ export const GET = withRouteHandler( return await handleLocalFile(cloudKey, userId, options, request.signal) } catch (error) { + if (error instanceof InternalUnauthenticatedError) { + logger.warn('Unauthorized file access attempt', { error: error.message }) + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + } + // An in-progress/incomplete doc source fails to compile — this is expected // mid-generation, not a server fault. Return 409 (not 500) so it isn't an // alarming error; the client re-fetches once the doc finishes (the serve @@ -194,6 +222,15 @@ export const GET = withRouteHandler( return NextResponse.json({ error: 'Document is still being generated' }, { status: 409 }) } + const orchestrationError = asOrchestrationError( + concealCrossTenantResourceError(error, 'File not found') + ) + if (orchestrationError?.code === 'not_found') { + const notFound = new FileNotFoundError('File not found') + logServeFailure('Error serving file:', notFound) + return createErrorResponse(notFound) + } + logServeFailure('Error serving file:', error) if (error instanceof FileNotFoundError) { @@ -205,6 +242,45 @@ export const GET = withRouteHandler( } ) +async function handleWorkspaceFile( + key: string, + principal: Principal, + options: ServeOptions, + request: NextRequest +): Promise { + const workspaceId = getWorkspaceIdForCompile(key) + if (!workspaceId) throw new FileNotFoundError(`File not found: ${key}`) + + const { file, content } = await readWorkspaceFileContentByKey.execute({ + principal, + input: { key, assertedWorkspaceId: workspaceId }, + request, + }) + const ownerKey = `user:${requirePrincipalSubjectUserId(principal)}` + const resolved = await resolveServableBytes({ + buffer: content, + filename: file.name, + storageKey: key, + workspaceId, + options, + ownerKey, + filePrincipal: principal, + signal: request.signal, + }) + + logger.info('Workspace file served', { + fileId: file.id, + workspaceId, + size: resolved.buffer.length, + }) + return createFileResponse({ + buffer: resolved.buffer, + contentType: resolved.contentType, + filename: file.name, + cacheControl: resolveServeCacheControl(options.versioned, 'workspace'), + }) +} + async function handleLocalFile( filename: string, userId: string, diff --git a/apps/sim/app/api/mcp/oauth/callback/route.ts b/apps/sim/app/api/mcp/oauth/callback/route.ts index 9d63191fb32..ff76af5b2a3 100644 --- a/apps/sim/app/api/mcp/oauth/callback/route.ts +++ b/apps/sim/app/api/mcp/oauth/callback/route.ts @@ -208,7 +208,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { try { // forceRefresh: skip any stale cache from before re-auth. await timedStep('discoverServerTools', 60_000, () => - mcpService.discoverServerTools(session.user.id, server.id, server.workspaceId, true) + mcpService.discoverServerTools(session.user.id, server.id, server.workspaceId, 'force') ) } catch (e) { logger.warn('Post-auth tools refresh failed', toError(e).message) diff --git a/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts b/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts index b1ceda9d016..90a91aeae7d 100644 --- a/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts +++ b/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts @@ -196,7 +196,7 @@ export const POST = withRouteHandler( userId, serverId, workspaceId, - true + 'force' ) logger.info( `[${requestId}] Discovered ${discoveredTools.length} tools from server ${serverId}` diff --git a/apps/sim/app/api/mcp/tools/discover/route.ts b/apps/sim/app/api/mcp/tools/discover/route.ts index 84acdad0b3d..a592dc116bb 100644 --- a/apps/sim/app/api/mcp/tools/discover/route.ts +++ b/apps/sim/app/api/mcp/tools/discover/route.ts @@ -65,8 +65,17 @@ export const GET = withRouteHandler( logger.info(`[${requestId}] Discovering MCP tools`, { serverId, workspaceId, forceRefresh }) const tools = serverId - ? await mcpService.discoverServerTools(userId, serverId, workspaceId, forceRefresh) - : await mcpService.discoverTools(userId, workspaceId, forceRefresh) + ? await mcpService.discoverServerTools( + userId, + serverId, + workspaceId, + forceRefresh ? 'force' : 'cache-aside' + ) + : await mcpService.discoverTools( + userId, + workspaceId, + forceRefresh ? 'force' : 'cache-aside' + ) const byServer: Record = {} for (const tool of tools) { @@ -115,7 +124,7 @@ export const POST = withRouteHandler( serverIds, MCP_REFRESH_DISCOVERY_CONCURRENCY, async (serverId: string) => { - const tools = await mcpService.discoverServerTools(userId, serverId, workspaceId, true) + const tools = await mcpService.discoverServerTools(userId, serverId, workspaceId, 'force') return { serverId, toolCount: tools.length } } ) diff --git a/apps/sim/app/api/mcp/tools/execute/route.ts b/apps/sim/app/api/mcp/tools/execute/route.ts index d045b407a54..cd371580e42 100644 --- a/apps/sim/app/api/mcp/tools/execute/route.ts +++ b/apps/sim/app/api/mcp/tools/execute/route.ts @@ -157,7 +157,7 @@ export const POST = withRouteHandler( userId, serverId, workspaceId, - false, + 'cache-aside', recordProvenance ) tool = tools.find((t) => t.name === toolName) ?? null diff --git a/apps/sim/app/api/pinned-items/route.ts b/apps/sim/app/api/pinned-items/route.ts index bf31285fb61..376bf510f60 100644 --- a/apps/sim/app/api/pinned-items/route.ts +++ b/apps/sim/app/api/pinned-items/route.ts @@ -2,40 +2,21 @@ import { db, pinnedItem } from '@sim/db' import { createLogger } from '@sim/logger' import { getPostgresErrorCode } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' -import { and, eq, ne } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import { createPinnedItemContract, listPinnedItemsContract, type PinnedItemApi, - pinnedResourceTypeSchema, } from '@/lib/api/contracts' import { parseRequest } from '@/lib/api/server' import { getSession } from '@/lib/auth' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { filterToActiveResources, pinnableResourceExists } from '@/lib/pinned-items/resources' +import { listPinnedItemsForUser } from '@/lib/pinned-items/queries' +import { pinnableResourceExists } from '@/lib/pinned-items/resources' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' const logger = createLogger('PinnedItemsAPI') -/** - * Narrows a stored row to the wire shape, dropping any row whose `resourceType` this build does - * not recognise. - * - * `pinned_item.resource_type` is plain `text` — deliberately, so the set of pinnable kinds can - * grow — while the contract is a closed enum. During a rolling deploy an older pod can therefore - * read a pin a newer one wrote. Returning it would fail response validation and take the WHOLE - * list down rather than the single row, so the unknown kind is skipped instead. - * - * `filterToActiveResources` already drops these as a side effect of not having a table to look - * them up in; this makes the guarantee explicit and compiler-checked at the wire boundary. - */ -function toPinnedItemApi(row: typeof pinnedItem.$inferSelect): PinnedItemApi | null { - const resourceType = pinnedResourceTypeSchema.safeParse(row.resourceType) - if (!resourceType.success) return null - return { ...row, resourceType: resourceType.data, pinnedAt: row.pinnedAt.toISOString() } -} - /** Lists the session user's pinned items in a workspace, optionally filtered to one `resourceType`. */ export const GET = withRouteHandler(async (request: NextRequest) => { const session = await getSession() @@ -52,30 +33,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { return NextResponse.json({ error: 'Access denied to this workspace' }, { status: 403 }) } - const rows = await db - .select() - .from(pinnedItem) - .where( - and( - eq(pinnedItem.userId, session.user.id), - eq(pinnedItem.workspaceId, workspaceId), - /** - * A `workspace` pin stores `workspaceId === resourceId`, so it would otherwise - * appear in this workspace's unscoped listing as a resource *inside* itself. - * It is read from the workspace-list payload instead, so it is excluded here - * rather than left for a future unscoped caller to mistake for a real resource. - */ - resourceType - ? eq(pinnedItem.resourceType, resourceType) - : ne(pinnedItem.resourceType, 'workspace') - ) - ) - - const activeRows = await filterToActiveResources(rows, workspaceId) - - const pinnedItems = activeRows - .map(toPinnedItemApi) - .filter((item): item is PinnedItemApi => item !== null) + const pinnedItems = await listPinnedItemsForUser(session.user.id, workspaceId, resourceType) return NextResponse.json({ pinnedItems }) }) diff --git a/apps/sim/app/api/table/[tableId]/columns/route.test.ts b/apps/sim/app/api/table/[tableId]/columns/route.test.ts index 6223c12bff6..24830309efc 100644 --- a/apps/sim/app/api/table/[tableId]/columns/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/columns/route.test.ts @@ -49,10 +49,12 @@ vi.mock('@/lib/table/columns/service', () => ({ updateColumnOptions: mockUpdateColumnOptions, updateColumnType: mockUpdateColumnType, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (c: unknown) => c, +})) vi.mock('@/app/api/table/utils', () => ({ accessError: () => new Response('denied', { status: 403 }), checkAccess: mockCheckAccess, - normalizeColumn: (c: unknown) => c, orchestrationOutcomeErrorResponse: ( outcome: { error?: string; errorCode?: OrchestrationErrorCode }, fallback: string diff --git a/apps/sim/app/api/table/[tableId]/columns/route.ts b/apps/sim/app/api/table/[tableId]/columns/route.ts index 54ca6de54e8..2b2aa60c131 100644 --- a/apps/sim/app/api/table/[tableId]/columns/route.ts +++ b/apps/sim/app/api/table/[tableId]/columns/route.ts @@ -13,10 +13,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumn, deleteColumn } from '@/lib/table' import { signalTableSchemaChanged } from '@/lib/table/events' import { performUpdateTableColumn } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, rootErrorMessage, tableLockErrorResponse, diff --git a/apps/sim/app/api/table/[tableId]/groups/route.test.ts b/apps/sim/app/api/table/[tableId]/groups/route.test.ts index cad09be8b65..7d628cfdeb3 100644 --- a/apps/sim/app/api/table/[tableId]/groups/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/groups/route.test.ts @@ -54,7 +54,7 @@ vi.mock('@/lib/table/application/groups', () => ({ updateTableGroupUseCase: mocks.useCases.update, })) -vi.mock('@/app/api/table/utils', () => ({ +vi.mock('@/lib/table/wire', () => ({ normalizeColumn: vi.fn(), })) diff --git a/apps/sim/app/api/table/[tableId]/groups/route.ts b/apps/sim/app/api/table/[tableId]/groups/route.ts index b1f9a1c4749..f8f14909ac4 100644 --- a/apps/sim/app/api/table/[tableId]/groups/route.ts +++ b/apps/sim/app/api/table/[tableId]/groups/route.ts @@ -12,7 +12,7 @@ import { } from '@/lib/table/application/groups' import { tableOperations } from '@/lib/table/application/operations' import type { TableDefinition } from '@/lib/table/types' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' const rateLimit = internalRateLimits.none({ reason: 'Existing authenticated table group mutations have no request-rate policy', diff --git a/apps/sim/app/api/table/[tableId]/route.test.ts b/apps/sim/app/api/table/[tableId]/route.test.ts index 43cbf68ae83..6e1b9a957c9 100644 --- a/apps/sim/app/api/table/[tableId]/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/route.test.ts @@ -51,9 +51,11 @@ vi.mock('@/lib/workspaces/permissions/utils', () => ({ vi.mock('@/app/api/table/utils', () => ({ accessError: () => new Response('denied', { status: 403 }), checkAccess: mockCheckAccess, - normalizeColumn: (column: unknown) => column, tableLockErrorResponse: () => null, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (column: unknown) => column, +})) import { GET, PATCH } from '@/app/api/table/[tableId]/route' diff --git a/apps/sim/app/api/table/[tableId]/route.ts b/apps/sim/app/api/table/[tableId]/route.ts index e14a3bdf775..4f61ce4ea12 100644 --- a/apps/sim/app/api/table/[tableId]/route.ts +++ b/apps/sim/app/api/table/[tableId]/route.ts @@ -18,11 +18,11 @@ import { performUpdateTableLocks, } from '@/lib/table/orchestration' import { TABLE_LOCK_FLAGS, TABLE_LOCK_KINDS } from '@/lib/table/types' +import { normalizeColumn } from '@/lib/table/wire' import { getWorkspaceWithOwner } from '@/lib/workspaces/permissions/utils' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, tableLockErrorResponse, } from '@/app/api/table/utils' diff --git a/apps/sim/app/api/table/import-csv/route.test.ts b/apps/sim/app/api/table/import-csv/route.test.ts index dae8f0c3d63..dea46a06a3a 100644 --- a/apps/sim/app/api/table/import-csv/route.test.ts +++ b/apps/sim/app/api/table/import-csv/route.test.ts @@ -34,7 +34,6 @@ vi.mock('@/app/api/table/utils', async () => { const { asOrchestrationError, messageForOrchestrationError, statusForOrchestrationError } = await import('@/lib/core/orchestration/types') return { - normalizeColumn: (column: unknown) => column, csvProxyBodyCapResponse: () => null, multipartErrorResponse: (error: { code: string; message: string }) => NextResponse.json( diff --git a/apps/sim/app/api/table/route.ts b/apps/sim/app/api/table/route.ts index 28714885cb5..be28a064fd1 100644 --- a/apps/sim/app/api/table/route.ts +++ b/apps/sim/app/api/table/route.ts @@ -15,8 +15,9 @@ import { type TableSchema, type TableScope, } from '@/lib/table' +import { normalizeColumn, toTableListItem, toWireTimestamp } from '@/lib/table/wire' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' -import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils' +import { orchestrationErrorResponse } from '@/app/api/table/utils' const logger = createLogger('TableAPI') @@ -140,14 +141,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => { maxRows: table.maxRows, folderId: table.folderId ?? null, locks: table.locks, - createdAt: - table.createdAt instanceof Date - ? table.createdAt.toISOString() - : String(table.createdAt), - updatedAt: - table.updatedAt instanceof Date - ? table.updatedAt.toISOString() - : String(table.updatedAt), + createdAt: toWireTimestamp(table.createdAt), + updatedAt: toWireTimestamp(table.updatedAt), }, message: 'Table created successfully', }, @@ -198,41 +193,10 @@ export const GET = withRouteHandler(async (request: NextRequest) => { logger.info(`[${requestId}] Listed ${tables.length} tables in workspace ${params.workspaceId}`) - const responseTables = tables.map((t) => { - const schemaData = t.schema as TableSchema - return { - id: t.id, - name: t.name, - description: t.description, - schema: { - columns: schemaData.columns.map(normalizeColumn), - }, - rowCount: t.rowCount, - maxRows: t.maxRows, - locks: t.locks, - workspaceId: t.workspaceId, - folderId: t.folderId ?? null, - createdBy: t.createdBy, - createdAt: t.createdAt instanceof Date ? t.createdAt.toISOString() : String(t.createdAt), - updatedAt: t.updatedAt instanceof Date ? t.updatedAt.toISOString() : String(t.updatedAt), - archivedAt: - t.archivedAt instanceof Date - ? t.archivedAt.toISOString() - : t.archivedAt - ? String(t.archivedAt) - : null, - jobStatus: t.jobStatus ?? null, - jobId: t.jobId ?? null, - jobType: t.jobType ?? null, - jobError: t.jobError ?? null, - jobRowsProcessed: t.jobRowsProcessed ?? 0, - } - }) - return NextResponse.json({ success: true, data: { - tables: responseTables, + tables: tables.map(toTableListItem), totalCount: tables.length, }, }) diff --git a/apps/sim/app/api/table/utils.ts b/apps/sim/app/api/table/utils.ts index e049b1a3f68..037c6c83cc8 100644 --- a/apps/sim/app/api/table/utils.ts +++ b/apps/sim/app/api/table/utils.ts @@ -17,7 +17,6 @@ import { import type { MultipartError } from '@/lib/core/utils/multipart' import type { ColumnDefinition, Filter, TableDefinition, TablePredicate } from '@/lib/table' import { buildFilterClause, getTableById, TableQueryValidationError } from '@/lib/table' -import { typeMetadataOf } from '@/lib/table/column-types' import { USER_TABLE_ROWS_SQL_NAME } from '@/lib/table/constants' import { TableLockedError } from '@/lib/table/mutation-locks' import { isTablePredicate } from '@/lib/table/query-builder/converters' @@ -358,21 +357,3 @@ export function serverErrorResponse(message = 'Internal server error') { export const CreateColumnSchema = createTableColumnBodySchema export const UpdateColumnSchema = updateTableColumnBodySchema export const DeleteColumnSchema = deleteTableColumnBodySchema - -export function normalizeColumn( - col: ColumnDefinition -): ColumnDefinition & { required: boolean; unique: boolean } { - return { - // Preserve the stable column id — it's the row-data storage key, so dropping - // it makes clients fall back to `name` and miss id-keyed cell values. - ...(col.id ? { id: col.id } : {}), - name: col.name, - type: col.type, - required: col.required ?? false, - unique: col.unique ?? false, - ...(col.workflowGroupId ? { workflowGroupId: col.workflowGroupId } : {}), - // Type-specific metadata is forwarded generically: naming keys here meant a - // new type's metadata was stored server-side but silently never returned. - ...typeMetadataOf(col), - } -} diff --git a/apps/sim/app/api/tools/windchill/route.test.ts b/apps/sim/app/api/tools/windchill/route.test.ts new file mode 100644 index 00000000000..7504e7e606c --- /dev/null +++ b/apps/sim/app/api/tools/windchill/route.test.ts @@ -0,0 +1,836 @@ +/** + * @vitest-environment node + */ +import { createMockRequest as createTestingRequest, resetEnvMock } from '@sim/testing' +import { NextResponse } from 'next/server' +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest' +import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' + +const { + MockInvalidBindingError, + MockWindchillProviderError, + mockAssertToolFileAccess, + mockBindDelegation, + mockCreateWindchillSession, + mockDownloadServableFileFromStorage, + mockDownloadWindchillContent, + mockGetSession, + mockResolveWindchillContentUrl, + mockProcessFilesToUserFiles, + mockUploadCopilotFile, + mockUploadExecutionFile, + mockUploadWindchillContent, + mockWindchillMutationRequest, +} = vi.hoisted(() => { + class MockInvalidBindingError extends Error {} + class MockWindchillProviderError extends Error { + constructor( + message: string, + readonly status: number + ) { + super(message) + this.name = 'WindchillProviderError' + } + } + + return { + MockInvalidBindingError, + MockWindchillProviderError, + mockAssertToolFileAccess: vi.fn(), + mockBindDelegation: vi.fn(), + mockCreateWindchillSession: vi.fn(), + mockDownloadServableFileFromStorage: vi.fn(), + mockDownloadWindchillContent: vi.fn(), + mockGetSession: vi.fn(), + mockResolveWindchillContentUrl: vi.fn(), + mockProcessFilesToUserFiles: vi.fn(), + mockUploadCopilotFile: vi.fn(), + mockUploadExecutionFile: vi.fn(), + mockUploadWindchillContent: vi.fn(), + mockWindchillMutationRequest: vi.fn(), + } +}) + +vi.mock('@/lib/auth', () => ({ getSession: mockGetSession })) +vi.mock('@/lib/auth/internal-delegation', () => ({ + bindInternalExecutorDelegation: mockBindDelegation, + InvalidInternalDelegationBindingError: MockInvalidBindingError, +})) +vi.unmock('@/lib/auth/internal') + +vi.mock('@/app/api/files/authorization', () => ({ + assertToolFileAccess: mockAssertToolFileAccess, +})) +vi.mock('@/lib/uploads/utils/file-utils', () => ({ + processFilesToUserFiles: mockProcessFilesToUserFiles, +})) +vi.mock('@/lib/uploads/utils/file-utils.server', () => ({ + downloadServableFileFromStorage: mockDownloadServableFileFromStorage, +})) +vi.mock('@/lib/uploads/utils/servable-file-response', () => ({ + docNotReadyResponse: vi.fn().mockReturnValue(null), +})) +vi.mock('@/lib/uploads/contexts/copilot', () => ({ + uploadCopilotFile: mockUploadCopilotFile, +})) +vi.mock('@/lib/uploads/contexts/execution', () => ({ + uploadExecutionFile: mockUploadExecutionFile, +})) +vi.mock('@/tools/windchill/utils.server', () => ({ + createWindchillSession: mockCreateWindchillSession, + downloadWindchillContent: mockDownloadWindchillContent, + resolveWindchillContentUrl: mockResolveWindchillContentUrl, + sanitizeWindchillError: (message: string) => message.replace(/https?:\/\/\S+/g, '[redacted URL]'), + uploadWindchillContent: mockUploadWindchillContent, + windchillDocumentUrl: (baseUrl: string, documentOid: string) => + `${baseUrl}/DocMgmt/Documents('${encodeURIComponent(documentOid)}')`, + windchillMutationRequest: mockWindchillMutationRequest, + WindchillProviderError: MockWindchillProviderError, +})) + +import { generateInternalDelegationToken, generateInternalToken } from '@/lib/auth/internal' +import { POST } from '@/app/api/tools/windchill/route' + +const BASE_BODY = { + baseUrl: 'https://windchill.example.com/Windchill/servlet/odata/v6', + username: 'windchill-user', + password: 'not-a-real-password', +} + +const DOCUMENT_OID = 'OR:wt.doc.WTDocument:1' +const SECOND_DOCUMENT_OID = 'OR:wt.doc.WTDocument:2' +let delegationToken = '' +let legacyInternalToken = '' + +function createMockRequest(method: string, body: unknown, headers: Record = {}) { + return createTestingRequest(method, body, { + authorization: `Bearer ${delegationToken}`, + ...headers, + }) +} + +const MUTATION_CASES = [ + { + operation: 'windchill_create_document', + input: { name: 'Specification', containerOid: 'OR:wt.pdmlink.PDMLinkProduct:1' }, + url: '/DocMgmt/Documents', + method: 'POST', + }, + { + operation: 'windchill_create_documents', + input: { + documents: [{ name: 'Specification', containerOid: 'OR:wt.pdmlink.PDMLinkProduct:1' }], + }, + url: '/DocMgmt/CreateDocuments', + method: 'POST', + }, + { + operation: 'windchill_update_document', + input: { documentOid: DOCUMENT_OID, attributes: { Title: 'Updated' } }, + url: '/DocMgmt/Documents(', + method: 'PATCH', + }, + { + operation: 'windchill_update_documents', + input: { documents: [{ id: DOCUMENT_OID, attributes: { Title: 'Updated' } }] }, + url: '/DocMgmt/UpdateDocuments', + method: 'POST', + }, + { + operation: 'windchill_update_common_properties', + input: { documentOid: DOCUMENT_OID, commonProperties: { Name: 'Renamed' } }, + url: '/PTC.DocMgmt.UpdateCommonProperties', + method: 'POST', + }, + { + operation: 'windchill_delete_document', + input: { documentOid: DOCUMENT_OID }, + url: '/DocMgmt/Documents(', + method: 'DELETE', + }, + { + operation: 'windchill_delete_documents', + input: { documentOids: [DOCUMENT_OID, SECOND_DOCUMENT_OID] }, + url: '/DocMgmt/DeleteDocuments', + method: 'POST', + }, + { + operation: 'windchill_check_out_document', + input: { documentOid: DOCUMENT_OID, checkOutNote: 'Editing' }, + url: '/PTC.DocMgmt.CheckOut', + method: 'POST', + }, + { + operation: 'windchill_check_out_documents', + input: { documentOids: [DOCUMENT_OID], checkOutNote: 'Editing' }, + url: '/DocMgmt/CheckOutDocuments', + method: 'POST', + }, + { + operation: 'windchill_check_in_document', + input: { documentOid: DOCUMENT_OID, checkInNote: 'Done', keepCheckedOut: false }, + url: '/PTC.DocMgmt.CheckIn', + method: 'POST', + }, + { + operation: 'windchill_check_in_documents', + input: { documentOids: [DOCUMENT_OID], checkInNote: 'Done' }, + url: '/DocMgmt/CheckInDocuments', + method: 'POST', + }, + { + operation: 'windchill_undo_check_out_document', + input: { documentOid: DOCUMENT_OID }, + url: '/PTC.DocMgmt.UndoCheckOut', + method: 'POST', + }, + { + operation: 'windchill_undo_check_out_documents', + input: { documentOids: [DOCUMENT_OID] }, + url: '/DocMgmt/UndoCheckOutDocuments', + method: 'POST', + }, + { + operation: 'windchill_revise_document', + input: { documentOid: DOCUMENT_OID, versionId: 'B' }, + url: '/PTC.DocMgmt.Revise', + method: 'POST', + }, + { + operation: 'windchill_revise_documents', + input: { documentOids: [DOCUMENT_OID] }, + url: '/DocMgmt/ReviseDocuments', + method: 'POST', + }, + { + operation: 'windchill_set_lifecycle_state', + input: { documentOid: DOCUMENT_OID, stateValue: 'RELEASED', stateDisplay: 'Released' }, + url: '/PTC.DocMgmt.SetState', + method: 'POST', + }, + { + operation: 'windchill_update_document_security_labels', + input: { + securityLabelUpdates: [{ id: DOCUMENT_OID, labels: { EXPORT_CONTROL: 'L1' } }], + }, + url: '/DocMgmt/EditDocumentsSecurityLabels', + method: 'POST', + }, +] as const + +const MUTATION_PAYLOAD_CASES = [ + { + operation: 'windchill_check_out_documents', + input: { documentOids: [DOCUMENT_OID], checkOutNote: 'Editing' }, + body: { Documents: [{ ID: DOCUMENT_OID }], CheckOutNote: 'Editing' }, + }, + { + operation: 'windchill_check_in_document', + input: { + documentOid: DOCUMENT_OID, + checkInNote: 'Done', + keepCheckedOut: false, + checkOutNote: 'Continue editing', + }, + body: { + CheckInNote: 'Done', + KeepCheckedOut: false, + CheckOutNote: 'Continue editing', + }, + }, + { + operation: 'windchill_revise_document', + input: { documentOid: DOCUMENT_OID, versionId: 'B' }, + body: { VersionId: 'B' }, + }, + { + operation: 'windchill_update_common_properties', + input: { + documentOid: DOCUMENT_OID, + commonProperties: { Name: 'Renamed', Number: 'DOC-001' }, + }, + body: { Updates: { Name: 'Renamed', Number: 'DOC-001' } }, + }, + { + operation: 'windchill_revise_documents', + input: { documentOids: [DOCUMENT_OID] }, + body: { Documents: [{ ID: DOCUMENT_OID }] }, + }, + { + operation: 'windchill_set_lifecycle_state', + input: { documentOid: DOCUMENT_OID, stateValue: 'RELEASED', stateDisplay: 'Released' }, + body: { State: { Display: 'Released', Value: 'RELEASED' } }, + }, + { + operation: 'windchill_update_document_security_labels', + input: { + securityLabelUpdates: [{ id: DOCUMENT_OID, labels: { EXPORT_CONTROL: 'L1' } }], + }, + body: { Documents: [{ EXPORT_CONTROL: 'L1', ID: DOCUMENT_OID }] }, + }, +] as const + +beforeAll(async () => { + delegationToken = await generateInternalDelegationToken({ + subjectUserId: 'user-1', + workflowId: '550e8400-e29b-41d4-a716-446655440001', + }) + legacyInternalToken = await generateInternalToken() +}) + +afterAll(resetEnvMock) + +beforeEach(() => { + vi.clearAllMocks() + mockGetSession.mockResolvedValue(null) + mockBindDelegation.mockImplementation(async (delegation, options) => ({ + kind: 'delegated', + serviceId: 'executor', + subjectUserId: delegation.subjectUserId, + workspaceId: '550e8400-e29b-41d4-a716-446655440000', + delegationId: delegation.delegationId, + audience: options.audience, + issuedAt: delegation.issuedAt, + expiresAt: delegation.expiresAt, + delegationContext: { + kind: 'workflow_execution', + workflowId: delegation.workflowId, + executionId: delegation.executionId, + }, + })) + mockCreateWindchillSession.mockResolvedValue({ + nonceHeader: 'CSRF_NONCE', + nonceValue: 'nonce-value', + cookie: 'JSESSIONID=session-value', + }) + mockWindchillMutationRequest.mockResolvedValue({ value: [{ ID: DOCUMENT_OID }] }) + mockAssertToolFileAccess.mockResolvedValue(null) + mockProcessFilesToUserFiles.mockReturnValue([ + { + key: 'workspace/workspace-1/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + ]) + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from('pdf'), + contentType: 'application/pdf', + }) + mockUploadWindchillContent.mockResolvedValue(['specification.pdf']) + mockResolveWindchillContentUrl.mockImplementation( + async ({ contentPath }: { contentPath: string }) => + `https://windchill.example.com/Windchill/servlet/WindchillGW/download?from=${encodeURIComponent(contentPath)}` + ) + mockDownloadWindchillContent.mockResolvedValue({ + buffer: Buffer.from('pdf'), + contentType: 'application/pdf', + contentDisposition: 'attachment; filename="specification.pdf"', + }) + mockUploadCopilotFile.mockResolvedValue({ + id: 'file-1', + name: 'specification.pdf', + url: '/api/files/serve?key=copilot/specification.pdf', + size: 3, + type: 'application/pdf', + key: 'copilot/specification.pdf', + }) +}) + +describe('POST /api/tools/windchill', () => { + it('authenticates before parsing the request body', async () => { + const response = await POST(createTestingRequest('POST', { operation: 'not-valid' })) + + expect(response.status).toBe(401) + expect(await response.json()).toEqual({ success: false, error: 'Unauthorized' }) + expect(mockCreateWindchillSession).not.toHaveBeenCalled() + }) + + it('binds executor identity and scope through the canonical delegation path', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_document', + documentOid: DOCUMENT_OID, + attributes: { Title: 'Updated' }, + }) + ) + + expect(response.status).toBe(200) + expect(mockBindDelegation).toHaveBeenCalledWith(expect.any(Object), { + audience: 'sim:windchill', + resourceScope: undefined, + }) + }) + + it('rejects browser sessions and legacy internal tokens', async () => { + mockGetSession.mockResolvedValueOnce({ + user: { id: 'user-1' }, + session: { id: 'session-1' }, + }) + + const sessionResponse = await POST(createTestingRequest('POST', BASE_BODY)) + const legacyResponse = await POST( + createTestingRequest('POST', BASE_BODY, { + authorization: `Bearer ${legacyInternalToken}`, + }) + ) + + expect(sessionResponse.status).toBe(401) + expect(legacyResponse.status).toBe(401) + expect(mockBindDelegation).not.toHaveBeenCalled() + }) + + it('rejects malformed operation inputs at the shared contract boundary', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_document', + documentOid: DOCUMENT_OID, + attributes: {}, + }) + ) + + expect(response.status).toBe(400) + expect(mockCreateWindchillSession).not.toHaveBeenCalled() + }) + + it('rejects an invalid service root before reading a protected upload', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + baseUrl: `${BASE_BODY.baseUrl}?token=secret`, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/workspace-1/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + }) + ) + + expect(response.status).toBe(400) + expect(mockAssertToolFileAccess).not.toHaveBeenCalled() + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it.each(MUTATION_CASES)( + 'dispatches $operation through one CSRF-protected transaction', + async ({ operation, input, url, method }) => { + const response = await POST(createMockRequest('POST', { ...BASE_BODY, operation, ...input })) + + expect(response.status).toBe(200) + expect((await response.json()).success).toBe(true) + expect(mockCreateWindchillSession).toHaveBeenCalledTimes(1) + expect(mockWindchillMutationRequest).toHaveBeenCalledTimes(1) + expect(mockWindchillMutationRequest.mock.calls[0][0].url).toContain(url) + expect(mockWindchillMutationRequest.mock.calls[0][0].method).toBe(method) + } + ) + + it.each(MUTATION_PAYLOAD_CASES)( + 'encodes the exact $operation action payload', + async ({ operation, input, body }) => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation, + ...input, + }) + ) + + expect(response.status).toBe(200) + expect(mockWindchillMutationRequest.mock.calls[0][0].body).toEqual(body) + } + ) + + it('maps create bindings and custom attributes without allowing them to replace bindings', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_create_document', + name: 'Specification', + containerOid: 'OR:wt.pdmlink.PDMLinkProduct:1', + folderOid: 'OR:wt.folder.SubFolder:2', + attributes: { CustomString: 'value' }, + }) + ) + + expect(response.status).toBe(200) + expect((await response.json()).output.affectedIds).toEqual([DOCUMENT_OID]) + expect(mockWindchillMutationRequest.mock.calls[0][0].body).toEqual({ + CustomString: 'value', + Name: 'Specification', + 'Context@odata.bind': "Containers('OR%3Awt.pdmlink.PDMLinkProduct%3A1')", + 'Folder@odata.bind': "Folders('OR%3Awt.folder.SubFolder%3A2')", + }) + }) + + it('returns operation-specific single, bulk, and delete mutation shapes', async () => { + const singleResponse = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_document', + documentOid: DOCUMENT_OID, + attributes: { Title: 'Updated' }, + }) + ) + const singleOutput = (await singleResponse.json()).output + expect(singleOutput.document).toMatchObject({ id: DOCUMENT_OID }) + expect(singleOutput).not.toHaveProperty('documents') + + const bulkResponse = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_documents', + documents: [{ id: DOCUMENT_OID, attributes: { Title: 'Updated' } }], + }) + ) + const bulkOutput = (await bulkResponse.json()).output + expect(bulkOutput.documents).toEqual([expect.objectContaining({ id: DOCUMENT_OID })]) + expect(bulkOutput).not.toHaveProperty('document') + + const deleteResponse = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_delete_document', + documentOid: DOCUMENT_OID, + }) + ) + const deleteOutput = (await deleteResponse.json()).output + expect(deleteOutput.affectedIds).toEqual([DOCUMENT_OID]) + expect(deleteOutput).not.toHaveProperty('document') + expect(deleteOutput).not.toHaveProperty('documents') + }) + + it('authorizes and reads a UserFile before starting the upload transaction', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/workspace-1/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + }) + ) + + expect(response.status).toBe(200) + expect(mockAssertToolFileAccess).toHaveBeenCalledWith( + 'workspace/workspace-1/specification.pdf', + 'user-1', + expect.any(String), + expect.anything() + ) + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledTimes(1) + expect(mockUploadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ + documentOid: DOCUMENT_OID, + primaryContent: true, + files: [ + expect.objectContaining({ + name: 'specification.pdf', + mimeType: 'application/pdf', + size: 3, + }), + ], + }) + ) + }) + + it('uploads multiple authorized files as attachments', async () => { + mockProcessFilesToUserFiles.mockReturnValueOnce([ + { + key: 'workspace/workspace-1/one.txt', + name: 'one.txt', + size: 3, + type: 'text/plain', + }, + { + key: 'workspace/workspace-1/two.txt', + name: 'two.txt', + size: 3, + type: 'text/plain', + }, + ]) + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from('txt'), + contentType: 'text/plain', + }) + mockUploadWindchillContent.mockResolvedValueOnce(['one.txt', 'two.txt']) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_attachments', + documentOid: DOCUMENT_OID, + attachmentFiles: [ + { key: 'workspace/workspace-1/one.txt', name: 'one.txt', size: 3 }, + { key: 'workspace/workspace-1/two.txt', name: 'two.txt', size: 3 }, + ], + }) + ) + + expect(response.status).toBe(200) + expect(mockAssertToolFileAccess).toHaveBeenCalledTimes(2) + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledTimes(2) + expect(mockUploadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ primaryContent: false }) + ) + expect(mockDownloadServableFileFromStorage.mock.calls[0][3]).toEqual({ + maxBytes: MAX_FILE_SIZE, + }) + expect(mockDownloadServableFileFromStorage.mock.calls[1][3]).toEqual({ + maxBytes: MAX_FILE_SIZE - 3, + }) + }) + + it('rejects attachment counts above the contract limit before reading storage', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_attachments', + documentOid: DOCUMENT_OID, + attachmentFiles: Array.from({ length: 11 }, (_, index) => ({ + key: `workspace/workspace-1/${index}.txt`, + name: `${index}.txt`, + size: 1, + })), + }) + ) + + expect(response.status).toBe(400) + expect(mockAssertToolFileAccess).not.toHaveBeenCalled() + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it('rejects declared aggregate upload size before reading storage', async () => { + mockProcessFilesToUserFiles.mockReturnValueOnce([ + { + key: 'workspace/workspace-1/oversized.bin', + name: 'oversized.bin', + size: MAX_FILE_SIZE + 1, + type: 'application/octet-stream', + }, + ]) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/workspace-1/oversized.bin', + name: 'oversized.bin', + size: MAX_FILE_SIZE + 1, + }, + }) + ) + + expect(response.status).toBe(413) + expect(mockAssertToolFileAccess).not.toHaveBeenCalled() + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it('stops an under-reported upload at the remaining aggregate byte budget', async () => { + mockProcessFilesToUserFiles.mockReturnValueOnce([ + { key: 'workspace/workspace-1/one.txt', name: 'one.txt', size: 1, type: 'text/plain' }, + { key: 'workspace/workspace-1/two.txt', name: 'two.txt', size: 1, type: 'text/plain' }, + { + key: 'workspace/workspace-1/three.txt', + name: 'three.txt', + size: 1, + type: 'text/plain', + }, + ]) + mockDownloadServableFileFromStorage + .mockResolvedValueOnce({ buffer: Buffer.from('one'), contentType: 'text/plain' }) + .mockRejectedValueOnce( + new PayloadSizeLimitError({ + label: 'Uploaded file', + maxBytes: MAX_FILE_SIZE - 3, + observedBytes: MAX_FILE_SIZE - 2, + }) + ) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_attachments', + documentOid: DOCUMENT_OID, + attachmentFiles: [ + { key: 'workspace/workspace-1/one.txt', name: 'one.txt', size: 1 }, + { key: 'workspace/workspace-1/two.txt', name: 'two.txt', size: 1 }, + { key: 'workspace/workspace-1/three.txt', name: 'three.txt', size: 1 }, + ], + }) + ) + + expect(response.status).toBe(413) + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledTimes(2) + expect(mockDownloadServableFileFromStorage.mock.calls[1][3]).toEqual({ + maxBytes: MAX_FILE_SIZE - 3, + }) + expect(mockUploadWindchillContent).not.toHaveBeenCalled() + }) + + it('stops before storage or Windchill when file ownership is denied', async () => { + mockAssertToolFileAccess.mockResolvedValueOnce( + NextResponse.json({ success: false, error: 'File not found' }, { status: 404 }) + ) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/other/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + }) + ) + + expect(response.status).toBe(404) + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + expect(mockUploadWindchillContent).not.toHaveBeenCalled() + }) + + it('stores downloads as a UserFile instead of returning inline bytes', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_download_primary_content', + documentOid: DOCUMENT_OID, + }) + ) + const data = await response.json() + + expect(response.status).toBe(200) + expect(mockResolveWindchillContentUrl).toHaveBeenCalledWith( + expect.objectContaining({ + contentPath: expect.stringContaining('/PrimaryContent'), + }) + ) + expect(mockResolveWindchillContentUrl.mock.calls[0][0].contentPath).not.toContain('$value') + expect(mockDownloadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ + url: expect.stringContaining('/WindchillGW/download'), + }) + ) + expect(mockUploadCopilotFile).toHaveBeenCalledWith( + expect.objectContaining({ + buffer: Buffer.from('pdf'), + fileName: 'specification.pdf', + contentType: 'application/pdf', + userId: 'user-1', + }) + ) + expect(data.output.file).toMatchObject({ key: 'copilot/specification.pdf' }) + expect(data.output.content).toBeUndefined() + }) + + it('downloads an attachment through its document-scoped content path', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_download_attachment', + documentOid: DOCUMENT_OID, + attachmentOid: 'OR:wt.content.ApplicationData:2', + }) + ) + + expect(response.status).toBe(200) + expect(mockResolveWindchillContentUrl).toHaveBeenCalledWith( + expect.objectContaining({ + contentPath: expect.stringContaining("/Attachments('OR%3Awt.content.ApplicationData%3A2')"), + }) + ) + expect(mockDownloadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ + url: expect.stringContaining('/WindchillGW/download'), + }) + ) + }) + + it('uses execution storage derived from the bound delegation principal', async () => { + mockBindDelegation.mockResolvedValueOnce({ + kind: 'delegated', + serviceId: 'executor', + subjectUserId: 'user-1', + workspaceId: '550e8400-e29b-41d4-a716-446655440000', + delegationId: 'delegation-1', + audience: 'sim:windchill', + issuedAt: new Date('2026-01-01T00:00:00.000Z'), + expiresAt: new Date('2027-01-01T00:00:00.000Z'), + delegationContext: { + kind: 'workflow_execution', + workflowId: '550e8400-e29b-41d4-a716-446655440001', + executionId: 'execution-1', + }, + }) + mockUploadExecutionFile.mockResolvedValueOnce({ + id: 'file-2', + name: 'specification.pdf', + url: '/api/files/serve?key=execution/specification.pdf', + size: 3, + type: 'application/pdf', + key: 'execution/specification.pdf', + }) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_download_primary_content', + documentOid: DOCUMENT_OID, + workspaceId: 'forged-workspace', + workflowId: 'forged-workflow', + executionId: 'forged-execution', + }) + ) + + expect(response.status).toBe(200) + expect(mockUploadExecutionFile).toHaveBeenCalledWith( + { + workspaceId: '550e8400-e29b-41d4-a716-446655440000', + workflowId: '550e8400-e29b-41d4-a716-446655440001', + executionId: 'execution-1', + }, + Buffer.from('pdf'), + 'specification.pdf', + 'application/pdf', + 'user-1' + ) + expect(mockUploadCopilotFile).not.toHaveBeenCalled() + }) + + it('preserves sanitized provider status codes', async () => { + mockWindchillMutationRequest.mockRejectedValueOnce( + new MockWindchillProviderError('Windchill rejected the transition', 409) + ) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_set_lifecycle_state', + documentOid: DOCUMENT_OID, + stateValue: 'RELEASED', + stateDisplay: 'Released', + }) + ) + + expect(response.status).toBe(409) + expect(await response.json()).toEqual({ + success: false, + error: 'Windchill rejected the transition', + }) + }) +}) diff --git a/apps/sim/app/api/tools/windchill/route.ts b/apps/sim/app/api/tools/windchill/route.ts new file mode 100644 index 00000000000..21eaee259ff --- /dev/null +++ b/apps/sim/app/api/tools/windchill/route.ts @@ -0,0 +1,641 @@ +import type { WorkflowExecutionDelegatedPrincipal } from '@sim/auth/principal' +import { createLogger } from '@sim/logger' +import { getErrorMessage } from '@sim/utils/errors' +import { type NextRequest, NextResponse } from 'next/server' +import type { + WindchillOperationBody, + WindchillOperationResponse, +} from '@/lib/api/contracts/tools/windchill' +import { windchillOperationContract } from '@/lib/api/contracts/tools/windchill' +import { getValidationErrorMessage, parseRequest } from '@/lib/api/server' +import { + createInternalSessionOrExecutorAuth, + InternalUnauthenticatedError, +} from '@/lib/api/server/routes' +import { generateRequestId } from '@/lib/core/utils/request' +import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { uploadCopilotFile } from '@/lib/uploads/contexts/copilot' +import { uploadExecutionFile } from '@/lib/uploads/contexts/execution' +import type { RawFileInput } from '@/lib/uploads/utils/file-schemas' +import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' +import { downloadServableFileFromStorage } from '@/lib/uploads/utils/file-utils.server' +import { docNotReadyResponse } from '@/lib/uploads/utils/servable-file-response' +import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' +import { assertToolFileAccess } from '@/app/api/files/authorization' +import { sanitizeFileName } from '@/executor/constants' +import type { UserFile } from '@/executor/types' +import { + encodeWindchillOid, + normalizeServiceRoot, + normalizeWindchillDocument, + normalizeWindchillDocuments, + sanitizeWindchillError, +} from '@/tools/windchill/utils' +import { + createWindchillSession, + downloadWindchillContent, + resolveWindchillContentUrl, + uploadWindchillContent, + WindchillProviderError, + type WindchillUploadFile, + windchillDocumentUrl, + windchillMutationRequest, +} from '@/tools/windchill/utils.server' + +export const dynamic = 'force-dynamic' +export const maxDuration = 900 + +const logger = createLogger('WindchillAPI') +const windchillSessionOrExecutorAuth = createInternalSessionOrExecutorAuth({ + audience: 'sim:windchill', +}) + +async function authenticateWindchillExecutor( + request: NextRequest +): Promise { + const principal = await windchillSessionOrExecutorAuth.authenticate(request, {}) + if ( + principal.kind !== 'delegated' || + principal.serviceId !== 'executor' || + !('delegationContext' in principal) + ) { + throw new InternalUnauthenticatedError('Authentication required') + } + return principal +} + +type WindchillRouteOutput = Extract['output'] +type MutationOperation = Exclude< + WindchillRouteOutput['operation'], + | 'windchill_download_attachment' + | 'windchill_download_primary_content' + | 'windchill_upload_attachments' + | 'windchill_upload_primary_content' +> + +const BULK_RESULT_OPERATIONS = [ + 'windchill_create_documents', + 'windchill_update_documents', + 'windchill_check_out_documents', + 'windchill_check_in_documents', + 'windchill_undo_check_out_documents', + 'windchill_revise_documents', + 'windchill_update_document_security_labels', +] as const satisfies readonly MutationOperation[] + +const DELETE_OPERATIONS = [ + 'windchill_delete_document', + 'windchill_delete_documents', +] as const satisfies readonly MutationOperation[] + +type BulkResultOperation = (typeof BULK_RESULT_OPERATIONS)[number] +type DeleteOperation = (typeof DELETE_OPERATIONS)[number] + +function isBulkResultOperation(operation: MutationOperation): operation is BulkResultOperation { + return BULK_RESULT_OPERATIONS.includes(operation as BulkResultOperation) +} + +function isDeleteOperation(operation: MutationOperation): operation is DeleteOperation { + return DELETE_OPERATIONS.includes(operation as DeleteOperation) +} + +function successResponse(output: WindchillRouteOutput) { + const body = { success: true, output } satisfies WindchillOperationResponse + return NextResponse.json(body) +} + +function failureResponse(error: string, status: number) { + const body = { + success: false, + error: sanitizeWindchillError(error), + } satisfies WindchillOperationResponse + return NextResponse.json(body, { status }) +} + +function documentsById(documentOids: string[]) { + return documentOids.map((ID) => ({ ID })) +} + +/** Keeps the media type and drops any `; charset=...` parameters Windchill cannot use. */ +function safeMimeType(value: string | undefined): string { + const mediaType = value?.split(';', 1)[0]?.trim() + if (mediaType && /^[A-Za-z0-9!#$&^_.+-]+\/[A-Za-z0-9!#$&^_.+-]+$/.test(mediaType)) { + return mediaType + } + return 'application/octet-stream' +} + +function mutationOutput( + operation: MutationOperation, + data: unknown, + fallbackIds: string[] +): WindchillRouteOutput { + const documents = normalizeWindchillDocuments(data) + const document = documents[0] ?? normalizeWindchillDocument(data) + const collectionIds = documents + .map((item) => item.id) + .filter((id): id is string => typeof id === 'string') + const returnedIds = + document?.id && !collectionIds.includes(document.id) + ? [document.id, ...collectionIds] + : collectionIds + const affectedIds = returnedIds.length > 0 ? returnedIds : fallbackIds + if (isDeleteOperation(operation)) return { operation, affectedIds: fallbackIds } + if (isBulkResultOperation(operation)) { + return { + operation, + affectedIds, + ...(documents.length > 0 ? { documents } : {}), + } + } + return { + operation, + affectedIds, + ...(document ? { document } : {}), + } +} + +async function executeMutation( + body: Exclude< + WindchillOperationBody, + | { operation: 'windchill_download_primary_content' } + | { operation: 'windchill_upload_primary_content' } + | { operation: 'windchill_download_attachment' } + | { operation: 'windchill_upload_attachments' } + >, + signal: AbortSignal +): Promise { + const session = await createWindchillSession(body, signal) + const root = normalizeServiceRoot(body.baseUrl) + + switch (body.operation) { + case 'windchill_create_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/Documents`, + method: 'POST', + body: { + ...(body.attributes ?? {}), + Name: body.name, + ...(body.number ? { Number: body.number } : {}), + ...(body.title ? { Title: body.title } : {}), + ...(body.description ? { Description: body.description } : {}), + 'Context@odata.bind': `Containers('${encodeWindchillOid(body.containerOid)}')`, + ...(body.folderOid + ? { 'Folder@odata.bind': `Folders('${encodeWindchillOid(body.folderOid)}')` } + : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, []) + } + case 'windchill_create_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/CreateDocuments`, + method: 'POST', + body: { + Documents: body.documents.map((document) => ({ + ...(document.attributes ?? {}), + Name: document.name, + ...(document.number ? { Number: document.number } : {}), + ...(document.title ? { Title: document.title } : {}), + ...(document.description ? { Description: document.description } : {}), + 'Context@odata.bind': `Containers('${encodeWindchillOid(document.containerOid)}')`, + ...(document.folderOid + ? { 'Folder@odata.bind': `Folders('${encodeWindchillOid(document.folderOid)}')` } + : {}), + })), + }, + signal, + }) + return mutationOutput(body.operation, data, []) + } + case 'windchill_update_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: windchillDocumentUrl(root, body.documentOid), + method: 'PATCH', + body: body.attributes, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_update_common_properties': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.UpdateCommonProperties`, + method: 'POST', + body: { Updates: body.commonProperties }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_update_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/UpdateDocuments`, + method: 'POST', + body: { + Documents: body.documents.map((document) => ({ + ...document.attributes, + ID: document.id, + })), + }, + signal, + }) + return mutationOutput( + body.operation, + data, + body.documents.map((document) => document.id) + ) + } + case 'windchill_delete_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: windchillDocumentUrl(root, body.documentOid), + method: 'DELETE', + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_delete_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/DeleteDocuments`, + method: 'POST', + body: { Documents: documentsById(body.documentOids) }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_check_out_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.CheckOut`, + method: 'POST', + body: { ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}) }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_check_out_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/CheckOutDocuments`, + method: 'POST', + body: { + Documents: documentsById(body.documentOids), + ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_check_in_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.CheckIn`, + method: 'POST', + body: { + ...(body.checkInNote ? { CheckInNote: body.checkInNote } : {}), + ...(body.keepCheckedOut !== undefined ? { KeepCheckedOut: body.keepCheckedOut } : {}), + ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_check_in_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/CheckInDocuments`, + method: 'POST', + body: { + Documents: documentsById(body.documentOids), + ...(body.checkInNote ? { CheckInNote: body.checkInNote } : {}), + ...(body.keepCheckedOut !== undefined ? { KeepCheckedOut: body.keepCheckedOut } : {}), + ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_undo_check_out_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.UndoCheckOut`, + method: 'POST', + body: {}, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_undo_check_out_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/UndoCheckOutDocuments`, + method: 'POST', + body: { Documents: documentsById(body.documentOids) }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_revise_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.Revise`, + method: 'POST', + body: { ...(body.versionId ? { VersionId: body.versionId } : {}) }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_revise_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/ReviseDocuments`, + method: 'POST', + body: { + Documents: documentsById(body.documentOids), + }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_set_lifecycle_state': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.SetState`, + method: 'POST', + body: { State: { Display: body.stateDisplay, Value: body.stateValue } }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_update_document_security_labels': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/EditDocumentsSecurityLabels`, + method: 'POST', + body: { + Documents: body.securityLabelUpdates.map((update) => ({ + ...update.labels, + ID: update.id, + })), + }, + signal, + }) + return mutationOutput( + body.operation, + data, + body.securityLabelUpdates.map((update) => update.id) + ) + } + } +} + +async function loadUploadFiles( + inputs: RawFileInput[], + userId: string, + requestId: string +): Promise { + let userFiles: UserFile[] + try { + userFiles = processFilesToUserFiles(inputs, requestId, logger) + } catch (error) { + return failureResponse(getErrorMessage(error, 'Invalid file input'), 400) + } + if (userFiles.length !== inputs.length) return failureResponse('Invalid file input', 400) + + const declaredTotal = userFiles.reduce((total, file) => total + file.size, 0) + if (declaredTotal > MAX_FILE_SIZE) { + return failureResponse('Combined Windchill upload exceeds the maximum file size', 413) + } + + const files: WindchillUploadFile[] = [] + let actualTotal = 0 + for (const userFile of userFiles) { + const denied = await assertToolFileAccess(userFile.key, userId, requestId, logger) + if (denied) return denied + try { + const servable = await downloadServableFileFromStorage(userFile, requestId, logger, { + maxBytes: MAX_FILE_SIZE - actualTotal, + }) + actualTotal += servable.buffer.length + if (actualTotal > MAX_FILE_SIZE) { + return failureResponse('Combined Windchill upload exceeds the maximum file size', 413) + } + files.push({ + name: sanitizeFileName(userFile.name), + mimeType: safeMimeType(servable.contentType || userFile.type), + size: servable.buffer.length, + buffer: servable.buffer, + }) + } catch (error) { + const notReady = docNotReadyResponse(error) + if (notReady) return notReady + return failureResponse( + getErrorMessage(error, 'Failed to read uploaded file'), + isPayloadSizeLimitError(error) ? 413 : 400 + ) + } + } + return files +} + +function contentDispositionFileName(value: string | null): string | null { + if (!value) return null + const encoded = value.match(/filename\*=UTF-8''([^;]+)/i)?.[1] + if (encoded) { + try { + return decodeURIComponent(encoded) + } catch { + return encoded + } + } + return ( + value.match(/filename\s*=\s*"([^"]+)"/i)?.[1] ?? + value.match(/filename\s*=\s*([^;]+)/i)?.[1]?.trim() ?? + null + ) +} + +async function storeDownloadedFile({ + principal, + buffer, + fileName, + contentType, +}: { + principal: WorkflowExecutionDelegatedPrincipal + buffer: Buffer + fileName: string + contentType: string +}): Promise { + const { workflowId, executionId } = principal.delegationContext + if (executionId) { + return uploadExecutionFile( + { + workspaceId: principal.workspaceId, + workflowId, + executionId, + }, + buffer, + fileName, + contentType, + principal.subjectUserId + ) + } + return uploadCopilotFile({ + buffer, + fileName, + contentType, + userId: principal.subjectUserId, + }) +} + +async function executeDownload( + body: Extract< + WindchillOperationBody, + | { operation: 'windchill_download_primary_content' } + | { operation: 'windchill_download_attachment' } + >, + principal: WorkflowExecutionDelegatedPrincipal, + signal: AbortSignal +): Promise { + const documentUrl = windchillDocumentUrl(body.baseUrl, body.documentOid) + const contentPath = + body.operation === 'windchill_download_primary_content' + ? `${documentUrl}/PrimaryContent` + : `${documentUrl}/Attachments('${encodeWindchillOid(body.attachmentOid)}')` + const contentUrl = await resolveWindchillContentUrl({ + params: body, + contentPath, + signal, + }) + const downloaded = await downloadWindchillContent({ + params: body, + url: contentUrl, + maxBytes: MAX_FILE_SIZE, + signal, + }) + const fallback = + body.operation === 'windchill_download_primary_content' + ? 'windchill-primary-content.bin' + : 'windchill-attachment.bin' + const fileName = sanitizeFileName( + body.fileName || contentDispositionFileName(downloaded.contentDisposition) || fallback + ) + const mimeType = safeMimeType(downloaded.contentType) + const file = await storeDownloadedFile({ + principal, + buffer: downloaded.buffer, + fileName, + contentType: mimeType, + }) + return { + operation: body.operation, + file: { ...file }, + fileName, + mimeType, + } +} + +export const POST = withRouteHandler( + async (request: NextRequest) => { + const requestId = generateRequestId() + let principal: WorkflowExecutionDelegatedPrincipal + try { + principal = await authenticateWindchillExecutor(request) + } catch (error) { + if (error instanceof InternalUnauthenticatedError) { + return failureResponse(error.message, 401) + } + throw error + } + + const parsed = await parseRequest( + windchillOperationContract, + request, + {}, + { + validationErrorResponse: (error) => + failureResponse(getValidationErrorMessage(error, 'Invalid Windchill request'), 400), + invalidJsonResponse: () => + failureResponse('Windchill request body must be valid JSON', 400), + payloadTooLargeResponse: () => failureResponse('Windchill request body is too large', 413), + } + ) + if (!parsed.success) return parsed.response + const body = parsed.data.body + + try { + if ( + body.operation === 'windchill_download_primary_content' || + body.operation === 'windchill_download_attachment' + ) { + return successResponse(await executeDownload(body, principal, request.signal)) + } + + if ( + body.operation === 'windchill_upload_primary_content' || + body.operation === 'windchill_upload_attachments' + ) { + const inputs = + body.operation === 'windchill_upload_primary_content' + ? [body.primaryFile] + : body.attachmentFiles + const files = await loadUploadFiles(inputs, principal.subjectUserId, requestId) + if (files instanceof NextResponse) return files + const uploadedFileNames = await uploadWindchillContent({ + params: body, + documentOid: body.documentOid, + files, + primaryContent: body.operation === 'windchill_upload_primary_content', + signal: request.signal, + }) + return successResponse({ + operation: body.operation, + affectedIds: [body.documentOid], + uploadedFileNames, + }) + } + + return successResponse(await executeMutation(body, request.signal)) + } catch (error) { + logger.error('Windchill operation failed', { + operation: body.operation, + error: sanitizeWindchillError(getErrorMessage(error, 'Windchill operation failed')), + }) + if (error instanceof WindchillProviderError) { + const status = error.status >= 400 && error.status <= 599 ? error.status : 502 + return failureResponse(error.message, status) + } + return failureResponse( + getErrorMessage(error, 'Windchill operation failed'), + isPayloadSizeLimitError(error) ? 413 : 500 + ) + } + }, + { + unhandledErrorResponse: () => failureResponse('Windchill operation failed', 500), + } +) diff --git a/apps/sim/app/api/users/me/usage-logs/cursor-route.test.ts b/apps/sim/app/api/users/me/usage-logs/cursor-route.test.ts new file mode 100644 index 00000000000..5d747a37d0c --- /dev/null +++ b/apps/sim/app/api/users/me/usage-logs/cursor-route.test.ts @@ -0,0 +1,50 @@ +/** + * @vitest-environment node + * + * The unresolvable-cursor rejection, end to end on the session-only ledger route. + * + * Deliberately a separate file from `route.test.ts`: that suite replaces + * `@/lib/billing/core/usage-log` with mocks, which is exactly the seam this case + * has to cross. Here the real query runs against the shared `@sim/db` chain mock, + * so the assertion covers the throw in billing core, `withRouteHandler`'s typed-error + * projection, and the message the caller reads — the path that answered 500 while the + * rejection was an `OrchestrationError` alone. + */ +import { authMockFns, createMockRequest, dbChainMockFns, resetDbChainMock } from '@sim/testing' +import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' +import { UNKNOWN_CURSOR_MESSAGE } from '@/lib/billing/core/usage-log' +import { GET } from '@/app/api/users/me/usage-logs/route' + +afterAll(() => { + resetDbChainMock() +}) + +describe('GET /api/users/me/usage-logs cursor rejection', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } }) + }) + + it('answers 400 when the cursor names no usage event', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([]) + + const response = await GET( + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/users/me/usage-logs?cursor=log-from-another-ledger' + ) + ) + + expect(response.status).toBe(400) + await expect(response.json()).resolves.toMatchObject({ error: UNKNOWN_CURSOR_MESSAGE }) + }) + + it('answers 200 for a request carrying no cursor', async () => { + const response = await GET(createMockRequest('GET')) + + expect(response.status).toBe(200) + }) +}) diff --git a/apps/sim/app/api/v1/admin/responses.ts b/apps/sim/app/api/v1/admin/responses.ts index 9308df895dc..3ecc5353b29 100644 --- a/apps/sim/app/api/v1/admin/responses.ts +++ b/apps/sim/app/api/v1/admin/responses.ts @@ -51,9 +51,7 @@ export function errorResponse( return NextResponse.json(body, { status }) } -// ============================================================================= // Common Error Responses -// ============================================================================= export function unauthorizedResponse(message = 'Authentication required'): NextResponse { return errorResponse('UNAUTHORIZED', message, 401) diff --git a/apps/sim/app/api/v1/admin/types.ts b/apps/sim/app/api/v1/admin/types.ts index 4256076d457..a6062ca8eee 100644 --- a/apps/sim/app/api/v1/admin/types.ts +++ b/apps/sim/app/api/v1/admin/types.ts @@ -20,9 +20,7 @@ import type { InferSelectModel } from 'drizzle-orm' import type { Edge } from 'reactflow' import type { BlockState, Loop, Parallel } from '@/stores/workflows/workflow/types' -// ============================================================================= // Database Model Types (inferred from schema) -// ============================================================================= export type DbUser = InferSelectModel export type DbWorkspace = InferSelectModel @@ -33,9 +31,7 @@ export type DbSubscription = InferSelectModel export type DbMember = InferSelectModel export type DbUserStats = InferSelectModel -// ============================================================================= // Pagination -// ============================================================================= export interface PaginationParams { limit: number @@ -74,9 +70,7 @@ export function createPaginationMeta(total: number, limit: number, offset: numbe } } -// ============================================================================= // API Response Types -// ============================================================================= export interface AdminListResponse { data: T[] @@ -95,9 +89,7 @@ export interface AdminErrorResponse { } } -// ============================================================================= // User Types -// ============================================================================= export interface AdminUser { id: string @@ -121,9 +113,7 @@ export function toAdminUser(dbUser: DbUser): AdminUser { } } -// ============================================================================= // Workspace Types -// ============================================================================= export interface AdminWorkspace { id: string @@ -148,9 +138,7 @@ export function toAdminWorkspace(dbWorkspace: DbWorkspace): AdminWorkspace { } } -// ============================================================================= // Folder Types -// ============================================================================= export interface AdminFolder { id: string @@ -179,9 +167,7 @@ export function toAdminFolder(dbFolder: DbWorkflowFolder): AdminFolder { } } -// ============================================================================= // Workflow Types -// ============================================================================= export interface AdminWorkflow { id: string @@ -233,9 +219,7 @@ export function toAdminWorkflow(dbWorkflow: AdminWorkflowSource): AdminWorkflow } } -// ============================================================================= // Workflow Variable Types -// ============================================================================= export type VariableType = 'string' | 'number' | 'boolean' | 'object' | 'array' | 'plain' @@ -246,9 +230,7 @@ export interface WorkflowVariable { value: unknown } -// ============================================================================= // Export/Import Types -// ============================================================================= export interface WorkflowExportState { blocks: Record @@ -296,9 +278,7 @@ export interface WorkspaceExportPayload { folders: FolderExportPayload[] } -// ============================================================================= // Import Types -// ============================================================================= export interface WorkflowImportRequest { workspaceId: string @@ -328,9 +308,7 @@ export interface WorkspaceImportResponse { results: ImportResult[] } -// ============================================================================= // Utility Functions -// ============================================================================= /** * Extract workflow metadata from various export formats. @@ -384,9 +362,7 @@ function getNestedString(obj: Record, path: string): string | u return typeof current === 'string' ? current : undefined } -// ============================================================================= // Organization Types -// ============================================================================= export interface AdminOrganization { id: string @@ -432,9 +408,7 @@ export function toAdminOrganization(dbOrg: AdminOrganizationSource): AdminOrgani } } -// ============================================================================= // Subscription Types -// ============================================================================= export interface AdminSubscription { id: string @@ -470,9 +444,7 @@ export function toAdminSubscription(dbSub: DbSubscription): AdminSubscription { } } -// ============================================================================= // Member Types -// ============================================================================= export interface AdminMember { id: string @@ -492,9 +464,7 @@ export interface AdminMemberDetail extends AdminMember { billingBlocked: boolean } -// ============================================================================= // Workspace Member Types -// ============================================================================= export interface AdminWorkspaceMember { id: string @@ -508,9 +478,7 @@ export interface AdminWorkspaceMember { userImage: string | null } -// ============================================================================= // User Billing Types -// ============================================================================= interface AdminUserBilling { userId: string @@ -539,9 +507,7 @@ export interface AdminUserBillingWithSubscription extends AdminUserBilling { }> } -// ============================================================================= // Organization Billing Summary Types -// ============================================================================= export interface AdminOrganizationBillingSummary { organizationId: string @@ -587,9 +553,7 @@ export interface AdminDeploymentVersion { deployedByName: string | null } -// ============================================================================= // Audit Log Types -// ============================================================================= export type DbAuditLog = InferSelectModel diff --git a/apps/sim/app/api/v1/auth.test.ts b/apps/sim/app/api/v1/auth.test.ts new file mode 100644 index 00000000000..90f0f2f1d76 --- /dev/null +++ b/apps/sim/app/api/v1/auth.test.ts @@ -0,0 +1,86 @@ +/** + * @vitest-environment node + */ + +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + authenticateApiKey: vi.fn(), + updateLastUsed: vi.fn(), +})) + +vi.mock('@/lib/core/config/env-flags', () => ({ isAuthDisabled: false })) +vi.mock('@/lib/api-key/service', () => ({ + authenticateApiKeyFromHeader: mocks.authenticateApiKey, + updateApiKeyLastUsed: mocks.updateLastUsed, +})) + +import { authenticateV1Request } from '@/app/api/v1/auth' + +describe('v1 API key authentication', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('constructs a personal API-key Principal from canonical key identity', async () => { + mocks.authenticateApiKey.mockResolvedValue({ + success: true, + userId: 'user-1', + keyId: 'key-1', + keyType: 'personal', + }) + + await expect( + authenticateV1Request( + new NextRequest('http://localhost/api/v1/files', { + headers: { 'x-api-key': 'secret' }, + }) + ) + ).resolves.toMatchObject({ + authenticated: true, + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, + }) + }) + + it('constructs a workspace API-key Principal without borrowing the creator identity', async () => { + mocks.authenticateApiKey.mockResolvedValue({ + success: true, + userId: 'creator-1', + keyId: 'key-1', + keyType: 'workspace', + workspaceId: 'workspace-1', + }) + + const result = await authenticateV1Request( + new NextRequest('http://localhost/api/v1/files', { + headers: { 'x-api-key': 'secret' }, + }) + ) + + expect(result.principal).toEqual({ + kind: 'workspace_api_key', + workspaceId: 'workspace-1', + keyId: 'key-1', + }) + expect(result.principal).not.toHaveProperty('userId') + }) + + it('fails closed when authenticated key identity is incomplete', async () => { + mocks.authenticateApiKey.mockResolvedValue({ + success: true, + userId: 'creator-1', + keyId: 'key-1', + keyType: 'workspace', + }) + + await expect( + authenticateV1Request( + new NextRequest('http://localhost/api/v1/files', { + headers: { 'x-api-key': 'secret' }, + }) + ) + ).resolves.toEqual({ authenticated: false, error: 'Authentication failed' }) + expect(mocks.updateLastUsed).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v1/auth.ts b/apps/sim/app/api/v1/auth.ts index 0f391889005..78c68e1f9dd 100644 --- a/apps/sim/app/api/v1/auth.ts +++ b/apps/sim/app/api/v1/auth.ts @@ -1,3 +1,4 @@ +import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { authenticateApiKeyFromHeader, updateApiKeyLastUsed } from '@/lib/api-key/service' @@ -11,6 +12,7 @@ export interface AuthResult { userId?: string workspaceId?: string keyType?: 'personal' | 'workspace' + principal?: PersonalApiKeyPrincipal | WorkspaceApiKeyPrincipal error?: string } @@ -20,6 +22,11 @@ export async function authenticateV1Request(request: NextRequest): Promise ({ mockCheckRateLimit: vi.fn(), mockValidateWorkspaceAccess: vi.fn(), mockGetWorkspaceFile: vi.fn(), - mockFetchServableWorkspaceFileBuffer: vi.fn(), + mockDownloadWorkspaceFileStream: vi.fn(), })) vi.mock('@/app/api/v1/middleware', () => ({ checkRateLimit: mockCheckRateLimit, createRateLimitResponse: () => new Response('rate limited', { status: 429 }), + requireRateLimitPrincipal: (rateLimit: { principal: unknown }) => rateLimit.principal, validateWorkspaceAccess: mockValidateWorkspaceAccess, v1ValidationErrorResponse: (e: { issues: unknown[] }) => NextResponse.json({ error: 'Validation error', details: e.issues }, { status: 400 }), })) vi.mock('@/lib/uploads/contexts/workspace', () => ({ getWorkspaceFile: mockGetWorkspaceFile, - fetchServableWorkspaceFileBuffer: mockFetchServableWorkspaceFileBuffer, +})) +vi.mock('@/lib/workspace-files/application/download-workspace-file', () => ({ + downloadWorkspaceFileStream: { execute: mockDownloadWorkspaceFileStream }, })) vi.mock('@/lib/workspace-files/orchestration', () => ({ performDeleteWorkspaceFileItems: vi.fn(), @@ -37,7 +40,7 @@ vi.mock('@sim/audit', () => ({ })) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn() })) -import { DocCompileUserError } from '@/lib/copilot/tools/server/files/doc-compile-error' +import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET } from '@/app/api/v1/files/[fileId]/route' const WORKSPACE_ID = 'ws-1' @@ -45,6 +48,11 @@ const FILE_ID = 'file-1' const context = { params: Promise.resolve({ fileId: FILE_ID }) } const DOCX_MIME = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' +const PRINCIPAL = { + kind: 'personal_api_key' as const, + userId: 'user-1', + keyId: 'key-1', +} function request() { return createMockRequest( @@ -71,16 +79,31 @@ function generatedDocument(name = 'report.docx') { } } +function renderedDownload(buffer: Buffer) { + return { + file: generatedDocument(), + stream: new ReadableStream({ + start(controller) { + controller.enqueue(buffer) + controller.close() + }, + }), + contentLength: buffer.length, + contentType: DOCX_MIME, + } +} + describe('v1 file download', () => { beforeEach(() => { vi.clearAllMocks() - mockCheckRateLimit.mockResolvedValue({ allowed: true, userId: 'user-1' }) + mockCheckRateLimit.mockResolvedValue({ + allowed: true, + userId: 'user-1', + principal: PRINCIPAL, + }) mockValidateWorkspaceAccess.mockResolvedValue(null) mockGetWorkspaceFile.mockResolvedValue(generatedDocument()) - mockFetchServableWorkspaceFileBuffer.mockResolvedValue({ - buffer: Buffer.from('PKrendered'), - contentType: DOCX_MIME, - }) + mockDownloadWorkspaceFileStream.mockResolvedValue(renderedDownload(Buffer.from('PKrendered'))) }) it('serves the rendered bytes and the rendered content type', async () => { @@ -104,10 +127,7 @@ describe('v1 file download', () => { it('reports Content-Length from the rendered bytes, not the declared source size', async () => { const rendered = Buffer.alloc(50_000) - mockFetchServableWorkspaceFileBuffer.mockResolvedValue({ - buffer: rendered, - contentType: DOCX_MIME, - }) + mockDownloadWorkspaceFileStream.mockResolvedValue(renderedDownload(rendered)) const response = await GET(request(), context) @@ -115,8 +135,8 @@ describe('v1 file download', () => { }) it('returns a retryable 409 while the artifact is still compiling', async () => { - mockFetchServableWorkspaceFileBuffer.mockRejectedValue( - new DocCompileUserError('Document is still being generated') + mockDownloadWorkspaceFileStream.mockRejectedValue( + new OrchestrationError('conflict', 'Document is still being generated') ) const response = await GET(request(), context) @@ -127,11 +147,17 @@ describe('v1 file download', () => { }) it('404s a file that does not exist', async () => { - mockGetWorkspaceFile.mockResolvedValue(null) + mockDownloadWorkspaceFileStream.mockRejectedValue( + new OrchestrationError('not_found', 'File not found') + ) const response = await GET(request(), context) expect(response.status).toBe(404) - expect(mockFetchServableWorkspaceFileBuffer).not.toHaveBeenCalled() + expect(mockDownloadWorkspaceFileStream).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: { fileId: FILE_ID, assertedWorkspaceId: WORKSPACE_ID }, + request: expect.anything(), + }) }) }) diff --git a/apps/sim/app/api/v1/files/[fileId]/route.ts b/apps/sim/app/api/v1/files/[fileId]/route.ts index 1e9b084e680..eb767ac5f3c 100644 --- a/apps/sim/app/api/v1/files/[fileId]/route.ts +++ b/apps/sim/app/api/v1/files/[fileId]/route.ts @@ -1,20 +1,18 @@ -import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' import { createLogger } from '@sim/logger' import { type NextRequest, NextResponse } from 'next/server' import { v1DeleteFileContract, v1DownloadFileContract } from '@/lib/api/contracts/v1/files' import { parseRequest } from '@/lib/api/server' +import { asOrchestrationError, statusForOrchestrationError } from '@/lib/core/orchestration/types' import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { captureServerEvent } from '@/lib/posthog/server' -import { - fetchServableWorkspaceFileBuffer, - getWorkspaceFile, -} from '@/lib/uploads/contexts/workspace' -import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' +import { getWorkspaceFile } from '@/lib/uploads/contexts/workspace' +import { downloadWorkspaceFileStream } from '@/lib/workspace-files/application/download-workspace-file' import { performDeleteWorkspaceFileItems } from '@/lib/workspace-files/orchestration' import { checkRateLimit, createRateLimitResponse, + requireRateLimitPrincipal, v1ValidationErrorResponse, validateWorkspaceAccess, } from '@/app/api/v1/middleware' @@ -38,7 +36,6 @@ export const GET = withRouteHandler(async (request: NextRequest, context: FileRo return createRateLimitResponse(rateLimit) } - const userId = rateLimit.userId! const parsed = await parseRequest(v1DownloadFileContract, request, context, { validationErrorResponse: v1ValidationErrorResponse, }) @@ -47,64 +44,43 @@ export const GET = withRouteHandler(async (request: NextRequest, context: FileRo const { fileId } = parsed.data.params const { workspaceId } = parsed.data.query - const accessError = await validateWorkspaceAccess(rateLimit, userId, workspaceId) - if (accessError) return accessError - - const fileRecord = await getWorkspaceFile(workspaceId, fileId) - if (!fileRecord) { - return NextResponse.json({ error: 'File not found' }, { status: 404 }) + const principal = requireRateLimitPrincipal(rateLimit) + const { file, stream, contentLength, contentType } = await downloadWorkspaceFileStream.execute({ + principal, + input: { fileId, assertedWorkspaceId: workspaceId }, + request, + }) + if (principal.kind === 'personal_api_key') { + captureServerEvent( + principal.userId, + 'file_downloaded', + { workspace_id: workspaceId, is_bulk: false, file_count: 1 }, + { groups: { workspace: workspaceId } } + ) } - // Generated docs store their generation source; serve the rendered artifact. - // Its content type is the rendered one, not the source MIME on the record. - const { buffer, contentType } = await fetchServableWorkspaceFileBuffer(fileRecord) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FILE_DOWNLOADED, - resourceType: AuditResourceType.FILE, - resourceId: fileRecord.id, - resourceName: fileRecord.name, - description: `Downloaded file "${fileRecord.name}" via API`, - metadata: { - fileId: fileRecord.id, - fileName: fileRecord.name, - bytes: buffer.length, - source: 'api_v1', + return new Response(stream, { + status: 200, + headers: { + 'Content-Type': contentType || file.type || 'application/octet-stream', + 'Content-Disposition': `attachment; filename="${file.name.replace(/[^\w.-]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(file.name)}`, + 'Content-Length': String(contentLength), + 'X-File-Id': file.id, + 'X-File-Name': encodeURIComponent(file.name), + 'X-Uploaded-At': + file.uploadedAt instanceof Date ? file.uploadedAt.toISOString() : String(file.uploadedAt), }, - request, }) - captureServerEvent( - userId, - 'file_downloaded', - { workspace_id: workspaceId, is_bulk: false, file_count: 1 }, - { groups: { workspace: workspaceId } } - ) - - // View, not copy — a second full copy would double peak memory for a large file. - return new Response( - new Uint8Array(buffer.buffer as ArrayBuffer, buffer.byteOffset, buffer.byteLength), - { - status: 200, - headers: { - 'Content-Type': contentType || fileRecord.type || 'application/octet-stream', - 'Content-Disposition': `attachment; filename="${fileRecord.name.replace(/[^\w.-]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(fileRecord.name)}`, - 'Content-Length': String(buffer.length), - 'X-File-Id': fileRecord.id, - 'X-File-Name': encodeURIComponent(fileRecord.name), - 'X-Uploaded-At': - fileRecord.uploadedAt instanceof Date - ? fileRecord.uploadedAt.toISOString() - : String(fileRecord.uploadedAt), - }, - } - ) } catch (error) { - // A generated doc whose artifact is still compiling is retryable, not a fault: - // without this the caller sees a 500 and has no reason to try again. - if (isDocNotReadyError(error)) { - return NextResponse.json({ error: docNotReadyMessage() }, { status: 409 }) + const orchestrationError = asOrchestrationError(error) + if (orchestrationError && orchestrationError.code !== 'internal') { + return NextResponse.json( + { + error: + orchestrationError.code === 'not_found' ? 'File not found' : orchestrationError.message, + }, + { status: statusForOrchestrationError(orchestrationError.code) } + ) } logger.error(`[${requestId}] Error downloading file:`, error) return NextResponse.json({ error: 'Failed to download file' }, { status: 500 }) diff --git a/apps/sim/app/api/v1/middleware.test.ts b/apps/sim/app/api/v1/middleware.test.ts index 94c0790b274..2a8e598a7b6 100644 --- a/apps/sim/app/api/v1/middleware.test.ts +++ b/apps/sim/app/api/v1/middleware.test.ts @@ -61,6 +61,7 @@ describe('checkRateLimit', () => { authenticated: true, userId: 'user-1', keyType: 'personal', + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, }) mockGetSubscription.mockResolvedValue({ plan: 'team' }) mockGetRateLimit.mockReturnValue(TEAM_BUCKET) @@ -78,6 +79,16 @@ describe('checkRateLimit', () => { expect(result.limit).not.toBe(TEAM_BUCKET.refillRate) }) + it('preserves the authenticated API-key Principal for application operations', async () => { + const result = await checkRateLimit(request(), 'workflows') + + expect(result.principal).toEqual({ + kind: 'personal_api_key', + userId: 'user-1', + keyId: 'key-1', + }) + }) + it('never reports more remaining than the limit', async () => { const result = await checkRateLimit(request(), 'workflows') @@ -196,6 +207,7 @@ describe('rate-limit snapshot context', () => { authenticated: true, userId: 'user-1', keyType: 'personal', + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, }) mockGetSubscription.mockResolvedValue({ plan: 'team' }) mockGetRateLimit.mockReturnValue(TEAM_BUCKET) diff --git a/apps/sim/app/api/v1/middleware.ts b/apps/sim/app/api/v1/middleware.ts index 6a9db50ec44..3f8d4878119 100644 --- a/apps/sim/app/api/v1/middleware.ts +++ b/apps/sim/app/api/v1/middleware.ts @@ -1,3 +1,4 @@ +import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { type PermissionType, permissionSatisfies } from '@sim/platform-authz/workspace' import { type NextRequest, NextResponse } from 'next/server' @@ -63,6 +64,7 @@ export interface RateLimitResult { userId?: string workspaceId?: string keyType?: 'personal' | 'workspace' + principal?: PersonalApiKeyPrincipal | WorkspaceApiKeyPrincipal error?: string } @@ -82,6 +84,18 @@ export function requireRateLimitUserId(rateLimit: RateLimitResult): string { return rateLimit.userId } +export function requireRateLimitPrincipal( + rateLimit: RateLimitResult +): PersonalApiKeyPrincipal | WorkspaceApiKeyPrincipal { + if (!rateLimit.allowed) { + throw new Error('Cannot authorize a denied public API request') + } + if (!rateLimit.principal) { + throw new Error('Allowed public API request is missing its Principal') + } + return rateLimit.principal +} + export async function checkRateLimit( request: NextRequest, endpoint: ApiEndpoint = 'logs' @@ -144,6 +158,7 @@ export async function checkRateLimit( userId, workspaceId: auth.workspaceId, keyType: auth.keyType, + principal: auth.principal, } } catch (error) { logger.error('Rate limit check error', { error }) diff --git a/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts b/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts index aa3f74d8157..ae8c00affab 100644 --- a/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts +++ b/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts @@ -12,10 +12,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumn, deleteColumn } from '@/lib/table' import { signalTableSchemaChanged } from '@/lib/table/events' import { performUpdateTableColumn } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationErrorResponse, orchestrationOutcomeErrorResponse, tableLockErrorResponse, diff --git a/apps/sim/app/api/v1/tables/[tableId]/route.ts b/apps/sim/app/api/v1/tables/[tableId]/route.ts index caaf87d8be7..5d46bdf619b 100644 --- a/apps/sim/app/api/v1/tables/[tableId]/route.ts +++ b/apps/sim/app/api/v1/tables/[tableId]/route.ts @@ -6,10 +6,10 @@ import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import type { TableSchema } from '@/lib/table' import { performDeleteTable } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, tableLockErrorResponse, } from '@/app/api/table/utils' diff --git a/apps/sim/app/api/v1/tables/route.test.ts b/apps/sim/app/api/v1/tables/route.test.ts index ded5f484e8d..f12bceb2334 100644 --- a/apps/sim/app/api/v1/tables/route.test.ts +++ b/apps/sim/app/api/v1/tables/route.test.ts @@ -35,9 +35,11 @@ vi.mock('@/app/api/v1/middleware', () => ({ })) vi.mock('@/app/api/table/utils', () => ({ - normalizeColumn: (column: unknown) => column, orchestrationErrorResponse: mocks.orchestrationErrorResponse, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (column: unknown) => column, +})) vi.mock('@/lib/table', () => ({ createTable: mocks.createTable, diff --git a/apps/sim/app/api/v1/tables/route.ts b/apps/sim/app/api/v1/tables/route.ts index e8a13eb9090..ecd742efb29 100644 --- a/apps/sim/app/api/v1/tables/route.ts +++ b/apps/sim/app/api/v1/tables/route.ts @@ -12,7 +12,8 @@ import { TableConflictError, type TableSchema, } from '@/lib/table' -import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' +import { orchestrationErrorResponse } from '@/app/api/table/utils' import { checkRateLimit, createRateLimitResponse, diff --git a/apps/sim/app/api/v2/[[...segments]]/route.test.ts b/apps/sim/app/api/v2/[[...segments]]/route.test.ts new file mode 100644 index 00000000000..4d4e14da713 --- /dev/null +++ b/apps/sim/app/api/v2/[[...segments]]/route.test.ts @@ -0,0 +1,50 @@ +/** + * @vitest-environment node + */ +import { NextRequest } from 'next/server' +import { describe, expect, it } from 'vitest' +import { DELETE, GET, PATCH, POST, PUT } from '@/app/api/v2/[[...segments]]/route' + +/** + * An unknown path under `/api/v2` used to fall through to the app's global + * `not-found` page, so a mistyped URL handed an API client a full HTML document + * — the one v2 response a JSON-parsing caller cannot read. + * + * The body must stay byte-identical to the rollout gate's 404 + * (`v2ApiGateError`), which answers 404 so an ungated caller cannot tell "not in + * the cohort" from "no such endpoint". A different body here would give that + * distinction straight back. + */ +describe('unknown /api/v2 path', () => { + const EXPECTED = { error: { code: 'NOT_FOUND', message: 'Not found' } } + + function probe(method: string) { + return new NextRequest('http://localhost/api/v2/nonexistent', { method }) + } + + it('answers JSON, not an HTML document', async () => { + const response = await GET(probe('GET'), undefined) + + expect(response.status).toBe(404) + expect(response.headers.get('content-type')).toContain('application/json') + expect(await response.json()).toEqual(EXPECTED) + }) + + it.each([ + ['POST', POST], + ['PUT', PUT], + ['PATCH', PATCH], + ['DELETE', DELETE], + ])('answers the same envelope for %s', async (method, handler) => { + const response = await handler(probe(method), undefined) + + expect(response.status).toBe(404) + expect(await response.json()).toEqual(EXPECTED) + }) + + it('does not require an API key, so probing a typo cannot become a 401', async () => { + const response = await GET(probe('GET'), undefined) + + expect(response.status).toBe(404) + }) +}) diff --git a/apps/sim/app/api/v2/[[...segments]]/route.ts b/apps/sim/app/api/v2/[[...segments]]/route.ts new file mode 100644 index 00000000000..893f455a7bc --- /dev/null +++ b/apps/sim/app/api/v2/[[...segments]]/route.ts @@ -0,0 +1,40 @@ +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { v2Error } from '@/app/api/v2/lib/response' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * JSON 404 for any `/api/v2` path that matches no route file. + * + * Without it a mistyped path falls through to the app's global `not-found` + * page and hands an API client a full HTML document, which is the one v2 + * response a JSON-parsing caller cannot read. Every other v2 failure — including + * the rollout gate's own 404 — is the `{ error: { code, message } }` envelope. + * + * The body is deliberately byte-identical to `v2ApiGateError`'s. The gate + * answers 404 so an ungated caller cannot distinguish "not in the rollout + * cohort" from "no such endpoint"; a different body here would reintroduce + * exactly that distinction. + * + * This is a documented raw-`withRouteHandler` route rather than a contract + * builder: it has no contract, no operation, and no authentication, because a + * caller probing an unknown path must get the same answer whether or not it + * holds a key — requiring auth first would turn the 404 into a 401 and confirm + * that the path is special. + * + * Next.js only routes a request here when no literal segment matches, so every + * real v2 route file is unaffected, however many there are. The optional form (`[[...segments]]`) also + * covers bare `/api/v2`. It cannot fix a 405 on a path that *does* have a route + * file but does not export that verb — Next generates that response itself, + * before any handler runs. + */ +const notFound = () => v2Error('NOT_FOUND', 'Not found') + +export const GET = withRouteHandler(notFound) +export const POST = withRouteHandler(notFound) +export const PUT = withRouteHandler(notFound) +export const PATCH = withRouteHandler(notFound) +export const DELETE = withRouteHandler(notFound) +export const HEAD = withRouteHandler(notFound) +export const OPTIONS = withRouteHandler(notFound) diff --git a/apps/sim/app/api/v2/billing/logs/route.test.ts b/apps/sim/app/api/v2/billing/logs/route.test.ts index 0c5e5f79387..b323e834a83 100644 --- a/apps/sim/app/api/v2/billing/logs/route.test.ts +++ b/apps/sim/app/api/v2/billing/logs/route.test.ts @@ -24,6 +24,8 @@ vi.mock('@/lib/billing/application/list-billing-logs', () => ({ listBillingLogs: { operation: { id: 'billing.logs.list' }, execute: mocks.execute }, })) +import { UNKNOWN_CURSOR_MESSAGE } from '@/lib/billing/core/usage-log' +import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET } from '@/app/api/v2/billing/logs/route' const auth = { @@ -95,6 +97,21 @@ describe('GET /api/v2/billing/logs', () => { }) }) + it('projects an unresolvable cursor as a 400 rather than an unpositioned first page', async () => { + mocks.execute.mockRejectedValueOnce( + new OrchestrationError('validation', UNKNOWN_CURSOR_MESSAGE) + ) + + const response = await GET( + new NextRequest('http://localhost:3000/api/v2/billing/logs?cursor=log-from-another-ledger') + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: UNKNOWN_CURSOR_MESSAGE }, + }) + }) + it('authenticates before rejecting invalid custom ranges', async () => { const response = await GET( new NextRequest('http://localhost:3000/api/v2/billing/logs?period=custom') diff --git a/apps/sim/app/api/v2/billing/status/route.test.ts b/apps/sim/app/api/v2/billing/status/route.test.ts index e664f896b36..d7ccd3d07e4 100644 --- a/apps/sim/app/api/v2/billing/status/route.test.ts +++ b/apps/sim/app/api/v2/billing/status/route.test.ts @@ -80,6 +80,19 @@ describe('GET /api/v2/billing/status', () => { expect(await response.json()).toEqual({ data: { ...result, credits: null, storage: null } }) }) + it.each(['workspaceID', 'workspace_id', 'workspace'])( + 'rejects %s rather than silently answering for the account payer', + async (key) => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/billing/status?${key}=workspace-1`) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ error: { code: 'BAD_REQUEST' } }) + expect(mocks.execute).not.toHaveBeenCalled() + } + ) + it('projects typed workspace-policy errors', async () => { mocks.execute.mockRejectedValueOnce( new OrchestrationError('forbidden', 'API key is not authorized for this workspace') diff --git a/apps/sim/app/api/v2/credentials/route.test.ts b/apps/sim/app/api/v2/credentials/route.test.ts index a1987f1b5af..a84ba1f6d68 100644 --- a/apps/sim/app/api/v2/credentials/route.test.ts +++ b/apps/sim/app/api/v2/credentials/route.test.ts @@ -27,6 +27,7 @@ vi.mock('@/lib/credentials/application/list-workspace-credentials', () => ({ }, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' import { GET } from '@/app/api/v2/credentials/route' const WORKSPACE_ID = '11111111-2222-4333-8444-555555555555' @@ -65,7 +66,12 @@ describe('GET /api/v2/credentials', () => { v2RouteMocks.gate.mockResolvedValue(null) v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) - mocks.execute.mockResolvedValue({ credentials: [credential] }) + mocks.execute.mockResolvedValue({ + credentials: [credential], + nextCursorKeys: null, + sortBy: 'createdAt', + sortOrder: 'desc', + }) }) it('authenticates and charges before validating workspace input', async () => { @@ -93,6 +99,9 @@ describe('GET /api/v2/credentials', () => { search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: V2_DEFAULT_PAGE_SIZE, + cursor: undefined, + cursorKeys: undefined, }, request, }) diff --git a/apps/sim/app/api/v2/credentials/route.ts b/apps/sim/app/api/v2/credentials/route.ts index 0312ea3a957..057ab1012be 100644 --- a/apps/sim/app/api/v2/credentials/route.ts +++ b/apps/sim/app/api/v2/credentials/route.ts @@ -8,6 +8,7 @@ import { import { listWorkspaceCredentials } from '@/lib/credentials/application/list-workspace-credentials' import { credentialOperations } from '@/lib/credentials/application/operations' import { toV2Credential } from '@/app/api/v2/credentials/utils' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -19,10 +20,15 @@ export const GET = defineV2JsonRoute({ operation: credentialOperations.listConnections, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + ...query, + cursorKeys: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + }), useCase: listWorkspaceCredentials, - present: ({ credentials }) => ({ + present: ({ credentials, nextCursorKeys, sortBy, sortOrder }) => ({ data: credentials.map(toV2Credential), - nextCursor: null, + nextCursor: nextCursorKeys + ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextCursorKeys) + : null, }), }) diff --git a/apps/sim/app/api/v2/custom-tools/route.test.ts b/apps/sim/app/api/v2/custom-tools/route.test.ts index f0d113e8961..b4609531cff 100644 --- a/apps/sim/app/api/v2/custom-tools/route.test.ts +++ b/apps/sim/app/api/v2/custom-tools/route.test.ts @@ -54,6 +54,7 @@ vi.mock('@/lib/custom-tools/application/use-cases', () => ({ }, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' import { GET, POST } from '@/app/api/v2/custom-tools/route' const WORKSPACE_ID = 'workspace-1' @@ -121,9 +122,10 @@ describe('/api/v2/custom-tools', () => { principal: PRINCIPAL, input: { workspaceId: WORKSPACE_ID, - search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: V2_DEFAULT_PAGE_SIZE, + cursorKeys: undefined, }, request: expect.anything(), }) diff --git a/apps/sim/app/api/v2/custom-tools/route.ts b/apps/sim/app/api/v2/custom-tools/route.ts index 0da00d8ce8f..88d691efd3a 100644 --- a/apps/sim/app/api/v2/custom-tools/route.ts +++ b/apps/sim/app/api/v2/custom-tools/route.ts @@ -14,6 +14,7 @@ import { listWorkspaceCustomToolsUseCase, } from '@/lib/custom-tools/application/use-cases' import { toV2CustomTool } from '@/app/api/v2/custom-tools/utils' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -25,9 +26,17 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + ...query, + cursorKeys: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + }), useCase: listWorkspaceCustomToolsUseCase, - present: ({ tools }) => ({ data: tools.map(toV2CustomTool), nextCursor: null }), + present: ({ tools, nextCursorKeys, sortBy, sortOrder }) => ({ + data: tools.map(toV2CustomTool), + nextCursor: nextCursorKeys + ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextCursorKeys) + : null, + }), }) /** POST /api/v2/custom-tools — Create a custom tool. */ diff --git a/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts index acef3a88a5f..db308de20b6 100644 --- a/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts @@ -1,15 +1,21 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ admit: vi.fn(), updateContent: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), getUserEmailsByIds: vi.fn(), })) @@ -25,22 +31,9 @@ vi.mock('@/lib/workspace-files/application/update-workspace-file-content', () => }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -94,17 +87,10 @@ const callPut = (body: unknown, contentLength?: number) => describe('PUT /api/v2/files/[fileId]/content', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.admit.mockResolvedValue(undefined) mocks.updateContent.mockResolvedValue({ file: record }) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) @@ -120,6 +106,15 @@ describe('PUT /api/v2/files/[fileId]/content', () => { expect(mocks.updateContent).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await callPut({ workspaceId: WORKSPACE_ID, content: 'id,name\n' }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('validates body fields after admission', async () => { const response = await callPut({ workspaceId: WORKSPACE_ID }) @@ -159,6 +154,7 @@ describe('PUT /api/v2/files/[fileId]/content', () => { uploadedByEmail: 'ada@example.com', uploadedAt: '2024-01-01T00:00:00.000Z', updatedAt: '2024-01-03T00:00:00.000Z', + deletedAt: null, }, }) expect(mocks.updateContent).toHaveBeenCalledWith({ @@ -171,7 +167,7 @@ describe('PUT /api/v2/files/[fileId]/content', () => { }, request, }) - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledWith( + expect(v2RouteMocks.operationRate).toHaveBeenCalledWith( 'v2:files.update_content:api-key:key-1', expect.anything() ) diff --git a/apps/sim/app/api/v2/files/[fileId]/content/route.ts b/apps/sim/app/api/v2/files/[fileId]/content/route.ts index 48d35dd94b7..c2a64201d23 100644 --- a/apps/sim/app/api/v2/files/[fileId]/content/route.ts +++ b/apps/sim/app/api/v2/files/[fileId]/content/route.ts @@ -8,7 +8,6 @@ import { } from '@/lib/workspace-files/application/update-workspace-file-content' import { MAX_WORKSPACE_FILE_INLINE_BODY_BYTES } from '@/lib/workspace-files/orchestration' import { toV2File } from '@/app/api/v2/files/utils' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -21,7 +20,6 @@ export const PUT = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.concealResourceAuthorization, parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), maxBodyBytes: MAX_WORKSPACE_FILE_INLINE_BODY_BYTES, }, beforeParse: async ({ principal, params }) => { diff --git a/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts index 3c26fadfaad..c947dff15a9 100644 --- a/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts @@ -1,14 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ readMetadata: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), getUserEmailsByIds: vi.fn(), })) @@ -19,22 +25,9 @@ vi.mock('@/lib/workspace-files/application/read-workspace-file-metadata', () => }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -89,17 +82,10 @@ const callGet = (query: string) => describe('GET /api/v2/files/[fileId]/metadata', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readMetadata.mockResolvedValue({ file: buildRecord(), share: SHARE }) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) }) @@ -108,11 +94,20 @@ describe('GET /api/v2/files/[fileId]/metadata', () => { const response = await callGet('') expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalled() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.readMetadata).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await callGet(`workspaceId=${WORKSPACE_ID}`) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('conceals cross-workspace authorization as not found', async () => { mocks.readMetadata.mockRejectedValue(new NoWorkspaceAccessError()) @@ -137,6 +132,7 @@ describe('GET /api/v2/files/[fileId]/metadata', () => { uploadedByEmail: 'ada@example.com', uploadedAt: '2024-01-01T00:00:00.000Z', updatedAt: '2024-01-02T00:00:00.000Z', + deletedAt: null, share: SHARE, }, }) diff --git a/apps/sim/app/api/v2/files/[fileId]/restore/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/restore/route.test.ts new file mode 100644 index 00000000000..926e0166a28 --- /dev/null +++ b/apps/sim/app/api/v2/files/[fileId]/restore/route.test.ts @@ -0,0 +1,156 @@ +/** + * @vitest-environment node + */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + restoreFile: vi.fn(), + getUserEmailsByIds: vi.fn(), +})) + +vi.mock('@/lib/workspace-files/application/restore-workspace-file', () => ({ + restoreWorkspaceFileOperation: { + operation: { id: 'files.restore', minimumRole: 'write', workspaceApiKey: 'allow' }, + execute: mocks.restoreFile, + }, +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/users/queries', () => ({ + getUserEmailsByIds: mocks.getUserEmailsByIds, + requireResolvedUserEmail: (emails: Map, userId: string) => emails.get(userId)!, +})) + +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { POST } from '@/app/api/v2/files/[fileId]/restore/route' + +const WORKSPACE_ID = 'workspace-1' +const FILE_ID = 'wf_1' + +const auth = { + principal: { + kind: 'workspace_api_key' as const, + workspaceId: WORKSPACE_ID, + keyId: 'key-1', + }, + rolloutUserId: 'billing-owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, + rateLimitSubscription: null, + keyType: 'workspace' as const, +} + +/** The post-restore record: renamed away from the taken name, back at the root. */ +const RESTORED_FILE = { + id: FILE_ID, + workspaceId: WORKSPACE_ID, + name: 'notes_restored.md', + key: `workspace/${WORKSPACE_ID}/notes.md`, + path: '/api/files/serve/notes.md?context=workspace', + size: 12, + type: 'text/markdown', + uploadedBy: 'user-1', + folderId: null, + folderPath: null, + deletedAt: null, + uploadedAt: new Date('2026-08-04T00:00:00.000Z'), + updatedAt: new Date('2026-08-07T00:00:00.000Z'), +} + +function restoreRequest(body: unknown): NextRequest { + return new NextRequest(`http://localhost:3000/api/v2/files/${FILE_ID}/restore`, { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) +} + +function post(body: unknown) { + return POST(restoreRequest(body), { params: Promise.resolve({ fileId: FILE_ID }) }) +} + +describe('POST /api/v2/files/[fileId]/restore', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.restoreFile.mockResolvedValue({ restored: true, file: RESTORED_FILE }) + mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) + }) + + it('returns the post-restore record so the caller sees the new name and root placement', async () => { + const response = await post({ workspaceId: WORKSPACE_ID }) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { + id: FILE_ID, + name: 'notes_restored.md', + size: 12, + type: 'text/markdown', + key: RESTORED_FILE.key, + folderPath: '/', + uploadedByEmail: 'ada@example.com', + uploadedAt: '2026-08-04T00:00:00.000Z', + updatedAt: '2026-08-07T00:00:00.000Z', + deletedAt: null, + }, + }) + expect(mocks.restoreFile).toHaveBeenCalledWith({ + principal: auth.principal, + input: { fileId: FILE_ID, assertedWorkspaceId: WORKSPACE_ID }, + request: expect.anything(), + }) + }) + + it('conceals a file in another workspace as 404 rather than confirming it exists', async () => { + mocks.restoreFile.mockRejectedValueOnce(new OrchestrationError('not_found', 'File not found')) + + const response = await post({ workspaceId: WORKSPACE_ID }) + + expect(response.status).toBe(404) + expect((await response.json()).error).toMatchObject({ + code: 'NOT_FOUND', + message: 'File not found', + }) + }) + + it('rejects an unknown body key instead of ignoring it', async () => { + const response = await post({ workspaceId: WORKSPACE_ID, folderPath: '/Engineering' }) + + expect(response.status).toBe(400) + expect(mocks.restoreFile).not.toHaveBeenCalled() + }) + + it('authenticates and charges before validating the body', async () => { + const response = await post({}) + + expect(response.status).toBe(400) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(mocks.restoreFile).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await post({ workspaceId: WORKSPACE_ID }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) +}) diff --git a/apps/sim/app/api/v2/files/[fileId]/restore/route.ts b/apps/sim/app/api/v2/files/[fileId]/restore/route.ts new file mode 100644 index 00000000000..050a49e0ccf --- /dev/null +++ b/apps/sim/app/api/v2/files/[fileId]/restore/route.ts @@ -0,0 +1,35 @@ +import { v2RestoreFileContract } from '@/lib/api/contracts/v2/files' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2FileErrorPolicies } from '@/lib/workspace-files/api' +import { fileOperations } from '@/lib/workspace-files/application/operations' +import { restoreWorkspaceFileOperation } from '@/lib/workspace-files/application/restore-workspace-file' +import { toV2File } from '@/app/api/v2/files/utils' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * POST /api/v2/files/[fileId]/restore — Bring an archived file back. + * + * `DELETE /api/v2/files/[fileId]` is a soft delete; this reverses it. Find the + * ids to pass here with `GET /api/v2/files?scope=archived`. + * + * Restore is not a pure undo: the file returns to the workspace root regardless + * of the folder it was deleted from, and it is renamed when its original name + * is no longer free. The response is therefore the post-restore record, not the + * one the caller deleted. Restoring an already-active file is a no-op that + * returns that file, so a retried request is safe. + */ +export const POST = defineV2JsonRoute({ + contract: v2RestoreFileContract, + auth: v2ApiKeyAuth, + operation: fileOperations.restore, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2FileErrorPolicies.concealResourceAuthorization, + mapInput: ({ params, body }) => ({ + fileId: params.fileId, + assertedWorkspaceId: body.workspaceId, + }), + useCase: restoreWorkspaceFileOperation, + present: async ({ file }) => ({ data: await toV2File(file) }), +}) diff --git a/apps/sim/app/api/v2/files/[fileId]/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/route.test.ts index a63dbb5c50a..d4405e2afd0 100644 --- a/apps/sim/app/api/v2/files/[fileId]/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/route.test.ts @@ -1,6 +1,15 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -8,9 +17,6 @@ const mocks = vi.hoisted(() => ({ download: vi.fn(), rename: vi.fn(), deleteFile: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), getUserEmailsByIds: vi.fn(), })) @@ -35,20 +41,9 @@ vi.mock('@/lib/workspace-files/application/delete-workspace-file', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: vi.fn().mockResolvedValue(null) })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -97,17 +92,10 @@ function fileRecord(overrides: Record = {}) { describe('v2 single-file routes', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.download.mockResolvedValue({ file: fileRecord(), stream: new Blob(['id,name\n']).stream(), @@ -141,6 +129,18 @@ describe('v2 single-file routes', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files/${FILE_ID}?workspaceId=${WORKSPACE_ID}`), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('encodes special characters in the extended download filename', async () => { mocks.download.mockResolvedValueOnce({ file: fileRecord({ name: "it's (final)* café.pdf" }), @@ -226,7 +226,11 @@ describe('v2 single-file routes', () => { expect(response.status).toBe(403) expect(await response.json()).toEqual({ - error: { code: 'FORBIDDEN', message: 'Insufficient workspace permissions' }, + error: { + code: 'FORBIDDEN', + message: 'Insufficient workspace permissions', + details: { code: 'INSUFFICIENT_WORKSPACE_ROLE' }, + }, }) }) diff --git a/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts index 42d6c30a8a2..7d33f8c91f9 100644 --- a/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts @@ -1,46 +1,26 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error { - constructor(message = 'Invalid API key') { - super(message) - this.name = 'V2ApiKeyUnauthenticatedError' - } - } - - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - getShare: vi.fn(), - updateShare: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, +const mocks = vi.hoisted(() => ({ + getShare: vi.fn(), + updateShare: vi.fn(), })) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), @@ -52,8 +32,6 @@ vi.mock('@/lib/core/utils/request', () => ({ getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) - vi.mock('@/lib/workspace-files/application/share-workspace-file', () => ({ getWorkspaceFileShare: { operation: { id: 'files.share.read', minimumRole: 'read', workspaceApiKey: 'allow' }, @@ -67,6 +45,7 @@ vi.mock('@/lib/workspace-files/application/share-workspace-file', () => ({ import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET, PATCH } from '@/app/api/v2/files/[fileId]/share/route' +import { v2Error } from '@/app/api/v2/lib/response' const WORKSPACE_ID = 'workspace-1' const FILE_ID = 'wf_1' @@ -74,16 +53,16 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, +const RATE_LIMIT_DENIED = { + allowed: false, limit: 100, - remaining: 99, + remaining: 0, resetAt: new Date('2024-01-01T01:00:00Z'), - retryAfterMs: 0, + retryAfterMs: 1000, } const SHARE = { id: 'shr_1', @@ -121,15 +100,15 @@ function callPatch(body: unknown) { describe('GET /api/v2/files/[fileId]/share', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) mocks.getShare.mockResolvedValue({ share: SHARE }) }) it('authenticates and rate-limits before parsing or executing', async () => { - mocks.authenticate.mockRejectedValueOnce( + v2RouteMocks.authenticate.mockRejectedValueOnce( new MockV2ApiKeyUnauthenticatedError('API key required') ) @@ -137,12 +116,11 @@ describe('GET /api/v2/files/[fileId]/share', () => { expect(response.status).toBe(401) expect(mocks.getShare).not.toHaveBeenCalled() - expect(mocks.operationRate).not.toHaveBeenCalled() + expect(v2RouteMocks.operationRate).not.toHaveBeenCalled() }) it('returns 404 when the v2 API surface flag is off', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) const response = await callGet() @@ -180,7 +158,7 @@ describe('GET /api/v2/files/[fileId]/share', () => { }) it('returns the rate-limit response when denied', async () => { - mocks.operationRate.mockResolvedValueOnce({ ...RATE_LIMIT_OK, allowed: false, remaining: 0 }) + v2RouteMocks.operationRate.mockResolvedValueOnce(RATE_LIMIT_DENIED) const response = await callGet() @@ -193,10 +171,10 @@ describe('GET /api/v2/files/[fileId]/share', () => { describe('PATCH /api/v2/files/[fileId]/share', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) mocks.updateShare.mockResolvedValue({ share: SHARE }) }) @@ -261,7 +239,7 @@ describe('PATCH /api/v2/files/[fileId]/share', () => { }) it('returns the rate-limit response when denied', async () => { - mocks.operationRate.mockResolvedValueOnce({ ...RATE_LIMIT_OK, allowed: false, remaining: 0 }) + v2RouteMocks.operationRate.mockResolvedValueOnce(RATE_LIMIT_DENIED) const response = await callPatch({ workspaceId: WORKSPACE_ID, isActive: true }) diff --git a/apps/sim/app/api/v2/files/bulk-delete/route.test.ts b/apps/sim/app/api/v2/files/bulk-delete/route.test.ts index a66f490f6c2..01206253399 100644 --- a/apps/sim/app/api/v2/files/bulk-delete/route.test.ts +++ b/apps/sim/app/api/v2/files/bulk-delete/route.test.ts @@ -1,35 +1,25 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockPreauth, mockOperationRate, mockGate, mockExecute } = vi.hoisted(() => ({ - mockPreauth: vi.fn(), - mockOperationRate: vi.fn(), - mockGate: vi.fn(), +const { mockExecute } = vi.hoisted(() => ({ mockExecute: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: vi.fn().mockResolvedValue({ - principal: { kind: 'workspace_api_key', workspaceId: 'workspace-1', keyId: 'key-1' }, - rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'], - rateLimitSubscription: null, - keyType: 'workspace', - }), - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mockPreauth - checkRateLimitDirectOrThrow = mockOperationRate - }, - getRateLimit: vi - .fn() - .mockReturnValue({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -38,7 +28,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mockGate })) vi.mock('@/lib/workspace-files/application/archive-workspace-file-items', () => ({ archiveWorkspaceFileItemsOperation: { operation: { id: 'files.delete', minimumRole: 'write', workspaceApiKey: 'allow' }, @@ -46,13 +35,22 @@ vi.mock('@/lib/workspace-files/application/archive-workspace-file-items', () => }, })) +import { OrchestrationError } from '@/lib/core/orchestration/types' import { POST } from '@/app/api/v2/files/bulk-delete/route' +import { v2Error } from '@/app/api/v2/lib/response' const WS = 'workspace-1' -const RATE_LIMIT_OK = { - allowed: true, +const AUTH = { + principal: { kind: 'workspace_api_key' as const, workspaceId: WS, keyId: 'key-1' }, + rolloutUserId: 'owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WS}`] as const, + rateLimitSubscription: null, + keyType: 'workspace' as const, +} +const RATE_LIMIT_DENIED = { + allowed: false, limit: 100, - remaining: 99, + remaining: 0, resetAt: new Date('2024-01-01T01:00:00Z'), retryAfterMs: 0, } @@ -69,15 +67,15 @@ const callDelete = (body: unknown) => describe('POST /api/v2/files/bulk-delete', () => { beforeEach(() => { vi.clearAllMocks() - mockPreauth.mockResolvedValue(RATE_LIMIT_OK) - mockOperationRate.mockResolvedValue(RATE_LIMIT_OK) - mockGate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mockExecute.mockResolvedValue({ deletedItems: { files: 3, folders: 0 } }) }) it('returns 404 when the v2 API surface flag is off', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mockGate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(404) expect(mockExecute).not.toHaveBeenCalled() @@ -91,19 +89,26 @@ describe('POST /api/v2/files/bulk-delete', () => { }) it('surfaces a forbidden collection operation', async () => { - const { OrchestrationError } = await import('@/lib/core/orchestration/types') mockExecute.mockRejectedValue(new OrchestrationError('forbidden', 'Access denied')) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(403) }) it('returns the rate-limit response when denied', async () => { - mockPreauth.mockResolvedValue({ ...RATE_LIMIT_OK, allowed: false, remaining: 0 }) + v2RouteMocks.preauthRate.mockResolvedValue(RATE_LIMIT_DENIED) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(429) expect((await res.json()).error.code).toBe('RATE_LIMITED') }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) + expect(res.status).toBe(401) + expect((await res.json()).error.code).toBe('UNAUTHORIZED') + expect(mockExecute).not.toHaveBeenCalled() + }) + it('deletes the selection and reports the file count', async () => { const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(200) @@ -114,7 +119,6 @@ describe('POST /api/v2/files/bulk-delete', () => { }) it('maps a not-found failure to 404', async () => { - const { OrchestrationError } = await import('@/lib/core/orchestration/types') mockExecute.mockRejectedValue(new OrchestrationError('not_found', 'File not found')) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_missing'] }) expect(res.status).toBe(404) diff --git a/apps/sim/app/api/v2/files/folders/route.test.ts b/apps/sim/app/api/v2/files/folders/route.test.ts index 50dd9b0c98c..bbdf3ffdf82 100644 --- a/apps/sim/app/api/v2/files/folders/route.test.ts +++ b/apps/sim/app/api/v2/files/folders/route.test.ts @@ -1,41 +1,28 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - listFolders: vi.fn(), - createFolder: vi.fn(), - updateFolder: vi.fn(), - deleteFolder: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), +const mocks = vi.hoisted(() => ({ + listFolders: vi.fn(), + createFolder: vi.fn(), + updateFolder: vi.fn(), + deleteFolder: vi.fn(), })) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -44,7 +31,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) vi.mock('@/lib/workspace-files/application/workspace-file-folders', () => ({ listWorkspaceFileFoldersOperation: { operation: { id: 'files.folders.list', minimumRole: 'read', workspaceApiKey: 'allow' }, @@ -75,17 +61,10 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, - limit: 100, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - retryAfterMs: 0, -} const folder = { id: 'folder-1', workspaceId: WORKSPACE_ID, @@ -114,10 +93,10 @@ function request(method: 'GET' | 'POST' | 'PATCH' | 'DELETE', url: string, body? describe('/api/v2/files/folders', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listFolders.mockResolvedValue({ folders: [folder] }) mocks.createFolder.mockResolvedValue({ folder }) mocks.updateFolder.mockResolvedValue({ folder }) @@ -294,11 +273,12 @@ describe('/api/v2/files/folders', () => { }) it('authenticates before parsing folder input', async () => { - mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) const response = await POST(request('POST', '/api/v2/files/folders', {}), context) expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.createFolder).not.toHaveBeenCalled() }) }) diff --git a/apps/sim/app/api/v2/files/move/route.test.ts b/apps/sim/app/api/v2/files/move/route.test.ts index a311ffcb614..9f8d95138f4 100644 --- a/apps/sim/app/api/v2/files/move/route.test.ts +++ b/apps/sim/app/api/v2/files/move/route.test.ts @@ -1,35 +1,25 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockPreauth, mockOperationRate, mockGate, mockExecute } = vi.hoisted(() => ({ - mockPreauth: vi.fn(), - mockOperationRate: vi.fn(), - mockGate: vi.fn(), +const { mockExecute } = vi.hoisted(() => ({ mockExecute: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: vi.fn().mockResolvedValue({ - principal: { kind: 'workspace_api_key', workspaceId: 'workspace-1', keyId: 'key-1' }, - rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'], - rateLimitSubscription: null, - keyType: 'workspace', - }), - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mockPreauth - checkRateLimitDirectOrThrow = mockOperationRate - }, - getRateLimit: vi - .fn() - .mockReturnValue({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -38,7 +28,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mockGate })) vi.mock('@/lib/workspace-files/application/move-workspace-file-items', () => ({ moveWorkspaceFileItemsOperation: { operation: { id: 'files.move', minimumRole: 'write', workspaceApiKey: 'allow' }, @@ -46,18 +35,26 @@ vi.mock('@/lib/workspace-files/application/move-workspace-file-items', () => ({ }, })) +import { OrchestrationError } from '@/lib/core/orchestration/types' import { WorkspaceFileMoveConflictError } from '@/lib/uploads/contexts/workspace/workspace-file-folder-manager' import { POST } from '@/app/api/v2/files/move/route' +import { v2Error } from '@/app/api/v2/lib/response' const WS = 'workspace-1' -const RATE_LIMIT_OK = { - allowed: true, +const auth = { + principal: { kind: 'workspace_api_key' as const, workspaceId: WS, keyId: 'key-1' }, + rolloutUserId: 'owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WS}`] as const, + rateLimitSubscription: null, + keyType: 'workspace' as const, +} +const RATE_LIMIT_DENIED = { + allowed: false, limit: 100, - remaining: 99, + remaining: 0, resetAt: new Date('2024-01-01T01:00:00Z'), - retryAfterMs: 0, + retryAfterMs: 1000, } -const RATE_LIMIT_DENIED = { ...RATE_LIMIT_OK, allowed: false, remaining: 0, retryAfterMs: 1000 } const callMove = (body: unknown) => POST( @@ -71,15 +68,22 @@ const callMove = (body: unknown) => describe('POST /api/v2/files/move', () => { beforeEach(() => { vi.clearAllMocks() - mockPreauth.mockResolvedValue(RATE_LIMIT_OK) - mockOperationRate.mockResolvedValue(RATE_LIMIT_OK) - mockGate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mockExecute.mockResolvedValue({ movedItems: { files: 2, folders: 0 } }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) + expect(res.status).toBe(401) + expect((await res.json()).error.code).toBe('UNAUTHORIZED') + }) + it('returns 404 when the v2 API surface flag is off', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mockGate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(404) expect(mockExecute).not.toHaveBeenCalled() @@ -93,7 +97,6 @@ describe('POST /api/v2/files/move', () => { }) it('surfaces a forbidden collection operation', async () => { - const { OrchestrationError } = await import('@/lib/core/orchestration/types') mockExecute.mockRejectedValue(new OrchestrationError('forbidden', 'Access denied')) const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(403) @@ -101,7 +104,7 @@ describe('POST /api/v2/files/move', () => { }) it('returns the rate-limit response when denied', async () => { - mockPreauth.mockResolvedValue(RATE_LIMIT_DENIED) + v2RouteMocks.preauthRate.mockResolvedValue(RATE_LIMIT_DENIED) const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(429) expect((await res.json()).error.code).toBe('RATE_LIMITED') diff --git a/apps/sim/app/api/v2/files/route.test.ts b/apps/sim/app/api/v2/files/route.test.ts index 8d62db0876c..242bfbf5aa4 100644 --- a/apps/sim/app/api/v2/files/route.test.ts +++ b/apps/sim/app/api/v2/files/route.test.ts @@ -1,17 +1,22 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), createFile: vi.fn(), queryFiles: vi.fn(), getUserEmailsByIds: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workspace-files/application/create-workspace-file', () => ({ @@ -28,20 +33,9 @@ vi.mock('@/lib/workspace-files/application/list-workspace-files', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -89,18 +83,10 @@ function createRequest(body: unknown): NextRequest { describe('/api/v2/files', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-04T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-04T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.queryFiles.mockResolvedValue({ files: [FILE], nextKeys: undefined, @@ -114,11 +100,22 @@ describe('/api/v2/files', () => { const response = await GET(new NextRequest('http://localhost:3000/api/v2/files')) expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalled() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.queryFiles).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}`) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('lists through the shared use case and v2 presenter', async () => { const request = new NextRequest( `http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&sortBy=name` @@ -138,6 +135,7 @@ describe('/api/v2/files', () => { uploadedByEmail: 'ada@example.com', uploadedAt: '2026-08-04T00:00:00.000Z', updatedAt: '2026-08-05T00:00:00.000Z', + deletedAt: null, }, ], nextCursor: null, @@ -146,6 +144,7 @@ describe('/api/v2/files', () => { principal: auth.principal, input: expect.objectContaining({ workspaceId: WORKSPACE_ID, + scope: 'active', sortBy: 'name', sortOrder: 'asc', limit: 100, @@ -154,6 +153,36 @@ describe('/api/v2/files', () => { }) }) + it('pages the archived set and dates each soft delete when asked for it', async () => { + mocks.queryFiles.mockResolvedValueOnce({ + files: [{ ...FILE, deletedAt: new Date('2026-08-06T00:00:00.000Z') }], + nextKeys: undefined, + cursorSort: 'uploadedAt:asc', + }) + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&scope=archived` + ) + ) + + expect(response.status).toBe(200) + expect((await response.json()).data[0].deletedAt).toBe('2026-08-06T00:00:00.000Z') + expect(mocks.queryFiles).toHaveBeenCalledWith({ + principal: auth.principal, + input: expect.objectContaining({ scope: 'archived' }), + request: expect.anything(), + }) + }) + + it('rejects an unimplemented scope instead of silently listing the active set', async () => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&scope=all`) + ) + + expect(response.status).toBe(400) + expect(mocks.queryFiles).not.toHaveBeenCalled() + }) + it('preserves escaped slashes in the containing folder path', async () => { mocks.queryFiles.mockResolvedValueOnce({ files: [{ ...FILE, folderId: 'folder-1', folderPath: 'Finance\\/Legal' }], @@ -216,8 +245,8 @@ describe('/api/v2/files', () => { ) expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalled() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.createFile).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/v2/files/route.ts b/apps/sim/app/api/v2/files/route.ts index 9b905ca9342..8a5a20ceee7 100644 --- a/apps/sim/app/api/v2/files/route.ts +++ b/apps/sim/app/api/v2/files/route.ts @@ -3,9 +3,7 @@ import { v2CreateFileContract, v2ListFilesContract, } from '@/lib/api/contracts/v2/files' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' import { v2FileErrorPolicies } from '@/lib/workspace-files/api' import { createWorkspaceFile } from '@/lib/workspace-files/application/create-workspace-file' @@ -13,12 +11,7 @@ import { queryWorkspaceFilePage } from '@/lib/workspace-files/application/list-w import { fileOperations } from '@/lib/workspace-files/application/operations' import { MAX_WORKSPACE_FILE_INLINE_BODY_BYTES } from '@/lib/workspace-files/orchestration' import { toV2File, toV2Files } from '@/app/api/v2/files/utils' -import { - cursorSortKey, - decodeSortedCursor, - encodeSortedCursor, - v2Error, -} from '@/app/api/v2/lib/response' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -30,23 +23,17 @@ export const GET = defineV2JsonRoute({ operation: fileOperations.list, rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.default, - mapInput: ({ query }) => { - const cursorSort = cursorSortKey(query.sortBy, query.sortOrder) - const decoded = decodeSortedCursor(query.cursor, cursorSort) - if (decoded.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - return { - workspaceId: query.workspaceId, - folderPath: query.folderPath, - search: query.search, - sortBy: query.sortBy, - sortOrder: query.sortOrder, - limit: query.limit, - after: decoded.status === 'ok' ? decoded.keys : undefined, - cursorSort, - } - }, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + scope: query.scope, + folderPath: query.folderPath, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + limit: query.limit, + after: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + cursorSort: cursorSortKey(query.sortBy, query.sortOrder), + }), useCase: queryWorkspaceFilePage, present: async ({ files, nextKeys, cursorSort }) => { const items: V2File[] = await toV2Files(files) @@ -62,7 +49,6 @@ export const POST = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.default, parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), maxBodyBytes: MAX_WORKSPACE_FILE_INLINE_BODY_BYTES, }, mapInput: ({ body }) => ({ diff --git a/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts b/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts index 8df882b0064..193aeee9c65 100644 --- a/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts +++ b/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts @@ -1,14 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ abort: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), })) vi.mock('@/lib/uploads/upload-session/application', () => ({ @@ -18,20 +24,9 @@ vi.mock('@/lib/uploads/upload-session/application', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: vi.fn().mockResolvedValue(null) })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/files/uploads/utils', () => ({ toV2FileUpload: vi.fn(async () => ({ @@ -78,17 +73,10 @@ function abortRequest() { describe('DELETE /api/v2/files/uploads/[uploadId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(AUTH) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.abort.mockResolvedValue({ id: UPLOAD_ID }) }) @@ -99,6 +87,16 @@ describe('DELETE /api/v2/files/uploads/[uploadId]', () => { expect(await response.json()).toMatchObject({ data: { id: UPLOAD_ID, status: 'aborted' } }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await DELETE(abortRequest(), context) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + expect(mocks.abort).not.toHaveBeenCalled() + }) + it('conceals a cross-tenant reach as a missing upload session', async () => { mocks.abort.mockRejectedValueOnce(new NoWorkspaceAccessError()) @@ -130,7 +128,11 @@ describe('DELETE /api/v2/files/uploads/[uploadId]', () => { expect(response.status).toBe(403) expect(await response.json()).toEqual({ - error: { code: 'FORBIDDEN', message: 'Insufficient workspace permissions' }, + error: { + code: 'FORBIDDEN', + message: 'Insufficient workspace permissions', + details: { code: 'INSUFFICIENT_WORKSPACE_ROLE' }, + }, }) }) }) diff --git a/apps/sim/app/api/v2/files/uploads/route.test.ts b/apps/sim/app/api/v2/files/uploads/route.test.ts index cfc19a29766..f1b6df19ad9 100644 --- a/apps/sim/app/api/v2/files/uploads/route.test.ts +++ b/apps/sim/app/api/v2/files/uploads/route.test.ts @@ -1,15 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), createUpload: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/uploads/upload-session/application', () => ({ @@ -19,20 +24,9 @@ vi.mock('@/lib/uploads/upload-session/application', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/files/uploads/utils', () => ({ toV2FileUpload: vi.fn(async () => ({ @@ -86,18 +80,10 @@ function request(body: Record) { describe('POST /api/v2/files/uploads', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(AUTH) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.createUpload.mockResolvedValue(UPLOAD_SESSION) }) @@ -139,8 +125,23 @@ describe('POST /api/v2/files/uploads', () => { const response = await request({ workspaceId: WORKSPACE_ID }).response expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalledTimes(1) - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalledTimes(1) + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(mocks.createUpload).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await request({ + workspaceId: WORKSPACE_ID, + name: 'file.csv', + contentType: 'text/csv', + size: 10, + }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.createUpload).not.toHaveBeenCalled() }) @@ -152,7 +153,7 @@ describe('POST /api/v2/files/uploads', () => { size: 0, }).response - expect(mocks.authenticateV2ApiKey).toHaveBeenCalledTimes(1) + expect(v2RouteMocks.authenticate).toHaveBeenCalledTimes(1) expect(mocks.createUpload).toHaveBeenCalledWith( expect.objectContaining({ principal: PRINCIPAL }) ) diff --git a/apps/sim/app/api/v2/files/utils.ts b/apps/sim/app/api/v2/files/utils.ts index d21514036e4..bff1477af38 100644 --- a/apps/sim/app/api/v2/files/utils.ts +++ b/apps/sim/app/api/v2/files/utils.ts @@ -30,6 +30,7 @@ function serializeV2File(record: WorkspaceFileRecord, uploadedByEmail: string): uploadedByEmail, uploadedAt: record.uploadedAt.toISOString(), updatedAt: record.updatedAt.toISOString(), + deletedAt: record.deletedAt?.toISOString() ?? null, } } diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.test.ts new file mode 100644 index 00000000000..272d20c5f52 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.test.ts @@ -0,0 +1,252 @@ +/** + * @vitest-environment node + */ +import { + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockReadDocument, mockUpdateDocument, mockDeleteDocument, mockCapture } = vi.hoisted( + () => ({ + mockReadDocument: vi.fn(), + mockUpdateDocument: vi.fn(), + mockDeleteDocument: vi.fn(), + mockCapture: vi.fn(), + }) +) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/knowledge/application/documents', () => ({ + readKnowledgeDocument: { + operation: { id: 'knowledge.documents.read' }, + execute: mockReadDocument, + }, + updateKnowledgeDocument: { + operation: { id: 'knowledge.documents.update' }, + execute: mockUpdateDocument, + }, + deleteKnowledgeDocument: { + operation: { id: 'knowledge.documents.delete' }, + execute: mockDeleteDocument, + }, +})) + +vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mockCapture })) + +import { GET, PATCH } from '@/app/api/v2/knowledge/[id]/documents/[documentId]/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const +const UPLOADED_AT = new Date('2025-06-18T16:45:00Z') + +const TAG_DEFINITIONS = [ + { + id: 'tag-def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: UPLOADED_AT, + updatedAt: UPLOADED_AT, + }, + { + id: 'tag-def-2', + knowledgeBaseId: 'kb-1', + tagSlot: 'number1', + displayName: 'priority', + fieldType: 'number', + createdAt: UPLOADED_AT, + updatedAt: UPLOADED_AT, + }, +] + +const DOCUMENT_ROW = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'support.txt', + fileSize: 5, + mimeType: 'text/plain', + processingStatus: 'completed' as const, + processingError: null, + processingStartedAt: UPLOADED_AT, + processingCompletedAt: UPLOADED_AT, + chunkCount: 2, + tokenCount: 10, + characterCount: 40, + enabled: true, + connectorId: null, + connectorType: null, + sourceUrl: null, + uploadedAt: UPLOADED_AT, + tag1: 'billing', + tag2: null, + number1: 2, + date1: null, + boolean1: null, + tag6: 'orphaned-slot-value', +} + +const context = { params: Promise.resolve({ id: 'kb-1', documentId: 'doc-1' }) } + +function buildGetRequest() { + return new NextRequest( + `http://localhost/api/v2/knowledge/kb-1/documents/doc-1?workspaceId=${WORKSPACE_ID}`, + { headers: { 'x-api-key': 'secret' } } + ) +} + +function buildPatchRequest(body: unknown) { + return new NextRequest('http://localhost/api/v2/knowledge/kb-1/documents/doc-1', { + method: 'PATCH', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) +} + +describe('/api/v2/knowledge/[id]/documents/[documentId]', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'], + rateLimitSubscription: null, + keyType: 'personal', + }) + mockReadDocument.mockResolvedValue({ + document: DOCUMENT_ROW, + tagDefinitions: TAG_DEFINITIONS, + workspaceId: WORKSPACE_ID, + }) + mockUpdateDocument.mockResolvedValue({ + kind: 'updated', + document: { ...DOCUMENT_ROW, filename: 'renamed.txt', enabled: false }, + tagDefinitions: TAG_DEFINITIONS, + updatedFields: ['filename', 'enabled'], + }) + }) + + it('keys document tag values by display name, falling back to the raw slot', async () => { + const response = await GET(buildGetRequest(), context) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data.tags).toEqual({ + category: 'billing', + priority: 2, + tag6: 'orphaned-slot-value', + }) + }) + + it('updates the whitelisted fields and returns the updated document with its tags', async () => { + const response = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + filename: 'renamed.txt', + enabled: false, + tag1: 'support', + }), + context + ) + + expect(response.status).toBe(200) + expect(mockUpdateDocument).toHaveBeenCalledWith( + expect.objectContaining({ + input: { + knowledgeBaseId: 'kb-1', + documentId: 'doc-1', + assertedWorkspaceId: WORKSPACE_ID, + updates: { filename: 'renamed.txt', enabled: false, tag1: 'support' }, + source: 'api', + }, + }) + ) + const body = await response.json() + expect(body.data).toEqual( + expect.objectContaining({ + id: 'doc-1', + filename: 'renamed.txt', + enabled: false, + tags: { category: 'billing', priority: 2, tag6: 'orphaned-slot-value' }, + }) + ) + }) + + it('acknowledges a processing retry without claiming settled indexing state', async () => { + mockUpdateDocument.mockResolvedValueOnce({ + kind: 'processing', + documentId: 'doc-1', + status: 'pending', + message: 'Document processing restarted', + }) + + const response = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, retryProcessing: true }), + context + ) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { + id: 'doc-1', + queued: true, + processingStatus: 'pending', + message: 'Document processing restarted', + }, + }) + expect(mockUpdateDocument).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ retryProcessing: true }), + }) + ) + expect(mockUpdateDocument.mock.calls[0][0].input).not.toHaveProperty('updates') + }) + + it('refuses to let a caller assert derived indexing state', async () => { + for (const body of [ + { workspaceId: WORKSPACE_ID, processingStatus: 'completed' }, + { workspaceId: WORKSPACE_ID, chunkCount: 99 }, + { workspaceId: WORKSPACE_ID, tokenCount: 99 }, + { workspaceId: WORKSPACE_ID, processingError: null }, + { workspaceId: WORKSPACE_ID, markFailedDueToTimeout: true }, + ]) { + const response = await PATCH(buildPatchRequest(body), context) + expect(response.status).toBe(400) + } + expect(mockUpdateDocument).not.toHaveBeenCalled() + }) + + it('rejects a retry combined with field updates instead of silently dropping them', async () => { + const response = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, retryProcessing: true, enabled: false }), + context + ) + + expect(response.status).toBe(400) + expect(await response.json()).toEqual({ + error: expect.objectContaining({ + message: expect.stringContaining('retryProcessing cannot be combined with enabled'), + }), + }) + expect(mockUpdateDocument).not.toHaveBeenCalled() + }) + + it('rejects an update that changes nothing', async () => { + const response = await PATCH(buildPatchRequest({ workspaceId: WORKSPACE_ID }), context) + + expect(response.status).toBe(400) + expect(mockUpdateDocument).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts index 2695d8e3984..3d9d8543e08 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts @@ -1,29 +1,56 @@ import { + V2_WRITABLE_TAG_SLOTS, + type V2UpdateKnowledgeDocumentBody, v2DeleteKnowledgeDocumentContract, v2GetKnowledgeDocumentContract, + v2UpdateKnowledgeDocumentContract, } from '@/lib/api/contracts/v2/knowledge' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { deleteKnowledgeDocument, readKnowledgeDocument, + type UpdateKnowledgeDocumentInput, + updateKnowledgeDocument, } from '@/lib/knowledge/application/documents' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { captureServerEvent } from '@/lib/posthog/server' import { serializeDate } from '@/app/api/v1/knowledge/utils' +import { + toV2DocumentSummary, + toV2DocumentTags, + toV2TaggedDocument, +} from '@/app/api/v2/knowledge/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 -function toProcessingStatus(status: string): 'pending' | 'processing' | 'completed' | 'failed' { - switch (status) { - case 'pending': - case 'processing': - case 'completed': - case 'failed': - return status - default: - throw new Error(`Unexpected knowledge document processing status: ${status}`) +type V2DocumentUpdates = Omit + +type UpdateKnowledgeDocumentUpdates = NonNullable + +/** + * Serializes the typed tag slots for the document writer. + * + * The wire takes each slot in its natural JSON type — a number for a number + * slot, `true`/`false` for a boolean one — because that is how a document read + * projects them. The writer's `convertTagValue` takes strings and parses back to + * the storage column's type, so the boundary hands it the canonical spelling. + * The contract has already rejected anything those parsers would answer `null` + * for, so nothing reaches storage silently cleared. + */ +function toTagSlotUpdates(updates: V2DocumentUpdates): UpdateKnowledgeDocumentUpdates { + const { filename, enabled, ...slots } = updates + const serialized: Record = {} + for (const slot of V2_WRITABLE_TAG_SLOTS) { + const value = slots[slot] + if (value === undefined) continue + serialized[slot] = typeof value === 'string' ? value : String(value) + } + return { + ...(filename === undefined ? {} : { filename }), + ...(enabled === undefined ? {} : { enabled }), + ...serialized, } } @@ -40,29 +67,60 @@ export const GET = defineV2JsonRoute({ assertedWorkspaceId: query.workspaceId, }), useCase: readKnowledgeDocument, - present: ({ document }) => ({ + present: ({ document, tagDefinitions }) => ({ data: { - id: document.id, - knowledgeBaseId: document.knowledgeBaseId, - filename: document.filename, - fileSize: document.fileSize, - mimeType: document.mimeType, - processingStatus: toProcessingStatus(document.processingStatus), + ...toV2DocumentSummary(document), + tags: toV2DocumentTags(document, tagDefinitions), processingError: document.processingError, processingStartedAt: serializeDate(document.processingStartedAt), processingCompletedAt: serializeDate(document.processingCompletedAt), - chunkCount: document.chunkCount, - tokenCount: document.tokenCount, - characterCount: document.characterCount, - enabled: document.enabled, connectorId: document.connectorId, connectorType: document.connectorType, sourceUrl: document.sourceUrl, - createdAt: serializeDate(document.uploadedAt), }, }), }) +/** + * PATCH /api/v2/knowledge/[id]/documents/[documentId] — Update a document. + * + * Renames, enables or disables, retags, or requeues processing. Derived + * indexing state is not writable; the contract records why. + * + * The updated document is returned without connector provenance because the + * update writes and returns the document row alone. A caller that needs the full + * detail re-reads it with GET. + */ +export const PATCH = defineV2JsonRoute({ + contract: v2UpdateKnowledgeDocumentContract, + auth: v2ApiKeyAuth, + operation: knowledgeOperations.updateDocument, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, + mapInput: ({ params, body }) => { + const { workspaceId, retryProcessing, ...updates } = body + return { + knowledgeBaseId: params.id, + documentId: params.documentId, + assertedWorkspaceId: workspaceId, + ...(retryProcessing ? { retryProcessing } : { updates: toTagSlotUpdates(updates) }), + source: 'api', + } + }, + useCase: updateKnowledgeDocument, + present: (result) => + result.kind === 'processing' + ? { + data: { + id: result.documentId, + queued: true as const, + processingStatus: result.status, + message: result.message, + }, + } + : { data: toV2TaggedDocument(result.document, result.tagDefinitions) }, +}) + /** DELETE /api/v2/knowledge/[id]/documents/[documentId] — Delete a document. */ export const DELETE = defineV2JsonRoute({ contract: v2DeleteKnowledgeDocumentContract, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/collection.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/collection.test.ts new file mode 100644 index 00000000000..b199cbc898e --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/collection.test.ts @@ -0,0 +1,310 @@ +/** + * @vitest-environment node + * + * Covers the JSON halves of the documents collection route (list and bulk + * update). The multipart upload half is covered in `route.test.ts`, which mocks + * the stream-limit helpers the JSON body parser also uses. + */ +import { + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockListDocuments, mockBulkUpdate } = vi.hoisted(() => ({ + mockListDocuments: vi.fn(), + mockBulkUpdate: vi.fn(), +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/knowledge/application/documents', () => ({ + listKnowledgeDocuments: { + operation: { id: 'knowledge.documents.list' }, + execute: mockListDocuments, + }, + bulkUpdateKnowledgeDocuments: { + operation: { id: 'knowledge.documents.bulk' }, + execute: mockBulkUpdate, + }, + admitKnowledgeDocumentUpload: { + operation: { id: 'knowledge.documents.upload' }, + execute: vi.fn(), + }, + uploadKnowledgeDocument: { + operation: { id: 'knowledge.documents.upload' }, + execute: vi.fn(), + }, +})) + +vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn() })) + +import { GET, PATCH } from '@/app/api/v2/knowledge/[id]/documents/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const +const UPLOADED_AT = new Date('2025-06-18T16:45:00Z') + +const TAG_DEFINITIONS = [ + { + id: 'tag-def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: UPLOADED_AT, + updatedAt: UPLOADED_AT, + }, +] + +const DOCUMENT = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'support.txt', + fileSize: 5, + mimeType: 'text/plain', + processingStatus: 'completed' as const, + chunkCount: 2, + tokenCount: 10, + characterCount: 40, + enabled: true, + uploadedAt: UPLOADED_AT, + tag1: 'billing', + tag2: null, +} + +const context = { params: Promise.resolve({ id: 'kb-1' }) } + +function buildListRequest(query: string) { + return new NextRequest(`http://localhost/api/v2/knowledge/kb-1/documents${query}`, { + headers: { 'x-api-key': 'secret' }, + }) +} + +function buildPatchRequest(body: unknown) { + return new NextRequest('http://localhost/api/v2/knowledge/kb-1/documents', { + method: 'PATCH', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) +} + +function authenticateAsPersonalKey() { + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'], + rateLimitSubscription: null, + keyType: 'personal', + }) +} + +describe('GET /api/v2/knowledge/[id]/documents', () => { + beforeEach(() => { + vi.clearAllMocks() + authenticateAsPersonalKey() + mockListDocuments.mockResolvedValue({ + documents: [DOCUMENT], + tagDefinitions: TAG_DEFINITIONS, + pagination: { total: 1, limit: 50, offset: 0, hasMore: false }, + cursorScope: 'scope', + workspaceId: WORKSPACE_ID, + }) + }) + + it('returns each document with its tag values keyed by display name', async () => { + const response = await GET(buildListRequest(`?workspaceId=${WORKSPACE_ID}`), context) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data[0]).toEqual( + expect.objectContaining({ id: 'doc-1', tags: { category: 'billing' } }) + ) + expect(body.nextCursor).toBeNull() + }) + + it('forwards display-named tag filters to the application use case', async () => { + const tagFilters = JSON.stringify([{ tagName: 'category', operator: 'eq', value: 'billing' }]) + + const response = await GET( + buildListRequest(`?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(tagFilters)}`), + context + ) + + expect(response.status).toBe(200) + expect(mockListDocuments).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + tagNameFilters: [{ tagName: 'category', operator: 'eq', value: 'billing' }], + }), + }) + ) + }) + + it('stamps the tag filters into the cursor scope so a replayed cursor cannot cross filters', async () => { + const tagFilters = JSON.stringify([{ tagName: 'category', operator: 'eq', value: 'billing' }]) + + await GET(buildListRequest(`?workspaceId=${WORKSPACE_ID}`), context) + await GET( + buildListRequest(`?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(tagFilters)}`), + context + ) + + const [unfiltered, filtered] = mockListDocuments.mock.calls.map( + ([call]) => call.input.cursorScope + ) + expect(unfiltered).not.toEqual(filtered) + }) + + it('rejects malformed and wrongly shaped tag filters with a 400', async () => { + const malformed = await GET( + buildListRequest(`?workspaceId=${WORKSPACE_ID}&tagFilters=not-json`), + context + ) + const wrongShape = await GET( + buildListRequest( + `?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(JSON.stringify([{ tagSlot: 'tag1' }]))}` + ), + context + ) + + expect(malformed.status).toBe(400) + expect(await malformed.json()).toEqual({ + error: expect.objectContaining({ + message: 'tagFilters must be a JSON-encoded array of tag filters', + }), + }) + expect(wrongShape.status).toBe(400) + expect(mockListDocuments).not.toHaveBeenCalled() + }) +}) + +describe('PATCH /api/v2/knowledge/[id]/documents', () => { + beforeEach(() => { + vi.clearAllMocks() + authenticateAsPersonalKey() + mockBulkUpdate.mockResolvedValue({ + operation: 'disable', + successCount: 2, + updatedDocuments: [ + { id: 'doc-1', enabled: false }, + { id: 'doc-2', enabled: false }, + ], + selectAll: false, + }) + }) + + /** + * `documentIds` is bounded by the request; `selectAll` is bounded by nothing. + * Echoing the identifiers for a knowledge base of 100k documents is a + * multi-megabyte array the caller never asked for, materialized and then + * element-wise validated by the response schema. + */ + it('omits the identifier echo for an unbounded selectAll update', async () => { + mockBulkUpdate.mockResolvedValueOnce({ + operation: 'disable', + successCount: 100_000, + updatedDocuments: Array.from({ length: 100_000 }, (_, index) => ({ + id: `doc-${index}`, + enabled: false, + })), + selectAll: true, + }) + + const response = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, operation: 'disable', selectAll: true }), + context + ) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { operation: 'disable', updatedCount: 100_000 }, + }) + }) + + it('disables the named documents and answers with one object, not a page', async () => { + const response = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'disable', + documentIds: ['doc-1', 'doc-2'], + }), + context + ) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { operation: 'disable', updatedCount: 2, documentIds: ['doc-1', 'doc-2'] }, + }) + expect(mockBulkUpdate).toHaveBeenCalledWith( + expect.objectContaining({ + input: { + knowledgeBaseId: 'kb-1', + assertedWorkspaceId: WORKSPACE_ID, + operation: 'disable', + documentIds: ['doc-1', 'doc-2'], + selectAll: undefined, + enabledFilter: undefined, + }, + }) + ) + }) + + it('does not expose an unaudited bulk delete', async () => { + const response = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'delete', + documentIds: ['doc-1'], + }), + context + ) + + expect(response.status).toBe(400) + expect(await response.json()).toEqual({ + error: expect.objectContaining({ + message: expect.stringContaining('operation: expected one of "enable" | "disable"'), + }), + }) + expect(mockBulkUpdate).not.toHaveBeenCalled() + }) + + it('requires exactly one selection and bounds an explicit list', async () => { + const neither = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, operation: 'enable' }), + context + ) + const both = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'enable', + documentIds: ['doc-1'], + selectAll: true, + }), + context + ) + const tooMany = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'enable', + documentIds: Array.from({ length: 101 }, (_, index) => `doc-${index}`), + }), + context + ) + + expect(neither.status).toBe(400) + expect(both.status).toBe(400) + expect(tooMany.status).toBe(400) + expect(mockBulkUpdate).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts index 1b03a060bbe..203817873d4 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts @@ -39,6 +39,10 @@ vi.mock('@/lib/knowledge/application/documents', () => ({ operation: { id: 'knowledge.documents.list' }, execute: vi.fn(), }, + bulkUpdateKnowledgeDocuments: { + operation: { id: 'knowledge.documents.bulk' }, + execute: vi.fn(), + }, admitKnowledgeDocumentUpload: { operation: { id: 'knowledge.documents.upload' }, execute: mockAdmitUpload, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts index 6ba62a1a941..1f11166b8f8 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts @@ -1,5 +1,6 @@ import { - type V2KnowledgeDocumentSummary, + parseV2KnowledgeTagFiltersParam, + v2BulkUpdateKnowledgeDocumentsContract, v2ListKnowledgeDocumentsContract, v2UploadKnowledgeDocumentContract, } from '@/lib/api/contracts/v2/knowledge' @@ -20,6 +21,7 @@ import { import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { admitKnowledgeDocumentUpload, + bulkUpdateKnowledgeDocuments, listKnowledgeDocuments, uploadKnowledgeDocument, } from '@/lib/knowledge/application/documents' @@ -28,42 +30,18 @@ import { KnowledgeDocumentUnsupportedMediaTypeError } from '@/lib/knowledge/appl import { captureServerEvent } from '@/lib/posthog/server' import { MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE } from '@/lib/uploads/shared/types' import { validateFileType } from '@/lib/uploads/utils/validation' -import { serializeDate } from '@/app/api/v1/knowledge/utils' -import { decodeOffsetCursor, encodeCursor } from '@/app/api/v2/lib/response' +import { toV2DocumentSummary, toV2TaggedDocument } from '@/app/api/v2/knowledge/utils' +import { + decodeOffsetCursor, + encodeOffsetCursor, + offsetCursorScope, +} from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 const MAX_FILE_SIZE = MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE -function toV2DocumentSummary(document: { - id: string - knowledgeBaseId: string - filename: string - fileSize: number - mimeType: string - processingStatus?: 'pending' | 'processing' | 'completed' | 'failed' - chunkCount: number - tokenCount: number - characterCount: number - enabled: boolean - uploadedAt: Date -}): V2KnowledgeDocumentSummary { - return { - id: document.id, - knowledgeBaseId: document.knowledgeBaseId, - filename: document.filename, - fileSize: document.fileSize, - mimeType: document.mimeType, - processingStatus: document.processingStatus ?? 'pending', - chunkCount: document.chunkCount, - tokenCount: document.tokenCount, - characterCount: document.characterCount, - enabled: document.enabled, - createdAt: serializeDate(document.uploadedAt), - } -} - /** GET /api/v2/knowledge/[id]/documents — List documents in a knowledge base. */ export const GET = defineV2JsonRoute({ contract: v2ListKnowledgeDocumentsContract, @@ -71,25 +49,93 @@ export const GET = defineV2JsonRoute({ operation: knowledgeOperations.listDocuments, rateLimit: v2RateLimits.publicApi, errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, - mapInput: ({ params, query }) => ({ - knowledgeBaseId: params.id, - assertedWorkspaceId: query.workspaceId, - enabledFilter: query.enabledFilter, - search: query.search, - limit: query.limit, - offset: decodeOffsetCursor(query.cursor), - sortBy: query.sortBy, - sortOrder: query.sortOrder, - }), + mapInput: ({ params, query }) => { + const tagFilters = parseV2KnowledgeTagFiltersParam(query.tagFilters) + if (!tagFilters.success) { + throw new OrchestrationError('validation', tagFilters.message) + } + /** + * The offset counts positions in the filtered, sorted document sequence, so + * every param that changes that sequence is stamped into the cursor and + * re-checked here. `limit` selects how much of the sequence to return, not + * what the sequence is, so it stays out. + */ + const cursorScope = offsetCursorScope({ + knowledgeBaseId: params.id, + enabledFilter: query.enabledFilter, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + tagFilters: query.tagFilters, + }) + return { + knowledgeBaseId: params.id, + assertedWorkspaceId: query.workspaceId, + enabledFilter: query.enabledFilter, + search: query.search, + limit: query.limit, + offset: decodeOffsetCursor(query.cursor, cursorScope), + sortBy: query.sortBy, + sortOrder: query.sortOrder, + tagNameFilters: tagFilters.filters, + cursorScope, + } + }, useCase: listKnowledgeDocuments, - present: ({ documents, pagination }) => ({ - data: documents.map(toV2DocumentSummary), + present: ({ documents, tagDefinitions, pagination, cursorScope }) => ({ + data: documents.map((document) => toV2TaggedDocument(document, tagDefinitions)), nextCursor: pagination.hasMore - ? encodeCursor({ offset: pagination.offset + pagination.limit }) + ? encodeOffsetCursor(cursorScope ?? '', pagination.offset + pagination.limit) : null, }), }) +/** + * PATCH /api/v2/knowledge/[id]/documents — Enable or disable many documents. + * + * Enable and disable only. Bulk delete is deliberately not offered: the bulk + * operation records no semantic audit, so a public bulk delete would empty a + * knowledge base leaving no `DOCUMENT_DELETED` entries, while the per-document + * DELETE audits every one. + */ +export const PATCH = defineV2JsonRoute({ + contract: v2BulkUpdateKnowledgeDocumentsContract, + auth: v2ApiKeyAuth, + operation: knowledgeOperations.bulkDocuments, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, + mapInput: ({ params, body }) => ({ + knowledgeBaseId: params.id, + assertedWorkspaceId: body.workspaceId, + operation: body.operation, + documentIds: body.documentIds, + selectAll: body.selectAll, + enabledFilter: body.enabledFilter, + }), + useCase: bulkUpdateKnowledgeDocuments, + present: (result) => { + if (result.operation === 'delete') { + throw new Error('Bulk knowledge document delete is not exposed on the public API') + } + /** + * `documentIds` is echoed only for an explicit-list request, which the body + * bounds. A `selectAll` request has no such bound: a knowledge base with + * 100k documents would otherwise materialize and element-wise validate a + * multi-megabyte identifier array nobody asked for. That caller reads + * `updatedCount` and re-lists if it needs the identifiers. + */ + return { + data: { + operation: result.operation, + updatedCount: result.successCount, + documentIds: result.selectAll + ? undefined + : result.updatedDocuments.map((document) => document.id), + }, + } + }, +}) + /** POST /api/v2/knowledge/[id]/documents — Upload a document to a knowledge base. */ export const POST = defineV2BodyLifecycleRoute({ contract: v2UploadKnowledgeDocumentContract, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts index fbf8e030418..0e48f8489e7 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts @@ -1,13 +1,10 @@ import type { NextResponse } from 'next/server' -import type { - V2KnowledgeDocumentSummary, - V2KnowledgeDocumentUpload, -} from '@/lib/api/contracts/v2/knowledge' +import type { V2KnowledgeDocumentUpload } from '@/lib/api/contracts/v2/knowledge' import { KnowledgeUsageLimitExceededError } from '@/lib/knowledge/application/billing' import { KnowledgeDocumentUnsupportedMediaTypeError } from '@/lib/knowledge/application/upload-sessions' import type { CreatedKnowledgeDocument } from '@/lib/knowledge/orchestration/documents' import type { UploadSessionRecord } from '@/lib/uploads/upload-session/service' -import { serializeDate } from '@/app/api/v1/knowledge/utils' +import { toV2DocumentSummary } from '@/app/api/v2/knowledge/utils' import { v2CaughtOrchestrationError, v2Error } from '@/app/api/v2/lib/response' export function v2KnowledgeDocumentUploadError(error: unknown): NextResponse | null { @@ -20,24 +17,6 @@ export function v2KnowledgeDocumentUploadError(error: unknown): NextResponse | n return v2CaughtOrchestrationError(error) } -export function toV2KnowledgeDocumentSummary( - document: CreatedKnowledgeDocument -): V2KnowledgeDocumentSummary { - return { - id: document.id, - knowledgeBaseId: document.knowledgeBaseId, - filename: document.filename, - fileSize: document.fileSize, - mimeType: document.mimeType, - processingStatus: document.processingStatus ?? 'pending', - chunkCount: document.chunkCount, - tokenCount: document.tokenCount, - characterCount: document.characterCount, - enabled: document.enabled, - createdAt: serializeDate(document.uploadedAt), - } -} - export function toV2KnowledgeDocumentUpload( session: UploadSessionRecord, document: CreatedKnowledgeDocument | null @@ -54,6 +33,6 @@ export function toV2KnowledgeDocumentUpload( size: session.fileSize, expiresAt: session.expiresAt.toISOString(), error: session.error, - document: document ? toV2KnowledgeDocumentSummary(document) : null, + document: document ? toV2DocumentSummary(document) : null, } } diff --git a/apps/sim/app/api/v2/knowledge/[id]/route.ts b/apps/sim/app/api/v2/knowledge/[id]/route.ts index b9bcee78298..6e2fdf65ba3 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/route.ts @@ -13,7 +13,6 @@ import { } from '@/lib/knowledge/application/knowledge-bases' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { toV2KnowledgeBase } from '@/app/api/v2/knowledge/utils' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -42,9 +41,6 @@ export const PATCH = defineV2JsonRoute({ operation: knowledgeOperations.update, rateLimit: v2RateLimits.publicApi, errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ params, body }) => ({ knowledgeBaseId: params.id, assertedWorkspaceId: body.workspaceId, diff --git a/apps/sim/app/api/v2/knowledge/[id]/tags/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/tags/route.test.ts new file mode 100644 index 00000000000..d6ab7343cc9 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/tags/route.test.ts @@ -0,0 +1,107 @@ +/** + * @vitest-environment node + */ +import { + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockListTags } = vi.hoisted(() => ({ + mockListTags: vi.fn(), +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/knowledge/application/tags', () => ({ + listKnowledgeTags: { operation: { id: 'knowledge.tags.list' }, execute: mockListTags }, +})) + +import { knowledgeOperations } from '@/lib/knowledge/application/operations' +import { GET } from '@/app/api/v2/knowledge/[id]/tags/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { kind: 'workspace_api_key', workspaceId: WORKSPACE_ID, keyId: 'key-1' } as const + +function buildRequest(query = `?workspaceId=${WORKSPACE_ID}`) { + return new NextRequest(`http://localhost/api/v2/knowledge/kb-1/tags${query}`, { + headers: { 'x-api-key': 'secret' }, + }) +} + +const context = { params: Promise.resolve({ id: 'kb-1' }) } + +describe('GET /api/v2/knowledge/[id]/tags', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'billing-owner', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], + rateLimitSubscription: null, + keyType: 'workspace', + }) + mockListTags.mockResolvedValue({ + tagDefinitions: [ + { + id: 'tag-def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: new Date('2025-01-10T09:00:00Z'), + updatedAt: new Date('2025-01-10T09:00:00Z'), + }, + ], + }) + }) + + it('returns the tag vocabulary as a full-set list', async () => { + const response = await GET(buildRequest(), context) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: [{ displayName: 'category', tagSlot: 'tag1', fieldType: 'text' }], + nextCursor: null, + }) + expect(mockListTags).toHaveBeenCalledWith( + expect.objectContaining({ + principal: PRINCIPAL, + input: { knowledgeBaseId: 'kb-1', assertedWorkspaceId: WORKSPACE_ID }, + }) + ) + expect(response.headers.get('cache-control')).toBe('private, no-store') + }) + + it('does not publish the tag definition identifier or its timestamps', async () => { + const response = await GET(buildRequest(), context) + + const [tag] = (await response.json()).data + expect(Object.keys(tag).sort()).toEqual(['displayName', 'fieldType', 'tagSlot']) + }) + + it('requires the workspace scope', async () => { + const response = await GET(buildRequest(''), context) + + expect(response.status).toBe(400) + expect(mockListTags).not.toHaveBeenCalled() + }) + + it('is reachable by a workspace API key, like its sibling knowledge reads', () => { + expect(knowledgeOperations.listTags.workspaceApiKey).toBe('allow') + expect(knowledgeOperations.listTags.principalKinds).toContain('workspace_api_key') + expect(knowledgeOperations.listTags.workspaceApiKey).toBe( + knowledgeOperations.listDocuments.workspaceApiKey + ) + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/tags/route.ts b/apps/sim/app/api/v2/knowledge/[id]/tags/route.ts new file mode 100644 index 00000000000..fadf77fe83a --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/tags/route.ts @@ -0,0 +1,35 @@ +import { v2ListKnowledgeTagsContract } from '@/lib/api/contracts/v2/knowledge' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' +import { knowledgeOperations } from '@/lib/knowledge/application/operations' +import { listKnowledgeTags } from '@/lib/knowledge/application/tags' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * GET /api/v2/knowledge/[id]/tags — List the knowledge base's tag vocabulary. + * + * Full-set list: a knowledge base has a fixed number of tag slots, so the whole + * vocabulary is one page and `nextCursor` is always null. + */ +export const GET = defineV2JsonRoute({ + contract: v2ListKnowledgeTagsContract, + auth: v2ApiKeyAuth, + operation: knowledgeOperations.listTags, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, + mapInput: ({ params, query }) => ({ + knowledgeBaseId: params.id, + assertedWorkspaceId: query.workspaceId, + }), + useCase: listKnowledgeTags, + present: ({ tagDefinitions }) => ({ + data: tagDefinitions.map((definition) => ({ + displayName: definition.displayName, + tagSlot: definition.tagSlot, + fieldType: definition.fieldType, + })), + nextCursor: null, + }), +}) diff --git a/apps/sim/app/api/v2/knowledge/folders/route.ts b/apps/sim/app/api/v2/knowledge/folders/route.ts index 45887f4c4b9..8392f343e68 100644 --- a/apps/sim/app/api/v2/knowledge/folders/route.ts +++ b/apps/sim/app/api/v2/knowledge/folders/route.ts @@ -18,7 +18,6 @@ import { relocateKnowledgeFolder, } from '@/lib/knowledge/application/folders' import { knowledgeOperations } from '@/lib/knowledge/application/operations' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -49,9 +48,6 @@ export const POST = defineV2JsonRoute({ operation: knowledgeOperations.createFolder, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, path: body.path, source: 'api' }), useCase: createKnowledgeFolder, present: ({ folder }) => ({ data: toFolderPathView(folder, folder.path) }), @@ -63,9 +59,6 @@ export const PATCH = defineV2JsonRoute({ operation: knowledgeOperations.relocateFolder, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, path: body.path, diff --git a/apps/sim/app/api/v2/knowledge/route.test.ts b/apps/sim/app/api/v2/knowledge/route.test.ts index 22084ec7aca..d2a9bf3e81a 100644 --- a/apps/sim/app/api/v2/knowledge/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/route.test.ts @@ -59,6 +59,7 @@ vi.mock('@/lib/users/queries', () => ({ requireResolvedUserEmail: (emails: Map, userId: string) => emails.get(userId)!, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' import { GET, POST } from '@/app/api/v2/knowledge/route' const WORKSPACE_ID = 'workspace-1' @@ -104,6 +105,9 @@ describe('/api/v2/knowledge route composition', () => { mockGetUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'owner@example.com']])) mockList.mockResolvedValue({ knowledgeBases: [{ knowledgeBase: buildKnowledgeBase(), folderPath: '/' }], + nextCursorKeys: null, + sortBy: 'name', + sortOrder: 'desc', }) mockCreate.mockResolvedValue({ knowledgeBase: buildKnowledgeBase(), folderPath: '/' }) }) @@ -125,6 +129,8 @@ describe('/api/v2/knowledge route composition', () => { search: 'support', sortBy: 'name', sortOrder: 'desc', + limit: V2_DEFAULT_PAGE_SIZE, + cursorKeys: undefined, }, request, }) diff --git a/apps/sim/app/api/v2/knowledge/route.ts b/apps/sim/app/api/v2/knowledge/route.ts index f8f0ee2e4a5..789db35d934 100644 --- a/apps/sim/app/api/v2/knowledge/route.ts +++ b/apps/sim/app/api/v2/knowledge/route.ts @@ -16,7 +16,7 @@ import { import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { captureServerEvent } from '@/lib/posthog/server' import { toV2KnowledgeBase, toV2KnowledgeBases } from '@/app/api/v2/knowledge/utils' -import { v2Error } from '@/app/api/v2/lib/response' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -34,11 +34,15 @@ export const GET = defineV2JsonRoute({ search: query.search, sortBy: query.sortBy, sortOrder: query.sortOrder, + limit: query.limit, + cursorKeys: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), }), useCase: listKnowledgeBases, - present: async ({ knowledgeBases }) => ({ + present: async ({ knowledgeBases, nextCursorKeys, sortBy, sortOrder }) => ({ data: await toV2KnowledgeBases(knowledgeBases), - nextCursor: null, + nextCursor: nextCursorKeys + ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextCursorKeys) + : null, }), }) @@ -49,9 +53,6 @@ export const POST = defineV2JsonRoute({ operation: knowledgeOperations.create, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, name: body.name, diff --git a/apps/sim/app/api/v2/knowledge/search/route.test.ts b/apps/sim/app/api/v2/knowledge/search/route.test.ts index cb7816ec799..e705128e562 100644 --- a/apps/sim/app/api/v2/knowledge/search/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/search/route.test.ts @@ -53,13 +53,16 @@ describe('POST /api/v2/knowledge/search', () => { mockSearch.mockResolvedValue({ results: [ { + embeddingId: 'embedding-1', + knowledgeBaseId: 'kb-1', documentId: 'doc-1', documentName: 'support.txt', sourceUrl: null, content: 'hello', chunkIndex: 0, - metadata: {}, + metadata: { category: 'billing' }, similarity: 0.9, + rerankerScore: 0.42, }, ], query: 'hello', @@ -92,6 +95,9 @@ describe('POST /api/v2/knowledge/search', () => { topK: 10, tagFilters: undefined, searchMode: 'hybrid', + rerankerEnabled: undefined, + rerankerModel: undefined, + rerankerInputCount: undefined, }, request, }) @@ -102,6 +108,121 @@ describe('POST /api/v2/knowledge/search', () => { expect(response.headers.get('x-ratelimit-limit')).toBe('100') }) + it('names the source knowledge base and the reranker score on every result', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1', 'kb-2'], + query: 'hello', + topK: 10, + }) + ) + ) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data.results[0]).toEqual({ + knowledgeBaseId: 'kb-1', + documentId: 'doc-1', + documentName: 'support.txt', + sourceUrl: null, + content: 'hello', + chunkIndex: 0, + metadata: { category: 'billing' }, + similarity: 0.9, + rerankerScore: 0.42, + }) + }) + + it('forwards reranker options and never a caller-supplied reranker key', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerInputCount: 40, + }) + ) + ) + + expect(response.status).toBe(200) + expect(mockSearch).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerInputCount: 40, + }), + }) + ) + const [{ input }] = mockSearch.mock.calls[0] + expect(input).not.toHaveProperty('rerankerApiKey') + expect(input).not.toHaveProperty('skipUsageBilling') + }) + + it('rejects an unsupported reranker model and an out-of-range candidate pool', async () => { + const unsupportedModel = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-does-not-exist', + }) + ) + ) + const oversizedPool = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerInputCount: 101, + }) + ) + ) + + expect(unsupportedModel.status).toBe(400) + expect(oversizedPool.status).toBe(400) + expect(await oversizedPool.json()).toEqual({ + error: expect.objectContaining({ + code: 'BAD_REQUEST', + message: expect.stringContaining('rerankerInputCount cannot exceed 100'), + }), + }) + expect(mockSearch).not.toHaveBeenCalled() + }) + + it('drops a caller-supplied reranker key instead of forwarding it', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerApiKey: 'secret-byok-key', + }) + ) + ) + + expect(response.status).toBe(200) + const [{ input }] = mockSearch.mock.calls[0] + expect(input).not.toHaveProperty('rerankerApiKey') + }) + it('forwards an opted-in hybrid search mode to the application use case', async () => { const response = await POST( buildRequest( diff --git a/apps/sim/app/api/v2/knowledge/search/route.ts b/apps/sim/app/api/v2/knowledge/search/route.ts index 070e1342015..d22a4be4f8c 100644 --- a/apps/sim/app/api/v2/knowledge/search/route.ts +++ b/apps/sim/app/api/v2/knowledge/search/route.ts @@ -3,7 +3,6 @@ import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/ import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { searchKnowledge } from '@/lib/knowledge/application/search' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -24,7 +23,6 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseUsageAuthorization, parseOptions: { maxBodyBytes: V2_KNOWLEDGE_SEARCH_MAX_BODY_BYTES, - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, @@ -35,7 +33,25 @@ export const POST = defineV2JsonRoute({ topK: body.topK, tagFilters: body.tagFilters, searchMode: body.searchMode, + rerankerEnabled: body.rerankerEnabled, + rerankerModel: body.rerankerModel, + rerankerInputCount: body.rerankerInputCount, }), useCase: searchKnowledge, - present: (result) => ({ data: result }), + /** + * Projected field by field rather than spread. The use-case result also + * carries `userId`, `workspaceId`, a `cost` breakdown with pricing internals, + * and a live resolved-secret trace registry; only Zod's default key-stripping + * keeps them off the wire today, so a single loosened or opaque field in the + * response schema would ship them. + */ + present: (result) => ({ + data: { + results: result.results, + query: result.query, + knowledgeBaseIds: result.knowledgeBaseIds, + topK: result.topK, + totalResults: result.totalResults, + }, + }), }) diff --git a/apps/sim/app/api/v2/knowledge/utils.test.ts b/apps/sim/app/api/v2/knowledge/utils.test.ts new file mode 100644 index 00000000000..9ccf8818312 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/utils.test.ts @@ -0,0 +1,92 @@ +/** + * @vitest-environment node + */ + +import { describe, expect, it } from 'vitest' +import { v2KnowledgeTaggedDocumentSchema } from '@/lib/api/contracts/v2/knowledge' +import type { DocumentTagDefinition } from '@/lib/knowledge/tags/types' +import { + toV2DocumentSummary, + toV2DocumentTags, + toV2TaggedDocument, +} from '@/app/api/v2/knowledge/utils' + +const uploadedAt = new Date('2026-08-01T00:00:00.000Z') + +const documentRow = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'invoice.pdf', + fileSize: 1024, + mimeType: 'application/pdf', + processingStatus: 'completed', + chunkCount: 4, + tokenCount: 512, + characterCount: 2048, + enabled: true, + uploadedAt, + tag1: 'billing', + number1: 7, +} + +const tagDefinitions: DocumentTagDefinition[] = [ + { + id: 'def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: uploadedAt, + updatedAt: uploadedAt, + } as DocumentTagDefinition, +] + +describe('toV2DocumentSummary', () => { + it('serializes the shared document fields', () => { + expect(toV2DocumentSummary(documentRow)).toEqual({ + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'invoice.pdf', + fileSize: 1024, + mimeType: 'application/pdf', + processingStatus: 'completed', + chunkCount: 4, + tokenCount: 512, + characterCount: 2048, + enabled: true, + createdAt: '2026-08-01T00:00:00.000Z', + }) + }) + + it('returns a null createdAt for a document with no upload timestamp', () => { + expect(toV2DocumentSummary({ ...documentRow, uploadedAt: null }).createdAt).toBeNull() + }) + + it('reads an absent processing status as pending', () => { + expect(toV2DocumentSummary({ ...documentRow, processingStatus: null }).processingStatus).toBe( + 'pending' + ) + }) +}) + +describe('toV2TaggedDocument', () => { + it('produces a contract-valid list item with tags keyed by display name', () => { + const projected = toV2TaggedDocument(documentRow, tagDefinitions) + expect(v2KnowledgeTaggedDocumentSchema.parse(projected)).toEqual(projected) + expect(projected.tags).toEqual({ category: 'billing', number1: 7 }) + }) + + it('does not throw when the document has no upload timestamp', () => { + const projected = toV2TaggedDocument({ ...documentRow, uploadedAt: null }, tagDefinitions) + expect(projected.createdAt).toBeNull() + expect(v2KnowledgeTaggedDocumentSchema.parse(projected)).toEqual(projected) + }) +}) + +describe('toV2DocumentTags', () => { + it('serializes a date-valued slot as an ISO string', () => { + expect(toV2DocumentTags({ date1: new Date('2026-08-02T00:00:00.000Z') }, [])).toEqual({ + date1: '2026-08-02T00:00:00.000Z', + }) + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/utils.ts b/apps/sim/app/api/v2/knowledge/utils.ts index 373f7b2dbd4..1751b673ee3 100644 --- a/apps/sim/app/api/v2/knowledge/utils.ts +++ b/apps/sim/app/api/v2/knowledge/utils.ts @@ -1,6 +1,113 @@ -import type { V2KnowledgeBase } from '@/lib/api/contracts/v2/knowledge' +import type { + V2KnowledgeBase, + V2KnowledgeDocumentSummary, + V2KnowledgeTaggedDocument, +} from '@/lib/api/contracts/v2/knowledge' +import { ALL_TAG_SLOTS, type AllTagSlot } from '@/lib/knowledge/constants' +import type { DocumentTagDefinition } from '@/lib/knowledge/tags/types' import type { KnowledgeBaseWithCounts } from '@/lib/knowledge/types' import { getUserEmailsByIds, requireResolvedUserEmail } from '@/lib/users/queries' +import { serializeDate } from '@/app/api/v1/knowledge/utils' + +/** + * Projects a document's tag slots onto a map keyed by tag display name, the same + * projection knowledge search applies to its result `metadata`. A slot holding a + * value with no definition keeps its raw slot name rather than disappearing. + */ +export function toV2DocumentTags( + document: Partial>, + tagDefinitions: readonly DocumentTagDefinition[] +): Record { + const displayNameBySlot = new Map( + tagDefinitions.map((definition) => [definition.tagSlot, definition.displayName]) + ) + const tags: Record = {} + for (const slot of ALL_TAG_SLOTS) { + const value = document[slot] + if (value === null || value === undefined) continue + const key = displayNameBySlot.get(slot) ?? slot + if (value instanceof Date) { + tags[key] = value.toISOString() + } else if ( + typeof value === 'string' || + typeof value === 'number' || + typeof value === 'boolean' + ) { + tags[key] = value + } + } + return tags +} + +const PROCESSING_STATUSES = ['pending', 'processing', 'completed', 'failed'] as const + +type V2DocumentProcessingStatus = (typeof PROCESSING_STATUSES)[number] + +/** + * Narrows a stored processing status onto the published enum. An absent value + * reads as `pending`, matching the column default; an unrecognised one is a + * producer bug rather than a caller-reachable failure, so it throws. + */ +export function toProcessingStatus(status: string | null | undefined): V2DocumentProcessingStatus { + if (status === null || status === undefined) return 'pending' + const known = PROCESSING_STATUSES.find((candidate) => candidate === status) + if (!known) throw new Error(`Unexpected knowledge document processing status: ${status}`) + return known +} + +/** + * The document columns every v2 document projection reads. `uploadedAt` is + * accepted as nullable because the column is nullable in storage. + */ +export interface V2DocumentSummarySource { + id: string + knowledgeBaseId: string + filename: string + fileSize: number + mimeType: string + processingStatus?: string | null + chunkCount: number + tokenCount: number + characterCount: number + enabled: boolean + uploadedAt: Date | string | null | undefined +} + +/** + * The single v2 document summary projection. Every v2 document response — list + * item, upload acknowledgement, detail — is this shape plus its own extras, so + * the shared field set is serialized in exactly one place. + */ +export function toV2DocumentSummary(document: V2DocumentSummarySource): V2KnowledgeDocumentSummary { + return { + id: document.id, + knowledgeBaseId: document.knowledgeBaseId, + filename: document.filename, + fileSize: document.fileSize, + mimeType: document.mimeType, + processingStatus: toProcessingStatus(document.processingStatus), + chunkCount: document.chunkCount, + tokenCount: document.tokenCount, + characterCount: document.characterCount, + enabled: document.enabled, + createdAt: serializeDate(document.uploadedAt), + } +} + +interface V2TaggedDocumentSource + extends V2DocumentSummarySource, + Partial> {} + +/** Serializes a document summary with its tag values keyed by display name. */ +export function toV2TaggedDocument( + document: V2TaggedDocumentSource, + tagDefinitions: readonly DocumentTagDefinition[] +): V2KnowledgeTaggedDocument { + return { + ...toV2DocumentSummary(document), + tags: toV2DocumentTags(document, tagDefinitions), + } +} interface KnowledgeBaseWithFolder { knowledgeBase: KnowledgeBaseWithCounts diff --git a/apps/sim/app/api/v2/lib/response.test.ts b/apps/sim/app/api/v2/lib/response.test.ts new file mode 100644 index 00000000000..f84dd84a203 --- /dev/null +++ b/apps/sim/app/api/v2/lib/response.test.ts @@ -0,0 +1,48 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { v2Error } from '@/app/api/v2/lib/response' + +describe('v2Error retry guidance', () => { + it('sends Retry-After on 503 so a client does not retry a degraded dependency immediately', () => { + const response = v2Error('SERVICE_UNAVAILABLE', 'Service temporarily unavailable') + + expect(response.status).toBe(503) + const retryAfter = response.headers.get('Retry-After') + expect(retryAfter).not.toBeNull() + expect(Number(retryAfter)).toBeGreaterThan(0) + expect(Number.isInteger(Number(retryAfter))).toBe(true) + }) + + it('lets a caller-supplied Retry-After win over the default', () => { + const response = v2Error('SERVICE_UNAVAILABLE', 'Service temporarily unavailable', { + headers: { 'Retry-After': '30' }, + }) + + expect(response.headers.get('Retry-After')).toBe('30') + }) + + it('does not invent Retry-After for failures a retry cannot fix', () => { + for (const code of ['BAD_REQUEST', 'NOT_FOUND', 'FORBIDDEN', 'CONFLICT'] as const) { + expect(v2Error(code, 'nope').headers.get('Retry-After')).toBeNull() + } + }) + + it('does not default Retry-After on 429, whose wait comes from the token bucket', () => { + expect(v2Error('RATE_LIMITED', 'API rate limit exceeded').headers.get('Retry-After')).toBeNull() + }) + + it('stays silent on retrying when the outcome is unknown rather than absent', () => { + const response = v2Error( + 'SERVICE_UNAVAILABLE', + 'Async execution queue acceptance unconfirmed', + { + omitRetryAfter: true, + } + ) + + expect(response.status).toBe(503) + expect(response.headers.get('Retry-After')).toBeNull() + }) +}) diff --git a/apps/sim/app/api/v2/lib/response.ts b/apps/sim/app/api/v2/lib/response.ts index c0567024d65..8228a8ee3d5 100644 --- a/apps/sim/app/api/v2/lib/response.ts +++ b/apps/sim/app/api/v2/lib/response.ts @@ -2,6 +2,8 @@ import { NextResponse } from 'next/server' import type { ZodError } from 'zod' import { type CursorKey, INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' import { getValidationErrorMessage, serializeZodIssues } from '@/lib/api/server' +import { ADMISSION_RETRY_AFTER_SECONDS } from '@/lib/core/admission/transient-failure' +import { forbiddenErrorDetails } from '@/lib/core/application' import { asOrchestrationError, OrchestrationError, @@ -59,6 +61,44 @@ const V2_CODE_BY_HTTP_STATUS: Partial> = Object.from */ const PRIVATE_NO_STORE = { 'Cache-Control': 'private, no-store' } as const +/** + * Seconds a caller should wait before retrying a transient v2 failure, for the + * statuses whose response carries no other timing signal. + * + * Keyed on the response status rather than the v2 error code because + * `Retry-After` is defined against the status, and the status is the only half + * of the pair a client actually sees. `v2Error` lets a caller override the + * status independently of the code, so keying on the code would let the two + * disagree. + * + * RFC 9110 §10.2.3 singles out 503 as the status whose `Retry-After` means "how + * long the service is expected to be unavailable to the client", and §15.6.4 + * permits one. Note the requirement level is `MAY`, so this is a deliberate + * improvement on the baseline rather than a conformance fix: without it a + * client's only defensible policy on a 503 is an immediate retry, which is + * exactly the traffic a degraded dependency cannot absorb. Sim raises 503 when + * the API-key store, the rollout gate, the rate-limit backend, or + * execution-identity allocation is briefly unavailable, and all four are made + * worse by an unthrottled retry storm. + * + * 429 is deliberately absent because every 429 already knows its own wait: the + * throttle path measures it from the caller's token bucket + * ({@link v2RateLimitError}), and an admission denial carries the descriptor's + * declared `retryAfterSeconds` through to the route. Defaulting it here would + * paper over a path that had simply dropped its value — which is exactly the + * bug that used to leave a concurrency denial with no `Retry-After` at all. + * + * The value is Sim's one transient-failure floor, shared with the admission + * descriptors so the execute route's capacity 429 and every other surface's 503 + * cannot drift apart. It is a floor, not a schedule: a fleet that retries at + * exactly this offset re-converges into a single burst, so callers should still + * add jitter — `backoffWithJitter` from `@sim/utils/retry` is what Sim's own + * clients use. + */ +const RETRY_AFTER_SECONDS_BY_STATUS: Partial> = { + 503: ADMISSION_RETRY_AFTER_SECONDS, +} + type RateLimitHeaderSource = Pick export function rateLimitHeaders(rateLimit?: RateLimitHeaderSource): Record { @@ -80,6 +120,21 @@ function successHeaders(options: V2SuccessOptions): Record { return { ...PRIVATE_NO_STORE, ...rateLimitHeaders(options.rateLimit), ...options.headers } } +/** + * The bodiless 200 a `HEAD` receives from a route whose `GET` is not safe. + * + * RFC 9110 §9.3.2 lets Next alias `HEAD` onto `GET` only because §9.2.1 defines + * `HEAD` as safe — "essentially read-only". A `GET` that opens an outbound + * connection or writes a row breaks that assumption, and an uptime monitor or + * link checker walking the documented URL list would drive those effects + * invisibly on every probe. Such a route answers the authorization and + * rate-limit questions and stops there. `HEAD` carries no body in any case, so + * nothing the caller can observe is fabricated. + */ +export function v2HeadNoEffect(options: V2SuccessOptions = {}): NextResponse { + return new NextResponse(null, { status: options.status ?? 200, headers: successHeaders(options) }) +} + /** `{ data }` (+ rate-limit headers). */ export function v2Data(data: T, options: V2SuccessOptions = {}): NextResponse { return NextResponse.json( @@ -104,6 +159,19 @@ interface V2ErrorOptions { status?: number details?: unknown headers?: Record + /** + * Suppresses the code's default `Retry-After` for a failure whose outcome is + * *unknown* rather than *absent*. + * + * A 503 normally means the work did not happen, so "come back in 5 seconds" + * is safe advice. The async enqueue that could not be confirmed + * (`ASYNC_ENQUEUE_AMBIGUOUS`) is the exception: it deliberately retains its + * execution-ID claim because a job may already exist. Telling that caller to + * retry invites a client with no `X-Run-Id` to start a second run of the same + * workflow, which bills twice. It must reconcile against the run id the + * response returns instead, so the response stays silent on retrying. + */ + omitRetryAfter?: boolean } /** `{ error: { code, message, details? } }`. */ @@ -114,11 +182,19 @@ export function v2Error( ): NextResponse { const error: { code: V2ErrorCode; message: string; details?: unknown } = { code, message } if (options.details !== undefined) error.details = options.details + const status = options.status ?? STATUS_BY_CODE[code] + const retryAfterSeconds = options.omitRetryAfter + ? undefined + : RETRY_AFTER_SECONDS_BY_STATUS[status] return NextResponse.json( { error }, { - status: options.status ?? STATUS_BY_CODE[code], - headers: { ...PRIVATE_NO_STORE, ...options.headers }, + status, + headers: { + ...PRIVATE_NO_STORE, + ...(retryAfterSeconds === undefined ? {} : { 'Retry-After': retryAfterSeconds.toString() }), + ...options.headers, + }, } ) } @@ -174,18 +250,56 @@ export function decodeCursor>(cursor: string): T | n } } +interface OffsetCursorPayload { + /** The query state the offset counts positions within. */ + scope: string + offset: number +} + /** - * Reads back an offset cursor minted by `encodeCursor({ offset })`. + * The filters and sort an offset cursor was minted under. + * + * An offset is only meaningful against one exact sequence, so everything that + * reorders or re-filters that sequence has to travel with it. Build the stamp + * from every such param; a value that does not affect ordering or membership + * (the page size itself) must stay out, or paging with a different `limit` + * would be rejected for no reason. + */ +export function offsetCursorScope(parts: Record): string { + return Object.keys(parts) + .sort() + .map((key) => `${key}=${parts[key] ?? ''}`) + .join('&') +} + +/** An offset cursor stamped with the query state that produced it. */ +export function encodeOffsetCursor(scope: string, offset: number): string { + return encodeCursor({ scope, offset } satisfies OffsetCursorPayload) +} + +/** + * Reads back an offset cursor, refusing one minted under different filters or a + * different sort. * * An absent cursor means page one. A cursor that is not valid base64-JSON, or * that does not carry a non-negative integer `offset`, is rejected rather than * coerced to 0: silently restarting at page one while the caller believes it is - * paging forward makes a paging client loop over the first page forever. The v2 - * error policies render the thrown validation error as the canonical 400. + * paging forward makes a paging client loop over the first page forever. + * + * The `scope` check is the offset counterpart of {@link decodeSortedCursor}'s + * sort stamp. A bare offset replayed against a newly filtered or re-sorted + * sequence names a different position in it, which silently skips rows, repeats + * them, or lands past the end and returns an empty page — the failure a keyset + * cursor is already protected from. The v2 error policies render the thrown + * validation error as the canonical 400. */ -export function decodeOffsetCursor(cursor: string | undefined): number { +export function decodeOffsetCursor(cursor: string | undefined, scope: string): number { if (!cursor) return 0 - const offset = decodeCursor<{ offset?: unknown }>(cursor)?.offset + const decoded = decodeCursor>(cursor) + if (!decoded || decoded.scope !== scope) { + throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) + } + const { offset } = decoded if (typeof offset !== 'number' || !Number.isInteger(offset) || offset < 0) { throw new OrchestrationError('validation', 'Invalid cursor') } @@ -241,9 +355,25 @@ export function decodeSortedCursor(cursor: string | undefined, sort: string): De return { status: 'ok', keys: decoded.keys } } -/** The 400 for a cursor that cannot be resumed under the request's sort. */ -export function v2CursorSortError(): NextResponse { - return v2Error('BAD_REQUEST', INVALID_CURSOR_MESSAGE) +/** + * The keyset a paged list should resume from, or `undefined` for page one. + * + * This is the `mapInput` half of every keyset list: it stamps the request's + * sort, reads the cursor back under it, and turns a cursor that was minted + * under a different sort into the canonical 400 rather than letting mismatched + * keys reach `keysetAfter`. Sharing it is what keeps "a bad cursor is a 400" + * from being re-decided per route. + */ +export function readSortedCursor( + cursor: string | undefined, + sortBy: string, + sortOrder: string +): CursorKey[] | undefined { + const decoded = decodeSortedCursor(cursor, cursorSortKey(sortBy, sortOrder)) + if (decoded.status === 'invalid') { + throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) + } + return decoded.status === 'ok' ? decoded.keys : undefined } const V2_CODE_BY_ORCHESTRATION_ERROR: Record = { @@ -278,9 +408,19 @@ export function v2ErrorForOrchestration( * Renders a thrown domain failure in the v2 envelope, or `null` when the error * carries no classification and the caller should log it and return its own * generic 500. The v2 counterpart of `orchestrationErrorResponse`. + * + * A refusal that names its cause carries it through as `error.details.code`. + * That projection lives here, on the one function every v2 error policy + * ultimately falls through to, rather than at each throw site — a route cannot + * then forget it, and the code cannot be attached to a status other than the + * one its failure class maps to. */ export function v2CaughtOrchestrationError(error: unknown): NextResponse | null { const classified = asOrchestrationError(error) if (!classified) return null - return v2ErrorForOrchestration(classified.code, classified.message) + return v2ErrorForOrchestration( + classified.code, + classified.message, + forbiddenErrorDetails(classified) + ) } diff --git a/apps/sim/app/api/v2/logs/route.test.ts b/apps/sim/app/api/v2/logs/route.test.ts index 74b5755c412..2b6bb177f72 100644 --- a/apps/sim/app/api/v2/logs/route.test.ts +++ b/apps/sim/app/api/v2/logs/route.test.ts @@ -128,6 +128,67 @@ describe('GET /api/v2/logs', () => { expect(mocks.execute).not.toHaveBeenCalled() }) + it.each([ + ['abc', 'startDate'], + ['2026-08-06', 'startDate'], + ['2026-08-06T00:00:00+02:00', 'startDate'], + ])('rejects %s as a window bound before it can reach the query', async (value, field) => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${field}=${encodeURIComponent(value)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('startDate') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('rejects an unparseable endDate', async () => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&endDate=abc`) + ) + + expect(response.status).toBe(400) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('forwards a UTC window bound as a Date', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&startDate=2026-08-06T00:00:00Z` + ) + ) + + expect(response.status).toBe(200) + expect(mocks.execute).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + filters: expect.objectContaining({ startDate: new Date('2026-08-06T00:00:00Z') }), + }), + }) + ) + }) + + it('rejects an inverted window instead of answering with an empty page', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&startDate=2026-08-06T00:00:00Z&endDate=2026-08-05T00:00:00Z` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { + code: 'BAD_REQUEST', + message: expect.stringContaining('startDate must be before or equal to endDate'), + }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + it('projects typed folder errors', async () => { mocks.execute.mockRejectedValueOnce(new OrchestrationError('not_found', 'Folder not found')) diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts b/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts index 01b9dffcb7e..d1c0836afdb 100644 --- a/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts +++ b/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ import type { mcpServers } from '@sim/db/schema' +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' import { @@ -9,38 +18,16 @@ import { NoWorkspaceAccessError, } from '@/lib/core/application' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - get: vi.fn(), - update: vi.fn(), - remove: vi.fn(), - capture: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), +const mocks = vi.hoisted(() => ({ + get: vi.fn(), + update: vi.fn(), + remove: vi.fn(), + capture: vi.fn(), })) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -49,7 +36,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) vi.mock('@/lib/mcp/application/use-cases', () => ({ getMcpServerUseCase: { operation: { id: 'mcp_servers.read' }, execute: mocks.get }, @@ -65,17 +51,10 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, - limit: 100, - remaining: 99, - resetAt: new Date('2026-01-01T00:00:00Z'), - retryAfterMs: 0, -} const server = { id: 'mcp-server-1', workspaceId: WORKSPACE_ID, @@ -122,10 +101,10 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/mcp-servers/[id]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.get.mockResolvedValue({ server }) mocks.update.mockResolvedValue({ server }) mocks.remove.mockResolvedValue({ server }) @@ -175,11 +154,12 @@ describe('/api/v2/mcp-servers/[id]', () => { }) it('authenticates before parsing an invalid update body', async () => { - mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) const response = await PATCH(request('PATCH', {}), context) expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.update).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/route.ts b/apps/sim/app/api/v2/mcp-servers/[id]/route.ts index 20027c2f474..1612b2269b0 100644 --- a/apps/sim/app/api/v2/mcp-servers/[id]/route.ts +++ b/apps/sim/app/api/v2/mcp-servers/[id]/route.ts @@ -3,12 +3,7 @@ import { v2GetMcpServerContract, v2UpdateMcpServerContract, } from '@/lib/api/contracts/v2/mcp-servers' -import { - createV2ResourceConcealmentPolicy, - defineV2JsonRoute, - v2ApiKeyAuth, - v2RateLimits, -} from '@/lib/api/server/routes' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { mcpServerOperations } from '@/lib/mcp/application/operations' import { deleteMcpServerUseCase, @@ -16,15 +11,11 @@ import { updateMcpServerUseCase, } from '@/lib/mcp/application/use-cases' import { captureServerEvent } from '@/lib/posthog/server' -import { toV2McpServer } from '@/app/api/v2/mcp-servers/utils' +import { mcpServerResourceErrorPolicy, toV2McpServer } from '@/app/api/v2/mcp-servers/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 -const mcpServerResourceErrorPolicy = createV2ResourceConcealmentPolicy({ - notFoundMessage: 'MCP server not found', -}) - /** GET /api/v2/mcp-servers/[id] — Fetch a single MCP server. */ export const GET = defineV2JsonRoute({ contract: v2GetMcpServerContract, diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.test.ts b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.test.ts new file mode 100644 index 00000000000..cf39d34b6a8 --- /dev/null +++ b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.test.ts @@ -0,0 +1,195 @@ +/** + * @vitest-environment node + */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + discover: vi.fn(), +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) +vi.mock('@/lib/api/server/rate-limit-context', () => ({ + recordRateLimitSnapshot: vi.fn(), + getRateLimitHeaders: vi.fn().mockReturnValue(null), +})) +vi.mock('@/lib/core/utils/request', () => ({ + generateRequestId: vi.fn().mockReturnValue('request-1'), + getClientIp: vi.fn().mockReturnValue('127.0.0.1'), +})) +vi.mock('@/lib/mcp/application/use-cases', () => ({ + discoverMcpServerToolsUseCase: { + operation: { id: 'mcp_servers.tools.discover' }, + execute: mocks.discover, + }, +})) + +import { WorkspaceApiKeyAuthorizationError } from '@/lib/core/application' +import { McpConnectionError, McpOauthAuthorizationRequiredError } from '@/lib/mcp/types' +import { GET } from '@/app/api/v2/mcp-servers/[id]/tools/route' + +const WORKSPACE_ID = 'workspace-1' +const SERVER_ID = 'mcp-3f7a9c21' +const PRINCIPAL = { kind: 'personal_api_key' as const, userId: 'user-1', keyId: 'key-1' } +const AUTH = { + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'] as const, + rateLimitSubscription: null, + keyType: 'personal' as const, +} +const TOOL = { + name: 'search_docs', + description: 'Search the internal documentation', + inputSchema: { + type: 'object' as const, + properties: { query: { type: 'string' } }, + required: ['query'], + }, + serverId: SERVER_ID, + serverName: 'Docs server', +} + +function request(query: string, method = 'GET') { + return new NextRequest(`http://localhost:3000/api/v2/mcp-servers/${SERVER_ID}/tools?${query}`, { + method, + headers: { 'x-api-key': 'key' }, + }) +} + +const context = { params: Promise.resolve({ id: SERVER_ID }) } + +describe('/api/v2/mcp-servers/[id]/tools', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.discover.mockResolvedValue({ tools: [TOOL] }) + }) + + it('returns a server tool inventory as a single page', async () => { + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), context) + const body = await response.json() + + expect(response.status).toBe(200) + expect(body).toEqual({ data: [TOOL], nextCursor: null }) + expect(mocks.discover).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: { workspaceId: WORKSPACE_ID, serverId: SERVER_ID, refresh: false }, + request: expect.anything(), + }) + }) + + it('forwards an explicit refresh so a caller can bypass the tool cache', async () => { + await GET(request(`workspaceId=${WORKSPACE_ID}&refresh=true`), { ...context }) + + expect(mocks.discover).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ refresh: true }) }) + ) + }) + + /** + * Next aliases a missing `HEAD` export onto `GET`, and RFC 9110 §9.2.1 defines + * `HEAD` as safe. Discovery is not: it opens a live connection to a + * third-party endpoint and writes the outcome onto the server row. An uptime + * monitor or link checker walking the documented URL list would otherwise + * drive both on every probe, invisibly. + */ + it('answers HEAD without connecting to the server or writing its status', async () => { + const response = await GET(request(`workspaceId=${WORKSPACE_ID}&refresh=true`, 'HEAD'), { + ...context, + }) + + expect(response.status).toBe(200) + expect(await response.text()).toBe('') + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('rejects a query param it does not implement', async () => { + const response = await GET(request(`workspaceId=${WORKSPACE_ID}&limit=10`), { ...context }) + + expect(response.status).toBe(400) + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('reports an unreachable server as a retryable 503, not a server fault', async () => { + mocks.discover.mockRejectedValueOnce(new McpConnectionError('ECONNREFUSED', 'Docs server')) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(503) + expect(body.error.code).toBe('SERVICE_UNAVAILABLE') + expect(response.headers.get('Retry-After')).not.toBeNull() + expect(JSON.stringify(body)).not.toContain('ECONNREFUSED') + }) + + it('reports a stale OAuth grant as a 409 a client can branch on, never as a Sim credential failure', async () => { + mocks.discover.mockRejectedValueOnce( + new McpOauthAuthorizationRequiredError(SERVER_ID, 'Docs server') + ) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(409) + expect(body.error.code).toBe('CONFLICT') + expect(body.error.details).toEqual({ code: 'MCP_SERVER_REAUTHORIZATION_REQUIRED' }) + }) + + it('does not blame the caller for an upstream protocol fault', async () => { + mocks.discover.mockRejectedValueOnce(new Error('MCP error -32602: Invalid params')) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(500) + expect(body.error.code).toBe('INTERNAL_ERROR') + expect(JSON.stringify(body)).not.toContain('Invalid params') + }) + + it('does not report a Sim-side response-schema defect as the caller`s bad request', async () => { + mocks.discover.mockResolvedValueOnce({ + tools: [{ ...TOOL, inputSchema: { ...TOOL.inputSchema, type: 'string' } }], + }) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(500) + expect(body.error.code).toBe('INTERNAL_ERROR') + }) + + it('rejects a workspace API key, which cannot supply the caller`s OAuth grant', async () => { + mocks.discover.mockRejectedValueOnce(new WorkspaceApiKeyAuthorizationError()) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(403) + expect(body.error.code).toBe('FORBIDDEN') + }) + + it('authenticates before parsing', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request(''), { ...context }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + expect(mocks.discover).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.ts b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.ts new file mode 100644 index 00000000000..137ef0fab8e --- /dev/null +++ b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.ts @@ -0,0 +1,39 @@ +import { v2ListMcpServerToolsContract } from '@/lib/api/contracts/v2/mcp-servers' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { mcpServerOperations } from '@/lib/mcp/application/operations' +import { discoverMcpServerToolsUseCase } from '@/lib/mcp/application/use-cases' +import { v2McpToolDiscoveryErrorPolicy } from '@/app/api/v2/mcp-servers/utils' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * GET /api/v2/mcp-servers/[id]/tools — List the tools a registered MCP server exposes. + * + * The path segment is static, so it can never shadow a server id: ids are minted + * as `mcp-` from the workspace and endpoint URL, and the registration + * contract requires a URL. + * + * Discovery is not a safe read: it opens a live connection to the registered + * endpoint and records the outcome on the server row. Next aliases `HEAD` onto + * `GET`, and RFC 9110 §9.2.1 defines `HEAD` as safe, so this route declares + * itself not head-safe — a `HEAD` is authenticated and rate-limited, then + * answered bodiless without connecting or writing. Without that, an uptime + * monitor or link checker walking the documented URL list would drive outbound + * third-party traffic and mutate rows on every probe. + */ +export const GET = defineV2JsonRoute({ + contract: v2ListMcpServerToolsContract, + operation: mcpServerOperations.discoverTools, + auth: v2ApiKeyAuth, + headSafe: false, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2McpToolDiscoveryErrorPolicy, + mapInput: ({ params, query }) => ({ + workspaceId: query.workspaceId, + serverId: params.id, + refresh: query.refresh, + }), + useCase: discoverMcpServerToolsUseCase, + present: ({ tools }) => ({ data: tools, nextCursor: null }), +}) diff --git a/apps/sim/app/api/v2/mcp-servers/route.test.ts b/apps/sim/app/api/v2/mcp-servers/route.test.ts index cf158e4cec2..027ae7272ed 100644 --- a/apps/sim/app/api/v2/mcp-servers/route.test.ts +++ b/apps/sim/app/api/v2/mcp-servers/route.test.ts @@ -2,40 +2,27 @@ * @vitest-environment node */ import type { mcpServers } from '@sim/db/schema' +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - list: vi.fn(), - create: vi.fn(), - capture: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), +const mocks = vi.hoisted(() => ({ + list: vi.fn(), + create: vi.fn(), + capture: vi.fn(), })) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -44,7 +31,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) vi.mock('@/lib/mcp/application/use-cases', () => ({ listMcpServersUseCase: { operation: { id: 'mcp_servers.list' }, execute: mocks.list }, @@ -59,17 +45,10 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, - limit: 100, - remaining: 99, - resetAt: new Date('2026-01-01T00:00:00Z'), - retryAfterMs: 0, -} const server = { id: 'mcp-server-1', workspaceId: WORKSPACE_ID, @@ -112,11 +91,16 @@ function request(method: 'GET' | 'POST', url: string, body?: unknown) { describe('/api/v2/mcp-servers', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ servers: [server] }) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.list.mockResolvedValue({ + servers: [server], + nextCursorKeys: null, + sortBy: 'createdAt', + sortOrder: 'desc', + }) mocks.create.mockResolvedValue({ server, updated: false }) }) @@ -134,11 +118,86 @@ describe('/api/v2/mcp-servers', () => { search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: 50, + cursor: undefined, + cursorKeys: undefined, }, request: expect.anything(), }) }) + it('bounds the server list by the requested limit', async () => { + await GET(request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=2`)) + + expect(mocks.list).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ limit: 2 }) }) + ) + }) + + it('mints a resumable cursor and replays it against the same sort', async () => { + mocks.list.mockResolvedValueOnce({ + servers: [server], + nextCursorKeys: [server.createdAt.toISOString(), server.id], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const first = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1`) + ) + const { nextCursor } = await first.json() + + expect(nextCursor).toEqual(expect.any(String)) + + const second = await GET( + request( + 'GET', + `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(second.status).toBe(200) + expect(mocks.list).toHaveBeenLastCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + cursorKeys: [server.createdAt.toISOString(), server.id], + }), + }) + ) + }) + + it('rejects a cursor minted under a different sort', async () => { + mocks.list.mockResolvedValueOnce({ + servers: [server], + nextCursorKeys: [server.createdAt.toISOString(), server.id], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const first = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1`) + ) + const { nextCursor } = await first.json() + + const response = await GET( + request( + 'GET', + `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1&sortBy=name&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(response.status).toBe(400) + }) + + it('rejects a fractional limit rather than paging on a fractional LIMIT', async () => { + const response = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1.5`) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + it('strictly creates an MCP server with the v2 source and status', async () => { const response = await POST( request('POST', '/api/v2/mcp-servers', { @@ -163,9 +222,10 @@ describe('/api/v2/mcp-servers', () => { }) it('keeps product analytics surface-specific for personal API keys', async () => { - mocks.authenticate.mockResolvedValueOnce({ + v2RouteMocks.authenticate.mockResolvedValueOnce({ ...AUTH, principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-personal' }, + rateLimitSubjectIds: ['api-key:key-personal', 'user:user-1'], keyType: 'personal', }) @@ -187,11 +247,12 @@ describe('/api/v2/mcp-servers', () => { }) it('authenticates before parsing create input', async () => { - mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) const response = await POST(request('POST', '/api/v2/mcp-servers', {})) expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.create).not.toHaveBeenCalled() }) }) diff --git a/apps/sim/app/api/v2/mcp-servers/route.ts b/apps/sim/app/api/v2/mcp-servers/route.ts index 5d31b41f9b9..53949d93738 100644 --- a/apps/sim/app/api/v2/mcp-servers/route.ts +++ b/apps/sim/app/api/v2/mcp-servers/route.ts @@ -11,6 +11,7 @@ import { import { mcpServerOperations } from '@/lib/mcp/application/operations' import { createMcpServerUseCase, listMcpServersUseCase } from '@/lib/mcp/application/use-cases' import { captureServerEvent } from '@/lib/posthog/server' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' import { toV2McpServer } from '@/app/api/v2/mcp-servers/utils' export const dynamic = 'force-dynamic' @@ -23,9 +24,21 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + limit: query.limit, + cursorKeys: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + }), useCase: listMcpServersUseCase, - present: ({ servers }) => ({ data: servers.map(toV2McpServer), nextCursor: null }), + present: ({ servers, nextCursorKeys, sortBy, sortOrder }) => ({ + data: servers.map(toV2McpServer), + nextCursor: nextCursorKeys + ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextCursorKeys) + : null, + }), }) /** POST /api/v2/mcp-servers — Register a new MCP server. */ diff --git a/apps/sim/app/api/v2/mcp-servers/utils.ts b/apps/sim/app/api/v2/mcp-servers/utils.ts index 7d186ff8770..a46f344ea57 100644 --- a/apps/sim/app/api/v2/mcp-servers/utils.ts +++ b/apps/sim/app/api/v2/mcp-servers/utils.ts @@ -1,7 +1,12 @@ -import type { NextResponse } from 'next/server' +import { UnauthorizedError } from '@modelcontextprotocol/sdk/client/auth.js' +import { StreamableHTTPError } from '@modelcontextprotocol/sdk/client/streamableHttp.js' +import { McpError as McpSdkError } from '@modelcontextprotocol/sdk/types.js' import { type V2McpServer, v2McpServerSchema } from '@/lib/api/contracts/v2/mcp-servers' +import { createV2ResourceConcealmentPolicy, type V2ErrorPolicy } from '@/lib/api/server/routes' +import { isTimeoutError } from '@/lib/core/execution-limits' import { projectMcpHeaders } from '@/lib/mcp/projection' import type { McpServerRow } from '@/lib/mcp/queries' +import { McpConnectionError, McpOauthAuthorizationRequiredError } from '@/lib/mcp/types' import { v2Error } from '@/app/api/v2/lib/response' /** @@ -26,25 +31,85 @@ export function toV2McpServer(row: McpServerRow): V2McpServer { }) } +export const mcpServerResourceErrorPolicy = createV2ResourceConcealmentPolicy({ + notFoundMessage: 'MCP server not found', +}) + /** - * Renders an MCP orchestration failure in the v2 error envelope. + * `error.details.code` on the 409 a stale MCP OAuth grant produces. * - * `forbidden` is the domain-allowlist / SSRF rejection and keeps its 403. - * `bad_gateway` is a DNS failure on the caller-supplied hostname — the caller's - * input is at fault, so it surfaces as a 400 rather than implying a Sim outage. + * 409 carries more than one cause across the v2 surface, so the discriminator is + * what lets a client branch without matching on prose. It is published in the + * operation description. */ -export function v2McpOrchestrationError( - errorCode: string | undefined, - message: string -): NextResponse { - switch (errorCode) { - case 'not_found': - return v2Error('NOT_FOUND', 'MCP server not found') - case 'forbidden': - return v2Error('FORBIDDEN', message) - case 'bad_gateway': - return v2Error('BAD_REQUEST', message) - default: - return v2Error('INTERNAL_ERROR', 'Internal server error') +export const MCP_SERVER_REAUTHORIZATION_REQUIRED = 'MCP_SERVER_REAUTHORIZATION_REQUIRED' + +/** + * Caller-safe wording for a third-party server that did not answer usefully. + * + * Every branch returns a constant, so an upstream message — which may quote a + * hostname, a token endpoint, or a stack — never reaches the caller. + */ +function unreachableServerMessage(error: unknown): string { + if (isTimeoutError(error)) return 'The MCP server took too long to respond' + if (error instanceof McpConnectionError && error.message.toLowerCase().includes('cooldown')) { + return 'The MCP server recently failed and is in cooldown' } + return 'The MCP server could not be reached' } + +/** + * Renders a tool-discovery failure. + * + * Discovery talks to a server the caller registered, so its failures are + * ordinary operating conditions rather than Sim faults: an unreachable, slow, or + * cooling-down server is a retryable 503 (`v2Error` stamps it with + * `Retry-After`), and a server whose stored OAuth grant no longer works is a 409 + * — the registration exists but its grant no longer does, which is a state + * conflict a human resolves by reauthorizing. Answering all of those with a bare + * 500 would make the endpoint that completes MCP onboarding indistinguishable + * from a Sim outage. + * + * The reauthorization case deliberately does **not** reuse 401. On this surface + * 401 means exactly one thing — the Sim API key is missing or invalid — and the + * published response description says so; a client that reacted to it by + * rotating or refreshing its Sim key would loop forever without touching the + * actual problem. It is also not a 403: the caller's rights on the Sim resource + * are fine. + * + * Classification is a typed dispatch over the MCP error families rather than + * `categorizeError`'s substring fallback. That fallback reaches 400 on any + * message containing `invalid`, which misattributed two different faults to the + * caller: an upstream JSON-RPC `Invalid params`, and — because the builder + * `.parse`s the response on the way out — a Sim-side response-schema defect, + * whose `ZodError` message carries `invalid_type`. The second is the worse of + * the two: answering it here suppressed the builder's 500 and its + * unhandled-error logging on the one v2 endpoint whose payload shape is authored + * by a third party. Anything unrecognised now returns `null` and keeps that + * generic 500. + */ +export const v2McpToolDiscoveryErrorPolicy = { + render(error) { + const orchestrated = mcpServerResourceErrorPolicy.render(error) + if (orchestrated) return orchestrated + + if (error instanceof McpOauthAuthorizationRequiredError || error instanceof UnauthorizedError) { + return v2Error( + 'CONFLICT', + 'The MCP server must be reauthorized in Sim before its tools can be listed', + { details: { code: MCP_SERVER_REAUTHORIZATION_REQUIRED } } + ) + } + + if ( + isTimeoutError(error) || + error instanceof McpConnectionError || + error instanceof McpSdkError || + error instanceof StreamableHTTPError + ) { + return v2Error('SERVICE_UNAVAILABLE', unreachableServerMessage(error)) + } + + return null + }, +} satisfies V2ErrorPolicy diff --git a/apps/sim/app/api/v2/secrets/route.test.ts b/apps/sim/app/api/v2/secrets/route.test.ts index 5c173b4d883..47147b25f91 100644 --- a/apps/sim/app/api/v2/secrets/route.test.ts +++ b/apps/sim/app/api/v2/secrets/route.test.ts @@ -46,6 +46,7 @@ vi.mock('@/lib/secrets/application/use-cases', () => ({ listSecretsUseCase: { operation: { id: 'secrets.list' }, execute: mocks.list }, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' import { GET } from '@/app/api/v2/secrets/route' const WORKSPACE_ID = 'workspace-1' @@ -88,7 +89,13 @@ describe('GET /api/v2/secrets', () => { mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ secrets: [secret], userId: 'user-1' }) + mocks.list.mockResolvedValue({ + secrets: [secret], + userId: 'user-1', + nextCursorKeys: null, + sortBy: 'name', + sortOrder: 'asc', + }) }) it('lists secret metadata without exposing values', async () => { @@ -121,6 +128,9 @@ describe('GET /api/v2/secrets', () => { search: undefined, sortBy: 'name', sortOrder: 'asc', + limit: V2_DEFAULT_PAGE_SIZE, + cursor: undefined, + cursorKeys: undefined, }, request: expect.anything(), }) diff --git a/apps/sim/app/api/v2/secrets/route.ts b/apps/sim/app/api/v2/secrets/route.ts index 62a1685dc88..c0b64d7f338 100644 --- a/apps/sim/app/api/v2/secrets/route.ts +++ b/apps/sim/app/api/v2/secrets/route.ts @@ -7,6 +7,7 @@ import { } from '@/lib/api/server/routes' import { secretOperations } from '@/lib/secrets/application/operations' import { listSecretsUseCase } from '@/lib/secrets/application/use-cases' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' import { toV2Secret } from '@/app/api/v2/secrets/utils' export const dynamic = 'force-dynamic' @@ -19,10 +20,15 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + ...query, + cursorKeys: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + }), useCase: listSecretsUseCase, - present: ({ secrets, userId }) => ({ + present: ({ secrets, userId, nextCursorKeys, sortBy, sortOrder }) => ({ data: secrets.map((secret) => toV2Secret(secret, userId)), - nextCursor: null, + nextCursor: nextCursorKeys + ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextCursorKeys) + : null, }), }) diff --git a/apps/sim/app/api/v2/skills/route.test.ts b/apps/sim/app/api/v2/skills/route.test.ts index 6d7770e05a0..6324aee87dc 100644 --- a/apps/sim/app/api/v2/skills/route.test.ts +++ b/apps/sim/app/api/v2/skills/route.test.ts @@ -53,6 +53,17 @@ vi.mock('@/lib/skills/application/use-cases', () => ({ import { GET, POST } from '@/app/api/v2/skills/route' const WORKSPACE_ID = 'workspace-1' + +/** + * The scope stamp the route mints for the default query. Written out rather + * than imported so the test pins the wire format a shipped cursor carries. + */ +const SCOPE = ({ + search = '', + sortBy = 'createdAt', + sortOrder = 'desc', +}: Record = {}) => + `search=${search}&sortBy=${sortBy}&sortOrder=${sortOrder}&workspaceId=${WORKSPACE_ID}` const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_ID, keyId: 'key-1' } const AUTH = { principal: PRINCIPAL, @@ -79,7 +90,7 @@ const skill = { updatedAt: new Date('2026-01-02T00:00:00Z'), } -function request(method: 'GET' | 'POST', url: string, body?: unknown) { +function request(method: 'GET' | 'POST' | 'HEAD', url: string, body?: unknown) { return new NextRequest(`http://localhost:3000${url}`, { method, headers: { @@ -97,7 +108,7 @@ describe('/api/v2/skills', () => { mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ skills: [skill] }) + mocks.list.mockResolvedValue({ skills: [skill], hasMore: false, offset: 0, limit: 50 }) mocks.create.mockResolvedValue({ skill }) }) @@ -105,7 +116,9 @@ describe('/api/v2/skills', () => { const response = await GET(request('GET', `/api/v2/skills?workspaceId=${WORKSPACE_ID}`)) expect(response.status).toBe(200) - expect((await response.json()).data[0]).not.toHaveProperty('content') + const body = await response.json() + expect(body.data[0]).not.toHaveProperty('content') + expect(body.nextCursor).toBeNull() expect(mocks.list).toHaveBeenCalledWith({ principal: PRINCIPAL, input: { @@ -113,11 +126,79 @@ describe('/api/v2/skills', () => { search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: 50, + cursor: undefined, + offset: 0, + cursorScope: SCOPE(), }, request: expect.anything(), }) }) + it('resumes from the offset cursor and mints the next one while pages remain', async () => { + mocks.list.mockResolvedValueOnce({ + skills: [skill], + hasMore: true, + offset: 2, + limit: 2, + cursorScope: SCOPE(), + }) + const cursor = Buffer.from(JSON.stringify({ scope: SCOPE(), offset: 2 })).toString('base64') + + const response = await GET( + request( + 'GET', + `/api/v2/skills?workspaceId=${WORKSPACE_ID}&limit=2&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(200) + expect((await response.json()).nextCursor).toBe( + Buffer.from(JSON.stringify({ scope: SCOPE(), offset: 4 })).toString('base64') + ) + expect(mocks.list).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ limit: 2, offset: 2 }) }) + ) + }) + + /** + * An offset means nothing against a sequence it was not counted in, so a + * cursor minted under one sort must not silently resume under another. + */ + it('rejects a cursor replayed under a different sort', async () => { + const cursor = Buffer.from(JSON.stringify({ scope: SCOPE(), offset: 2 })).toString('base64') + + const response = await GET( + request( + 'GET', + `/api/v2/skills?workspaceId=${WORKSPACE_ID}&sortBy=name&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + + /** + * The guard itself is unit-tested in `definition.test.ts`; this proves the + * pairing end-to-end, on a real v2 read that used to reply 500 to a plain HEAD. + */ + it('serves HEAD through the GET handler instead of throwing', async () => { + const response = await GET(request('HEAD', `/api/v2/skills?workspaceId=${WORKSPACE_ID}`)) + + expect(response.status).toBe(200) + expect(mocks.list).toHaveBeenCalled() + }) + + it('rejects a malformed cursor rather than silently restarting at page one', async () => { + const response = await GET( + request('GET', `/api/v2/skills?workspaceId=${WORKSPACE_ID}&cursor=not-a-cursor`) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + it('creates a skill with the v2 source and status', async () => { const response = await POST( request('POST', '/api/v2/skills', { diff --git a/apps/sim/app/api/v2/skills/route.ts b/apps/sim/app/api/v2/skills/route.ts index b49c685fe90..e1356ef7ea0 100644 --- a/apps/sim/app/api/v2/skills/route.ts +++ b/apps/sim/app/api/v2/skills/route.ts @@ -8,8 +8,28 @@ import { import { captureServerEvent } from '@/lib/posthog/server' import { skillOperations } from '@/lib/skills/application/operations' import { createSkillUseCase, listSkillsUseCase } from '@/lib/skills/application/use-cases' +import { + decodeOffsetCursor, + encodeOffsetCursor, + offsetCursorScope, +} from '@/app/api/v2/lib/response' import { toV2Skill, toV2SkillSummary } from '@/app/api/v2/skills/utils' +/** The query state a skills offset cursor is only valid within. */ +function skillCursorScope(query: { + workspaceId: string + search?: string + sortBy: string + sortOrder: string +}): string { + return offsetCursorScope({ + workspaceId: query.workspaceId, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + }) +} + export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -20,9 +40,25 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => { + /** + * The offset counts positions in the merged, filtered, sorted sequence, so + * every param that changes that sequence is stamped into the cursor and + * re-checked here. `limit` is deliberately absent — it selects how much of + * the sequence to return, not what the sequence is. + */ + const scope = skillCursorScope(query) + return { + ...query, + offset: decodeOffsetCursor(query.cursor, scope), + cursorScope: scope, + } + }, useCase: listSkillsUseCase, - present: ({ skills }) => ({ data: skills.map(toV2SkillSummary), nextCursor: null }), + present: ({ skills, hasMore, offset, limit, cursorScope }) => ({ + data: skills.map(toV2SkillSummary), + nextCursor: hasMore ? encodeOffsetCursor(cursorScope, offset + limit) : null, + }), }) /** POST /api/v2/skills — Create a skill. */ diff --git a/apps/sim/app/api/v2/skills/utils.ts b/apps/sim/app/api/v2/skills/utils.ts index a1cc30ceb02..eb07440b91e 100644 --- a/apps/sim/app/api/v2/skills/utils.ts +++ b/apps/sim/app/api/v2/skills/utils.ts @@ -1,18 +1,16 @@ import type { skill } from '@sim/db/schema' -import type { NextResponse } from 'next/server' import type { V2Skill, V2SkillSummary } from '@/lib/api/contracts/v2/skills' -import type { SkillOrchestrationErrorCode } from '@/lib/skills/orchestration' import { isBuiltinSkillId } from '@/lib/workflows/skills/builtin-skills' -import { v2Error } from '@/app/api/v2/lib/response' - -/** - * Shared serialization + error mapping for the v2 skills surface. - */ +import type { SkillSummaryRow } from '@/lib/workflows/skills/operations' +/** Shared serialization for the v2 skills surface. */ type SkillRow = typeof skill.$inferSelect -/** List projection — no `content`; skill bodies are fetched per skill. */ -export function toV2SkillSummary(row: SkillRow): V2SkillSummary { +/** + * List projection — no `content`; skill bodies are fetched per skill. It takes + * the body-less row so the list query never has to load one. + */ +export function toV2SkillSummary(row: SkillSummaryRow): V2SkillSummary { return { id: row.id, name: row.name, @@ -27,22 +25,3 @@ export function toV2SkillSummary(row: SkillRow): V2SkillSummary { export function toV2Skill(row: SkillRow): V2Skill { return { ...toV2SkillSummary(row), content: row.content } } - -/** Renders a skill orchestration failure in the v2 error envelope. */ -export function v2SkillOrchestrationError( - errorCode: SkillOrchestrationErrorCode | undefined, - message: string -): NextResponse { - switch (errorCode) { - case 'validation': - return v2Error('BAD_REQUEST', message) - case 'forbidden': - return v2Error('FORBIDDEN', message) - case 'not_found': - return v2Error('NOT_FOUND', 'Skill not found') - case 'conflict': - return v2Error('CONFLICT', message) - default: - return v2Error('INTERNAL_ERROR', 'Internal server error') - } -} diff --git a/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts index 17d013606d6..4ed78156035 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), cancelRuns: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, })) @@ -49,16 +45,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} function call(body: unknown) { const request = new NextRequest('http://localhost/api/v2/tables/table-1/cancel-runs', { @@ -75,10 +65,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/cancel-runs', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.cancelRuns.mockResolvedValue({ table: { id: 'table-1' }, cancelled: 4 }) }) @@ -146,4 +136,13 @@ describe('POST /api/v2/tables/[tableId]/cancel-runs', () => { expect(contradictory.status).toBe(400) expect(mocks.cancelRuns).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID, scope: 'all' }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts index 4ca1f73bf67..5fedb51afac 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts @@ -2,31 +2,27 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), add: vi.fn(), update: vi.fn(), remove: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/columns', () => ({ addTableColumnUseCase: { operation: { id: 'tables.columns.add' }, execute: mocks.add }, updateTableColumnUseCase: { operation: { id: 'tables.columns.update' }, execute: mocks.update }, @@ -45,16 +41,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const table = { id: 'table-1', name: 'Contacts', @@ -77,10 +67,10 @@ function request(method: 'POST' | 'PATCH' | 'DELETE', body: unknown) { describe('/api/v2/tables/[tableId]/columns', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.add.mockResolvedValue({ table }) mocks.update.mockResolvedValue({ table, changed: false }) mocks.remove.mockResolvedValue({ table }) @@ -108,6 +98,49 @@ describe('/api/v2/tables/[tableId]/columns', () => { }) }) + it('forwards required on both the add and the update column write', async () => { + await POST( + request('POST', { + workspaceId: WORKSPACE_ID, + column: { name: 'Name', type: 'string', required: true }, + }), + context + ) + await PATCH( + request('PATCH', { + workspaceId: WORKSPACE_ID, + columnName: 'Name', + updates: { required: false }, + }), + context + ) + + expect(mocks.add).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + column: { name: 'Name', type: 'string', required: true }, + }), + }) + ) + expect(mocks.update).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ updates: { required: false } }) }) + ) + }) + + it('rejects an unrecognized key on the column delete body', async () => { + const response = await DELETE( + request('DELETE', { + workspaceId: WORKSPACE_ID, + columnName: 'Other', + columnNames: ['Other'], + }), + context + ) + + expect(response.status).toBe(400) + expect(mocks.remove).not.toHaveBeenCalled() + }) + it('maps typed application validation failures without inspecting messages', async () => { mocks.update.mockRejectedValueOnce(new OrchestrationError('validation', 'Invalid column')) @@ -133,4 +166,16 @@ describe('/api/v2/tables/[tableId]/columns', () => { expect(response.status).toBe(200) expect(mocks.remove).toHaveBeenCalledOnce() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await POST( + request('POST', { workspaceId: WORKSPACE_ID, column: { name: 'Name', type: 'string' } }), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts index 03030a64729..4440d408b74 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts @@ -12,7 +12,7 @@ import { updateTableColumnUseCase, } from '@/lib/table/application/columns' import { tableOperations } from '@/lib/table/application/operations' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' export const dynamic = 'force-dynamic' export const revalidate = 0 diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts index 83b77f98923..e9257648ab5 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), startRun: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, })) @@ -49,16 +45,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} function call(body: unknown) { const request = new NextRequest('http://localhost/api/v2/tables/table-1/columns/run', { @@ -75,10 +65,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/columns/run', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.startRun.mockResolvedValue({ table: { id: 'table-1' }, dispatchId: 'dispatch-1' }) }) @@ -143,4 +133,13 @@ describe('POST /api/v2/tables/[tableId]/columns/run', () => { expect(response.status).toBe(400) expect(mocks.startRun).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID, groupIds: ['group-1'] }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts index ad7a2d1bc2f..b2c700871c9 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts @@ -2,31 +2,27 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), create: vi.fn(), read: vi.fn(), download: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/tables/presenters', () => ({ presentV2TableExport: (tableExport: unknown) => ({ data: tableExport }), })) @@ -53,16 +49,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const tableExport = { id: 'export-1', tableId: 'table-1', @@ -79,10 +69,10 @@ const tableExport = { describe('v2 table exports', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) }) it('creates an export through the authorized use case', async () => { @@ -140,4 +130,18 @@ describe('v2 table exports', () => { request: downloadRequest, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + const request = new NextRequest('http://localhost:3000/api/v2/tables/table-1/exports', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ workspaceId: WORKSPACE_ID, format: 'csv' }), + }) + + const response = await POST(request, { params: Promise.resolve({ tableId: 'table-1' }) }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts index 35acd0b7739..94bfb5816c6 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), update: vi.fn(), remove: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/groups', () => ({ listTableGroupsUseCase: { operation: { id: 'tables.groups.list' }, execute: mocks.list }, createTableGroupUseCase: { operation: { id: 'tables.groups.create' }, execute: mocks.create }, @@ -47,16 +43,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const group = { id: 'group-1', workflowId: 'workflow-1', @@ -93,10 +83,10 @@ function writeRequest(method: 'POST' | 'PATCH' | 'DELETE', body: unknown) { describe('/api/v2/tables/[tableId]/groups', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.list.mockResolvedValue({ groups: [group] }) mocks.create.mockResolvedValue({ table, group }) mocks.update.mockResolvedValue({ table, group, changed: true, startAutoRun: false }) @@ -118,6 +108,20 @@ describe('/api/v2/tables/[tableId]/groups', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables/table-1/groups?workspaceId=${WORKSPACE_ID}` + ), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('defaults create autoRun off and delegates all execution initiation to the application layer', async () => { const req = writeRequest('POST', { workspaceId: WORKSPACE_ID, diff --git a/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts b/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts index 8910958f55a..b92dfb69d5f 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts @@ -13,7 +13,7 @@ import { updateTableGroupUseCase, } from '@/lib/table/application/groups' import { tableOperations } from '@/lib/table/application/operations' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' export const dynamic = 'force-dynamic' export const revalidate = 0 diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/count/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.test.ts new file mode 100644 index 00000000000..576b3925505 --- /dev/null +++ b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.test.ts @@ -0,0 +1,181 @@ +/** + * @vitest-environment node + */ + +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { + class MockTableRowsValidationError extends Error { + constructor( + message: string, + readonly details?: unknown + ) { + super(message) + } + } + return { + mocks: { + queryRows: vi.fn(), + }, + MockTableRowsValidationError, + } +}) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) +vi.mock('@/lib/table/application/rows', () => ({ + TableRowsValidationError: MockTableRowsValidationError, + queryTableRows: { operation: { id: 'tables.rows.query' }, execute: mocks.queryRows }, +})) + +import { POST } from '@/app/api/v2/tables/[tableId]/query/count/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { + kind: 'workspace_api_key' as const, + workspaceId: WORKSPACE_ID, + keyId: 'key-1', +} +const AUTH = { + principal: PRINCIPAL, + rolloutUserId: 'owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], + rateLimitSubscription: null, + keyType: 'workspace' as const, +} +const TABLE = { + id: 'table-1', + workspaceId: WORKSPACE_ID, + schema: { columns: [{ id: 'column-name', name: 'name', type: 'string' as const }] }, +} +const ROW = { + id: 'row-1', + data: { 'column-name': 'Ada' }, + createdAt: new Date('2026-01-01T00:00:00Z'), + updatedAt: new Date('2026-01-02T00:00:00Z'), +} + +function call(body: unknown) { + const request = new NextRequest('http://localhost/api/v2/tables/table-1/query/count', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) + return { + request, + response: POST(request, { params: Promise.resolve({ tableId: 'table-1' }) }), + } +} + +describe('POST /api/v2/tables/[tableId]/query/count', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.queryRows.mockResolvedValue({ + table: TABLE, + rows: [ROW], + rowCount: 1, + totalCount: 4321, + nextCursor: 'cursor-1', + }) + }) + + it('counts the predicate matches across the whole table, not the page', async () => { + const predicate = { all: [{ field: 'name', op: 'eq', value: 'Ada' }] } + const invocation = call({ workspaceId: WORKSPACE_ID, predicate }) + const response = await invocation.response + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ data: { totalCount: 4321 } }) + expect(mocks.queryRows).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: { + tableId: 'table-1', + assertedWorkspaceId: WORKSPACE_ID, + predicate, + limit: 1, + includeTotal: true, + }, + request: invocation.request, + }) + }) + + it('counts the whole table when no predicate is sent', async () => { + mocks.queryRows.mockResolvedValue({ + table: TABLE, + rows: [], + rowCount: 0, + totalCount: 0, + nextCursor: null, + }) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ data: { totalCount: 0 } }) + expect(mocks.queryRows).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ predicate: undefined }) }) + ) + }) + + it('rejects the paging controls a count has no use for', async () => { + const response = await call({ workspaceId: WORKSPACE_ID, limit: 10, cursor: 'x' }).response + + expect(response.status).toBe(400) + expect(mocks.queryRows).not.toHaveBeenCalled() + }) + + it('keeps a malformed predicate as a structured 400', async () => { + mocks.queryRows.mockRejectedValue( + new MockTableRowsValidationError('Unknown column "nope"', { code: 'INVALID_PREDICATE' }) + ) + + const response = await call({ + workspaceId: WORKSPACE_ID, + predicate: { all: [{ field: 'nope', op: 'eq', value: 1 }] }, + }).response + + expect(response.status).toBe(400) + expect((await response.json()).error.details).toEqual({ code: 'INVALID_PREDICATE' }) + }) + + it('never presents a fabricated zero when no total was computed', async () => { + mocks.queryRows.mockResolvedValue({ + table: TABLE, + rows: [ROW], + rowCount: 1, + totalCount: null, + nextCursor: null, + }) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(500) + expect(await response.json()).toEqual({ + error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, + }) + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) +}) diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/count/route.ts b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.ts new file mode 100644 index 00000000000..f71712224dc --- /dev/null +++ b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.ts @@ -0,0 +1,45 @@ +import { TABLE_QUERY_MAX_BODY_BYTES } from '@/lib/api/contracts/tables' +import { v2QueryRowsCountContract } from '@/lib/api/contracts/v2/tables' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2TableRowsErrorPolicy } from '@/lib/table/api/row-route-policies' +import { tableOperations } from '@/lib/table/application/operations' +import { queryTableRows } from '@/lib/table/application/rows' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * Counts the rows a predicate matches. + * + * The same `queryTableRows` read the paged endpoints use, asked for its total + * instead of its page: `includeTotal` runs a COUNT over the full predicate view + * (not the page's keyset window), and `limit: 1` keeps the row drain that runs + * alongside it to a single row rather than a full default page. + * + * `totalCount` is `number | null` on the use-case result because callers may ask + * for a page without a total. This route always asks for one, so a null here is + * a broken invariant rather than a reachable outcome — it fails loudly instead + * of being coerced into a plausible-looking zero. + */ +export const POST = defineV2JsonRoute({ + contract: v2QueryRowsCountContract, + operation: tableOperations.queryRows, + auth: v2ApiKeyAuth, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2TableRowsErrorPolicy, + parseOptions: { maxBodyBytes: TABLE_QUERY_MAX_BODY_BYTES }, + mapInput: ({ params, body }) => ({ + tableId: params.tableId, + assertedWorkspaceId: body.workspaceId, + predicate: body.predicate, + limit: 1, + includeTotal: true, + }), + useCase: queryTableRows, + present: ({ totalCount }) => { + if (totalCount === null) { + throw new Error('Table row count requested with includeTotal but no total was computed') + } + return { data: { totalCount } } + }, +}) diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts index 88ab015bf3c..d32782c9bb2 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -16,28 +25,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { } return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), queryRows: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, queryTableRows: { operation: { id: 'tables.rows.query' }, execute: mocks.queryRows }, @@ -54,16 +50,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -91,10 +81,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/query', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.queryRows.mockResolvedValue({ table: TABLE, rows: [ROW], nextCursor: null }) }) @@ -138,12 +128,22 @@ describe('POST /api/v2/tables/[tableId]/query', () => { ) }) + it('rejects a v1-shaped filter key instead of answering with an unfiltered page', async () => { + const response = await call({ + workspaceId: WORKSPACE_ID, + filter: { name: { $eq: 'Ada' } }, + }).response + + expect(response.status).toBe(400) + expect(mocks.queryRows).not.toHaveBeenCalled() + }) + it('rejects an invalid page limit after admission and before delegation', async () => { const response = await call({ workspaceId: WORKSPACE_ID, limit: 5000 }).response expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalledOnce() - expect(mocks.operationRate).toHaveBeenCalledOnce() + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(AUTH.rateLimitSubjectIds.length) expect(mocks.queryRows).not.toHaveBeenCalled() }) @@ -165,4 +165,13 @@ describe('POST /api/v2/tables/[tableId]/query', () => { expect(response.status).toBe(413) expect(mocks.queryRows).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/route.test.ts index da221f15235..5060a47a762 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/route.test.ts @@ -2,14 +2,19 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), read: vi.fn(), update: vi.fn(), remove: vi.fn(), @@ -18,18 +23,9 @@ const mocks = vi.hoisted(() => ({ getMaxRowsPerTable: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) vi.mock('@/lib/table/application/tables', () => ({ readTableUseCase: { operation: { id: 'tables.read' }, execute: mocks.read }, @@ -57,16 +53,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const table = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -107,10 +97,10 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/tables/[tableId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['owner-1', 'owner@example.com']])) mocks.getMaxRowsPerTable.mockResolvedValue(5000) mocks.read.mockResolvedValue({ table, folderPath: '/' }) @@ -147,6 +137,15 @@ describe('/api/v2/tables/[tableId]', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET'), context) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('preserves a successful no-op PATCH response', async () => { const response = await PATCH( request('PATCH', { workspaceId: WORKSPACE_ID, name: 'Contacts' }), diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts index c0e1306fe6a..31671eb4fa4 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), startRun: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, })) @@ -50,16 +46,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} function call(body: unknown) { const request = new NextRequest( @@ -81,10 +71,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.startRun.mockResolvedValue({ table: { id: 'table-1' }, dispatchId: 'dispatch-1' }) }) @@ -116,6 +106,15 @@ describe('POST /api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]', () = expect(await response.json()).toEqual({ data: { dispatchId: null } }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('rejects a missing workspace before delegation', async () => { const response = await call({}).response diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts index 8a2395c073a..276751f0723 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,10 +18,6 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), readRow: vi.fn(), updateRow: vi.fn(), deleteRow: vi.fn(), @@ -21,18 +26,9 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, readTableRow: { operation: { id: 'tables.rows.read' }, execute: mocks.readRow }, @@ -53,16 +49,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -93,10 +83,10 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/tables/[tableId]/rows/[rowId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readRow.mockResolvedValue({ table: TABLE, row: ROW }) mocks.updateRow.mockResolvedValue({ table: TABLE, row: ROW, changed: true }) mocks.deleteRow.mockResolvedValue({ table: TABLE, deletedRowId: ROW.id }) @@ -120,6 +110,15 @@ describe('/api/v2/tables/[tableId]/rows/[rowId]', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET'), CONTEXT) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('updates through the shared use case with the exact patch', async () => { const req = request('PATCH', { workspaceId: WORKSPACE_ID, data: { name: 'Ada' } }) const response = await PATCH(req, CONTEXT) diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts index e86f657c272..293935428a1 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,33 +18,21 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), findRows: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, findTableRows: { operation: { id: 'tables.rows.find' }, execute: mocks.findRows }, })) +import { v2Error } from '@/app/api/v2/lib/response' import { POST } from '@/app/api/v2/tables/[tableId]/rows/find/route' const WORKSPACE_ID = 'workspace-1' @@ -47,16 +44,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -78,10 +69,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/rows/find', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.findRows.mockResolvedValue({ table: TABLE, matches: [{ ordinal: 3, rowId: 'row-1', column: 'column-name' }], @@ -119,17 +110,25 @@ describe('POST /api/v2/tables/[tableId]/rows/find', () => { const response = await call({ workspaceId: WORKSPACE_ID, q: '' }).response expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() expect(mocks.findRows).not.toHaveBeenCalled() }) it('stops at the rollout gate before the shared use case', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mocks.gate.mockResolvedValue(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValue(v2Error('NOT_FOUND', 'Not found')) const response = await call({ workspaceId: WORKSPACE_ID, q: 'ada' }).response expect(response.status).toBe(404) expect(mocks.findRows).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID, q: 'ada' }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts index 9c4eaa74426..3d7ef8d5271 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,10 +18,6 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), listRows: vi.fn(), createRows: vi.fn(), updateRows: vi.fn(), @@ -22,18 +27,9 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, listTableRows: { operation: { id: 'tables.rows.list' }, execute: mocks.listRows }, @@ -53,16 +49,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -90,10 +80,10 @@ function request(method: 'GET' | 'POST' | 'PATCH' | 'DELETE', body?: unknown, qu describe('/api/v2/tables/[tableId]/rows', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listRows.mockResolvedValue({ table: TABLE, rows: [ROW], nextCursor: null }) mocks.createRows.mockResolvedValue({ kind: 'single', table: TABLE, row: ROW }) mocks.updateRows.mockResolvedValue({ @@ -139,6 +129,18 @@ describe('/api/v2/tables/[tableId]/rows', () => { expect((await response.json()).nextCursor).toBe('next-native-cursor') }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + request('GET', undefined, `?workspaceId=${WORKSPACE_ID}&limit=25`), + CONTEXT + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('delegates single and batch creation through one semantic use case', async () => { const single = request('POST', { workspaceId: WORKSPACE_ID, data: { name: 'Ada' } }) expect((await (await POST(single, CONTEXT)).json()).data.id).toBe('row-1') diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts index ddcd5106649..96e633d0834 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), upsertRow: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, upsertTableRow: { operation: { id: 'tables.rows.upsert' }, execute: mocks.upsertRow }, @@ -47,16 +43,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -72,10 +62,10 @@ const ROW = { describe('POST /api/v2/tables/[tableId]/rows/upsert', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.upsertRow.mockResolvedValue({ table: TABLE, row: ROW, operation: 'update' }) }) @@ -117,6 +107,26 @@ describe('POST /api/v2/tables/[tableId]/rows/upsert', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const request = new NextRequest('http://localhost/api/v2/tables/table-1/rows/upsert', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + data: { email: 'ada@example.com' }, + conflictTarget: 'email', + }), + }) + const response = await POST(request, { + params: Promise.resolve({ tableId: 'table-1' }), + }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('rejects an empty conflict target before delegation', async () => { const request = new NextRequest('http://localhost/api/v2/tables/table-1/rows/upsert', { method: 'POST', diff --git a/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts index fa253c093f4..9fdecfd9985 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), read: vi.fn(), update: vi.fn(), remove: vi.fn(), email: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/views', () => ({ readTableViewUseCase: { operation: { id: 'tables.views.read' }, execute: mocks.read }, updateTableViewUseCase: { operation: { id: 'tables.views.update' }, execute: mocks.update }, @@ -46,16 +42,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const view = { id: 'view-1', tableId: 'table-1', @@ -82,10 +72,10 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/tables/[tableId]/views/[viewId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.read.mockResolvedValue({ view }) mocks.update.mockResolvedValue({ view, changed: false }) mocks.remove.mockResolvedValue({ viewId: 'view-1' }) @@ -122,4 +112,13 @@ describe('/api/v2/tables/[tableId]/views/[viewId]', () => { expect(await response.json()).toEqual({ data: { id: 'view-1', deleted: true } }) expect(mocks.remove).toHaveBeenCalledOnce() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET'), context) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts index 3f2e6a20288..e89fe4a4fa2 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), emails: vi.fn(), email: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/views', () => ({ listTableViewsUseCase: { operation: { id: 'tables.views.list' }, execute: mocks.list }, createTableViewUseCase: { operation: { id: 'tables.views.create' }, execute: mocks.create }, @@ -49,16 +45,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const view = { id: 'view-1', tableId: 'table-1', @@ -74,10 +64,10 @@ const context = { params: Promise.resolve({ tableId: 'table-1' }) } describe('/api/v2/tables/[tableId]/views', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.list.mockResolvedValue({ views: [view] }) mocks.create.mockResolvedValue({ view }) mocks.emails.mockResolvedValue(new Map([['user-1', 'user@example.com']])) @@ -129,4 +119,18 @@ describe('/api/v2/tables/[tableId]/views', () => { request: req, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables/table-1/views?workspaceId=${WORKSPACE_ID}` + ), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/folders/route.test.ts b/apps/sim/app/api/v2/tables/folders/route.test.ts index 866438a058c..c0f2f244068 100644 --- a/apps/sim/app/api/v2/tables/folders/route.test.ts +++ b/apps/sim/app/api/v2/tables/folders/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), update: vi.fn(), remove: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/folders', () => ({ listTableFoldersUseCase: { operation: { id: 'tables.folders.list' }, execute: mocks.list }, createTableFolderUseCase: { operation: { id: 'tables.folders.create' }, execute: mocks.create }, @@ -46,16 +42,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const folder = { id: 'folder-1', workspaceId: WORKSPACE_ID, @@ -85,10 +75,10 @@ function request(method: 'GET' | 'POST' | 'PATCH' | 'DELETE', url: string, body? describe('/api/v2/tables/folders', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.list.mockResolvedValue({ folders: [folder], index }) mocks.create.mockResolvedValue({ folder, index, path: '/Reports' }) mocks.update.mockResolvedValue({ @@ -155,4 +145,13 @@ describe('/api/v2/tables/folders', () => { }, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET', `/api/v2/tables/folders?workspaceId=${WORKSPACE_ID}`)) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts b/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts index 6964d818d6b..e9e12f184a1 100644 --- a/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts +++ b/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts @@ -2,29 +2,25 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), complete: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/tables/presenters', () => ({ presentV2TableImport: (tableImport: unknown) => ({ data: tableImport }), })) @@ -46,24 +42,18 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} describe('POST /api/v2/tables/imports/[importId]/complete', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) }) it('delegates idempotent completion to the authorized import use case', async () => { @@ -101,4 +91,19 @@ describe('POST /api/v2/tables/imports/[importId]/complete', () => { request, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await POST( + new NextRequest( + `http://localhost:3000/api/v2/tables/imports/import-1/complete?workspaceId=${WORKSPACE_ID}`, + { method: 'POST', headers: { 'upload-token': 'signed-upload-token' } } + ), + { params: Promise.resolve({ importId: 'import-1' }) } + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/imports/route.test.ts b/apps/sim/app/api/v2/tables/imports/route.test.ts index fd1f7672193..44fe854fa3a 100644 --- a/apps/sim/app/api/v2/tables/imports/route.test.ts +++ b/apps/sim/app/api/v2/tables/imports/route.test.ts @@ -2,29 +2,25 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), create: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/tables/presenters', () => ({ presentV2CreateTableImport: (tableImport: unknown) => ({ data: tableImport }), })) @@ -43,25 +39,19 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const timestamp = '2026-01-01T00:00:00.000Z' describe('POST /api/v2/tables/imports', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) }) it.each([ @@ -145,8 +135,27 @@ describe('POST /api/v2/tables/imports', () => { ) expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalled() - expect(mocks.operationRate).toHaveBeenCalled() + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalled() expect(mocks.create).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await POST( + new NextRequest('http://localhost:3000/api/v2/tables/imports', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + source: { type: 'workspace_file', fileId: 'file-1' }, + target: { type: 'new', name: 'imported_data' }, + }), + }) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/route.test.ts b/apps/sim/app/api/v2/tables/route.test.ts index 831a3c69d9c..e27af471956 100644 --- a/apps/sim/app/api/v2/tables/route.test.ts +++ b/apps/sim/app/api/v2/tables/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), getUserEmailsByIds: vi.fn(), getMaxRowsPerTable: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/tables', () => ({ listTablesUseCase: { operation: { id: 'tables.list' }, execute: mocks.list }, createTableUseCase: { operation: { id: 'tables.create' }, execute: mocks.create }, @@ -51,16 +47,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const table = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -89,10 +79,10 @@ const table = { describe('/api/v2/tables', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['owner-1', 'owner@example.com']])) mocks.getMaxRowsPerTable.mockResolvedValue(5000) mocks.list.mockResolvedValue({ @@ -134,13 +124,24 @@ describe('/api/v2/tables', () => { const response = await GET(new NextRequest('http://localhost:3000/api/v2/tables')) expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalled() - expect(mocks.operationRate).toHaveBeenCalled() + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalled() expect(mocks.list).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25`) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('maps operation rate-limit infrastructure failures to service unavailable', async () => { - mocks.operationRate.mockRejectedValueOnce(new Error('rate store unavailable')) + v2RouteMocks.operationRate.mockRejectedValueOnce(new Error('rate store unavailable')) const response = await GET( new NextRequest(`http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25`) @@ -181,7 +182,7 @@ describe('/api/v2/tables', () => { }) }) - it('rejects required in a table column before calling the use case', async () => { + it('forwards required on a table column to the use case', async () => { const request = new NextRequest('http://localhost:3000/api/v2/tables', { method: 'POST', headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, @@ -193,6 +194,28 @@ describe('/api/v2/tables', () => { }) const response = await POST(request) + expect(response.status).toBe(201) + expect(mocks.create).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + schema: { columns: [{ name: 'Name', type: 'string', required: true }] }, + }), + }) + ) + }) + + it('rejects an unrecognized key in a table column before calling the use case', async () => { + const request = new NextRequest('http://localhost:3000/api/v2/tables', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + name: 'Contacts', + schema: { columns: [{ name: 'Name', type: 'string', requried: true }] }, + }), + }) + const response = await POST(request) + expect(response.status).toBe(400) expect(mocks.create).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/v2/tables/route.ts b/apps/sim/app/api/v2/tables/route.ts index b9515fdb91a..9099a1f51b9 100644 --- a/apps/sim/app/api/v2/tables/route.ts +++ b/apps/sim/app/api/v2/tables/route.ts @@ -1,11 +1,9 @@ import { v2CreateTableContract, v2ListTablesContract } from '@/lib/api/contracts/v2/tables' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { v2TableErrorPolicies } from '@/lib/table/api' import { tableOperations } from '@/lib/table/application/operations' import { createTableUseCase, listTablesUseCase } from '@/lib/table/application/tables' -import { cursorSortKey, decodeSortedCursor, encodeSortedCursor } from '@/app/api/v2/lib/response' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' import { toApiTable, toApiTables } from '@/app/api/v2/tables/utils' export const dynamic = 'force-dynamic' @@ -18,22 +16,15 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2TableErrorPolicies.default, - mapInput: ({ query }) => { - const sort = cursorSortKey(query.sortBy, query.sortOrder) - const decoded = decodeSortedCursor(query.cursor, sort) - if (decoded.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - return { - workspaceId: query.workspaceId, - folderPath: query.folderPath, - search: query.search, - sortBy: query.sortBy, - sortOrder: query.sortOrder, - limit: query.limit, - after: decoded.status === 'ok' ? decoded.keys : undefined, - } - }, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + folderPath: query.folderPath, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + limit: query.limit, + after: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + }), present: async ({ tables, nextKeys, sortBy, sortOrder }) => ({ data: await toApiTables(tables), nextCursor: nextKeys ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextKeys) : null, diff --git a/apps/sim/app/api/v2/tables/utils.ts b/apps/sim/app/api/v2/tables/utils.ts index 04d02c3264e..d262e015001 100644 --- a/apps/sim/app/api/v2/tables/utils.ts +++ b/apps/sim/app/api/v2/tables/utils.ts @@ -14,8 +14,9 @@ import { import { predicateToStorage } from '@/lib/table/select-values' import type { Filter, TableLockKind } from '@/lib/table/types' import type { TableView } from '@/lib/table/views/service' +import { normalizeColumn } from '@/lib/table/wire' import { getUserEmailsByIds, requireResolvedUserEmail } from '@/lib/users/queries' -import { CSV_IMPORT_PROXY_BODY_CAP_BYTES, normalizeColumn } from '@/app/api/table/utils' +import { CSV_IMPORT_PROXY_BODY_CAP_BYTES } from '@/app/api/table/utils' import { v2Error, v2ErrorForOrchestration } from '@/app/api/v2/lib/response' /** diff --git a/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts index 8e77d94b1dd..21e6ad41708 100644 --- a/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts @@ -30,6 +30,12 @@ import { workflowOperations } from '@/lib/workflows/application/operations' import { DELETE, POST } from '@/app/api/v2/workflows/[id]/deploy/route' describe('/api/v2/workflows/[id]/deploy route definitions', () => { + /** + * Both the malformed-body 400 and the oversized-body 413 are v2 builder + * defaults, so neither belongs on the route. The envelope they produce is + * asserted once against the builder in + * `lib/api/server/routes/v2-error-envelope.test.ts`. + */ it('keeps an omitted deploy body valid and binds the authorized deployment use case', async () => { expect(v2DeployWorkflowContract.body?.parse(undefined)).toEqual({}) expect(POST).toMatchObject({ @@ -46,15 +52,7 @@ describe('/api/v2/workflows/[id]/deploy route definitions', () => { }) ) - const invalidJsonResponse = Reflect.get( - Reflect.get(POST, 'parseOptions'), - 'invalidJsonResponse' - )() - expect(invalidJsonResponse.status).toBe(400) - expect(await invalidJsonResponse.json()).toEqual({ - error: { code: 'BAD_REQUEST', message: 'Request body must be valid JSON' }, - }) - + expect(Reflect.get(Reflect.get(POST, 'parseOptions'), 'invalidJsonResponse')).toBeUndefined() expect( Reflect.get(Reflect.get(POST, 'parseOptions'), 'payloadTooLargeResponse') ).toBeUndefined() diff --git a/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts b/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts index a72e3838f7f..93f0e6a8cdb 100644 --- a/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts @@ -8,7 +8,6 @@ import { captureServerEvent } from '@/lib/posthog/server' import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { deployWorkflow, undeployWorkflow } from '@/lib/workflows/application/deployments' import { workflowOperations } from '@/lib/workflows/application/operations' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const runtime = 'nodejs' @@ -22,7 +21,6 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, parseOptions: { optionalJsonBody: true, - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), }, mapInput: ({ params, body }) => ({ workflowId: params.id, diff --git a/apps/sim/app/api/v2/workflows/[id]/deployment/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/deployment/route.test.ts new file mode 100644 index 00000000000..54f7bc7664d --- /dev/null +++ b/apps/sim/app/api/v2/workflows/[id]/deployment/route.test.ts @@ -0,0 +1,207 @@ +/** + * @vitest-environment node + */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + resolvePermission: vi.fn(), + resolveWorkflowContext: vi.fn(), + getWorkflowDeploymentSummary: vi.fn(), + checkNeedsRedeployment: vi.fn(), +})) + +vi.mock('@sim/platform-authz/workspace', () => ({ + permissionSatisfies: (actual: string | null, required: string) => { + const rank = { read: 1, write: 2, admin: 3 } as const + return ( + actual !== null && rank[actual as keyof typeof rank] >= rank[required as keyof typeof rank] + ) + }, + resolveEffectiveWorkspacePermission: mocks.resolvePermission, +})) +vi.mock('@/lib/workflows/application/context', () => ({ + resolveActiveWorkflowApplicationContext: mocks.resolveWorkflowContext, +})) +vi.mock('@/lib/workflows/orchestration/deploy', () => ({ + getWorkflowDeploymentSummary: mocks.getWorkflowDeploymentSummary, + performActivateVersion: vi.fn(), + performFullDeploy: vi.fn(), + performFullUndeploy: vi.fn(), + performRevertToVersion: vi.fn(), +})) +vi.mock('@/lib/workflows/deployment-status', () => ({ + checkNeedsRedeployment: mocks.checkNeedsRedeployment, +})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +import { GET } from '@/app/api/v2/workflows/[id]/deployment/route' + +const auth = { + principal: { + kind: 'personal_api_key' as const, + userId: 'user-1', + keyId: 'personal-key-1', + }, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:personal-key-1', 'user:user-1'] as const, + rateLimitSubscription: null, + keyType: 'personal' as const, +} + +const activeDeployment = { + deploymentVersionId: 'depver-2', + version: 2, + deployedAt: '2026-08-01T00:00:00.000Z', +} + +const latestDeploymentAttempt = { + id: 'op-2', + deploymentVersionId: 'depver-2', + version: 2, + action: 'deploy' as const, + status: 'active' as const, + isCurrent: true, + readiness: { + webhooks: 'not_applicable' as const, + schedules: 'not_applicable' as const, + mcp: 'not_applicable' as const, + }, + requestedAt: '2026-08-01T00:00:00.000Z', + activatedAt: '2026-08-01T00:00:01.000Z', + error: null, +} + +/** + * `workflow.deployedAt` carries a stale timestamp from a deployment that was + * later undeployed — the presenter must never fall back to it. + */ +const workflowContext = { + workspaceId: 'workspace-1', + workspaceOrganizationId: null, + allowPersonalApiKeys: true, + billedAccountUserId: 'billing-owner-1', + workflowId: 'workflow-1', + workflow: { + id: 'workflow-1', + workspaceId: 'workspace-1', + deployedAt: new Date('2025-01-01T00:00:00.000Z'), + }, +} + +async function get() { + const request = new NextRequest('http://localhost/api/v2/workflows/workflow-1/deployment') + return GET(request, { params: Promise.resolve({ id: 'workflow-1' }) }) +} + +describe('GET /api/v2/workflows/[id]/deployment', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.resolvePermission.mockResolvedValue('read') + mocks.resolveWorkflowContext.mockResolvedValue(workflowContext) + mocks.getWorkflowDeploymentSummary.mockResolvedValue({ + activeDeployment, + latestDeploymentAttempt, + warnings: undefined, + }) + mocks.checkNeedsRedeployment.mockResolvedValue(true) + }) + + it('publishes draft-versus-live drift and the latest attempt after canonical authorization', async () => { + const response = await get() + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { + id: 'workflow-1', + isDeployed: true, + needsRedeployment: true, + deployedAt: '2026-08-01T00:00:00.000Z', + warnings: [], + activeDeployment, + latestDeploymentAttempt, + }, + }) + expect(mocks.resolveWorkflowContext).toHaveBeenCalledBefore(mocks.getWorkflowDeploymentSummary) + }) + + it('carries the failed attempt error payload when nothing is live', async () => { + mocks.getWorkflowDeploymentSummary.mockResolvedValue({ + activeDeployment: null, + latestDeploymentAttempt: { + ...latestDeploymentAttempt, + status: 'failed' as const, + activatedAt: null, + error: { + code: 'webhook_conflict', + message: 'Webhook path already in use', + retryable: false, + }, + }, + warnings: ['Deployment attempt failed'], + }) + + const response = await get() + const body = await response.json() + + expect(response.status).toBe(200) + expect(body.data.isDeployed).toBe(false) + expect(body.data.needsRedeployment).toBe(false) + expect(body.data.deployedAt).toBeNull() + expect(body.data.warnings).toEqual(['Deployment attempt failed']) + expect(body.data.latestDeploymentAttempt.error).toEqual({ + code: 'webhook_conflict', + message: 'Webhook path already in use', + retryable: false, + }) + expect(mocks.checkNeedsRedeployment).not.toHaveBeenCalled() + }) + + it('never reports a deploy time from the stale workflow column once nothing is live', async () => { + mocks.getWorkflowDeploymentSummary.mockResolvedValue({ + activeDeployment: null, + latestDeploymentAttempt: null, + warnings: undefined, + }) + + const response = await get() + const body = await response.json() + + expect(response.status).toBe(200) + expect(body.data.deployedAt).toBeNull() + }) + + it('conceals a workflow the caller cannot reach as 404', async () => { + mocks.resolvePermission.mockResolvedValue(null) + + const response = await get() + + expect(response.status).toBe(404) + expect((await response.json()).error.code).toBe('NOT_FOUND') + expect(mocks.getWorkflowDeploymentSummary).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await get() + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) +}) diff --git a/apps/sim/app/api/v2/workflows/[id]/deployment/route.ts b/apps/sim/app/api/v2/workflows/[id]/deployment/route.ts new file mode 100644 index 00000000000..52224636031 --- /dev/null +++ b/apps/sim/app/api/v2/workflows/[id]/deployment/route.ts @@ -0,0 +1,44 @@ +import { v2GetWorkflowDeploymentContract } from '@/lib/api/contracts/v2/workflows' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' +import { readWorkflowDeploymentStatus } from '@/lib/workflows/application/deployments' +import { workflowOperations } from '@/lib/workflows/application/operations' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * GET /api/v2/workflows/[id]/deployment — Read current deployment state. + * + * The deploy, undeploy, and rollback responses are the only other place this + * state is published, so a caller that lost one — or that polls from a + * different process — had no way to ask. `needsRedeployment` is exposed here + * only: it compares the draft against the live version, so it is meaningless on + * the response of the mutation that just made them equal. + * + * `deployedAt` comes from the active deployment version, which always carries + * one. The workflow's own `deployed_at` column is deliberately not used as a + * fallback: it retains the timestamp of a deployment that has since been + * undeployed, so reading it would report a deploy time alongside + * `isDeployed: false`. + */ +export const GET = defineV2JsonRoute({ + contract: v2GetWorkflowDeploymentContract, + auth: v2ApiKeyAuth, + operation: workflowOperations.read, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, + mapInput: ({ params }) => ({ workflowId: params.id }), + useCase: readWorkflowDeploymentStatus, + present: (result) => ({ + data: { + id: result.workflow.id, + isDeployed: result.isDeployed, + needsRedeployment: result.needsRedeployment, + deployedAt: result.activeDeployment?.deployedAt ?? null, + warnings: result.warnings ?? [], + activeDeployment: result.activeDeployment ?? null, + latestDeploymentAttempt: result.latestDeploymentAttempt ?? null, + }, + }), +}) diff --git a/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts index 9d5f5aeeb17..8d2abfda872 100644 --- a/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts @@ -536,6 +536,39 @@ describe('POST /api/v2/workflows/[id]/execute', () => { expect((await res.json()).error.code).toBe('RATE_LIMITED') }) + it('tells a client how long to wait when a dependency is briefly unavailable', async () => { + mockPreprocessExecution.mockResolvedValue({ + success: false, + error: { + message: 'Workflow execution identity is temporarily unavailable', + statusCode: 503, + }, + }) + + const res = await callExecute({ input: {} }) + + expect(res.status).toBe(503) + expect(Number(res.headers.get('Retry-After'))).toBeGreaterThan(0) + }) + + it('never advises a retry when an enqueue may already have started a run', async () => { + mockPreprocessExecution.mockResolvedValue({ + success: false, + error: { + message: 'Async execution queue acceptance could not be confirmed', + statusCode: 503, + code: 'ASYNC_ENQUEUE_AMBIGUOUS', + }, + }) + + const res = await callExecute({ input: {} }) + + expect(res.status).toBe(503) + // Retrying without X-Run-Id would start, and bill, a second run of the same workflow. + expect(res.headers.get('Retry-After')).toBeNull() + expect((await res.json()).error.details.code).toBe('ASYNC_ENQUEUE_AMBIGUOUS') + }) + it('runs the anonymous public path sync but refuses async', async () => { dbChainMockFns.limit.mockReset() dbChainMockFns.limit.mockResolvedValueOnce([ diff --git a/apps/sim/app/api/v2/workflows/[id]/execute/route.ts b/apps/sim/app/api/v2/workflows/[id]/execute/route.ts index b9fe5eab1e9..8e44e5f35f5 100644 --- a/apps/sim/app/api/v2/workflows/[id]/execute/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/execute/route.ts @@ -12,6 +12,7 @@ import { import { parseRequest } from '@/lib/api/server' import { admitOptionalV2Request, + V2_PARSE_DEFAULTS, V2RouteInfrastructureError, v2ApiKeyAuth, v2RateLimits, @@ -19,6 +20,7 @@ import { import type { V2ApiKeyPrincipal } from '@/lib/api/server/routes/v2-api-key-auth' import { tryAdmit } from '@/lib/core/admission/gate' import { ADMISSION_ERROR_DESCRIPTOR } from '@/lib/core/admission/transient-failure' +import type { ForbiddenDetailCode } from '@/lib/core/application' import { generateRequestId } from '@/lib/core/utils/request' import { getBaseUrl } from '@/lib/core/utils/urls' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' @@ -82,6 +84,8 @@ function serviceFailureResponse(failure: ExecuteWorkflowServiceFailure) { return v2Error(code, isRunIdConflict ? 'Run ID has already been used' : failure.message, { status: failure.statusCode, headers, + /** An unconfirmed enqueue may already have started a run — reconcile on `runId`, never retry blind. */ + omitRetryAfter: failure.code === 'ASYNC_ENQUEUE_AMBIGUOUS', details: detailCode || failure.executionId ? { @@ -185,6 +189,7 @@ export const POST = withRouteHandler( try { const parsed = await parseRequest(v2ExecuteWorkflowContract, req, context, { + ...V2_PARSE_DEFAULTS, maxBodyBytes: 10 * 1024 * 1024, validationErrorResponse: v2ValidationError, }) @@ -270,7 +275,9 @@ export const POST = withRouteHandler( return v2Error('NOT_FOUND', 'Workflow not found') } if (workflowAuthorization.status === 403) { - return v2Error('FORBIDDEN', 'Insufficient workspace permissions') + return v2Error('FORBIDDEN', 'Insufficient workspace permissions', { + details: { code: 'INSUFFICIENT_WORKSPACE_ROLE' satisfies ForbiddenDetailCode }, + }) } throw new Error( `Unexpected workflow authorization status: ${workflowAuthorization.status}` diff --git a/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts index 2bd018a3df9..e06621ad0cc 100644 --- a/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts @@ -25,6 +25,12 @@ import { workflowOperations } from '@/lib/workflows/application/operations' import { POST } from '@/app/api/v2/workflows/[id]/rollback/route' describe('/api/v2/workflows/[id]/rollback route definition', () => { + /** + * Both the malformed-body 400 and the oversized-body 413 are v2 builder + * defaults, so neither belongs on the route. The envelope they produce is + * asserted once against the builder in + * `lib/api/server/routes/v2-error-envelope.test.ts`. + */ it('keeps an omitted rollback body valid and delegates version selection to the use case', async () => { expect(v2RollbackWorkflowContract.body?.parse(undefined)).toEqual({}) expect(POST).toMatchObject({ @@ -41,15 +47,7 @@ describe('/api/v2/workflows/[id]/rollback route definition', () => { }) ) - const invalidJsonResponse = Reflect.get( - Reflect.get(POST, 'parseOptions'), - 'invalidJsonResponse' - )() - expect(invalidJsonResponse.status).toBe(400) - expect(await invalidJsonResponse.json()).toEqual({ - error: { code: 'BAD_REQUEST', message: 'Request body must be valid JSON' }, - }) - + expect(Reflect.get(Reflect.get(POST, 'parseOptions'), 'invalidJsonResponse')).toBeUndefined() expect( Reflect.get(Reflect.get(POST, 'parseOptions'), 'payloadTooLargeResponse') ).toBeUndefined() diff --git a/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts b/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts index c0b9b6c20e0..3c40d92b4ba 100644 --- a/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts @@ -4,7 +4,6 @@ import { generateRequestId } from '@/lib/core/utils/request' import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { activateWorkflowVersion } from '@/lib/workflows/application/deployments' import { workflowOperations } from '@/lib/workflows/application/operations' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const runtime = 'nodejs' @@ -18,7 +17,6 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, parseOptions: { optionalJsonBody: true, - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), }, mapInput: ({ params, body }) => ({ workflowId: params.id, diff --git a/apps/sim/app/api/v2/workflows/[id]/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/route.test.ts index d16cabfc429..72003b5574f 100644 --- a/apps/sim/app/api/v2/workflows/[id]/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/route.test.ts @@ -1,17 +1,22 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), readWorkflow: vi.fn(), updateWorkflow: vi.fn(), deleteWorkflow: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workflows/application/read-workflow', () => ({ @@ -23,18 +28,9 @@ vi.mock('@/lib/workflows/application/update-workflow', () => ({ vi.mock('@/lib/workflows/application/delete-workflow', () => ({ deleteWorkflow: { operation: { id: 'workflows.delete' }, execute: mocks.deleteWorkflow }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { NoWorkspaceAccessError, PersonalApiKeysDisabledError } from '@/lib/core/application' import { DELETE, GET, PATCH } from '@/app/api/v2/workflows/[id]/route' @@ -71,18 +67,10 @@ const routeContext = { params: Promise.resolve({ id: WORKFLOW_ID }) } describe('/api/v2/workflows/[id]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readWorkflow.mockResolvedValue({ workflow, workspaceId: WORKSPACE_ID, @@ -176,4 +164,16 @@ describe('/api/v2/workflows/[id]', () => { request, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost/api/v2/workflows/${WORKFLOW_ID}`), + routeContext + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts index 2d726aa208d..87d14c90168 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts @@ -45,6 +45,7 @@ vi.mock('@/lib/api/server/routes', () => { V2RouteInfrastructureError, v2ApiKeyAuth: { kind: 'v2-api-key' }, v2RateLimits: { publicApi: { kind: 'public-api' } }, + V2_PARSE_DEFAULTS: {}, v2OrchestrationErrorPolicy: { render: renderOrchestrationError }, } }) diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts index cc0938d8232..5e459182179 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts @@ -8,6 +8,7 @@ import { import { parseRequest } from '@/lib/api/server' import { admitV2Request, + V2_PARSE_DEFAULTS, V2RouteInfrastructureError, v2ApiKeyAuth, v2RateLimits, @@ -52,6 +53,7 @@ export const POST = withRouteHandler( if (!admission.success) return admission.response const parsed = await parseRequest(v2ResumeWorkflowContract, request, context, { + ...V2_PARSE_DEFAULTS, maxBodyBytes: 10 * 1024 * 1024, validationErrorResponse: v2ValidationError, }) diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts index 14ca04fbc5d..e67b9ddf739 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts @@ -1,40 +1,27 @@ /** * @vitest-environment node */ -import { createMockRequest } from '@sim/testing' +import { + createMockRequest, + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - MockV2ApiKeyUnauthenticatedError, - mocks: { - authenticate: vi.fn(), - cancel: vi.fn(), - capture: vi.fn(), - checkOperationRate: vi.fn(), - checkPreAuthRate: vi.fn(), - readRun: vi.fn(), - }, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkPreAuthRate - checkRateLimitDirectOrThrow = mocks.checkOperationRate - }, +const mocks = vi.hoisted(() => ({ + cancel: vi.fn(), + capture: vi.fn(), + readRun: vi.fn(), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) @@ -98,6 +85,17 @@ const baseStatus = { blockOutputs: null, } +/** + * Local denial fixture — the harness only publishes the allowed shapes, and the + * cancel adapter must surface `retryAfterMs` as a `Retry-After` header. + */ +const OPERATION_RATE_LIMIT_DENIED = { + allowed: false, + remaining: 0, + resetAt: new Date('2026-08-05T01:00:00Z'), + retryAfterMs: 5_000, +} as const + const successfulCancellation = { success: true, executionId: 'run-1', @@ -113,17 +111,10 @@ const successfulCancellation = { describe('v2 run detail and cancel adapters', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.checkPreAuthRate.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) - mocks.checkOperationRate.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readRun.mockResolvedValue(baseStatus) mocks.cancel.mockResolvedValue(successfulCancellation) }) @@ -223,13 +214,14 @@ describe('v2 run detail and cancel adapters', () => { }) it('rejects missing API keys before reading the run', async () => { - mocks.authenticate.mockRejectedValueOnce( + v2RouteMocks.authenticate.mockRejectedValueOnce( new MockV2ApiKeyUnauthenticatedError('API key required') ) const response = await callStatus() expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.readRun).not.toHaveBeenCalled() }) @@ -249,8 +241,8 @@ describe('v2 run detail and cancel adapters', () => { input: { workflowId: 'workflow-1', runId: 'run-1' }, request: expect.anything(), }) - expect(mocks.checkOperationRate).toHaveBeenCalledTimes(2) - expect(mocks.checkOperationRate).toHaveBeenCalledWith( + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.operationRate).toHaveBeenCalledWith( 'v2:workflows.runs.cancel:api-key:key-1', expect.anything() ) @@ -258,18 +250,9 @@ describe('v2 run detail and cancel adapters', () => { }) it('keeps cancellation request-rate admission separate from run control', async () => { - mocks.checkOperationRate - .mockResolvedValueOnce({ - allowed: false, - remaining: 0, - resetAt: new Date('2026-08-05T01:00:00Z'), - retryAfterMs: 5_000, - }) - .mockResolvedValueOnce({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) + v2RouteMocks.operationRate + .mockResolvedValueOnce(OPERATION_RATE_LIMIT_DENIED) + .mockResolvedValueOnce(V2_OPERATION_RATE_LIMIT_ALLOWED) const response = await cancelPost(createMockRequest('POST', undefined, {}), { params: Promise.resolve({ id: 'workflow-1', runId: 'run-1' }), @@ -296,7 +279,7 @@ describe('v2 run detail and cancel adapters', () => { }) it('projects cancellation analytics only after a successful personal-key result', async () => { - mocks.authenticate.mockResolvedValueOnce({ + v2RouteMocks.authenticate.mockResolvedValueOnce({ ...auth, principal: { kind: 'personal_api_key', userId: 'key-user', keyId: 'personal-key' }, rolloutUserId: 'key-user', diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts index f3714d3f563..de409d6a69b 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts @@ -1,32 +1,25 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - checkPreAuthRate: vi.fn(), - checkOperationRate: vi.fn(), listRuns: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkPreAuthRate - checkRateLimitDirectOrThrow = mocks.checkOperationRate - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/workflows/application/list-workflow-runs', () => ({ listWorkflowRuns: { @@ -85,17 +78,10 @@ const EXECUTIONS = [ describe('GET /api/v2/workflows/[id]/runs', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.checkPreAuthRate.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) - mocks.checkOperationRate.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listRuns.mockResolvedValue({ data: EXECUTIONS, nextCursor: null, @@ -160,8 +146,8 @@ describe('GET /api/v2/workflows/[id]/runs', () => { const response = await callGet('?cursor=not-a-cursor') expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalledOnce() - expect(mocks.checkOperationRate).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.listRuns).not.toHaveBeenCalled() }) @@ -242,4 +228,13 @@ describe('GET /api/v2/workflows/[id]/runs', () => { error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await callGet() + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts index 909c823e864..fd19cec176f 100644 --- a/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts @@ -1,17 +1,22 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), resolvePermission: vi.fn(), resolveWorkflowContext: vi.fn(), readVersion: vi.fn(), - gate: vi.fn(), })) vi.mock('@sim/platform-authz/workspace', () => ({ @@ -60,18 +65,9 @@ vi.mock('@/blocks/registry', () => ({ outputs: {}, }), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { GET } from '@/app/api/v2/workflows/[id]/versions/[version]/route' @@ -136,20 +132,12 @@ function versionState() { describe('GET /api/v2/workflows/[id]/versions/[version]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.resolvePermission.mockResolvedValue('admin') mocks.resolveWorkflowContext.mockResolvedValue(workflowContext) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) mocks.readVersion.mockResolvedValue({ id: 'version-2', version: 2, @@ -194,4 +182,13 @@ describe('GET /api/v2/workflows/[id]/versions/[version]', () => { expect(JSON.stringify(subBlocks)).not.toContain('sk-tool-plaintext-secret') expect(JSON.stringify(subBlocks)).not.toContain('table-plaintext-secret') }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await get() + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts index aca18ee8ede..f039ed8ba39 100644 --- a/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts @@ -1,15 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), listVersions: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workflows/application/list-workflow-versions', () => ({ @@ -18,18 +23,9 @@ vi.mock('@/lib/workflows/application/list-workflow-versions', () => ({ execute: mocks.listVersions, }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { GET } from '@/app/api/v2/workflows/[id]/versions/route' @@ -49,18 +45,10 @@ const context = { params: Promise.resolve({ id: 'workflow-1' }) } describe('GET /api/v2/workflows/[id]/versions', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listVersions.mockResolvedValue({ versions: [ { @@ -116,4 +104,16 @@ describe('GET /api/v2/workflows/[id]/versions', () => { expect(response.status).toBe(400) expect(mocks.listVersions).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest('http://localhost/api/v2/workflows/workflow-1/versions?limit=10'), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/route.test.ts b/apps/sim/app/api/v2/workflows/route.test.ts index 6a3e094b23c..c93ae286fa8 100644 --- a/apps/sim/app/api/v2/workflows/route.test.ts +++ b/apps/sim/app/api/v2/workflows/route.test.ts @@ -1,16 +1,21 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), createWorkflow: vi.fn(), listWorkflows: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workflows/application/create-workflow', () => ({ @@ -21,20 +26,9 @@ vi.mock('@/lib/workflows/application/list-workflows', () => ({ listWorkflows: { operation: { id: 'workflows.list' }, execute: mocks.listWorkflows }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { GET, POST } from '@/app/api/v2/workflows/route' @@ -82,18 +76,10 @@ const personalAuth = { describe('/api/v2/workflows', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(workspaceAuth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(workspaceAuth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listWorkflows.mockResolvedValue({ workflows: [WORKFLOW], nextCursorKeys: null, @@ -107,8 +93,8 @@ describe('/api/v2/workflows', () => { const response = await GET(new NextRequest('http://localhost/api/v2/workflows')) expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalledOnce() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.listWorkflows).not.toHaveBeenCalled() }) @@ -148,7 +134,7 @@ describe('/api/v2/workflows', () => { }) it('creates through a personal-key principal with the exact 201 contract', async () => { - mocks.authenticateV2ApiKey.mockResolvedValue(personalAuth) + v2RouteMocks.authenticate.mockResolvedValue(personalAuth) const request = new NextRequest('http://localhost/api/v2/workflows', { method: 'POST', headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, @@ -176,4 +162,15 @@ describe('/api/v2/workflows', () => { error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}`) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/route.ts b/apps/sim/app/api/v2/workflows/route.ts index f4dc34c3ef7..8fd65da66ab 100644 --- a/apps/sim/app/api/v2/workflows/route.ts +++ b/apps/sim/app/api/v2/workflows/route.ts @@ -1,17 +1,15 @@ import type { V2WorkflowListItem } from '@/lib/api/contracts/v2/workflows' import { v2CreateWorkflowContract, v2ListWorkflowsContract } from '@/lib/api/contracts/v2/workflows' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' import { defineV2JsonRoute, v2ApiKeyAuth, v2OrchestrationErrorPolicy, v2RateLimits, } from '@/lib/api/server/routes' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { createWorkflow } from '@/lib/workflows/application/create-workflow' import { listWorkflows } from '@/lib/workflows/application/list-workflows' import { workflowOperations } from '@/lib/workflows/application/operations' -import { cursorSortKey, decodeSortedCursor, encodeSortedCursor } from '@/app/api/v2/lib/response' +import { cursorSortKey, encodeSortedCursor, readSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -22,23 +20,16 @@ export const GET = defineV2JsonRoute({ operation: workflowOperations.list, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => { - const sort = cursorSortKey(query.sortBy, query.sortOrder) - const decoded = decodeSortedCursor(query.cursor, sort) - if (decoded.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - return { - workspaceId: query.workspaceId, - folderPath: query.folderPath, - deployedOnly: query.deployedOnly, - search: query.search, - sortBy: query.sortBy, - sortOrder: query.sortOrder, - cursorKeys: decoded.status === 'ok' ? decoded.keys : undefined, - limit: query.limit, - } - }, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + folderPath: query.folderPath, + deployedOnly: query.deployedOnly, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + cursorKeys: readSortedCursor(query.cursor, query.sortBy, query.sortOrder), + limit: query.limit, + }), useCase: listWorkflows, present: ({ workflows, nextCursorKeys, sortBy, sortOrder }) => ({ data: workflows.map( diff --git a/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts b/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts index 495044427bb..2b95cd7e007 100644 --- a/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts +++ b/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts @@ -30,6 +30,7 @@ describe('GET /api/workspaces/[id]/files/inline', () => { mockReadInline.mockResolvedValue({ file: { name: 'photo.png', type: 'image/png', size: PNG.length }, stream: new Blob([new Uint8Array(PNG)]).stream(), + contentAddressed: false, }) }) @@ -43,6 +44,7 @@ describe('GET /api/workspaces/[id]/files/inline', () => { input: { workspaceId: 'ws-1', fileId: 'wf_abc' }, }) ) + // A file id names the FILE, whose bytes move under it on every edit — so it must revalidate. expect(res.headers.get('Cache-Control')).toBe('private, no-cache, must-revalidate') expect(res.headers.get('Content-Disposition')).toBe('inline; filename="photo.png"') }) @@ -58,6 +60,24 @@ describe('GET /api/workspaces/[id]/files/inline', () => { }) }) + /** + * A storage key names one object and a content write never rewrites one, so these bytes can never + * change. Revalidating them meant re-downloading every embedded image on every open — a document is + * rendered by two editors (the read-only placeholder, then the live one) and each renders the image + * twice, so the image was fetched again on every one of those passes. + */ + it('lets the browser keep an image whose URL names the object that was streamed', async () => { + mockReadInline.mockResolvedValue({ + file: { name: 'photo.png', type: 'image/png', size: PNG.length }, + stream: new Blob([new Uint8Array(PNG)]).stream(), + contentAddressed: true, + }) + + const res = await GET(req('key=workspace%2Fws-1%2Fphoto.png'), params) + + expect(res.headers.get('Cache-Control')).toBe('private, max-age=31536000, immutable') + }) + it('returns the concealed 404 response for an unauthorized or missing file', async () => { mockReadInline.mockRejectedValue( new OrchestrationError('forbidden', 'Insufficient permissions') diff --git a/apps/sim/app/api/workspaces/[id]/files/inline/route.ts b/apps/sim/app/api/workspaces/[id]/files/inline/route.ts index ad3780eb4be..0a3f20bf4ec 100644 --- a/apps/sim/app/api/workspaces/[id]/files/inline/route.ts +++ b/apps/sim/app/api/workspaces/[id]/files/inline/route.ts @@ -10,6 +10,24 @@ import { encodeFilenameForHeader, getSecureFileHeaders } from '@/app/api/files/u export const dynamic = 'force-dynamic' +/** + * How long the browser may reuse an embedded image, decided by whether the URL names the exact object + * that was streamed (see {@link ReadWorkspaceInlineFileResult.contentAddressed}). + * + * A content write never rewrites a storage object, so a URL that names one addresses bytes that can + * never change and the browser needs no round trip — which is the difference between an embedded image + * reappearing instantly and being downloaded again. Every document render asks for the same image at + * least twice (ProseMirror's own DOM, then the React node view) and every editor mounts twice (the + * read-only placeholder, then the live editor), so revalidating each time meant re-fetching the whole + * image on every open and reload — measured at ~1 MB per open on a real document, with the image area + * blank until it landed. `private` keeps it out of shared caches: the bytes are authorized per user. + * + * Anything else — a request that names the FILE, whose bytes move under it, or one whose object was + * rotated away mid-request — keeps revalidating. + */ +const IMMUTABLE_CACHE_CONTROL = 'private, max-age=31536000, immutable' +const REVALIDATE_CACHE_CONTROL = 'private, no-cache, must-revalidate' + /** * GET /api/workspaces/[id]/files/inline?key=|fileId= * @@ -29,12 +47,12 @@ export const GET = defineInternalBinaryRoute({ fileId: query.fileId, }), useCase: readWorkspaceInlineFile, - present: ({ file, stream }) => { + present: ({ file, stream, contentAddressed }) => { const secure = getSecureFileHeaders(file.name, file.type) const headers = new Headers({ 'Content-Type': secure.contentType, 'Content-Disposition': `${secure.disposition}; ${encodeFilenameForHeader(file.name)}`, - 'Cache-Control': 'private, no-cache, must-revalidate', + 'Cache-Control': contentAddressed ? IMMUTABLE_CACHE_CONTROL : REVALIDATE_CACHE_CONTROL, 'X-Content-Type-Options': 'nosniff', }) if (secure.contentType === 'image/svg+xml') { diff --git a/apps/sim/app/workspace/[workspaceId]/files/[fileId]/loading.tsx b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/loading.tsx new file mode 100644 index 00000000000..da22416378e --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/loading.tsx @@ -0,0 +1,35 @@ +'use client' + +import { File as FileIcon } from '@sim/emcn/icons' +import { noop } from '@sim/utils/helpers' +import { + type BreadcrumbItem, + ResourceChromeFallback, +} from '@/app/workspace/[workspaceId]/components' +import { FOLDERED_RESOURCE_HEADERS } from '@/app/workspace/[workspaceId]/components/folders/foldered-resources' + +const FILES_HEADER = FOLDERED_RESOURCE_HEADERS.file + +/** + * Transcribes the trail the loaded page shows while its record resolves (`loadingBreadcrumbs` in + * `files.tsx`): the root crumb plus a terminal `…`, with no icon on the leaf — so the fallback and + * the page paint the same two crumbs and only the label changes. + */ +const BREADCRUMBS: BreadcrumbItem[] = [ + { label: FILES_HEADER.rootLabel, icon: FileIcon, onClick: noop }, + { label: '…', terminal: true }, +] + +/** + * Fallback for the file DETAIL route. Without it the segment inherits the Files list fallback, which + * paints an options bar and a table header row that a document page does not have — chrome that has + * to be torn down a frame later. A detail page is header + body, so this is the header alone. + * + * Header actions are deliberately omitted: they are a function of the open file (a previewable + * non-markdown file gets a mode toggle, an editable one gets Share/Delete), which is exactly what is + * not yet known here. Chips appearing beside the title reads as content arriving; chips appearing + * and then changing reads as a glitch. + */ +export default function FilesFileLoading() { + return +} diff --git a/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx index 590e94b0816..b4808ec50dd 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx @@ -1,17 +1,48 @@ import { Suspense } from 'react' +import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' +import { getQueryClient } from '@/app/_shell/providers/get-query-client' +import FilesFileLoading from '@/app/workspace/[workspaceId]/files/[fileId]/loading' import { Files } from '@/app/workspace/[workspaceId]/files/files' -import FilesLoading from '@/app/workspace/[workspaceId]/files/loading' +import { prefetchFilesBrowser } from '@/app/workspace/[workspaceId]/files/prefetch' export const metadata: Metadata = { title: 'Files', robots: { index: false }, } -export default function FilesFilePage() { +/** + * File detail entry. `Files` resolves the open file out of the workspace file LIST, so this route + * needs the same prefetch its sibling list page does — without it the server can only ever render + * the "resolving the record" spinner, and the real header (breadcrumbs, actions) has to pop in a + * frame later on the client. + * + * It also removes a whole class of hydration mismatch: which branch `Files` renders is decided by + * whether that list is in the cache, so a server render without it and a client render with it + * disagree on the header's markup (a static `…` crumb vs. the file's dropdown crumb). Prefetching + * here makes both sides read the same cache and pick the same branch by construction. + * + * `Files` reads URL query params via nuqs (`useSearchParams` internally), so it must sit under a + * Suspense boundary; the fallback is the detail chrome, matching the route's own `loading.tsx`. + */ +export default async function FilesFilePage({ + params, +}: { + params: Promise<{ workspaceId: string; fileId: string }> +}) { + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) + + const queryClient = getQueryClient() + if (session?.user?.id) { + await prefetchFilesBrowser(queryClient, workspaceId, session.user.id) + } + return ( - }> - - + + }> + + + ) } diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts index 88f54d6c0b7..708bb00444c 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts @@ -2,10 +2,8 @@ import type { Editor } from '@tiptap/core' import { Node as PMNode } from '@tiptap/pm/model' import { initProseMirrorDoc, updateYFragment, ySyncPluginKey } from '@tiptap/y-tiptap' import * as Y from 'yjs' -import { parseMarkdownToDoc } from '../markdown-parse' - -/** The Yjs fragment name TipTap's Collaboration extension binds to (its default `field`). */ -const COLLAB_DOC_FIELD = 'default' +import { COLLAB_DOC_FIELD } from '@/lib/collab-doc/field' +import { editorNormalForm } from '../markdown-parse' /** * Transaction origin for agent-streamed writes into a live collaborative doc. It is deliberately NOT @@ -63,7 +61,11 @@ export function applyAgentStreamFrame( ): boolean { const binding = ySyncPluginKey.getState(editor.state)?.binding if (!binding) return false - const target = PMNode.fromJSON(editor.schema, parseMarkdownToDoc(body)) + // Through the editor's normal form, like every other writer to the shared document. A frame whose + // body ends on a list, heading, table, or rule parses WITHOUT the editor's trailing paragraph, so + // reconciling toward the bare parse deletes the one the seed put there — and the next client to bind + // writes it back, which is the divergence this normalization exists to prevent. + const target = PMNode.fromJSON(editor.schema, editorNormalForm(body)) let delta: Uint8Array | null = null const capture = (update: Uint8Array, origin: unknown) => { if (origin === AGENT_STREAM_ORIGIN) delta = update diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts index b8996800923..46183927ec1 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts @@ -164,7 +164,7 @@ describe('collab streaming integration — moving pieces', () => { reopened.destroy() }) - it('EMPTY-COLLAPSE ON THE STREAM PATH: an agent body with a huge blank run does not strand empties', () => { + it('EMPTY-BOUND ON THE STREAM PATH: an agent body with a huge blank run does not strand empties', () => { const A = makeCollabEditor() A.editor.commands.setContent(parseMarkdownToDoc('# Title\n\nintro'), { contentType: 'json' }) const session = beginAgentStream(A.editor)! @@ -173,9 +173,11 @@ describe('collab streaming integration — moving pieces', () => { endAgentStream(session) console.log( - `\n[STREAM-COLLAPSE] text=${JSON.stringify(A.editor.state.doc.textContent)} empty=${emptyParas(A.editor)}` + `\n[STREAM-BOUND] text=${JSON.stringify(A.editor.state.doc.textContent)} empty=${emptyParas(A.editor)}` ) - expect(emptyParas(A.editor)).toBe(0) // collapse protects the live streaming path, not just static open + // ~200 blank paragraphs' worth of run arrives; the parse bound caps what reaches the live doc, so the + // streaming path is protected exactly like a static open — no unbounded node explosion in the CRDT. + expect(emptyParas(A.editor)).toBe(20) expect(A.editor.state.doc.textContent).toContain('tail paragraph') }) @@ -197,4 +199,45 @@ describe('collab streaming integration — moving pieces', () => { expect(D.editor.state.doc.textContent).toContain('streamed body') expect(emptyParas(D.editor)).toBe(0) }) + + /** + * Every writer to the shared document has to produce the editor's normal form, or the one that does + * not silently removes what the others add. A frame whose body ends on a list, heading, table, or rule + * parses WITHOUT the trailing paragraph the seed puts there — reconciling toward that bare parse + * deleted it from the live room, and the next client to bind wrote it back, reopening the + * placeholder-vs-live divergence the seed normalization exists to close. + */ + it.each([ + ['ends on a list', ['# T\n\nintro\n\n- a', '# T\n\nintro\n\n- a\n- b']], + ['ends on a heading', ['# T\n\nintro\n\n## Sec', '# T\n\nintro\n\n## Section']], + ['ends on a table', ['# T\n\n| a |\n| --- |\n| 1 |']], + ])('AGENT STREAM KEEPS THE EDITOR NORMAL FORM: %s', (_label, frames) => { + const doc = markdownToYDoc('# T\n\nintro\n\n- seed') + const awareness = new Awareness(doc) + const editor = new Editor({ + extensions: createMarkdownEditorExtensions({ + placeholder: '', + collaboration: { + doc, + awareness, + user: { name: 'U', color: '#fff', clientId: doc.clientID }, + }, + }), + }) + const trailingIsEmptyParagraph = () => { + const fragment = doc.getXmlFragment('default') + const last = fragment.get(fragment.length - 1) + return last instanceof Y.XmlElement && last.nodeName === 'paragraph' && last.length === 0 + } + expect(trailingIsEmptyParagraph()).toBe(true) + + const session = beginAgentStream(editor)! + for (const frame of frames) applyAgentStreamFrame(editor, session, frame) + endAgentStream(session) + + expect(trailingIsEmptyParagraph()).toBe(true) + editor.destroy() + awareness.destroy() + doc.destroy() + }) }) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts index 689c2ee6c77..baa4d97b592 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts @@ -98,6 +98,50 @@ describe('FileDocProvider', () => { expect(emittedMessages(emit)).toHaveLength(0) }) + /** + * A tab that outlived its room can be offered a DIFFERENT document for the same file. Yjs would union + * the two — the file twice, on both sides, and the relay persists it — and there is no un-merge. So + * the sync must not happen at all; the fatal path leaves the editor read-only on what it already + * shows, and a reload binds a fresh document. + */ + it('refuses to sync into a document it does not recognize', () => { + const { provider, doc, emit, fire } = createProvider(true) + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.docIdKey, 'doc-original') + const joinError = vi.fn() + provider.on('join-error', joinError) + emit.mockClear() + + fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1', docId: 'doc-rebuilt' }) + + expect(emittedMessages(emit)).toHaveLength(0) + expect(provider.synced).toBe(false) + expect(provider.joinError).toMatchObject({ code: 'DOCUMENT_REPLACED', retryable: false }) + expect(joinError).toHaveBeenCalledTimes(1) + }) + + it('syncs when the room holds the document it already has', () => { + const { doc, emit, fire } = createProvider(true) + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.docIdKey, 'doc-original') + emit.mockClear() + + fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1', docId: 'doc-original' }) + + expect(emittedMessages(emit).length).toBeGreaterThan(0) + }) + + it('syncs when either side carries no identity (a fresh doc, or a room seeded before identities)', () => { + const fresh = createProvider(true) + fresh.emit.mockClear() + fresh.fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1', docId: 'doc-rebuilt' }) + expect(emittedMessages(fresh.emit).length).toBeGreaterThan(0) + + const unnamedRoom = createProvider(true) + unnamedRoom.doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.docIdKey, 'doc-original') + unnamedRoom.emit.mockClear() + unnamedRoom.fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1' }) + expect(emittedMessages(unnamedRoom.emit).length).toBeGreaterThan(0) + }) + it('applies a server sync step 2 and becomes synced', () => { const { provider, doc, fire } = createProvider(true) const synced = vi.fn() diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts index 84bab0733bc..79884c1e81c 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts @@ -174,19 +174,11 @@ export class FileDocProvider extends ObservableV2 { */ private handleReadinessDeadline = () => { this.readinessTimer = null - if ((this.synced && this.isSeeded()) || this.fatal || this.disposed) return - const error: JoinFileDocError = { - fileId: this.fileId, - error: 'Realtime document was not ready in time', - code: 'READINESS_TIMEOUT', - retryable: false, - } - this.fatal = true - this.joinError = error - // Drop `synced` so the editor's `synced && seeded` gate stays closed → the fallback renders the - // stored content read-only rather than becoming editable on a doc the server never seeded. - this.setSynced(false) - this.emit('join-error', [error]) + if (this.synced && this.isSeeded()) return + // Dropping `synced` (see {@link failFatally}) is what keeps the editor's `synced && seeded` gate + // closed, so the fallback renders the stored content read-only rather than becoming editable on a + // document the server never seeded. + this.failFatally('Realtime document was not ready in time', 'READINESS_TIMEOUT') } private clearReadinessTimer() { @@ -215,13 +207,55 @@ export class FileDocProvider extends ObservableV2 { /** * Handle the join ack. The server registers the room before acking, so an earlier * send could be dropped — the initial sync + local awareness exchange begins here. + * + * Unless the room holds a DIFFERENT document than ours. Two documents built from the same markdown + * are not the same document to Yjs — their items carry different client ids — so syncing one into the + * other appends the file to itself, on both sides, and the server persists the result. A document is + * rebuilt only when the room AND the shared stream are both gone (a tab that slept through it), which + * is precisely when a stale tab reconnects. There is no way to un-merge afterwards, so the sync never + * happens: take the fatal path, which leaves the editor read-only on the content it already shows. + * A reload binds a fresh document and recovers. */ private handleJoinSuccess = (data: JoinFileDocSuccess) => { if (data.fileId !== this.fileId) return + const local = this.docId() + if (local !== undefined && data.docId !== undefined && data.docId !== local) { + this.failFatally( + 'This document was reloaded on the server; refresh to continue editing', + 'DOCUMENT_REPLACED' + ) + return + } this.sendSyncStep1() this.sendLocalAwareness() } + /** The identity of the document we hold, once the server seed has named one. */ + private docId(): string | undefined { + const docId = this.doc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.docIdKey) + return typeof docId === 'string' ? docId : undefined + } + + /** + * Give up on this document, non-retryably: latch fatal so nothing more is applied or relayed, drop + * `synced` so the editor's gate closes, and surface the rejection to the owner (which falls back to a + * read-only view of the stored content). + */ + private failFatally(message: string, code: string) { + if (this.fatal || this.disposed) return + const error: JoinFileDocError = { + fileId: this.fileId, + error: message, + code, + retryable: false, + } + this.fatal = true + this.joinError = error + this.clearReadinessTimer() + this.setSynced(false) + this.emit('join-error', [error]) + } + /** * Handle a join rejection. A non-retryable rejection (access denied, invalid) * won't succeed on retry, so latch {@link fatal} to stop (re)joining and let the @@ -247,18 +281,7 @@ export class FileDocProvider extends ObservableV2 { */ private handleAccessRevoked = (data: RoomAccessRevokedBroadcast) => { if (data.room?.type !== ROOM_TYPES.WORKSPACE_FILE_DOC || data.room.id !== this.fileId) return - if (this.fatal || this.disposed) return - const error: JoinFileDocError = { - fileId: this.fileId, - error: data.message, - code: 'ACCESS_REVOKED', - retryable: false, - } - this.fatal = true - this.joinError = error - this.clearReadinessTimer() - this.setSynced(false) - this.emit('join-error', [error]) + this.failFatally(data.message, 'ACCESS_REVOKED') } private handleMessage = (data: unknown) => { diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts index 39bb68590fc..667638a78ea 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts @@ -4,11 +4,20 @@ import { describe, expect, it } from 'vitest' import { type CollabReadinessInputs, nextCollabReadiness } from './readiness' +/** An observation, with the healthy defaults filled in so each case states only what it exercises. */ +const at = (input: Partial): CollabReadinessInputs => ({ + synced: false, + seeded: false, + offlineSeed: false, + fatal: false, + ...input, +}) + /** Drive a sequence of observations through the latch, returning the readiness at each step. */ -function run(steps: CollabReadinessInputs[]): boolean[] { +function run(steps: Partial[]): boolean[] { let syncedOnce = false - return steps.map((input) => { - const next = nextCollabReadiness(syncedOnce, input) + return steps.map((step) => { + const next = nextCollabReadiness(syncedOnce, at(step)) syncedOnce = next.syncedOnce return next.ready }) @@ -16,21 +25,27 @@ function run(steps: CollabReadinessInputs[]): boolean[] { describe('nextCollabReadiness', () => { it('is not ready before syncing or seeding', () => { - const { syncedOnce, ready } = nextCollabReadiness(false, { - synced: false, - seeded: false, - offlineSeed: false, - }) + const { syncedOnce, ready } = nextCollabReadiness( + false, + at({ + synced: false, + seeded: false, + offlineSeed: false, + }) + ) expect(syncedOnce).toBe(false) expect(ready).toBe(false) }) it('is not ready when synced but not yet seeded', () => { - const { syncedOnce, ready } = nextCollabReadiness(false, { - synced: true, - seeded: false, - offlineSeed: false, - }) + const { syncedOnce, ready } = nextCollabReadiness( + false, + at({ + synced: true, + seeded: false, + offlineSeed: false, + }) + ) expect(syncedOnce).toBe(true) // latched expect(ready).toBe(false) // waits for the seed }) @@ -50,11 +65,14 @@ describe('nextCollabReadiness', () => { it('opens even if the seed lands before we ever observed synced (server seed proves a sync)', () => { // If the flap beat our first observation, the seed flag alone (not the offline fallback) proves a // completed sync happened. - const { syncedOnce, ready } = nextCollabReadiness(false, { - synced: false, - seeded: true, - offlineSeed: false, - }) + const { syncedOnce, ready } = nextCollabReadiness( + false, + at({ + synced: false, + seeded: true, + offlineSeed: false, + }) + ) expect(syncedOnce).toBe(true) expect(ready).toBe(true) }) @@ -74,4 +92,33 @@ describe('nextCollabReadiness', () => { ]) expect(readiness).toEqual([true, true]) }) + /** + * The reported bug. A brand-new file syncs EMPTY (latching `syncedOnce`), its server seed never + * lands, and the readiness deadline fires: the provider goes fatal and drops `synced` precisely so + * this gate closes. The offline fallback then seeds locally — and the sticky latch used to re-open + * the gate on that, handing back an editable editor bound to a document the provider had abandoned. + * Every keystroke was dropped (the provider ignores frames and never rejoins) and client autosave + * stayed off (collaboration is nominally on), so the edits vanished on reload with no error shown. + */ + it('stays read-only after the readiness deadline goes fatal, even though a sync was latched', () => { + const readiness = run([ + { synced: false }, + { synced: true }, // initial EMPTY sync — latches syncedOnce + { synced: false, fatal: true }, // deadline: provider drops synced and gives up + { seeded: true, offlineSeed: true, fatal: true }, // fallback seeds locally + ]) + expect(readiness).toEqual([false, false, false, false]) + }) + + /** + * The same revocation on an ALREADY-ready doc: access is withdrawn mid-session, the provider goes + * fatal, and readiness must be taken back rather than left latched open. + */ + it('revokes readiness when a live document turns fatal', () => { + const readiness = run([ + { synced: true, seeded: true }, // ready + { synced: false, seeded: true, fatal: true }, // access revoked mid-session + ]) + expect(readiness).toEqual([true, false]) + }) }) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts index 343b301b394..9974de6ed43 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts @@ -22,16 +22,32 @@ export interface CollabReadinessInputs { seeded: boolean /** Whether the seed flag was set by the offline fallback (no server sync) rather than the server. */ offlineSeed: boolean + /** + * Whether the provider has GIVEN UP on this document — a non-retryable rejection, an access + * revocation, or the readiness deadline lapsing. A fatal provider ignores every inbound frame and + * never rejoins, so nothing typed after this point reaches the server. + */ + fatal: boolean } /** * Pure transition for the readiness latch. `syncedOnce` is the sticky prior state — pass the returned * `syncedOnce` back in on the next call. `ready` is whether the doc is synced-and-seeded. + * + * `fatal` overrides the latch, and that override is the whole reason it is an input. The latch is + * sticky on purpose, but stickiness must not outlive the document: a doc that syncs empty and never + * receives its server seed trips the readiness deadline, and the provider answers by dropping `synced` + * so this gate closes. The latch ignored that — `syncedOnce` was already set by the empty sync — so the + * offline fallback's seed flag re-opened the gate and handed back an EDITABLE editor on a document the + * provider had already abandoned. Nothing typed into it could persist: the provider drops every frame + * and never rejoins, and the client's own autosave stays gated off because collaboration is nominally + * on. The user types, sees no error, and loses the edits on reload. Revoking readiness on `fatal` is + * what makes the fallback what it is documented to be — a READ-ONLY view of the stored content. */ export function nextCollabReadiness( syncedOnce: boolean, input: CollabReadinessInputs ): { syncedOnce: boolean; ready: boolean } { const next = syncedOnce || input.synced || (input.seeded && !input.offlineSeed) - return { syncedOnce: next, ready: next && input.seeded } + return { syncedOnce: next, ready: next && input.seeded && !input.fatal } } diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts index 43466f49861..4470187fefa 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts @@ -173,10 +173,11 @@ function stripEmptyListItemLines(markdown: string): string { * round-trip ({@link stripEmptyListItemLines}), restores callout markers the serializer * backslash-escapes (`> \[!NOTE\]` → `> [!NOTE]`), and collapses trailing blank lines to a single * newline. Interior blank runs are NOT collapsed here — blank lines inside a fenced code block (or a - * verbatim raw-markdown-snippet) are significant, and a global collapse would corrupt them. Spurious - * interior blank runs between top-level blocks are removed upstream instead, by - * {@link parseMarkdownToDoc} stripping empty paragraphs, so a doc that has been through the editor - * never serializes with an interior blank run outside code in the first place. The table serializer's + * verbatim raw-markdown-snippet) are significant, and a global collapse would corrupt them. An interior + * run between top-level blocks is significant too: it is how an empty paragraph is written, and + * {@link parseMarkdownToDoc} reads exactly the count back out, so collapsing it here would delete the + * document's spacing. Only the TRAILING run is collapsed — it can carry no paragraph (see + * `clampEmptyParagraphs`) and would otherwise churn the file on every save. The table serializer's * spurious surrounding blank lines are trimmed at the source (PipeSafeTable), so no global * leading-newline strip is needed here — avoiding clobbering content that legitimately begins with * whitespace. diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts index 0067ef31f47..94c645fc2e2 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts @@ -91,45 +91,101 @@ describe('parseMarkdownToDoc (chunked)', () => { expect(splitMarkdownBlocks('\n\n \n')).toEqual([]) }) - // Asserts the collapse documented on `stripEmptyParagraphs` — at document edges, between blocks, for - // one or many blank lines, and around lists. (Blank runs are insignificant in markdown, so a collapsed - // file renders identically everywhere it's viewed; the pathological case is a run of thousands.) - describe('collapses blank-line runs to markdown-standard spacing', () => { - /** Block-type shape of a doc after `parseMarkdownToDoc`, `∅` for any surviving empty paragraph. */ - function shapeOf(md: string): string { - return (parseMarkdownToDoc(md).content ?? []) - .map((n) => (isEmptyPara(n) ? '∅' : n.type)) - .join(',') - } + /** Block-type shape of a doc after `parseMarkdownToDoc`, `∅` for each empty paragraph. */ + function shapeOf(md: string): string { + return (parseMarkdownToDoc(md).content ?? []) + .map((n) => (isEmptyPara(n) ? '∅' : n.type)) + .join(',') + } + // A blank line an author left between two blocks is part of the document, so parse must read back the + // exact count the serializer wrote (`blocks.join('\n\n')` ⇒ an empty paragraph costs TWO blank lines, + // the first separator is free). Getting this wrong is visible: the static placeholder is built from + // markdown while the live collaborative doc is the CRDT, so any drift shows up as the doc reflowing + // its spacing a beat after the file appears. + describe('preserves authored blank lines', () => { it.each([ - ['one blank gap between paragraphs', 'a\n\n\n\nb', 'paragraph,paragraph'], - ['many blank lines between paragraphs', 'a\n\n\n\n\n\n\n\nb', 'paragraph,paragraph'], - ['leading blank lines', '\n\n\n\na', 'paragraph'], - ['leading + interior', '\n\n\na\n\n\n\nb', 'paragraph,paragraph'], - ['blank gap between a heading and text', '# H\n\n\n\ntext', 'heading,paragraph'], - ['blank gap after a tight list', '- a\n- b\n\n\n\ntext', 'bulletList,paragraph'], - ['blank gap before a tight list', 'text\n\n\n\n- a\n- b', 'paragraph,bulletList'], - // Line-ending variants normalize first, so `\r`-only / CRLF blank runs collapse identically. - ['CRLF between blocks', 'a\r\n\r\n\r\n\r\nb', 'paragraph,paragraph'], - ['CR-only (classic Mac) between blocks', 'a\r\r\r\rb', 'paragraph,paragraph'], - ])('collapses to no empty paragraphs: %s', (_label, md, expected) => { + ['single separator — no empty paragraph', 'a\n\nb', 'paragraph,paragraph'], + ['odd blank line is insignificant', 'a\n\n\nb', 'paragraph,paragraph'], + ['one authored blank line', 'a\n\n\n\nb', 'paragraph,∅,paragraph'], + ['three authored blank lines', 'a\n\n\n\n\n\n\n\nb', 'paragraph,∅,∅,∅,paragraph'], + ['leading blank lines', '\n\n\n\na', '∅,∅,paragraph'], + ['leading + interior', '\n\n\na\n\n\n\nb', '∅,paragraph,∅,paragraph'], + ['between a heading and text', '# H\n\n\n\ntext', 'heading,∅,paragraph'], + ['after a tight list', '- a\n- b\n\n\n\ntext', 'bulletList,∅,paragraph'], + ['before a tight list', 'text\n\n\n\n- a\n- b', 'paragraph,∅,bulletList'], + // Line-ending variants normalize first, so `\r`-only / CRLF runs count identically. + ['CRLF between blocks', 'a\r\n\r\n\r\n\r\nb', 'paragraph,∅,paragraph'], + ['CR-only (classic Mac) between blocks', 'a\r\r\r\rb', 'paragraph,∅,paragraph'], + ])('%s', (_label, md, expected) => { + expect(shapeOf(md)).toBe(expected) + }) + + // A loose list's own internal blank lines are absorbed into its merged block, so they stay list + // spacing rather than becoming top-level paragraphs that would split the list in two. + it('a loose list keeps its internal blank lines as one list', () => { + expect(shapeOf('- a\n\n- b\n\n- c')).toBe('bulletList') + }) + + // …but a gap WIDE enough to carry an empty paragraph is a top-level block boundary: the serializer + // only writes one by emitting the two sides as separate blocks, so re-merging them made parse stop + // inverting serialize. That swallowed the paragraph, fused the two blocks, and — because the file + // then never reached a fixpoint — silently opened it READ-ONLY. + it.each([ + ['between two bullet lists', '- a\n\n\n\n- b', 'bulletList,∅,bulletList'], + ['between two blockquotes', '> a\n\n\n\n> b', 'blockquote,∅,blockquote'], + ['before an indented continuation', 'a\n\n\n\n indented', 'paragraph,∅,paragraph'], + ])('a gap that carries a paragraph breaks the merge: %s', (_label, md, expected) => { expect(shapeOf(md)).toBe(expected) }) it('a pathological blank run does not explode into empty paragraph nodes', () => { // The production incident: an agent/paste artifact with a huge blank run became ~1959 empty - // paragraphs baked into the doc. Collapsing on parse neutralizes any such source. + // paragraphs baked into the doc. The run is bounded on parse, so no source can reach that. const body = `Para A${'\n'.repeat(4000)}Para B` const content = parseMarkdownToDoc(body).content ?? [] - expect(content.filter(isEmptyPara).length).toBe(0) - expect(content.map((n) => n.type)).toEqual(['paragraph', 'paragraph']) + expect(content.filter(isEmptyPara).length).toBe(20) + expect(content.length).toBe(22) + }) + + // The per-gap ceiling bounds one run; the realistic artifact shape is a moderate run between EVERY + // paragraph, which scales with file size. Without a document budget an 86KB body produced ~40k empty + // paragraphs — twenty times the incident the per-gap ceiling exists to prevent. + it('many blank runs cannot explode the document either', () => { + const body = `${'x'.padEnd(1)}${`${'\n'.repeat(42)}x`.repeat(2000)}` + const content = parseMarkdownToDoc(body).content ?? [] + expect(content.filter(isEmptyPara).length).toBe(500) + }) + + // The bounds have to be fixpoints too, or a clamped file would churn on every save. + it.each([ + ['one huge run', `Para A${'\n'.repeat(4000)}Para B`], + ['many runs past the document budget', `x${`${'\n'.repeat(42)}x`.repeat(2000)}`], + ])('a bounded document re-serializes to itself: %s', (_label, md) => { + const once = serializeMarkdownBody(md) + expect(serializeMarkdownBody(once)).toBe(once) + }) + + // The whole-document path hands blank runs to @tiptap/markdown, which keeps them after a paragraph + // but swallows them after a heading/ordered list/table. Preserving only some would break the fixpoint + // for the same document, so that path keeps none — consistently zero, which IS a fixpoint. + it.each([ + ['block HTML', '# H\n\n\n\ntext\n\n
x
', 'heading,paragraph,rawHtmlBlock'], + [ + 'a reference definition', + '# H\n\n\n\nsee [y][r]\n\n[r]: https://e.com', + 'heading,paragraph', + ], + ])('a document that must parse whole keeps no empty paragraphs: %s', (_label, md, expected) => { + expect(shapeOf(md)).toBe(expected) + const once = serializeMarkdownBody(md) + expect(serializeMarkdownBody(once)).toBe(once) }) }) - // Regression: a file with blank lines (leading, interior, or trailing) must stay EDITABLE. Collapsing - // blank runs keeps serialize→parse idempotent, so the round-trip-safety probe reaches a fixed point - // instead of flipping the file read-only. + // Regression: a file with blank lines (leading, interior, or trailing) must stay EDITABLE — parse and + // serialize have to agree on the blank count, or the round-trip-safety probe never reaches a fixed + // point and the file silently opens read-only. describe('blank lines stay editable (regression)', () => { it.each([ ['plain paragraph', 'abc\n\n'], @@ -137,15 +193,27 @@ describe('parseMarkdownToDoc (chunked)', () => { ['three trailing newlines', 'hello\n\n\n'], ['two paragraphs', 'para one\n\npara two\n\n'], ['interior blank run + trailing', 'a\n\n\n\nb\n\n'], + ['many interior blank runs', '# T\n\n\n\na\n\n\n\n\n\nb\n\n\n\n- x\n- y\n\n'], + ['leading blank run', '\n\n\n\nabc\n'], + // These regressed to read-only when a gap carrying a paragraph was still merged away: the merge + // fused the two blocks, so the second pass produced different markdown from the first. + ['gap before a list glued to a lead-in line', 'text\n1. one\n\n\n\n- bullet'], + ['gap between two glued list kinds', 'text\n- bullet\n\n\n\n1. one'], + ['gap between two blockquotes after a lead-in', 'text\n> a\n\n\n\n> b'], + [ + 'changelog shape', + '## v2\n\nHighlights:\n1. faster\n2. smaller\n\n\n\n- also: fixed a crash\n', + ], ])('a file with blank lines is round-trip-safe: %s', (_label, md) => { expect(isRoundTripSafe(md)).toBe(true) }) - it('removes only structurally-empty paragraphs — a paragraph with content survives', () => { - // The shape suite above already proves leading/interior/trailing blank runs collapse to zero empty - // paragraphs; this pins the complementary guarantee — a real (non-empty) paragraph is never dropped. + // Trailing empties are the one kind that cannot round-trip: `postProcessSerializedMarkdown` + // collapses trailing blank lines, so keeping them would make the doc differ from its own output. + it('drops trailing empty paragraphs', () => { + expect(shapeOf('abc\n\n')).toBe('paragraph') + expect(shapeOf('abc\n\n\n\n\n\n')).toBe('paragraph') const trailing = parseMarkdownToDoc('abc\n\n').content ?? [] - expect(trailing.at(-1)?.type).toBe('paragraph') expect(isEmptyPara(trailing.at(-1) ?? {})).toBe(false) }) }) @@ -247,16 +315,25 @@ const FUZZ_BLOCKS: Array<(r: () => number) => string> = [ () => 'See [the docs][ref].\n\n[ref]: https://example.com/docs', ] -function buildFuzzDoc(seed: number): string { +/** + * `blankRuns` widens the separator from a single blank line to a run of up to three, so the corpus + * exercises authored spacing. The single-separator corpus structurally could not: every document it + * built was `parts.join('\n\n')`, which is exactly the one gap width that carries no empty paragraph — + * so the whole blank-line design was invisible to the property test that claims to cover any input. + */ +function buildFuzzDoc(seed: number, blankRuns: boolean): string { const r = rng(seed) const count = 2 + Math.floor(r() * 8) const parts: string[] = [] - for (let i = 0; i < count; i++) parts.push(FUZZ_BLOCKS[Math.floor(r() * FUZZ_BLOCKS.length)](r)) - return parts.join('\n\n') + for (let i = 0; i < count; i++) { + if (i > 0) parts.push('\n'.repeat(blankRuns ? 2 + Math.floor(r() * 4) : 2)) + parts.push(FUZZ_BLOCKS[Math.floor(r() * FUZZ_BLOCKS.length)](r)) + } + return parts.join('') } describe('chunked parse — property test over randomized documents', () => { - it('chunked === one-shot for every document, and idempotent for every editable one', () => { + it('chunked === one-shot on single-separator documents, and idempotent for every editable one', () => { const failures: Array<{ seed: number; kind: string }> = [] // Compare modulo trailing whitespace: `parseMarkdownToDoc` strips trailing empty paragraphs (they // can't be serialized stably — postProcess collapses trailing newlines — so keeping them would flip @@ -264,17 +341,48 @@ describe('chunked parse — property test over randomized documents', () => { // intended and invisible after save; interior/leading fidelity is still compared exactly. const trimEnd = (md: string) => md.replace(/\n+$/, '') for (let seed = 1; seed <= 400; seed++) { - const body = buildFuzzDoc(seed) + const body = buildFuzzDoc(seed, false) const chunked = serializeMarkdownBody(body) - // Fidelity is the load-bearing invariant — chunked must never diverge from the whole-document - // parse, for ANY input; idempotency only needs to hold where the doc is editable (raw HTML is - // non-idempotent in the underlying editor regardless of chunking, which is why it opens read-only). + // On documents with no authored blank run the two paths must still agree exactly. They are allowed + // to differ once a gap carries an empty paragraph: the chunked path reconstructs it and the + // whole-document path deliberately keeps none (see `parseMarkdownToDoc`), and only ONE path ever + // runs for a given document. Idempotency is the invariant that must hold for both, and it is + // asserted for every editable document in the blank-run corpus below. if (trimEnd(chunked) !== trimEnd(oneShot(body))) failures.push({ seed, kind: 'fidelity' }) else if (isRoundTripSafe(body) && serializeMarkdownBody(chunked) !== chunked) { failures.push({ seed, kind: 'idempotency' }) } } expect(failures).toEqual([]) - // 400 docs each parsed+serialized twice — generous timeout so it can't flake under parallel load. - }, 30000) + // 400 docs each parsed+serialized twice. Measured ~10s alone; the whole-suite run gives each worker + // a fraction of a core, and at 30s BOTH property tests in this file timed out there while passing + // standalone. Sized off the loaded number, not the isolated one. + }, 60000) + + /** + * Idempotency is what keeps a file editable: `isRoundTripSafe` opens a document read-only unless + * serializing twice is byte-identical. Preserving blank lines put every gap width on that path, and a + * merge rule that swallowed a gap silently flipped ordinary documents (a changelog, a lead-in line + * followed by a list) to read-only. Fuzz the separator width so that class cannot come back. + * + * Gated on `isRoundTripSafe` for the same reason the single-separator test above is: a document the + * probe rejects opens read-only and is never re-serialized, so its instability is contained by design. + * This corpus does surface such documents — a blank run INSIDE a loose list parses to an empty + * paragraph nested in a list item, which `getMarkdown` writes as an indented `' '` marker line rather + * than a blank one, and that does not round-trip. That defect predates blank-line preservation (it + * reproduces identically with the empty-paragraph strip in place) and is only reachable through a gap + * width the old corpus could not generate; the probe correctly holds those files read-only. + */ + it('stays idempotent with authored blank runs of every width', () => { + const failures: Array<{ seed: number; body: string }> = [] + for (let seed = 1; seed <= 400; seed++) { + const body = buildFuzzDoc(seed, true) + if (!isRoundTripSafe(body)) continue + const once = serializeMarkdownBody(body) + if (serializeMarkdownBody(once) !== once) failures.push({ seed, body }) + } + expect(failures).toEqual([]) + // Same budget as the corpus above, for the same reason — this is the second ~10s property test in + // the file, and adding it is what pushed both past 30s under whole-suite parallelism. + }, 60000) }) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts index 6cdeeabf4ae..fd8cb706a34 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts @@ -47,14 +47,64 @@ const FENCE_CLOSE = /^ {0,3}(`{3,}|~{3,})[ \t]*$/ const LIST_MARKER = /^[ ]{0,3}(?:[-*+]|\d+[.)])\s/ const BLOCKQUOTE = /^[ ]{0,3}>/ +/** + * Ceiling on the empty paragraphs one gap may carry. Deliberate spacing is a handful of blank lines; a + * run of thousands is an agent/paste artifact, and baking a node per blank would put thousands of empty + * paragraphs in the document forever (the reported incident: ~1959 nodes from one 4000-newline run). + * Well past any spacing a person types, low enough that no single gap can explode. + */ +const MAX_CONSECUTIVE_EMPTY_PARAGRAPHS = 20 + +/** + * Ceiling on a document's TOTAL empty paragraphs, enforced by {@link boundEmptyParagraphs}. The per-gap + * ceiling alone bounds nothing at document scale — the realistic artifact shape is a moderate blank run + * between every paragraph, not one giant run, and that scales linearly with file size. Generous enough + * that no hand-spaced document reaches it, finite so a machine-generated one cannot grow the node count + * without limit. + */ +const MAX_EMPTY_PARAGRAPHS_PER_DOC = 500 + +/** + * How many empty paragraphs a run of `blankLines` between two blocks carries. + * + * The serializer joins top-level blocks with a blank line (`blocks.join('\n\n')`), so an empty + * paragraph costs TWO blank lines — its own, plus the separator that follows it — while the first + * separator is free. Inverting that join is the whole rule: an interior gap of `b` blank lines carries + * `(b - 1) / 2` empty paragraphs, a leading gap (no preceding block, so no free separator) carries + * `b / 2`, and both round down. A hand-authored odd blank line is insignificant in markdown and + * collapses, exactly as every standard renderer shows it; a gap the editor itself wrote reconstructs + * exactly, which is what makes parse ∘ serialize a fixed point. + * + * The count is computed here rather than delegated to `@tiptap/markdown`, whose own blank-run handling + * is not self-consistent: after a paragraph, list, blockquote, code fence, rule, or image it follows the + * same `(b - 1) / 2`, but after a heading, an ordered list, or a table the token swallows the whole run + * and yields nothing. Delegating would mean a blank line after a heading could never survive a save. + * + * Bounded here as well as in {@link clampEmptyParagraphs} so a pathological run is never materialized + * in the first place — a megabyte of newlines would otherwise allocate half a million throwaway nodes + * on its way to being clamped back down to {@link MAX_CONSECUTIVE_EMPTY_PARAGRAPHS}. + */ +function emptyBlockCount(blankLines: number, leading: boolean): number { + const count = Math.floor((blankLines - (leading ? 0 : 1)) / 2) + return Math.max(0, Math.min(count, MAX_CONSECUTIVE_EMPTY_PARAGRAPHS)) +} + /** * Split a markdown body into top-level blocks that can each be parsed independently and reassembled - * without changing meaning. Blank lines separate candidate groups (fenced code blocks stay atomic), - * then adjacent groups are merged back together whenever they could form one logical block: any - * indented (continuation) group, and consecutive list/blockquote groups (which would otherwise be a - * single loose list/quote). Merging is intentionally conservative — over-merging only yields a larger - * chunk, whereas under-merging would shatter a structure — and every block is parsed by - * `@tiptap/markdown`'s own lexer, so block boundaries always match the parser. + * (by `join('\n\n')`) without changing meaning. Blank lines separate candidate groups (fenced code + * blocks stay atomic), then adjacent groups are merged back together whenever they could form one + * logical block: any indented (continuation) group, and consecutive list/blockquote groups (which + * would otherwise be a single loose list/quote). Merging is intentionally conservative — over-merging + * only yields a larger chunk, whereas under-merging would shatter a structure — and every non-empty + * block is parsed by `@tiptap/markdown`'s own lexer, so block boundaries always match the parser. + * + * An EMPTY string in the result is a blank line the author left between two blocks — the exact inverse + * of the serializer's block join (see {@link emptyBlockCount}), so a document's deliberate spacing + * survives the round-trip instead of being silently dropped. {@link parseMarkdownToDoc} turns each into + * an empty paragraph; a run is bounded there by {@link clampEmptyParagraphs}. Gaps are measured before + * merging, so blank lines absorbed INTO a merged block (a loose list's own internal spacing) never + * become paragraphs — only gaps between the final top-level blocks do. Trailing blank lines carry + * nothing: the serializer collapses them to a single newline, so keeping them would never round-trip. * * The indent-merge rule is load-bearing for fenced code indented past 3 spaces (e.g. inside a list * item): {@link FENCE_OPEN} only tracks fences at the document margin, so a nested fence's interior @@ -67,10 +117,17 @@ export function splitMarkdownBlocks(body: string): string[] { // block (defeating the chunker). The editor normalizes `\r` on parse anyway, so meaning is unchanged. const lines = body.replace(/\r\n?/g, '\n').split('\n') const groups: string[] = [] + /** Blank lines immediately preceding `groups[i]`, parallel to it. */ + const gaps: number[] = [] + let blanks = 0 let current: string[] = [] let fence: string | null = null const flush = () => { - if (current.length > 0) groups.push(current.join('\n')) + if (current.length > 0) { + groups.push(current.join('\n')) + gaps.push(blanks) + blanks = 0 + } current = [] } for (const line of lines) { @@ -87,7 +144,9 @@ export function splitMarkdownBlocks(body: string): string[] { continue } if (line.trim() === '') { + // Flush BEFORE counting: `blanks` is the gap that preceded the group being closed here. flush() + blanks++ continue } current.push(line) @@ -97,18 +156,33 @@ export function splitMarkdownBlocks(body: string): string[] { // Build continuation runs and join each once — concatenating onto the growing block per group would be // O(n²) for one long loose list. A group continues the run when indented, or when its first line and the // group open the same marker kind (list or blockquote) — i.e. they form one loose list/quote. - const runs: string[][] = [] - for (const group of groups) { - const head = runs.length > 0 ? runs[runs.length - 1][0] : null + const runs: Array<{ empties: number; parts: string[] }> = [] + for (let index = 0; index < groups.length; index++) { + const group = groups[index] + const previous = runs.length > 0 ? runs[runs.length - 1] : null + const head = previous?.parts[0] ?? null + // A gap wide enough to carry an empty paragraph IS a top-level block boundary: the serializer only + // writes one by emitting the two sides as separate blocks, so merging across it swallowed the + // paragraph AND fused the two blocks (`- a` ∅ `- b` became one list, `> a` ∅ `> b` one quote, an + // indented continuation absorbed the gap). Parse then stopped inverting serialize, so the file never + // reached a fixpoint and silently opened READ-ONLY on the next open. + const empties = emptyBlockCount(gaps[index], index === 0) const continues = head !== null && + empties === 0 && (/^\s/.test(group) || (LIST_MARKER.test(head) && LIST_MARKER.test(group)) || (BLOCKQUOTE.test(head) && BLOCKQUOTE.test(group))) - if (continues) runs[runs.length - 1].push(group) - else runs.push([group]) + if (continues) previous?.parts.push(group) + else runs.push({ empties, parts: [group] }) } - return runs.map((run) => run.join('\n\n')) + + const blocks: string[] = [] + for (const run of runs) { + for (let n = run.empties; n > 0; n--) blocks.push('') + blocks.push(run.parts.join('\n\n')) + } + return blocks } /** @@ -121,11 +195,18 @@ export function splitMarkdownBlocks(body: string): string[] { * Documents whose constructs span blocks ({@link NON_CHUNKABLE}) parse whole, and any failure falls * back to a single whole-document parse, so correctness never depends on the splitter. * - * Runs of blank lines take the fast chunked path too: the chunker parses each block stripped of the - * blank lines between them, which drops the empty paragraphs `@tiptap/markdown` reconstructs from a - * blank run — exactly what {@link stripEmptyParagraphs} does to the whole-parse output anyway. A blank - * run between blocks is insignificant in markdown, so collapsing it is the intended normalization (see - * {@link stripEmptyParagraphs}), and both parse paths converge on the same empty-paragraph-free result. + * A blank line the author left between two blocks is part of the document, not noise: the chunker hands + * it back as an empty block (see {@link splitMarkdownBlocks}) and it becomes an empty paragraph here, so + * the editor renders the spacing that is actually in the file — on the very first paint, with no reflow + * once a collaborative doc settles. + * + * The whole-document path CANNOT do that. It hands blank runs to `@tiptap/markdown`, whose handling is + * not self-consistent (see {@link emptyBlockCount}), so a blank line there survives after a paragraph but + * is swallowed after a heading, an ordered list, or a table. Preserving it on only some of those would + * make parse stop inverting serialize for the same document — the file would never reach a fixpoint and + * would open read-only. So that path keeps NO empty paragraphs: consistently zero is a fixpoint, and a + * document whose spacing cannot be represented is better rendered the way every other markdown renderer + * shows it than rendered one way and saved another. */ export function parseMarkdownToDoc(body: string): JSONContent { const manager = markdownManager() @@ -133,22 +214,23 @@ export function parseMarkdownToDoc(body: string): JSONContent { // the chunker and parser do — a classic `\r`-only body would otherwise slip past the reference-def / // block-HTML guard and be chunked, shattering a construct that must parse whole. const normalized = body.replace(/\r\n?/g, '\n') - let doc: JSONContent - if (NON_CHUNKABLE.test(normalized)) { - doc = manager.parse(normalized) - } else { - try { - const content: JSONContent[] = [] - for (const block of splitMarkdownBlocks(normalized)) { - // `MarkdownManager.parse` always returns a doc node with a `content` array; spread its blocks. - content.push(...(manager.parse(block).content ?? [])) + if (NON_CHUNKABLE.test(normalized)) return boundEmptyParagraphs(manager.parse(normalized), 0) + try { + const content: JSONContent[] = [] + for (const block of splitMarkdownBlocks(normalized)) { + // An empty block is the chunker's marker for an authored blank line, and + // `MarkdownManager.parse('')` yields a doc with no blocks — so materialize the node directly. + if (block === '') { + content.push({ type: 'paragraph' }) + continue } - doc = { type: 'doc', content } - } catch { - doc = manager.parse(normalized) + // `MarkdownManager.parse` always returns a doc node with a `content` array; spread its blocks. + content.push(...(manager.parse(block).content ?? [])) } + return boundEmptyParagraphs({ type: 'doc', content }, MAX_EMPTY_PARAGRAPHS_PER_DOC) + } catch { + return boundEmptyParagraphs(manager.parse(normalized), 0) } - return stripEmptyParagraphs(doc) } /** An empty paragraph node — the shape a blank line reconstructs to (no content, or `content: []`). */ @@ -157,26 +239,72 @@ function isEmptyParagraph(node: JSONContent): boolean { } /** - * Drop ALL top-level empty paragraphs from a parsed doc — leading, interior, and trailing. In markdown - * a run of blank lines between blocks is insignificant (CommonMark collapses it), so `@tiptap/markdown` - * reconstructing each blank as an empty paragraph node is not fidelity: it makes the editor render the - * file differently from every standard renderer (GitHub, the download, our own static preview), and a - * pathological blank run (an agent/paste artifact) explodes into thousands of empty nodes that persist - * forever and reflow the doc on open. Collapsing them here keeps normal single-blank-line block spacing - * while removing the spurious gaps, and stays idempotent so the round-trip-safety probe still passes: a - * doc parsed this way has no empty paragraphs, so re-serializing it never re-emits an interior blank run - * (the serializer is intentionally left alone — a blank line inside a fenced code block IS significant), - * and a second parse is a fixed point. Only TOP-LEVEL paragraphs are touched, so blank lines that carry - * meaning inside a construct (e.g. a loose list) are left to the block parser. TipTap re-adds its own - * trailing filler paragraph on `setContent`, so the editor still has a place to type. + * Bound the top-level empty paragraphs of a parsed doc to `budget` in total, and drop trailing ones + * entirely. `budget` is 0 for the whole-document path, which cannot represent them at all. + * + * The per-gap ceiling in {@link emptyBlockCount} bounds one run; this bounds the DOCUMENT. Without it the + * ceiling buys nothing against the shape a real artifact takes — an export that puts a moderate blank run + * between every paragraph, rather than one giant run. Measured before this budget existed: an 86KB body + * of `x` + 42 newlines produced 39,980 empty paragraphs, twenty times the incident the ceiling cites. + * + * Trailing empties cannot round-trip — `postProcessSerializedMarkdown` collapses trailing blank lines to + * a single newline, so a trailing empty paragraph would be re-serialized away and the doc would differ + * from its own output, flipping the file read-only. Dropping them here is what keeps the probe stable + * (TipTap re-adds its own trailing filler paragraph on `setContent`, so there is still somewhere to + * type). Interior and leading empties DO round-trip exactly, so they are kept. + * + * Only TOP-LEVEL paragraphs are considered — blank lines that carry meaning inside a construct (a loose + * list, a blockquote) live below the doc root and belong to the block parser. Returns the doc untouched, + * with no array copy, when nothing needs bounding (the overwhelmingly common case). */ -function stripEmptyParagraphs(doc: JSONContent): JSONContent { +function boundEmptyParagraphs(doc: JSONContent, budget: number): JSONContent { const content = doc.content if (!content || content.length === 0) return doc - // The dominant (chunked) parse already emits no top-level empty paragraphs, so scan before allocating: - // return the doc untouched — no array copy — unless there is actually something to strip. + // Most documents carry no empty paragraph at all, so scan before allocating anything. if (!content.some(isEmptyParagraph)) return doc - return { ...doc, content: content.filter((node) => !isEmptyParagraph(node)) } + let end = content.length + while (end > 0 && isEmptyParagraph(content[end - 1])) end-- + const kept: JSONContent[] = [] + let remaining = budget + for (let index = 0; index < end; index++) { + const node = content[index] + if (!isEmptyParagraph(node)) { + kept.push(node) + continue + } + if (remaining > 0) { + remaining-- + kept.push(node) + } + } + return kept.length === content.length ? doc : { ...doc, content: kept } +} + +/** + * The markdown parse in the form the EDITOR settles on — the only shape that may enter the shared + * document. + * + * ProseMirror appends an empty paragraph to any document that does not end in one, so a parse ending on + * a list, heading, table, or rule is NOT what a bound editor holds. Seeding the CRDT with the + * un-normalized shape means the first client to bind writes that paragraph back into the SHARED + * document — and because a trailing blank line does not survive serialization + * (`postProcessSerializedMarkdown` collapses it) the file never records it, so nothing reconciles the + * two and a client that seeds without seeing another's contribution adds one more. Measured on a + * heavily-reopened document: 18 stacked empty paragraphs in the live doc against the placeholder's 1 — + * the pane growing several hundred pixels the instant the live editor took over. + * + * Opt-in rather than folded into {@link parseMarkdownToDoc}, because only a writer to the SHARED + * document has to agree with the editor. Every other consumer of the parse (paste, the round-trip + * probe, the read-only placeholder) is rendered through a real editor that applies this itself, and + * baking it into the parse changes what those surfaces assert. Every CRDT writer — the seed, the agent + * merge, and the streaming frame reconciler — must go through here, or the one that does not silently + * removes what the others add. + */ +export function editorNormalForm(markdown: string): JSONContent { + const json = parseMarkdownToDoc(markdown) + const content = json.content ?? [] + if (content[content.length - 1]?.type === 'paragraph') return json + return { ...json, content: [...content, { type: 'paragraph' }] } } /** diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx index 4bc6285eedb..0a53b21cce5 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx @@ -721,6 +721,15 @@ export function LoadedRichMarkdownEditor({ * is latched, so a fatal rejection that fired before this subscription is not missed. */ useEffect(() => { + /** + * Readiness is a protocol fact, never a timing guess: the relay attaches a client only once its + * room holds the whole document (it awaits the shared-stream catch-up and the server seed before + * answering a join), so a completed sync IS the finished document and revealing on it cannot show + * an intermediate state. This deliberately does NOT wait for the document to "stop moving" — a + * quiet-frame gate was tried and it is unsound in both directions: it delays the reveal of a + * document that was already correct, and it opens mid-flight anyway whenever the updates arrive + * more than a frame apart (which is what a remote Redis and a long room history produce). + */ const setReady = (ready: boolean) => { // Child-local: gates editability (a user must never type into an unsynced/unseeded doc). setCollabReady(ready) @@ -766,12 +775,21 @@ export function LoadedRichMarkdownEditor({ const report = () => { const synced = provider.synced const seeded = config.get(FILE_DOC_SEED.flag) === true - const next = nextCollabReadiness(syncedOnce, { synced, seeded, offlineSeed }) + // `joinError` is latched ONLY on the provider's fatal paths (non-retryable rejection, access + // revocation, readiness deadline), so it is exactly "this document is abandoned". + const fatal = provider.joinError !== null + const next = nextCollabReadiness(syncedOnce, { synced, seeded, offlineSeed, fatal }) syncedOnce = next.syncedOnce setReady(next.ready) } + /** + * Re-report unconditionally, not just when the fallback seeds. A fatal that arrives on an ALREADY + * seeded doc (access revoked mid-session) leaves `seedFromLoaded` a no-op, so nothing else would + * fire an observer and the editor would stay editable on a document the provider has abandoned. + */ const onJoinError = (error: JoinFileDocError) => { if (error.retryable === false) seedFromLoaded() + report() } // A server edit that changes ONLY the frontmatter (e.g. copilot) updates the config map but not diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts index f8dac76bbdd..db6206c43b1 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts @@ -178,7 +178,15 @@ export function useEditableFileContent({ file.id, file.key, GENERATED_SOURCE_FILE_TYPES.has(file.type), - { refetchInterval: reconcileRefetchInterval } + { + refetchInterval: reconcileRefetchInterval, + // `canAutosave: false` on this surface means a server-side owner holds durability — the + // collaborative relay, which projects the live document to markdown itself and merges + // external writes INTO that document. There is nothing a focus refetch of the durable bytes + // can teach the editor that the shared document does not already have; all it does is + // re-read a storage key the relay's last save has already rotated away from. + refetchOnWindowFocus: canAutosave, + } ) /** diff --git a/apps/sim/app/workspace/[workspaceId]/files/page.tsx b/apps/sim/app/workspace/[workspaceId]/files/page.tsx index 389b4d17ded..11dc9fd450e 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/page.tsx @@ -23,9 +23,7 @@ export default async function FilesPage({ params }: { params: Promise<{ workspac const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - if (session?.user?.id) { - await prefetchFilesBrowser(queryClient, workspaceId, session.user.id) - } + await prefetchFilesBrowser(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts index 250a6e5f713..6ed88de422d 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts @@ -1,53 +1,48 @@ import type { QueryClient } from '@tanstack/react-query' import { listWorkspaceFileFolders } from '@/lib/uploads/contexts/workspace/workspace-file-folder-manager' -import { listWorkspaceFilesWithShares } from '@/lib/workspace-files/queries' import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' import { WORKSPACE_FILE_FOLDERS_STALE_TIME, workspaceFileFolderKeys, } from '@/hooks/queries/workspace-file-folders' -import { - WORKSPACE_FILES_LIST_STALE_TIME, - workspaceFilesKeys, -} from '@/hooks/queries/workspace-files' /** - * Prefetches everything the Files browser needs to paint a complete, correctly-ordered - * first frame: workspace files, file folders, and (via {@link prefetchResourceListChrome}) - * the pinned ids that drive row order plus the members behind the Owner column — - * under the same query keys their client hooks (`useWorkspaceFiles`, - * `useWorkspaceFileFolders`) use (scope `active`), so the browser paints - * populated on first render. + * Prefetches what the Files browser needs on top of the workspace layout's own prefetch, so the + * first frame is complete and correctly ordered: file folders, and (via + * {@link prefetchResourceListChrome}) the pinned ids that drive row order plus the members behind + * the Owner column — under the same query keys their client hooks (`useWorkspaceFileFolders`) use + * (scope `active`), so the browser paints populated on first render. + * + * The FILE LIST itself is deliberately not here: the sidebar reads it on every workspace route, so + * it is prefetched by `prefetchWorkspaceSidebar` in the layout — the only boundary that renders + * before the sidebar registers the query. Prefetching it again here would re-read it per request + * and still not reach the server render (`HydrationBoundary` defers an already-seen query to an + * effect, which SSR never runs). See the note on that entry. * - * Files and folders read the data layer; both payloads are shaped to their route contract so - * a hydrated entry matches a client fetch. Everything else still goes through its route — - * see {@link prefetchInternalJson}. + * Folders and the chrome reads all go through the data layer, shaped to their route contracts so a + * hydrated entry matches a client fetch. * - * Those two reads carry no authorization of their own, so the viewer is proved first. This - * reuses the layout's `cache`d host-context lookup rather than re-deriving the permission, - * so it costs no additional queries; a viewer without access caches nothing and the client - * fetch reaches the route for the real 403. + * That read carries no authorization of its own, so the viewer is proved first. This reuses the + * layout's `cache`d host-context lookup rather than re-deriving the permission, so it costs no + * additional queries; a viewer without access caches nothing and the client fetch reaches the + * route for the real 403. */ export async function prefetchFilesBrowser( queryClient: QueryClient, workspaceId: string, - userId: string + userId: string | undefined ): Promise { + if (!userId) return const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) if (!hostContext) return await Promise.all([ - queryClient.prefetchQuery({ - queryKey: workspaceFilesKeys.list(workspaceId, 'active'), - queryFn: () => listWorkspaceFilesWithShares(workspaceId, 'active'), - staleTime: WORKSPACE_FILES_LIST_STALE_TIME, - }), queryClient.prefetchQuery({ queryKey: workspaceFileFolderKeys.list(workspaceId, 'active'), queryFn: () => listWorkspaceFileFolders(workspaceId, { scope: 'active' }), staleTime: WORKSPACE_FILE_FOLDERS_STALE_TIME, }), - prefetchResourceListChrome(queryClient, workspaceId, 'file'), + prefetchResourceListChrome(queryClient, workspaceId, 'file', userId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/home/page.tsx b/apps/sim/app/workspace/[workspaceId]/home/page.tsx index b7a6cc4ea95..cfb87f8ce04 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/home/page.tsx @@ -1,11 +1,8 @@ import { Suspense } from 'react' -import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' import { redirect } from 'next/navigation' import { getSession } from '@/lib/auth' import { isChatEnabled } from '@/lib/core/config/env-flags' -import { getQueryClient } from '@/app/_shell/providers/get-query-client' -import { prefetchHomeLists } from '@/app/workspace/[workspaceId]/home/prefetch' import { resolveTableViewsEnabled } from '@/app/workspace/[workspaceId]/home/resolve-table-views-flag' import { Home } from './home' import { HomeFallback } from './home-fallback' @@ -23,23 +20,19 @@ export default async function HomePage({ params }: { params: Promise<{ workspace redirect(`/workspace/${workspaceId}`) } - const queryClient = getQueryClient() - const listsPrefetch = prefetchHomeLists(queryClient, workspaceId) - + /** + * Home prefetches nothing of its own. Both lists it reads — workflow folders and + * the workspace file list — are hydrated by `prefetchWorkspaceSidebar` in the + * layout under the same keys, and re-reading them here would cost a second query + * per request without reaching the server render. + */ const session = await getSession() const userId = session?.user?.id const tableViewsEnabled = await resolveTableViewsEnabled(workspaceId, userId) - await listsPrefetch return ( - - }> - - - + }> + + ) } diff --git a/apps/sim/app/workspace/[workspaceId]/home/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/home/prefetch.ts deleted file mode 100644 index f08791c0bbd..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/home/prefetch.ts +++ /dev/null @@ -1,50 +0,0 @@ -import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts' -import type { ListWorkspaceFilesResponse } from '@/lib/api/contracts/workspace-files' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' -import { - WORKSPACE_FILES_LIST_STALE_TIME, - workspaceFilesKeys, -} from '@/hooks/queries/workspace-files' - -/** - * Prefetches the home page's secondary lists — folders and workspace files — - * under the same query keys their client hooks (`useFolders`, - * `useWorkspaceFiles`) use, so the home view paints populated on first render. - * - * The workflow list (`workflowKeys.list(ws, 'active')`) is already hydrated by - * the workspace sidebar prefetch and is intentionally not repeated here. - * - * Folders are fetched through the route and mapped with the same `mapFolder` - * the hook applies, matching its cached shape (string dates → `Date`). Files - * carry `Date` fields, so they go through the route and cache the serialized - * wire shape — see {@link prefetchInternalJson}. - */ -export async function prefetchHomeLists( - queryClient: QueryClient, - workspaceId: string -): Promise { - await Promise.all([ - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'workflow'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=workflow` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - queryClient.prefetchQuery({ - queryKey: workspaceFilesKeys.list(workspaceId, 'active'), - queryFn: async () => { - const data = await prefetchInternalJson( - `/api/workspaces/${workspaceId}/files?scope=active` - ) - return data.success ? data.files : [] - }, - staleTime: WORKSPACE_FILES_LIST_STALE_TIME, - }), - ]) -} diff --git a/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx b/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx index 48b7934bb13..8bef3b5b32f 100644 --- a/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx @@ -1,6 +1,7 @@ import { Suspense } from 'react' import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' import { getQueryClient } from '@/app/_shell/providers/get-query-client' import { FOLDERED_RESOURCE_HEADERS } from '@/app/workspace/[workspaceId]/components/folders/foldered-resources' import { Knowledge } from '@/app/workspace/[workspaceId]/knowledge/knowledge' @@ -24,10 +25,9 @@ export default async function KnowledgePage({ }: { params: Promise<{ workspaceId: string }> }) { - const { workspaceId } = await params - + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - await prefetchKnowledgeBases(queryClient, workspaceId) + await prefetchKnowledgeBases(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts index 7c9d45cb668..7aad80a5bca 100644 --- a/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts @@ -1,9 +1,10 @@ import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts/folders' -import type { KnowledgeBaseData } from '@/lib/api/contracts/knowledge' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import { listKnowledgeBasesContract } from '@/lib/api/contracts/knowledge' +import { internalSessionAuth } from '@/lib/api/server/routes' +import { internalKnowledgePresenters } from '@/lib/knowledge/api/internal-route' +import { listInternalKnowledgeBases } from '@/lib/knowledge/application/knowledge-bases' +import { prefetchResourceFolders } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-folders' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' import { KNOWLEDGE_BASE_LIST_STALE_TIME, knowledgeKeys } from '@/hooks/queries/utils/knowledge-keys' /** @@ -16,35 +17,43 @@ import { KNOWLEDGE_BASE_LIST_STALE_TIME, knowledgeKeys } from '@/hooks/queries/u * beside, so prefetching one without the other still flashes an ungrouped list — and a * `?folderId=` deep link renders an empty breadcrumb until the folders arrive. * - * The list carries `Date` fields, so it goes through the `/api/knowledge` route and caches the - * serialized wire shape — see {@link prefetchInternalJson}. Folders are mapped with the same - * `mapFolder` the hook applies, so the hydrated entry matches a client fetch exactly. + * The bases list runs the same `listInternalKnowledgeBases` application use case + * `GET /api/knowledge` runs, authenticated with the same `internalSessionAuth` policy, and is + * projected through the same `internalKnowledgePresenters.list` presenter and the contract's + * response schema. Nothing about authorization moves here: the use case still loads the + * canonical workspace context and authorizes the session principal against + * `knowledgeOperations.list`. An unauthenticated or unauthorized viewer throws inside the + * query function, which caches nothing and leaves the client fetch to reach the route for the + * real 401/403. + * + * Folders read the data layer and are mapped with the same `mapFolder` the hook applies, + * matching the workspace sidebar prefetch. That read carries no authorization of its own, so + * the viewer is proved first; `getWorkspaceHostContextForViewer` is `cache`d and the layout has + * already resolved it for this request, so it costs no additional queries. */ export async function prefetchKnowledgeBases( queryClient: QueryClient, - workspaceId: string + workspaceId: string, + userId: string | undefined ): Promise { + if (!userId) return + await Promise.all([ queryClient.prefetchQuery({ queryKey: knowledgeKeys.list(workspaceId, 'active'), queryFn: async () => { - const result = await prefetchInternalJson<{ data: KnowledgeBaseData[] }>( - `/api/knowledge?workspaceId=${workspaceId}&scope=active` - ) - return result.data + const principal = await internalSessionAuth.authenticate() + const result = await listInternalKnowledgeBases.execute({ + principal, + input: { workspaceId, scope: 'active' }, + }) + return listKnowledgeBasesContract.response.schema.parse( + internalKnowledgePresenters.list(result) + ).data }, staleTime: KNOWLEDGE_BASE_LIST_STALE_TIME, }), - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'knowledge_base'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=knowledge_base` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - prefetchResourceListChrome(queryClient, workspaceId, 'knowledge_base'), + prefetchResourceFolders(queryClient, workspaceId, 'knowledge_base', userId), + prefetchResourceListChrome(queryClient, workspaceId, 'knowledge_base', userId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts deleted file mode 100644 index 4ba194395e6..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts +++ /dev/null @@ -1,25 +0,0 @@ -import { headers } from 'next/headers' -import { getInternalApiBaseUrl } from '@/lib/core/utils/urls' - -/** - * Server-side GET against an internal `/api` route, forwarding the incoming - * request's cookie so the route authenticates as the current user. - * - * The legacy path. Reading the data layer and shaping the result through the - * route's response contract — as `files/prefetch.ts` does — is canonical: it - * drops a server-to-server request and its duplicate auth, and the contract - * parse is what guarantees the hydrated entry matches a client fetch. Prefetches - * still on this helper have not been converted; a converted one must prove the - * viewer itself, since the route's own authorization no longer runs. - */ -export async function prefetchInternalJson(path: string): Promise { - const cookie = (await headers()).get('cookie') - // boundary-raw-fetch: server-side RSC prefetch forwarding the session cookie to an internal API route; requestJson is client-only and cannot run here - const response = await fetch(`${getInternalApiBaseUrl()}${path}`, { - headers: cookie ? { cookie } : {}, - }) - if (!response.ok) { - throw new Error(`Prefetch failed for ${path}: ${response.status}`) - } - return response.json() as Promise -} diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-folders.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-folders.ts new file mode 100644 index 00000000000..ecd08257221 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-folders.ts @@ -0,0 +1,37 @@ +import type { QueryClient } from '@tanstack/react-query' +import type { FolderResourceType } from '@/lib/api/contracts/folders' +import { listFoldersForWorkspace } from '@/lib/folders/queries' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' + +/** + * Prefetches one resource family's folder tree under the same key its client + * `useFolders` hook reads, mapped with the same `mapFolder` the hook applies so a + * hydrated entry matches a client fetch. + * + * Shared by the resource list pages so the key, stale time, and mapper cannot + * drift apart across them. Self-guarding like {@link prefetchResourceListChrome}: + * the read carries no authorization of its own, so an unproven viewer caches + * nothing and their client fetch reaches the route for the real 403. + * `getWorkspaceHostContextForViewer` is `cache`d and the layout has already + * resolved it for this request, so the proof costs no additional queries. + */ +export async function prefetchResourceFolders( + queryClient: QueryClient, + workspaceId: string, + resourceType: FolderResourceType, + userId: string | undefined +): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + + await queryClient.prefetchQuery({ + queryKey: folderKeys.list(workspaceId, 'active', resourceType), + queryFn: async () => { + const rows = await listFoldersForWorkspace(workspaceId, 'active', resourceType) + return rows.map(mapFolder) + }, + staleTime: FOLDER_LIST_STALE_TIME, + }) +} diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts index 5d9241aa23f..96b324b9d86 100644 --- a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts +++ b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts @@ -1,12 +1,10 @@ import type { QueryClient } from '@tanstack/react-query' -import type { PinnedItemApi, PinnedResourceType } from '@/lib/api/contracts/pinned-items' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import type { PinnedResourceType } from '@/lib/api/contracts/pinned-items' +import { listPinnedItemsForUser } from '@/lib/pinned-items/queries' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { getWorkspaceMemberProfiles } from '@/lib/workspaces/permissions/utils' import { PINNED_ITEMS_STALE_TIME, pinnedItemKeys } from '@/hooks/queries/utils/pinned-item-keys' -import { - WORKSPACE_MEMBERS_STALE_TIME, - type WorkspaceMember, - workspaceKeys, -} from '@/hooks/queries/workspace' +import { WORKSPACE_MEMBERS_STALE_TIME, workspaceKeys } from '@/hooks/queries/workspace' /** * Prefetches the two lists every foldered resource page needs to paint a row completely, @@ -19,21 +17,28 @@ import { * * Members back the Owner column; without them every owner cell paints empty and fills in * after. Both are cheap and shared with the page's own list prefetch in one `Promise.all`. + * + * Both read the data layer through the same functions their routes call, so a hydrated entry + * matches what a client fetch would parse out of the response. Neither read carries + * authorization of its own, so the viewer is proved first — `getWorkspaceHostContextForViewer` + * is `cache`d and the layout has already resolved it for this request, so it costs no + * additional queries. A viewer without access caches nothing and the client fetch reaches the + * route for the real 403. */ export async function prefetchResourceListChrome( queryClient: QueryClient, workspaceId: string, - resourceType: PinnedResourceType + resourceType: PinnedResourceType, + userId: string | undefined ): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + const prefetchPinned = (type: PinnedResourceType) => queryClient.prefetchQuery({ queryKey: pinnedItemKeys.list(workspaceId, type), - queryFn: async () => { - const { pinnedItems } = await prefetchInternalJson<{ pinnedItems: PinnedItemApi[] }>( - `/api/pinned-items?workspaceId=${workspaceId}&resourceType=${type}` - ) - return pinnedItems - }, + queryFn: () => listPinnedItemsForUser(userId, workspaceId, type), staleTime: PINNED_ITEMS_STALE_TIME, }) @@ -42,12 +47,7 @@ export async function prefetchResourceListChrome( prefetchPinned('folder'), queryClient.prefetchQuery({ queryKey: workspaceKeys.members(workspaceId), - queryFn: async () => { - const { members } = await prefetchInternalJson<{ members: WorkspaceMember[] }>( - `/api/workspaces/${workspaceId}/members` - ) - return members - }, + queryFn: () => getWorkspaceMemberProfiles(workspaceId), staleTime: WORKSPACE_MEMBERS_STALE_TIME, }), ]) diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts index 7d701d22066..dee79520936 100644 --- a/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts @@ -5,29 +5,90 @@ import { QueryClient } from '@tanstack/react-query' import { beforeEach, describe, expect, it, vi } from 'vitest' const { + mockAuthenticate, mockGetWorkspaceHostContextForViewer, + mockGetWorkspaceMemberProfiles, + mockKnowledgePresenterList, + mockListFoldersForWorkspace, + mockListInternalKnowledgeBases, + mockListPinnedItemsForUser, + mockListWorkflowsForUser, + mockListWorkspacesForViewer, + mockGetUserProfile, + mockGetWorkspacePermissions, + mockListMothershipChats, + mockListTables, mockListWorkspaceFileFolders, mockListWorkspaceFilesWithShares, - mockPrefetchInternalJson, } = vi.hoisted(() => ({ + mockAuthenticate: vi.fn(), mockGetWorkspaceHostContextForViewer: vi.fn(), + mockGetWorkspaceMemberProfiles: vi.fn(), + mockKnowledgePresenterList: vi.fn(), + mockListFoldersForWorkspace: vi.fn(), + mockListInternalKnowledgeBases: vi.fn(), + mockListPinnedItemsForUser: vi.fn(), + mockListWorkflowsForUser: vi.fn(), + mockListWorkspacesForViewer: vi.fn(), + mockGetUserProfile: vi.fn(), + mockGetWorkspacePermissions: vi.fn(), + mockListMothershipChats: vi.fn(), + mockListTables: vi.fn(), mockListWorkspaceFileFolders: vi.fn(), mockListWorkspaceFilesWithShares: vi.fn(), - mockPrefetchInternalJson: vi.fn(), })) vi.mock('@/lib/workspaces/host-context', () => ({ getWorkspaceHostContextForViewer: mockGetWorkspaceHostContextForViewer, })) +vi.mock('@/lib/folders/queries', () => ({ + listFoldersForWorkspace: mockListFoldersForWorkspace, +})) vi.mock('@/lib/workspace-files/queries', () => ({ listWorkspaceFilesWithShares: mockListWorkspaceFilesWithShares, })) vi.mock('@/lib/uploads/contexts/workspace/workspace-file-folder-manager', () => ({ listWorkspaceFileFolders: mockListWorkspaceFileFolders, })) - -vi.mock('@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch', () => ({ - prefetchInternalJson: mockPrefetchInternalJson, +vi.mock('@/lib/pinned-items/queries', () => ({ + listPinnedItemsForUser: mockListPinnedItemsForUser, +})) +vi.mock('@/lib/workspaces/permissions/utils', () => ({ + getWorkspaceMemberProfiles: mockGetWorkspaceMemberProfiles, + getWorkspacePermissionsForAuthorizedViewer: mockGetWorkspacePermissions, +})) +vi.mock('@/lib/workflows/queries', () => ({ + listWorkflowsForUser: mockListWorkflowsForUser, +})) +vi.mock('@/lib/workspaces/list', () => ({ + listWorkspacesForViewer: mockListWorkspacesForViewer, +})) +vi.mock('@/lib/users/queries', () => ({ + getUserProfile: mockGetUserProfile, +})) +vi.mock('@/lib/copilot/chat/list-mothership-chats', () => ({ + listMothershipChats: mockListMothershipChats, +})) +vi.mock('@/lib/table/service', () => ({ + listTables: mockListTables, +})) +/** + * `typeMetadataOf` is the one leaf of the real wire projection that reaches the + * column-type registry, and through it every type module's icon and editor. Stub + * that leaf only, so `toTableListItem`'s timestamp, `metadata`, and job + * normalization stay under test rather than being mocked away wholesale. + */ +vi.mock('@/lib/table/column-types', () => ({ + typeMetadataOf: () => ({}), +})) +vi.mock('@/lib/api/server/routes', () => ({ + internalSessionAuth: { authenticate: mockAuthenticate }, +})) +vi.mock('@/lib/knowledge/application/knowledge-bases', () => ({ + listInternalKnowledgeBases: { execute: mockListInternalKnowledgeBases }, +})) +vi.mock('@/lib/knowledge/api/internal-route', () => ({ + internalKnowledgePresenters: { list: mockKnowledgePresenterList }, })) vi.mock('@sim/emcn', () => ({ @@ -35,8 +96,8 @@ vi.mock('@sim/emcn', () => ({ })) import { prefetchFilesBrowser } from '@/app/workspace/[workspaceId]/files/prefetch' -import { prefetchHomeLists } from '@/app/workspace/[workspaceId]/home/prefetch' import { prefetchKnowledgeBases } from '@/app/workspace/[workspaceId]/knowledge/prefetch' +import { prefetchWorkspaceSidebar } from '@/app/workspace/[workspaceId]/prefetch' import { prefetchTables } from '@/app/workspace/[workspaceId]/tables/prefetch' import { folderKeys } from '@/hooks/queries/utils/folder-keys' import { knowledgeKeys } from '@/hooks/queries/utils/knowledge-keys' @@ -57,58 +118,207 @@ describe('workspace list prefetches', () => { beforeEach(() => { vi.clearAllMocks() mockGetWorkspaceHostContextForViewer.mockResolvedValue({ viewer: { permission: 'admin' } }) + mockListFoldersForWorkspace.mockResolvedValue([]) mockListWorkspaceFilesWithShares.mockResolvedValue([]) mockListWorkspaceFileFolders.mockResolvedValue([]) + mockListPinnedItemsForUser.mockResolvedValue([]) + mockListWorkflowsForUser.mockResolvedValue([]) + mockGetUserProfile.mockResolvedValue({ id: USER_ID, name: 'Ada', email: 'a@b.c' }) + mockGetWorkspacePermissions.mockResolvedValue({ users: [] }) + mockListMothershipChats.mockResolvedValue([]) + mockListWorkspacesForViewer.mockResolvedValue({ + workspaces: [], + lastActiveWorkspaceId: null, + pinnedWorkspaceIds: [], + creationPolicy: null, + }) + mockGetWorkspaceMemberProfiles.mockResolvedValue([]) + mockListTables.mockResolvedValue([]) + mockAuthenticate.mockResolvedValue({ kind: 'session', userId: USER_ID, sessionId: 'sess-1' }) + mockListInternalKnowledgeBases.mockResolvedValue({ knowledgeBases: [] }) + mockKnowledgePresenterList.mockReturnValue({ success: true, data: [] }) + }) + + describe.each([ + { + name: 'prefetchKnowledgeBases', + run: (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID), + resourceType: 'knowledge_base' as const, + }, + { + name: 'prefetchTables', + run: (client: QueryClient) => prefetchTables(client, WORKSPACE_ID, USER_ID), + resourceType: 'table' as const, + }, + ])('$name folder reads', ({ run, resourceType }) => { + it('reads folders from the data layer rather than over the wire', async () => { + const folderRow = { + id: 'fld-1', + name: 'Folder', + userId: 'u-1', + workspaceId: WORKSPACE_ID, + parentId: null, + resourceType, + locked: false, + sortOrder: 0, + createdAt: '2026-01-01T00:00:00.000Z', + updatedAt: '2026-01-02T00:00:00.000Z', + deletedAt: null, + } + mockListFoldersForWorkspace.mockResolvedValue([folderRow]) + const client = makeClient() + + await run(client) + + expect(mockListFoldersForWorkspace).toHaveBeenCalledWith(WORKSPACE_ID, 'active', resourceType) + const cached = client.getQueryData( + folderKeys.list(WORKSPACE_ID, 'active', resourceType) + ) as Array<{ + resourceType: string + createdAt: Date + }> + expect(cached).toHaveLength(1) + expect(cached[0].resourceType).toBe(resourceType) + expect(cached[0].createdAt).toBeInstanceOf(Date) + }) + + it('skips the folder read when the viewer cannot be proved', async () => { + mockGetWorkspaceHostContextForViewer.mockResolvedValue(null) + const client = makeClient() + + await run(client) + + expect(mockListFoldersForWorkspace).not.toHaveBeenCalled() + expect( + client.getQueryData(folderKeys.list(WORKSPACE_ID, 'active', resourceType)) + ).toBeUndefined() + }) }) describe('prefetchKnowledgeBases', () => { - it('primes the exact key useKnowledgeBasesQuery reads and unwraps data', async () => { - const bases = [{ id: 'kb-1' }] - mockPrefetchInternalJson.mockResolvedValue({ data: bases }) + /** + * The bases list is a protected read behind an application operation, so the prefetch runs + * the same use case the route declares, with a principal from the same auth policy — + * rather than reaching past it to a manager. + */ + it('runs the route’s own use case with a session principal', async () => { const client = makeClient() - await prefetchKnowledgeBases(client, WORKSPACE_ID) + await prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/knowledge?workspaceId=${WORKSPACE_ID}&scope=active` - ) - expect(client.getQueryData(knowledgeKeys.list(WORKSPACE_ID, 'active'))).toEqual(bases) + expect(mockAuthenticate).toHaveBeenCalled() + expect(mockListInternalKnowledgeBases).toHaveBeenCalledWith({ + principal: { kind: 'session', userId: USER_ID, sessionId: 'sess-1' }, + input: { workspaceId: WORKSPACE_ID, scope: 'active' }, + }) + expect(client.getQueryData(knowledgeKeys.list(WORKSPACE_ID, 'active'))).toEqual([]) + }) + + it('caches nothing when the session principal cannot be built', async () => { + mockAuthenticate.mockRejectedValue(new Error('Unauthorized')) + const client = makeClient() + + await prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID) + + expect(mockListInternalKnowledgeBases).not.toHaveBeenCalled() + expect(client.getQueryData(knowledgeKeys.list(WORKSPACE_ID, 'active'))).toBeUndefined() }) }) describe('prefetchTables', () => { - it('primes the exact key useTablesList reads and unwraps data.tables', async () => { - const tables = [{ id: 't-1' }] - mockPrefetchInternalJson.mockResolvedValue({ data: { tables } }) + const TABLE_ROW = { + id: 't-1', + name: 'people', + description: null, + schema: { columns: [{ id: 'c1', name: 'name', type: 'string' }] }, + metadata: { columnWidths: { c1: 120 } }, + rowCount: 3, + maxRows: 10_000, + workspaceId: WORKSPACE_ID, + folderId: null, + createdBy: 'u-1', + locks: { + schemaLocked: false, + insertLocked: false, + updateLocked: false, + deleteLocked: false, + }, + archivedAt: null, + createdAt: new Date('2026-01-01T00:00:00.000Z'), + updatedAt: new Date('2026-01-02T00:00:00.000Z'), + } + + it('reads tables from the data layer', async () => { + mockListTables.mockResolvedValue([TABLE_ROW]) const client = makeClient() - await prefetchTables(client, WORKSPACE_ID) + await prefetchTables(client, WORKSPACE_ID, USER_ID) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/table?workspaceId=${WORKSPACE_ID}&scope=active` - ) - expect(client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active'))).toEqual(tables) + expect(mockListTables).toHaveBeenCalledWith(WORKSPACE_ID, { scope: 'active' }) }) - }) + /** + * `listTablesContract`'s response schema is a passthrough, so a client fetch caches the + * route's JSON verbatim. Seeding the raw data-layer row would put `Date`s and the + * server-only `metadata` field under a key the hook never sees them on. + */ + it('seeds the wire shape a client fetch caches, not the raw data-layer row', async () => { + mockListTables.mockResolvedValue([TABLE_ROW]) + const client = makeClient() + + await prefetchTables(client, WORKSPACE_ID, USER_ID) + + const [cached] = client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active')) as Array< + Record + > + expect(cached.createdAt).toBe('2026-01-01T00:00:00.000Z') + expect(cached.updatedAt).toBe('2026-01-02T00:00:00.000Z') + expect(cached.archivedAt).toBeNull() + expect(cached).not.toHaveProperty('metadata') + expect(cached.jobStatus).toBeNull() + expect(cached.jobRowsProcessed).toBe(0) + }) + + it('caches no tables when the viewer cannot be proved', async () => { + mockGetWorkspaceHostContextForViewer.mockResolvedValue(null) + const client = makeClient() + + await prefetchTables(client, WORKSPACE_ID, USER_ID) + + expect(mockListTables).not.toHaveBeenCalled() + expect(client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active'))).toBeUndefined() + }) + }) describe('prefetchFilesBrowser', () => { - it('primes both file + folder keys the client hooks read', async () => { - const files = [{ id: 'f-1' }] + it('primes the folder key the client hook reads', async () => { const folders = [{ id: 'folder-1' }] - mockListWorkspaceFilesWithShares.mockResolvedValue(files) mockListWorkspaceFileFolders.mockResolvedValue(folders) const client = makeClient() await prefetchFilesBrowser(client, WORKSPACE_ID, USER_ID) - expect(mockListWorkspaceFilesWithShares).toHaveBeenCalledWith(WORKSPACE_ID, 'active') expect(mockListWorkspaceFileFolders).toHaveBeenCalledWith(WORKSPACE_ID, { scope: 'active' }) - expect(client.getQueryData(workspaceFilesKeys.list(WORKSPACE_ID, 'active'))).toEqual(files) expect(client.getQueryData(workspaceFileFolderKeys.list(WORKSPACE_ID, 'active'))).toEqual( folders ) }) + /** + * The FILE LIST is deliberately not primed here — `prefetchWorkspaceSidebar` owns it, because the + * sidebar reads that query on every workspace route and therefore registers it before any page + * renders. `HydrationBoundary` hands an already-seen query to a `useEffect`, which SSR never runs, + * so a page-level prefetch of this key costs a request per render and still cannot reach the server + * render. Restoring it here would reintroduce exactly that. + */ + it('leaves the file list to the layout rather than re-reading it per page', async () => { + const client = makeClient() + + await prefetchFilesBrowser(client, WORKSPACE_ID, USER_ID) + + expect(mockListWorkspaceFilesWithShares).not.toHaveBeenCalled() + expect(client.getQueryData(workspaceFilesKeys.list(WORKSPACE_ID, 'active'))).toBeUndefined() + }) + /** * The reads bypass the route that used to authorize them, so a viewer without workspace * access must prime nothing and let the client fetch reach the route for the real 403. @@ -120,7 +330,7 @@ describe('workspace list prefetches', () => { await prefetchFilesBrowser(client, WORKSPACE_ID, USER_ID) expect(client.getQueryCache().getAll()).toHaveLength(0) - expect(mockListWorkspaceFilesWithShares).not.toHaveBeenCalled() + expect(mockListWorkspaceFileFolders).not.toHaveBeenCalled() }) }) @@ -138,86 +348,133 @@ describe('workspace list prefetches', () => { }, { name: 'tables', - run: (client: QueryClient) => prefetchTables(client, WORKSPACE_ID), + run: (client: QueryClient) => prefetchTables(client, WORKSPACE_ID, USER_ID), resourceType: 'table' as const, }, { name: 'knowledge', - run: (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID), + run: (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID), resourceType: 'knowledge_base' as const, }, ] for (const { name, run, resourceType } of chromeCases) { it(`primes pinned ids (${resourceType} + folder) and members for ${name}`, async () => { - const pinnedItems = [{ id: 'p-1', resourceId: 'r-1' }] - const members = [{ userId: 'u-1', name: 'Ada' }] - mockPrefetchInternalJson.mockImplementation(async (path: string) => { - if (path.startsWith('/api/pinned-items')) return { pinnedItems } - if (path.endsWith('/members')) return { members } - if (path.includes('/folders')) return { folders: [] } - return { success: true, files: [], data: { tables: [] } } - }) + /** + * Distinct fixtures per key: identical ones would still pass if the two pin + * namespaces were crossed. + */ + const resourcePins = [{ id: 'p-1', resourceType, resourceId: 'r-1' }] + const folderPins = [{ id: 'p-2', resourceType: 'folder' as const, resourceId: 'fld-1' }] + const members = [{ userId: 'u-1', name: 'Ada', image: null }] + mockListPinnedItemsForUser.mockImplementation( + async (_userId: string, _workspaceId: string, type: string) => + type === 'folder' ? folderPins : resourcePins + ) + mockGetWorkspaceMemberProfiles.mockResolvedValue(members) const client = makeClient() await run(client) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/pinned-items?workspaceId=${WORKSPACE_ID}&resourceType=${resourceType}` - ) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/pinned-items?workspaceId=${WORKSPACE_ID}&resourceType=folder` - ) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/workspaces/${WORKSPACE_ID}/members` - ) + expect(mockListPinnedItemsForUser).toHaveBeenCalledWith(USER_ID, WORKSPACE_ID, resourceType) + expect(mockListPinnedItemsForUser).toHaveBeenCalledWith(USER_ID, WORKSPACE_ID, 'folder') + expect(mockGetWorkspaceMemberProfiles).toHaveBeenCalledWith(WORKSPACE_ID) expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, resourceType))).toEqual( - pinnedItems - ) - expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, 'folder'))).toEqual( - pinnedItems + resourcePins ) + expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, 'folder'))).toEqual(folderPins) expect(client.getQueryData(workspaceKeys.members(WORKSPACE_ID))).toEqual(members) }) + + it(`caches no chrome for ${name} when the viewer cannot be proved`, async () => { + mockGetWorkspaceHostContextForViewer.mockResolvedValue(null) + const client = makeClient() + + await run(client) + + expect(mockListPinnedItemsForUser).not.toHaveBeenCalled() + expect(mockGetWorkspaceMemberProfiles).not.toHaveBeenCalled() + expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, resourceType))).toBeUndefined() + expect(client.getQueryData(workspaceKeys.members(WORKSPACE_ID))).toBeUndefined() + }) } }) - describe('prefetchHomeLists', () => { - it('primes folder + file keys, mapping folder rows to the client shape', async () => { - const folderRow = { - id: 'folder-1', - name: 'Docs', - userId: 'u-1', - workspaceId: WORKSPACE_ID, - parentId: null, - resourceType: 'workflow', - locked: false, - sortOrder: 0, - createdAt: '2026-01-01T00:00:00.000Z', - updatedAt: '2026-01-02T00:00:00.000Z', - deletedAt: null, - } - const files = [{ id: 'f-1' }] - mockPrefetchInternalJson.mockImplementation(async (path: string) => - path.startsWith('/api/folders') ? { folders: [folderRow] } : { success: true, files } - ) + describe('prefetchWorkspaceSidebar / seedWorkspaceList', () => { + const HOST_CONTEXT = { + workspace: { id: WORKSPACE_ID }, + viewer: { permission: 'admin' }, + } as never + + const WORKSPACE_ROW = { + id: WORKSPACE_ID, + name: 'GTM', + ownerId: USER_ID, + organizationId: null, + workspaceMode: 'personal', + permissions: 'admin', + } + + const LIST_PAYLOAD = { + workspaces: [WORKSPACE_ROW], + lastActiveWorkspaceId: null, + pinnedWorkspaceIds: [], + creationPolicy: null, + } + + /** + * The load-bearing contract: an empty list must leave the key UNSET so the client + * fetch reaches `GET /api/workspaces`' default-workspace creation path. Seeding an + * empty array instead would suppress it and strand a brand-new viewer. + */ + it('seeds nothing when the viewer has no workspaces', async () => { + mockListWorkspacesForViewer.mockResolvedValue({ ...LIST_PAYLOAD, workspaces: [] }) + const client = makeClient() + + await prefetchWorkspaceSidebar(client, WORKSPACE_ID, USER_ID, HOST_CONTEXT, null) + + expect(client.getQueryData(workspaceKeys.list('active'))).toBeUndefined() + }) + + it('seeds the workspace list when the viewer has one', async () => { + mockListWorkspacesForViewer.mockResolvedValue(LIST_PAYLOAD) const client = makeClient() - await prefetchHomeLists(client, WORKSPACE_ID) + await prefetchWorkspaceSidebar(client, WORKSPACE_ID, USER_ID, HOST_CONTEXT, null) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/folders?workspaceId=${WORKSPACE_ID}&scope=active&resourceType=workflow` + const cached = client.getQueryData(workspaceKeys.list('active')) as + | { workspaces: Array<{ id: string }> } + | undefined + expect(cached).toBeDefined() + expect(cached?.workspaces.map((w) => w.id)).toEqual([WORKSPACE_ID]) + }) + + /** A failed seed is an optimization loss, not a render failure. */ + it('does not throw when the workspace read rejects, and seeds nothing', async () => { + mockListWorkspacesForViewer.mockRejectedValue(new Error('500')) + const client = makeClient() + + await expect( + prefetchWorkspaceSidebar(client, WORKSPACE_ID, USER_ID, HOST_CONTEXT, null) + ).resolves.toBeUndefined() + expect(client.getQueryData(workspaceKeys.list('active'))).toBeUndefined() + }) + + /** Guards the mismatch check that keeps one workspace's data out of another's cache. */ + it('seeds nothing when the host context is for a different workspace', async () => { + mockListWorkspacesForViewer.mockResolvedValue(LIST_PAYLOAD) + const client = makeClient() + + await prefetchWorkspaceSidebar( + client, + WORKSPACE_ID, + USER_ID, + { workspace: { id: 'other-ws' }, viewer: { permission: 'admin' } } as never, + null ) - const cachedFolders = client.getQueryData(folderKeys.list(WORKSPACE_ID, 'active')) as Array<{ - id: string - resourceType: string - createdAt: Date - }> - expect(cachedFolders).toHaveLength(1) - expect(cachedFolders[0].resourceType).toBe('workflow') - // The wire shape carries ISO strings; the client shape carries Dates. - expect(cachedFolders[0].createdAt).toBeInstanceOf(Date) - expect(client.getQueryData(workspaceFilesKeys.list(WORKSPACE_ID, 'active'))).toEqual(files) + + expect(client.getQueryCache().getAll()).toHaveLength(0) + expect(mockListWorkspacesForViewer).not.toHaveBeenCalled() }) }) @@ -225,29 +482,35 @@ describe('workspace list prefetches', () => { it.each([ [ 'prefetchKnowledgeBases', - (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID), + (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID), knowledgeKeys.list(WORKSPACE_ID, 'active'), ], [ 'prefetchTables', - (client: QueryClient) => prefetchTables(client, WORKSPACE_ID), + (client: QueryClient) => prefetchTables(client, WORKSPACE_ID, USER_ID), tableKeys.list(WORKSPACE_ID, 'active'), ], [ - 'prefetchHomeLists', - (client: QueryClient) => prefetchHomeLists(client, WORKSPACE_ID), - folderKeys.list(WORKSPACE_ID, 'active'), - ], - [ + /** + * Asserted against the folder key, not the file list: `prefetchFilesBrowser` + * deliberately never seeds `workspaceFilesKeys` (the layout owns it), so an + * assertion on that key would hold no matter what this function did. + */ 'prefetchFilesBrowser', (client: QueryClient) => prefetchFilesBrowser(client, WORKSPACE_ID, USER_ID), - workspaceFilesKeys.list(WORKSPACE_ID, 'active'), + workspaceFileFolderKeys.list(WORKSPACE_ID, 'active'), ], ] as const)( '%s does not throw when the fetcher rejects (page still renders, client refetches)', async (_name, prefetch, queryKey) => { - mockPrefetchInternalJson.mockRejectedValue(new Error('500')) - mockListWorkspaceFilesWithShares.mockRejectedValue(new Error('500')) + const boom = new Error('500') + mockListWorkspaceFilesWithShares.mockRejectedValue(boom) + mockListFoldersForWorkspace.mockRejectedValue(boom) + mockListTables.mockRejectedValue(boom) + mockListInternalKnowledgeBases.mockRejectedValue(boom) + mockListPinnedItemsForUser.mockRejectedValue(boom) + mockGetWorkspaceMemberProfiles.mockRejectedValue(boom) + mockListWorkspaceFileFolders.mockRejectedValue(boom) const client = makeClient() await expect(prefetch(client)).resolves.toBeUndefined() diff --git a/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/trace-view/trace-view.tsx b/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/trace-view/trace-view.tsx index 6365ec2e8ce..80c4fea424f 100644 --- a/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/trace-view/trace-view.tsx +++ b/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/trace-view/trace-view.tsx @@ -42,11 +42,12 @@ import { getDisplayName, hasErrorInTree, hasUnhandledErrorInTree, - iconColorClass, isIterationType, parseTime, } from '@/app/workspace/[workspaceId]/logs/components/log-details/utils' +import { BlockTile } from '@/blocks/block-tile' import { isCustomBlockType } from '@/blocks/custom/build-config' +import { getTileIconColorClass } from '@/blocks/icon-color' import { useCodeViewerFeatures } from '@/hooks/use-code-viewer' const DEFAULT_TREE_PANE_WIDTH = 240 @@ -331,12 +332,12 @@ const TraceTreeRow = memo(function TraceTreeRow({
)} {!isIterationType(span.type) && ( -
- {BlockIcon && } -
+ )} @@ -711,7 +712,9 @@ const TraceDetailPane = memo(function TraceDetailPane({ span }: { span: TraceSpa className='mt-[2px] flex size-[18px] flex-shrink-0 items-center justify-center overflow-hidden rounded-sm [&_img]:size-full' style={{ background: bgColor }} > - {BlockIcon && } + {BlockIcon && ( + + )}
)}
diff --git a/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/utils.ts b/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/utils.ts index 4b993050ecd..b3dc95416bb 100644 --- a/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/utils.ts +++ b/apps/sim/app/workspace/[workspaceId]/logs/components/log-details/utils.ts @@ -86,12 +86,6 @@ export function getBlockIconAndColor( */ const MAX_YIQ_SUM = 255_000 -/** Returns 'text-white' for dark backgrounds, dark text for light ones. */ -export function iconColorClass(bgColor: string): string { - const brightness = perceivedBrightness(bgColor) - return brightness !== null && brightness > 160_000 / MAX_YIQ_SUM ? 'text-[#111111]' : 'text-white' -} - /** * Near-black bgColors disappear against the dark-mode surface (--bg: #1b1b1b). * Below the brightness threshold we fall back to the neutral block color used diff --git a/apps/sim/app/workspace/[workspaceId]/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/prefetch.ts index fe69e488fae..c24198fb51f 100644 --- a/apps/sim/app/workspace/[workspaceId]/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/prefetch.ts @@ -1,13 +1,16 @@ +import { createLogger } from '@sim/logger' +import { getErrorMessage } from '@sim/utils/errors' import type { QueryClient } from '@tanstack/react-query' import { listWorkspacesContract, type WorkspaceHostContext } from '@/lib/api/contracts/workspaces' import { listMothershipChats } from '@/lib/copilot/chat/list-mothership-chats' import { isChatEnabled } from '@/lib/core/config/env-flags' -import { listFoldersForWorkspace } from '@/lib/folders/queries' import { getUserProfile } from '@/lib/users/queries' import { listWorkflowsForUser } from '@/lib/workflows/queries' +import { listWorkspaceFilesWithShares } from '@/lib/workspace-files/queries' import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' import { listWorkspacesForViewer } from '@/lib/workspaces/list' import { getWorkspacePermissionsForAuthorizedViewer } from '@/lib/workspaces/permissions/utils' +import { prefetchResourceFolders } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-folders' import { MOTHERSHIP_CHAT_LIST_STALE_TIME, mapChat, @@ -18,14 +21,14 @@ import { USER_PROFILE_STALE_TIME, userProfileKeys, } from '@/hooks/queries/user-profile' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' import { workflowKeys } from '@/hooks/queries/utils/workflow-keys' import { mapWorkflow, WORKFLOW_LIST_STALE_TIME } from '@/hooks/queries/utils/workflow-list-query' -import { - normalizeWorkspacesResponse, - WORKSPACE_LIST_STALE_TIME, -} from '@/hooks/queries/utils/workspace-list-query' +import { normalizeWorkspacesResponse } from '@/hooks/queries/utils/workspace-list-query' import { WORKSPACE_PERMISSIONS_STALE_TIME, workspaceKeys } from '@/hooks/queries/workspace' +import { + WORKSPACE_FILES_LIST_STALE_TIME, + workspaceFilesKeys, +} from '@/hooks/queries/workspace-files' import { WORKSPACE_HOST_CONTEXT_STALE_TIME, workspaceHostKeys, @@ -47,22 +50,77 @@ export function prefetchWorkspaceHostContext( }) } +const logger = createLogger('WorkspacePrefetch') + +/** + * Seeds the viewer's workspace list, which the switcher reads. + * + * Seeded rather than prefetched so the empty-list case can decline to create a + * cache entry at all: the route's default-workspace creation path must run on + * the client, and an entry — even an empty one — would suppress it. Expressing + * that as an absent seed also keeps a routine state out of the error channel, + * where it read as a failure rather than as "nothing to seed". + */ +async function seedWorkspaceList( + queryClient: QueryClient, + userId: string, + activeOrganizationId: string | null +): Promise { + try { + const payload = await listWorkspacesForViewer({ + userId, + activeOrganizationId, + scope: 'active', + }) + if (payload.workspaces.length === 0) return + /** + * Parsing through the route contract's response schema strips the same + * server-only fields `requestJson` strips on the client, guaranteeing the + * seeded shape is identical to a client fetch. + */ + queryClient.setQueryData( + workspaceKeys.list('active'), + normalizeWorkspacesResponse(listWorkspacesContract.response.schema.parse(payload)) + ) + } catch (error) { + /** + * Swallowed rather than rethrown — this read is an optimization; the layout + * renders fine without it and the client fetch reaches the route instead. + * Logged because contract drift between the read and the response schema + * would otherwise degrade silently into every viewer waterfalling. + */ + logger.warn('Workspace list seed failed; client will fetch', { + error: getErrorMessage(error), + }) + } +} + /** * Prefetches the sidebar's workflow, chat, folder, workspace-permissions, * workspace, and viewer-profile reads for a workspace and stores them under the * same query keys + mappers the client hooks use, so the persistent sidebar - * (including the workspace switcher header and the footer's profile row) paints - * populated on the first server render - * instead of flashing skeletons on a cold load (e.g. after the browser - * discards an idle tab). Calls the data layer directly — the same functions - * the API routes use — with no internal HTTP hop. + * (including the workspace switcher header and the footer's profile row) is + * populated without a client-side request waterfall on a cold load (e.g. after + * the browser discards an idle tab). Calls the data layer directly — the same + * functions the API routes use — with no internal HTTP hop. * * The host context is the authorization proof for this server-render pass, so * permission prefetch can reuse its effective permission without repeating * workspace and membership reads. It also proves the viewer has at least one - * accessible workspace, which is why the workspace-list prefetch can safely - * skip the route's empty-list default-workspace creation path — and the - * route's orphaned-workflow repair, which still runs on client refetches. + * accessible workspace, so this pass skips the route's orphaned-workflow + * repair, which still runs on client refetches. + * + * All reads run concurrently and are awaited together, so every pane is settled + * in the cache before `dehydrate` and the sidebar still paints populated rather + * than flashing skeletons that stream in behind the shell. + * + * The workspace list is seeded rather than prefetched. An empty or failed read + * seeds nothing, leaving the client fetch to reach `GET /api/workspaces`' + * default-workspace creation path — the same outcome a rejecting `queryFn` used + * to produce, without routing a normal state through the error channel. That + * matters because only a settled query is dehydrated: an unawaited read would be + * dropped from the payload entirely, so the switcher would waterfall on every + * cold load rather than paint populated. */ export async function prefetchWorkspaceSidebar( queryClient: QueryClient, @@ -72,6 +130,7 @@ export async function prefetchWorkspaceSidebar( activeOrganizationId: string | null ): Promise { if (hostContext.workspace.id !== workspaceId) return + await Promise.all([ queryClient.prefetchQuery({ queryKey: workflowKeys.list(workspaceId, 'active'), @@ -93,34 +152,22 @@ export async function prefetchWorkspaceSidebar( }), ] : []), + prefetchResourceFolders(queryClient, workspaceId, 'workflow', userId), + /** + * The sidebar reads the workspace's files for its search modal, on EVERY workspace route — so this + * query is registered by sidebar chrome before any page renders. That ordering is why it has to be + * prefetched HERE and not only by the Files pages: `HydrationBoundary` hydrates a query the cache + * has already seen from a `useEffect`, which never runs during SSR, so a page-level boundary can + * only ever hand this entry to the client. Seeding it with the layout's own boundary — the first + * one to render — is what lets the server paint the Files browser and the open file's header + * populated instead of shipping a spinner and resolving it a beat later on the client. + * + * Same key + shape as {@link prefetchFilesBrowser}, so whichever runs is a no-op for the other. + */ queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'workflow'), - queryFn: async () => { - const rows = await listFoldersForWorkspace(workspaceId, 'active', 'workflow') - return rows.map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - queryClient.prefetchQuery({ - queryKey: workspaceKeys.list('active'), - queryFn: async () => { - const payload = await listWorkspacesForViewer({ - userId, - activeOrganizationId, - scope: 'active', - }) - // An empty list means GET /api/workspaces' default-workspace creation - // path must run — throw so prefetchQuery caches nothing and the client - // fetch reaches the route. - if (payload.workspaces.length === 0) { - throw new Error('Empty workspace list requires the route creation path') - } - // Parsing through the route contract's response schema strips the same - // server-only fields `requestJson` strips on the client, guaranteeing the - // cached shape is identical to a client fetch. - return normalizeWorkspacesResponse(listWorkspacesContract.response.schema.parse(payload)) - }, - staleTime: WORKSPACE_LIST_STALE_TIME, + queryKey: workspaceFilesKeys.list(workspaceId, 'active'), + queryFn: () => listWorkspaceFilesWithShares(workspaceId, 'active'), + staleTime: WORKSPACE_FILES_LIST_STALE_TIME, }), queryClient.prefetchQuery({ queryKey: workspaceKeys.permissions(workspaceId), @@ -148,5 +195,6 @@ export async function prefetchWorkspaceSidebar( }, staleTime: USER_PROFILE_STALE_TIME, }), + seedWorkspaceList(queryClient, userId, activeOrganizationId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx b/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx index 2f84f225401..22d0fbc4f9a 100644 --- a/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx @@ -25,7 +25,7 @@ import { } from '@/app/workspace/[workspaceId]/settings/navigation' import { resolveWorkspaceGroup } from '@/ee/access-control/utils/permission-check' import { isForkingAvailableForWorkspace } from '@/ee/workspace-forking/lib/lineage/authz' -import { prefetchGeneralSettings, prefetchUserProfile } from './prefetch' +import { prefetchGeneralSettings } from './prefetch' import { SettingsPage } from './settings' interface WorkspaceSettingsSectionPageProps { @@ -170,8 +170,13 @@ export default async function WorkspaceSettingsSectionPage({ } const queryClient = getQueryClient() - void prefetchGeneralSettings(queryClient) - void prefetchUserProfile(queryClient) + /** + * Awaited, not fired and forgotten: only a settled query is dehydrated, so an unawaited + * prefetch is dropped from the payload and the panel waterfalls anyway. The viewer's + * profile is already seeded by the workspace layout under the same key, so it is not + * repeated here. + */ + await prefetchGeneralSettings(queryClient) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts index 059690a037b..9cbcf3d5f61 100644 --- a/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts @@ -1,24 +1,21 @@ import type { QueryClient } from '@tanstack/react-query' -import { headers } from 'next/headers' import { getSession } from '@/lib/auth' -import { getInternalApiBaseUrl } from '@/lib/core/utils/urls' -import { getUserProfile, getUserSettings } from '@/lib/users/queries' +import { getUserSettings } from '@/lib/users/queries' import { GENERAL_SETTINGS_STALE_TIME, generalSettingsKeys, mapGeneralSettingsResponse, } from '@/hooks/queries/general-settings' -import { SUBSCRIPTION_DATA_STALE_TIME, subscriptionKeys } from '@/hooks/queries/subscription' -import { - mapUserProfileResponse, - USER_PROFILE_STALE_TIME, - userProfileKeys, -} from '@/hooks/queries/user-profile' /** * Prefetch general settings server-side via the shared data layer. - * Uses the same query keys as the client `useGeneralSettings` hook - * so data is shared via HydrationBoundary. + * + * Uses the same query key and mapper as the client `useGeneralSettings` hook, so the + * hydrated entry is indistinguishable from one a client fetch produced. + * + * Callers must `await` this. Only a settled query is dehydrated, so an unawaited prefetch + * is dropped from the payload entirely and the panel waterfalls on every load as if it had + * never been prefetched. */ export function prefetchGeneralSettings(queryClient: QueryClient) { return queryClient.prefetchQuery({ @@ -31,48 +28,3 @@ export function prefetchGeneralSettings(queryClient: QueryClient) { staleTime: GENERAL_SETTINGS_STALE_TIME, }) } - -/** - * Prefetch subscription data server-side. Unlike the other prefetches this goes - * through the internal billing API rather than calling the data layer directly: - * the billing summary contains `Date` fields (and an untyped `metadata` blob) that - * `NextResponse.json` serializes to the string wire shape the client caches. Going - * through the route yields that exact shape, avoiding a Date-vs-string mismatch - * between server-hydrated and client-fetched data. Uses the same query key as the - * client `useSubscriptionData` hook (with includeOrg=false) so data is shared via - * HydrationBoundary. - */ -export function prefetchSubscriptionData(queryClient: QueryClient) { - return queryClient.prefetchQuery({ - queryKey: subscriptionKeys.user(false), - queryFn: async () => { - const h = await headers() - const cookie = h.get('cookie') - const response = await fetch(`${getInternalApiBaseUrl()}/api/billing?context=user`, { - headers: cookie ? { cookie } : {}, - }) - if (!response.ok) throw new Error(`Subscription prefetch failed: ${response.status}`) - return response.json() - }, - staleTime: SUBSCRIPTION_DATA_STALE_TIME, - }) -} - -/** - * Prefetch user profile server-side via the shared data layer. - * Uses the same query keys as the client `useUserProfile` hook - * so data is shared via HydrationBoundary. - */ -export function prefetchUserProfile(queryClient: QueryClient) { - return queryClient.prefetchQuery({ - queryKey: userProfileKeys.profile(), - queryFn: async () => { - const session = await getSession() - if (!session?.user?.id) throw new Error('Unauthorized') - const user = await getUserProfile(session.user.id) - if (!user) throw new Error('User not found') - return mapUserProfileResponse(user) - }, - staleTime: USER_PROFILE_STALE_TIME, - }) -} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx index c8995487beb..677cd5eedf1 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx @@ -7,10 +7,10 @@ import type { EnrichmentProviderOutcome, EnrichmentRunDetail } from '@/lib/table import { adjustBgForContrast, getBlockIconAndColor, - iconColorClass, } from '@/app/workspace/[workspaceId]/logs/components/log-details/utils' import { useLogDetailsResize } from '@/app/workspace/[workspaceId]/logs/hooks' import { formatDate } from '@/app/workspace/[workspaceId]/logs/utils' +import { getTileIconColorClass } from '@/blocks/icon-color' import { useEnrichmentDetail } from '@/hooks/queries/tables' import { formatCost } from '@/providers/utils' import { useLogDetailsUIStore } from '@/stores/logs/store' @@ -255,7 +255,9 @@ function EnrichmentDetailsContent({ style={{ background: bgColor }} > {ProviderIcon && ( - + )}
diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx index 9dbb32fce28..d2d486561c5 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx @@ -1,6 +1,5 @@ 'use client' -import type React from 'react' import { useMemo, useState } from 'react' import { Button, @@ -18,7 +17,7 @@ import { Tooltip, toast, } from '@sim/emcn' -import { ArrowLeft, ChevronDown, Repeat, Split, SquareArrowUpRight, X } from '@sim/emcn/icons' +import { ArrowLeft, ChevronDown, SquareArrowUpRight, X } from '@sim/emcn/icons' import { toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' import { useMutation, useQueryClient } from '@tanstack/react-query' @@ -57,8 +56,7 @@ import { RequiredLabel, } from '@/app/workspace/[workspaceId]/tables/[tableId]/components/sidebar-fields' import { PreviewWorkflow } from '@/app/workspace/[workspaceId]/w/components/preview' -import { getBlock } from '@/blocks' -import { getTileIconColorClass } from '@/blocks/icon-color' +import { BlockTile } from '@/blocks/block-tile' import { useAddWorkflowGroup, useUpdateColumn, @@ -140,8 +138,6 @@ interface BlockOutputGroup { blockId: string blockName: string blockType: string - blockIcon: string | React.ComponentType<{ className?: string }> - blockColor: string paths: string[] } @@ -164,25 +160,6 @@ function tableColumnTypeToInputType(colType: ColumnDefinition['type'] | undefine return columnTypeById(colType).workflowInputType } -const TagIcon: React.FC<{ - icon: string | React.ComponentType<{ className?: string }> - color: string -}> = ({ icon, color }) => ( -
- {typeof icon === 'string' ? ( - {icon} - ) : ( - (() => { - const IconComponent = icon - return - })() - )} -
-) - /** * Right-edge sidebar for workflow group configuration. Three flows: * - create a new group (workflow + outputs + deps), @@ -468,20 +445,10 @@ export function WorkflowSidebarBody({ for (const f of flat) { let group = groupsByBlockId.get(f.blockId) if (!group) { - const blockConfig = getBlock(f.blockType) - const blockColor = blockConfig?.bgColor || '#2F55FF' - let blockIcon: string | React.ComponentType<{ className?: string }> = f.blockName - .charAt(0) - .toUpperCase() - if (blockConfig?.icon) blockIcon = blockConfig.icon - else if (f.blockType === 'loop') blockIcon = Repeat - else if (f.blockType === 'parallel') blockIcon = Split group = { blockId: f.blockId, blockName: f.blockName, blockType: f.blockType, - blockIcon, - blockColor, paths: [], } groupsByBlockId.set(f.blockId, group) @@ -504,7 +471,11 @@ export function WorkflowSidebarBody({ section: group.blockName, sectionElement: (
- + {group.blockName}
), diff --git a/apps/sim/app/workspace/[workspaceId]/tables/page.tsx b/apps/sim/app/workspace/[workspaceId]/tables/page.tsx index 0e9390a5d95..1e9cb1f0592 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/page.tsx @@ -1,6 +1,7 @@ import { Suspense } from 'react' import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' import { getQueryClient } from '@/app/_shell/providers/get-query-client' import TablesLoading from '@/app/workspace/[workspaceId]/tables/loading' import { prefetchTables } from '@/app/workspace/[workspaceId]/tables/prefetch' @@ -17,10 +18,9 @@ export const metadata: Metadata = { * route-level `loading.tsx` covers the navigation/chunk-load transition. */ export default async function TablesPage({ params }: { params: Promise<{ workspaceId: string }> }) { - const { workspaceId } = await params - + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - await prefetchTables(queryClient, workspaceId) + await prefetchTables(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts index 5a548885511..a937a26e753 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts @@ -1,9 +1,9 @@ import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts/folders' -import type { TableDefinition } from '@/lib/table' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import { listTables } from '@/lib/table/service' +import { toTableListItem } from '@/lib/table/wire' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { prefetchResourceFolders } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-folders' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' import { TABLE_LIST_STALE_TIME, tableKeys } from '@/hooks/queries/utils/table-keys' /** @@ -14,33 +14,36 @@ import { TABLE_LIST_STALE_TIME, tableKeys } from '@/hooks/queries/utils/table-ke * only placed correctly relative to the folder rows it sits beside, so * prefetching one without the other still flashes an ungrouped list. * - * Table definitions carry `Date` fields, so the list goes through the - * `/api/table` route and caches the serialized wire shape — see - * {@link prefetchInternalJson}. Folders are mapped with the same `mapFolder` the - * hook applies so the hydrated entry matches a client fetch exactly. + * The list goes through {@link toTableListItem}, the projection `GET /api/table` itself + * returns, because `listTablesContract`'s response schema is a passthrough that neither + * coerces nor strips — the client caches the route's JSON verbatim, so seeding raw rows + * would put `Date` objects and the server-only `metadata` field under that key. + * + * The read carries no authorization of its own, so the viewer is proved first. + * `getWorkspaceHostContextForViewer` resolves the same effective workspace permission the + * route's own check does (both bottom out in `checkWorkspaceAccess`), and it is `cache`d and + * already resolved by the layout for this request, so it costs no additional queries. A viewer + * without access caches nothing and the client fetch reaches the route for the real 403. */ -export async function prefetchTables(queryClient: QueryClient, workspaceId: string): Promise { +export async function prefetchTables( + queryClient: QueryClient, + workspaceId: string, + userId: string | undefined +): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + await Promise.all([ queryClient.prefetchQuery({ queryKey: tableKeys.list(workspaceId, 'active'), queryFn: async () => { - const response = await prefetchInternalJson<{ data: { tables: TableDefinition[] } }>( - `/api/table?workspaceId=${workspaceId}&scope=active` - ) - return response.data.tables + const tables = await listTables(workspaceId, { scope: 'active' }) + return tables.map(toTableListItem) }, staleTime: TABLE_LIST_STALE_TIME, }), - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'table'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=table` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - prefetchResourceListChrome(queryClient, workspaceId, 'table'), + prefetchResourceFolders(queryClient, workspaceId, 'table', userId), + prefetchResourceListChrome(queryClient, workspaceId, 'table', userId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx b/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx index 188e3a6523f..f2079037598 100644 --- a/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx +++ b/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx @@ -120,6 +120,22 @@ describe('useUpgradeState', () => { }) }) + it('shows checkout admission failures through the standard error toast', async () => { + mockHandleUpgrade.mockRejectedValueOnce( + new Error('Your subscription payment is still processing.') + ) + + await act(async () => { + root.render() + }) + + await act(async () => { + await currentState?.doUpgrade('team', 25000) + }) + + expect(mockToastError).toHaveBeenCalledWith('Your subscription payment is still processing.') + }) + it('includes the routed workspace when switching the host billing interval', async () => { await act(async () => { root.render() diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx index c2882134000..c053ec1b50e 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx @@ -66,10 +66,30 @@ const ACTION_BUTTON_STYLES = [ * 25.11px of period. Writing 24/26 directly would render ~3.5% wide and drift * out of the squares' rhythm across the row. * + * Each edge ramps over 0.75px rather than switching colour at a single offset, + * which is why the stops come in pairs 0.375px either side of the mark's two + * edges. A gradient is sampled once per pixel with no coverage term, so a hard + * stop on a 15°-off-vertical edge can only ever land wholly on one side or the + * other — the marks came out visibly stepped, which is the one thing a shape + * this thin cannot hide. Ramping across roughly a device pixel gives the + * rasterizer the intermediate values antialiasing would have produced, and + * measured edge deviation drops from 0.28 device px (pure quantization) to 0.05. + * + * The period runs centre-of-mark to centre-of-mark (11.59 → 36.7) rather than + * starting at an edge, because a repeating gradient truncates at its own wrap: + * anchored at 0, the ramp leaving the mark would have run 24.735 → 25.485 and + * been cut at 25.11, so that edge got half the feather and the gap came out + * 1.93 → 1.75px. Both ramps have to sit strictly inside the period. The list + * still tiles backwards from its first stop, so the marks land exactly where + * anchoring at 0 put them — same 26px pitch, same phase against the squares. + * + * Widening the feather further would keep smoothing, but the gap is only 1.93px + * of stop, so it comes straight out of the mark's dark core. + * * `--surface-2` is the same fill the slots used; only where it is painted moved. */ const RUNNING_FILL = - 'bg-[repeating-linear-gradient(75deg,var(--surface-2)_0_23.18px,transparent_23.18px_25.11px)]' + 'bg-[repeating-linear-gradient(75deg,var(--surface-2)_11.59px_22.805px,transparent_23.555px_24.735px,var(--surface-2)_25.485px_36.7px)]' /** Left edge of the fill: clears the run/stop button, which stays live mid-run. */ const RUNNING_FILL_INSET_SWELL = 'left-[42px]' @@ -84,12 +104,14 @@ const RUNNING_FILL_INSET_PLAIN = 'left-[26px]' * inside it at the bottom — the fill visibly ran off the block. The per-slot * version never did, because each button's own clip contained it. * - * Same taper, read off that path: the edge sits 16.67px in from the row's right - * at the overlay's top (y=4) and 3.33px at its bottom (y=20), a slope of 20/24. - * Changing the end silhouette means changing these two numbers with it. + * Same taper, read off that path. Its straight run — (22.4, 2.88) to + * (36.59, 19.9) in the slot's own 40×24 box — has a slope of 20/24, so across + * the full row it moves from 20px in at the top to flush at the bottom. The + * overlay spans the row, so those are its two numbers; they are the slot's own + * edge continued, which is what puts the hatch's end exactly where a hovered + * slot's fill ends. Changing the end silhouette means changing them with it. */ -const RUNNING_FILL_END_TAPER = - '[clip-path:polygon(0_0,calc(100%_-_16.67px)_0,calc(100%_-_3.33px)_100%,0_100%)]' +const RUNNING_FILL_END_TAPER = '[clip-path:polygon(0_0,calc(100%_-_20px)_0,100%_100%,0_100%)]' const ICON_SIZE = 'size-[14px]' @@ -415,7 +437,11 @@ export const ActionBar = memo(