fix(webapp): dashboard agent maintenance moves into the agent project - #4599
Open
kathiekiwi wants to merge 10 commits into
Open
fix(webapp): dashboard agent maintenance moves into the agent project#4599kathiekiwi wants to merge 10 commits into
kathiekiwi wants to merge 10 commits into
Conversation
Drop the eval retention, soft-deleted-chat retention and investigation sweeps, plus the retention half of the watch sweep. They now run in the agent trigger project. Watch finalize/deliver, batch re-arm and the org purge stay.
Tests hand-rolled the drizzle migration replay against a testcontainers Postgres. Export it once from the package as `@internal/dashboard-agent-db/testing`, taking whatever client the suite already has.
Two scheduled tasks, the project's first crons: - `dashboard-agent-maintenance` (daily, 03:00) is retention: judged turns and soft-deleted chats past 30 days, terminal watches and their submission ledger past 7 days. Each pass drains in bounded batches, so a backlog doesn't wait a day per batch, and a failing pass is reported at the end instead of masking the others. - `dashboard-agent-investigation-sweep` (every 5 min) settles cards left `in_progress` past 30 minutes, force-abandoning a poison row past the attempt cap. The two differ on purpose in how they find their database. Retention reads only `DASHBOARD_AGENT_DATABASE_URL` and no-ops when it is unset: a delete sweep must never guess which database it deletes from. The investigation sweep deletes nothing, so it reuses the project's existing `getWatchDb()`, which still falls back to `DATABASE_URL`. The org-deletion purge stays in the webapp.
Keep a cron-less catalog entry with a no-op handler so the queued items left behind ack instead of logging a missing schema every run. Also restore the settlement-card coupling test and trim the release note.
…atch cap Ports the hard-delete cascade test back from the webapp suite, logs when a retention pass ends on a full batch, and drops the unused exports.
|
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
🚧 Files skipped from review as they are similar to previous changes (3)
📜 Recent review details⏰ Context from checks skipped due to timeout. (34)
🧰 Additional context used📓 Path-based instructions (5)**/*.{ts,tsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
**/*.{ts,tsx,js,jsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
**/*.ts📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
Files:
internal-packages/**/*.{ts,tsx}📄 CodeRabbit inference engine (AGENTS.md)
Files:
**/*.{test,spec}.{ts,tsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
🧠 Learnings (53)📓 Common learnings📚 Learning: 2026-08-08T08:29:21.685ZApplied to files:
📚 Learning: 2026-08-08T08:30:57.294ZApplied to files:
📚 Learning: 2026-05-14T16:39:02.759ZApplied to files:
📚 Learning: 2026-08-08T08:30:23.116ZApplied to files:
📚 Learning: 2026-07-02T19:14:58.851ZApplied to files:
📚 Learning: 2026-08-12T08:49:18.799ZApplied to files:
📚 Learning: 2026-04-30T13:05:56.593ZApplied to files:
📚 Learning: 2026-08-12T06:32:24.127ZApplied to files:
📚 Learning: 2026-06-01T12:05:44.112ZApplied to files:
📚 Learning: 2026-08-12T07:30:37.802ZApplied to files:
📚 Learning: 2026-08-08T08:29:46.703ZApplied to files:
📚 Learning: 2026-08-08T08:33:22.847ZApplied to files:
📚 Learning: 2026-08-11T10:53:21.099ZApplied to files:
📚 Learning: 2026-05-15T08:05:57.683ZApplied to files:
📚 Learning: 2026-08-08T08:33:52.819ZApplied to files:
📚 Learning: 2026-03-26T10:02:25.354ZApplied to files:
📚 Learning: 2026-06-17T17:13:49.929ZApplied to files:
📚 Learning: 2026-04-13T21:44:00.032ZApplied to files:
📚 Learning: 2026-06-30T14:58:42.838ZApplied to files:
📚 Learning: 2026-05-12T14:13:17.114ZApplied to files:
📚 Learning: 2026-06-16T13:14:20.816ZApplied to files:
📚 Learning: 2026-05-12T21:04:01.335ZApplied to files:
📚 Learning: 2026-07-26T13:14:05.711ZApplied to files:
📚 Learning: 2026-07-18T13:08:54.672ZApplied to files:
📚 Learning: 2026-08-08T12:49:17.489ZApplied to files:
📚 Learning: 2026-08-08T12:22:34.925ZApplied to files:
📚 Learning: 2026-08-11T22:35:29.301ZApplied to files:
📚 Learning: 2026-08-08T08:33:38.878ZApplied to files:
📚 Learning: 2026-08-08T08:34:41.913ZApplied to files:
📚 Learning: 2026-03-22T13:26:12.060ZApplied to files:
📚 Learning: 2026-03-22T19:24:14.403ZApplied to files:
📚 Learning: 2026-05-18T08:21:27.694ZApplied to files:
📚 Learning: 2026-05-18T08:21:27.694ZApplied to files:
📚 Learning: 2026-06-13T19:53:13.759ZApplied to files:
📚 Learning: 2026-06-23T13:04:21.413ZApplied to files:
📚 Learning: 2026-06-04T18:16:35.386ZApplied to files:
📚 Learning: 2026-06-09T17:58:04.699ZApplied to files:
📚 Learning: 2026-08-11T22:35:59.403ZApplied to files:
📚 Learning: 2026-03-02T12:43:25.254ZApplied to files:
📚 Learning: 2026-07-18T18:31:37.633ZApplied to files:
📚 Learning: 2026-07-18T18:17:16.266ZApplied to files:
📚 Learning: 2026-04-16T13:45:22.317ZApplied to files:
📚 Learning: 2025-11-27T16:26:37.432ZApplied to files:
📚 Learning: 2026-07-27T15:07:14.579ZApplied to files:
📚 Learning: 2026-08-12T08:49:25.047ZApplied to files:
📚 Learning: 2026-06-16T09:19:47.637ZApplied to files:
📚 Learning: 2026-07-13T14:51:32.938ZApplied to files:
📚 Learning: 2025-11-27T16:26:37.432ZApplied to files:
📚 Learning: 2026-07-13T14:51:40.805ZApplied to files:
📚 Learning: 2026-08-08T17:53:06.947ZApplied to files:
📚 Learning: 2026-05-18T14:40:02.173ZApplied to files:
📚 Learning: 2026-08-05T16:10:45.358ZApplied to files:
🔇 Additional comments (3)
WalkthroughDashboard-agent retention now runs in the dashboard-agent package through a daily scheduled task. Investigation settlement runs through a separate five-minute scheduled task. Both tasks use dashboard-agent database wiring and include new integration coverage. The webapp retention sweeps and related watch purge logic were removed, and the former maintenance worker entry became a deprecated no-op. Documentation and changelog entries describe the new schedules. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
kathiekiwi
changed the title
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
kathiekiwi
changed the title
Collaborator
Author
|
Re the CodeRabbit nitpicks, all in 8f64cc5: the chat-retention test now uses |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Drops 23 hand-rolled copies of the drizzle migration replayer in favour of applyDashboardAgentMigrations.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Comment on lines
+122
to
+128
| run: async (): Promise<RetentionResult | undefined> => { | ||
| if (!watchConnectionString()) { | ||
| logger.warn( | ||
| "dashboard-agent maintenance skipped: no DASHBOARD_AGENT_DATABASE_URL or DATABASE_URL" | ||
| ); | ||
| return undefined; | ||
| } |
Contributor
There was a problem hiding this comment.
🔍 Retention does fall back to DATABASE_URL, contradicting the PR description
The PR description states "When DASHBOARD_AGENT_DATABASE_URL is unset the task does nothing — a delete sweep must never guess its database, so there is no DATABASE_URL fallback here." The code does the opposite: the guard and the connection both go through watchConnectionString() (internal-packages/dashboard-agent/src/watch-task-adapters.ts:21-24), which is DASHBOARD_AGENT_DATABASE_URL || DATABASE_URL. So retention will connect to (and hard-delete from) the main database's trigger_dashboard_agent schema whenever only DATABASE_URL is set. The updated GUIDEBOOK (internal-packages/dashboard-agent/GUIDEBOOK.md:385-392) and the webapp's own resolution (apps/webapp/app/services/dashboardAgentDb.server.ts:16) both use the same fallback, so the code is self-consistent — but the description's stated safety property is not implemented. Worth confirming which behaviour is intended before merge, since this pass issues destructive deletes.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
9
to
14
| /** | ||
| * How long a soft-deleted chat is kept before it and its children are hard-deleted. | ||
| * Long enough that an accidental delete can still be investigated; org deletion soft- | ||
| * deletes the org's chats, so those are removed the same way once the window passes. | ||
| */ | ||
| export const CHAT_SOFT_DELETE_RETENTION_MS = 30 * 24 * 60 * 60 * 1000; | ||
|
|
||
| /** Per-run cap. Retention is one bounded statement, not a row-at-a-time loop. */ | ||
| const RETENTION_BATCH_LIMIT = 500; | ||
|
|
||
| export type ChatRetentionResult = { | ||
| /** Soft-deleted chats past the retention window dropped this run. */ | ||
| purged: number; | ||
| failed: number; | ||
| }; | ||
|
|
||
| export type ChatRetentionDeps = { | ||
| now?: () => Date; | ||
| limit?: number; | ||
| /** Hard-delete chats soft-deleted before `before`. Returns how many went. */ | ||
| purge?: (params: { before: Date; limit: number }) => Promise<number>; | ||
| }; | ||
|
|
||
| export async function sweepDashboardAgentSoftDeletedChats( | ||
| deps: ChatRetentionDeps = {} | ||
| ): Promise<ChatRetentionResult> { | ||
| const now = deps.now?.() ?? new Date(); | ||
| const limit = deps.limit ?? RETENTION_BATCH_LIMIT; | ||
| const purge = | ||
| deps.purge ?? ((params) => hardDeleteChatsSoftDeletedBefore(dashboardAgentDb, params)); | ||
|
|
||
| const result: ChatRetentionResult = { purged: 0, failed: 0 }; | ||
|
|
||
| try { | ||
| result.purged = await purge({ | ||
| before: new Date(now.getTime() - CHAT_SOFT_DELETE_RETENTION_MS), | ||
| limit, | ||
| }); | ||
| } catch (error) { | ||
| result.failed++; | ||
| logger.error("Dashboard agent chat retention failed", { error }); | ||
| } | ||
|
|
||
| if (result.failed > 0) { | ||
| throw new Error("The dashboard agent chat retention pass failed"); | ||
| } | ||
|
|
||
| return result; | ||
| } | ||
|
|
||
| /** | ||
| * Soft-delete every chat belonging to a deleted organization. The retention sweep above | ||
| * hard-deletes them once the window passes, so the org-deletion request never runs a | ||
| * cross-database hard delete. | ||
| * Soft-delete every chat belonging to a deleted organization. Retention hard-deletes them | ||
| * once the window passes, so the org-deletion request never runs a cross-database hard delete. | ||
| */ | ||
| export async function purgeDashboardAgentChatsForOrganization(params: { | ||
| organizationId: string; |
Contributor
There was a problem hiding this comment.
🔍 Chat hard-delete now depends entirely on the agent project being deployed
purgeDashboardAgentChatsForOrganization still only soft-deletes on org deletion, and the webapp no longer has any hard-delete path (sweepDashboardAgentSoftDeletedChats is gone). The eventual erasure of a deleted organization's chat transcripts now happens only if the separate agent Trigger project is deployed and its 03:00 UTC task runs. For a deployment where the dashboard agent datastore exists but the agent project isn't deployed (or its schedule is disabled), soft-deleted chats — including org-deletion purges — are retained indefinitely. Worth confirming that's acceptable for the data-deletion guarantee.
(Refers to lines 9-17)
Was this helpful? React with 👍 or 👎 to provide feedback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The dashboard agent's upkeep — retention deletes and the investigation sweep — ran as cron jobs on the webapp's common worker, even though it only touches the agent's own datastore. This moves that upkeep into the agent's Trigger project as scheduled tasks (TRI-13182).
What's inside
Retention —
internal-packages/dashboard-agent/src/maintenance.ts, a daily task (03:00 UTC). Deletes turn evals older than 30 days, hard-deletes chats soft-deleted more than 30 days ago, and purges terminal watches and submission rows older than 7 days. It used to run every 5 minutes; nothing needs a hard delete that fast, so it is daily now, draining in bounded batches and warning if it hits the cap. It retries (3 attempts) because the next run is a day away. It connects withDASHBOARD_AGENT_DATABASE_URL, falling back toDATABASE_URLlike every other task in the package (the deletes are confined to the agent's own Postgres schema), and skips when neither is set.Investigation sweep —
src/investigation-sweep.ts, every 5 minutes, same as before: settles investigation cards stuckin_progress(30-minute window, attempt cap, force-abandon note). It keeps the fast cadence because it fixes live state the UI is showing.What stays in the webapp. The watch finalize/deliver sweep and batch rearm: they cover a dead agent-side tick chain — a backstop can't live inside the thing it backstops — and they need the main database and the alerts worker. The org-deletion chat purge also stays: deletion must not depend on the agent project being deployed. The removed cron job keeps a cron-less tombstone entry so already-queued items drain cleanly; remove it in a follow-up.
Test plumbing — the drizzle migration replayer that webapp tests hand-rolled is now exported once from
@internal/dashboard-agent-db/testing; the moved tests live in the agent package assrc/*.test.tsagainst real Postgres.Testing
Agent package: retention passes (backlog drain, batch cap, no-op guard, chat-delete cascade) and the sweep, on testcontainers Postgres. Webapp: the watch/chat suites, plus a test that a settlement card stops the dashboard spinner. Full typecheck on both.