Skip to content

feat(kubernetes): add sandbox disruption protection - #2692

Open
varshaprasad96 wants to merge 3 commits into
NVIDIA:mainfrom
varshaprasad96:feat/kubernetes-sandbox-pdb
Open

feat(kubernetes): add sandbox disruption protection#2692
varshaprasad96 wants to merge 3 commits into
NVIDIA:mainfrom
varshaprasad96:feat/kubernetes-sandbox-pdb

Conversation

@varshaprasad96

@varshaprasad96 varshaprasad96 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds opt-in, time-bounded PodDisruptionBudget management for Kubernetes sandboxes. Operators enable the capability and set a maximum duration, while individual sandbox requests choose whether to use it and for how long.

Related Issue

Partially addresses #2616 by implementing optional per-sandbox PDB protection. Disruption recovery status, checkpointing, and portable-storage recovery remain outside this PR.

Changes

  • Add disabled-by-default Kubernetes driver and Helm configuration for disruption protection, with a configurable maximum duration.
  • Accept per-sandbox durations such as 30m or 4h, validate them against the operator maximum, and persist an absolute UTC deadline on the Sandbox and PDB.
  • Create a policy/v1 PDB with minAvailable: 1, unhealthyPodEvictionPolicy: AlwaysAllow, exact sandbox pod labels, and a Sandbox owner reference.
  • Reconcile expired, missing, orphaned, and partially owned PDBs; clean up directly during sandbox deletion with owner-reference garbage collection as a fallback.
  • Add the required Kubernetes RBAC, Helm tests, driver unit tests, a focused kind-based Kubernetes e2e test, and operator/user documentation.

Testing

  • mise run pre-commit passes
  • mise run ci passes
  • Unit tests added/updated
  • E2E tests added/updated: focused kubernetes_disruption_protection test passed on a dedicated kind cluster

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mrunalp mrunalp added this to the OpenShell Beta milestone Aug 10, 2026
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
@varshaprasad96
varshaprasad96 marked this pull request as ready for review August 11, 2026 23:47
@varshaprasad96
varshaprasad96 marked this pull request as draft August 11, 2026 23:47
@varshaprasad96
varshaprasad96 marked this pull request as ready for review August 11, 2026 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants