Skip to content

build(cuda.core): spike scikit-build-core migration - #2607

Draft
kkraus14 wants to merge 9 commits into
NVIDIA:mainfrom
kkraus14:agent/cuda-core-scikit-build-core-spike
Draft

build(cuda.core): spike scikit-build-core migration#2607
kkraus14 wants to merge 9 commits into
NVIDIA:mainfrom
kkraus14:agent/cuda-core-scikit-build-core-spike

Conversation

@kkraus14

@kkraus14 kkraus14 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

What

This spike moves only cuda-core extension compilation and package staging from setuptools to scikit-build-core/CMake. It includes one small cuda-bindings packaging adjustment so editable installs expose their Cython declarations through setuptools' native static .pth strategy.

  • add an explicit CMake/Cython build graph for all 47 POSIX extensions (46 on Windows), including the two handwritten C++ helpers and Windows AOTI import shim
  • keep the target inventory reviewable and fail CMake configuration if it no longer matches the package's .pyx files
  • use target-scoped C++17, include paths, compile definitions, visibility, coverage, and free-threaded settings instead of directory-global compiler state
  • keep a thin in-tree PEP 517 wrapper for the CUDA-major-specific cuda-bindings==<major>.* build requirement and the deprecated pre-2.0 debug compatibility alias; all native scikit-build-core settings pass through untouched
  • select the toolkit from CUDA_PATH (preferred) or CUDA_HOME, optionally accept CUDA_CORE_BUILD_MAJOR, and have CMake parse the exact cuda.h being compiled and reject mismatches
  • move coverage, parallel-build forwarding, and editable build defaults into native scikit-build-core TOML configuration
  • make cuda-bindings editable wheels add their source root using setuptools' built-in data-only .pth, removing all bindings-specific path discovery and runtime helper code from cuda-core
  • model the generated Windows AOTI import library as an imported CMake target while retaining the necessary LIB.exe generation command
  • preserve the existing coverage, free-threaded, editable, package-data, and dual-CUDA wheel-merge contracts
  • move version/readme/package configuration to scikit-build-core and remove the obsolete setup.py and MANIFEST.in
  • update Pixi build dependencies/lock data and source/sdist CI coverage

Python still determines the CUDA major before CMake configures because PEP 517 calls get_requires_for_build_wheel before installing the returned, major-specific bindings dependency. The backend contains no generic CMake-settings parser: it reads only CUDA_PATH/CUDA_HOME and the optional CUDA_CORE_BUILD_MAJOR environment override needed for dependency selection, plus a narrow translator that preserves the documented debug option through the 1.x series with a removal-in-2.0 warning. Compilation and arbitrary CMake settings remain scikit-build-core/CMake concerns.

The target-per-extension, build-tree Cython generation, depfile, target-scoping, and package-relative install patterns follow the useful parts of RAPIDS RMM/cuDF practice without taking a dependency on RAPIDS build infrastructure.

Related to #605.

Validation

  • focused build-backend tests: 37 cuda-core tests and 2 real setuptools wheel/RECORD tests for cuda-bindings
  • normal CPython 3.13 wheel: 47 extensions, including the current _copy_attributes and _copy_ops modules, with no Cython/C++ sources, .pth, bytecode, or stale in-tree binaries
  • coverage wheel: 47 .pyx, 17 .pxi, and 49 .cpp files (47 generated plus 2 helpers) alongside 47 extensions
  • editable wheel: 47 extensions; isolated import cuda.core, both new memory modules, and copy_batch all pass
  • editable cuda-bindings wheel: native static source-root .pth; plain downstream Cython resolves declarations without an explicit -I
  • explicit -Ccmake.build-type=Debug wheel: Debug CMake configuration, Cython --gdb, and _GLIBCXX_ASSERTIONS verified
  • legacy -Cdebug=True wheel: visible FutureWarning, Debug CMake configuration, all 47 extensions, and a valid archive; the setting also succeeds through CUDA-independent sdist hooks
  • two byte-identical CUDA-independent sdists from the current tree, containing all 47 package .pyx files and no generated/test/native artifacts; wheel-from-sdist also exercised
  • persistent CMake build directory reconfigured from CUDA 13 to CUDA 12 without retaining the prior root or major
  • the formerly failing Pixi source-build import cuda.core smoke reproduced locally and now passes against current main
  • AOTI imported-target graph validated with Ninja and Ninja Multi-Config against the declared CMake 3.21 minimum
  • pixi lock --check, Ruff check/format, CMake configure, archive integrity, TOML/YAML/workflow lint, and applicable commit hooks

Draft gaps

  • Windows/MSVC (including actual AOTI shim generation and the free-threaded ABI define) and Linux aarch64 require CI validation.
  • The broader CPython/cibuildwheel matrix remains CI-only.
  • The local merge smoke used structurally equivalent relabeled builds; CI must exercise genuinely distinct CUDA 12 and CUDA 13 artifacts.
  • This spike retains a small backend wrapper. Eliminating all custom backend Python would require a different way to express the CUDA-major-dependent build requirement and, before 2.0, would break the documented legacy debug option.

@kkraus14 kkraus14 added this to the cuda.core next milestone Aug 12, 2026
@kkraus14 kkraus14 added CI/CD CI/CD infrastructure packaging Anything related to wheels or Conda packages cuda.core Everything related to the cuda.core module experiment Describes an investigation or measurement labels Aug 12, 2026
@copy-pr-bot

copy-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.bindings Everything related to the cuda.bindings module label Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module experiment Describes an investigation or measurement packaging Anything related to wheels or Conda packages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant