Migrate from pybind11 to nanobind

[rdspring1]

Library Size -- Nanobind is 52.6% smaller than PyBind11

Size (MB)	Library
1.060	Nanobind
2.238	Pybind11

[github-actions]

Review updated until commit 853280b

Description

• Migrate from pybind11 to nanobind library

• Reduce library size by 52.6% (1.060 MB vs 2.238 MB)

• Update binding framework across the codebase

• Maintain existing functionality with new binding system

PR Reviewer Guide

Here are some key observations to aid the review process:

🧪 PR contains tests

⚡ Recommended focus areas for review

Potential Logic Change in convertToVal In the convertToVal function around line 36, there's an additional null check added: dtype == std::nullopt || v == nullptr ||. This changes the error condition logic and could potentially mask bugs or change behavior when v is nullptr. Need to verify this change is intentional and doesn't introduce regressions. dtype == std::nullopt || v == nullptr || std::get<PrimDataType>(v->dtype().type) == dtype.value()); Complex Number Binding Restructuring The complex number binding around line 1603-1608 has been restructured. The Val* version is now explicitly defined with nb::rv_policy::reference, while the TensorView* version follows the standard pattern. Need to verify this doesn't break complex number operations or change their behavior. ops.def( "complex", [](Val* lhs, Val* rhs) -> Val* { return static_cast<Val* (*)(Val*, Val*)>(complex)(lhs, rhs); }, nb::rv_policy::reference); Type Detection Changes The toPolymorphicValue function has been completely rewritten to use nanobind type detection instead of pybind11. The complex number detection changed from PyComplex_Check to nb::isinstance<std::complex>. This could potentially miss some complex number types or change behavior. Need to verify type detection works correctly for all expected input types. PolymorphicValue toPolymorphicValue(const nb::handle& obj) { if (nb::isinstance<nb::ndarray<nb::pytorch>>(obj)) { return PolymorphicValue(nb::cast<at::Tensor>(obj)); } else if (nb::isinstance<nb::bool_>(obj)) { return PolymorphicValue(nb::cast<bool>(obj)); } else if (nb::isinstance<nb::int_>(obj)) { return PolymorphicValue(nb::cast<int64_t>(obj)); } else if (nb::isinstance<nb::float_>(obj)) { return PolymorphicValue(nb::cast<double>(obj)); } else if (nb::isinstance<std::complex<double>>(obj)) { return PolymorphicValue(nb::cast<std::complex<double>>(obj)); } NVF_THROW("Cannot convert provided nb::handle to a PolymorphicValue."); }

[rdspring1]

!test

[rdspring1]

!test