python: nanobind tidying, auto-build, CI testing

[lgritz]

A collection of changes related to our switch from pybind11 to nanobind. None of these change the binding code itself! Mostly related to build and testing odds and ends.

Sorry for the large number of things mixed together; each change individually either does nothing or doesn't work on its own, they are a collection of things that are all needed to fully enable working nanobind across all our CI tests.

• CI: Build and test both python back ends! Except for Windows, only do nanobind there, since it's really slow to build the pybind11 bindings.

• auto-builder for nanobind if not found

  • src/cmake/build_nanobind.cmake, mirroring build_pybind11.cmake
  • With auto-build of nanobind, no need to use pip to install nanobind.
  • Take care to share robin-map with OIIO core (both OIIO and nanobind need robin-map).

• CMakeLists.txt : Change OIIO_PYTHON_BINDINGS_BACKEND initialization to use set_cache() to allow env variables to easily control it.

• Testing :

  • Fixes that make the tests run properly when only nanobind is built (it previously worked for "both" and of course for "pybind11").
  • Add tests to the nanobind set that were overlooked: filters, docs-examples-python, and the hwy-enabled python-imagebufalgo variant.
  • Add missing PYTHONPATH to python-imagebufalgo.hwy and openexr-copy tests, which relied on the ambient shell already having it set (true in CI via ci-startup.bash, not for a plain local ctest run).
  • Semi-related cleanup: Don't have docs-examples-python depend on docs-examples-cpp having already run to create its ref symlink; create it ourselves if missing. Fixes failures when running testing subset that includes docs-examples-python but not docs-examples-cpp.

• pythonutils.cmake: comment out a now-redundant find_package call for nanobind, since externalpackages.cmake already resolves it before this macro runs.

• Fix discover_nanobind_cmake_dir(): it was a macro, so its return could abort the entire including file (externalpackages.cmake) rather than just itself, whenever a stale cached nanobind_DIR was present -- silently skipping every dependency check after the Python section. Convert to function.

• Robin-map:

  • Raise robinmap minimum to 1.3, as needed by nanobind.
  • Remove our bespoke FindRobinmap.cmake module in favor of using modern robinmap's exported cmake configs (including cleanup of the changed target names).

• Suppress nanobind reference leak warnings for python 3.9, allow for 3.10+. We were only getting these for 3.9, not newer Python, so discussion with Wenzel concluded that it's probably just an old python cleanup problem and not worth tracking down unless we see it surface with a newer Python release.

Assisted-by: Claude Code / Sonnet 5

[lgritz]

@soswow I would love your review of this, since it builds directly on your work.

[soswow]

@soswow I would love your review of this, since it builds directly on your work.

by the end of my today 👍

[lgritz]

This is in a sense not directly related to the present PR, but people using both OIIO and OSL may be interested in AcademySoftwareFoundation/OpenShadingLanguage#2142 that is relevant to how we're going to migrate the projects safely to nanobind, since mixing and matching an OIIO and OSL where one uses pybind11 and the other uses nanobind will fail if you use any OSL API calls that involve types imported from OIIO (luckily, there is only one such call!).

[lgritz]

@soswow Ping. No rush, but also... I would like to incorporate this as soon as we can so that we are fully testing both bindings in CI.

[soswow]

my bad. looking at it right now.

[soswow]

Just to confirm - you said in PR description that we don't want to build both for windows builds. But we do both here.

[lgritz]

I think I updated the code subsequently. I will edit the commit message when I merge.

[soswow]

src/build-scripts/ci-requirements-nanobind.txt can be killed now, since we don't do pip install any more.
Also, we can update INSTALL.md part:

nanobind discoverable by CMake, or installed in the active Python 
environment so `python -m nanobind --cmake_dir` works

to something else, like

* [nanobind](https://github.com/wjakob/nanobind) >= 2.8 (tested through 2.13). 
If not found at build time, this will be automatically downloaded and built 
(same pattern as pybind11 / fmt). A system or pip install is still fine if you 
already have one discoverable by CMake (`nanobind_DIR`, or `python -m nanobind --cmake_dir`).

[soswow]

Do we really want it to be PUBLIC?
would target_compile_options (nanobind-static PRIVATE -Wno-error=format-nonliteral) do?
I am not good at this. Asking LLMs to explain this to me, but I understand this might hide (not even warning) actual warnings even in non-third-party code?

[lgritz]

I found PUBLIC necessary, because the warning is encountered in the header, so it needs to be suppressed both in the build of nanobind itself (that's what PRIVATE would cover), and also in our files where we include the nanobind headers. Note that this declaration only suppresses the warning within our python bindings (the only part that uses the nanobind-static target), not across the whole codebase.

But I will double check one more time before merging, to see if there is a more localized way to suppress the warning. I'm not super worried about it. The fact that we weren't getting warned before means that we aren't doing it in our code currently -- anywhere -- which makes it seems unlikely that we will start abusing it now. I assume this is something nanobind will eventually fix on its end, but in the mean time, it's really hard to control for things that 3rd party libraries do that happen to warn at the high warning levels we prefer.

[soswow]

LGTM. up to you if you want to act on any of my comments.