diff --git a/docs/how-to-guides/shared-libraries.rst b/docs/how-to-guides/shared-libraries.rst index 55a9b2c10..b3112ae6e 100644 --- a/docs/how-to-guides/shared-libraries.rst +++ b/docs/how-to-guides/shared-libraries.rst @@ -187,29 +187,20 @@ strategies for folding a library built in a subproject into a wheel built with to be within the Python package's tree, or rely on ``meson-python`` to fold it into the wheel when it'd otherwise be installed to ``libdir``. -Option (1) tends to be easier, so unless the library of interest cannot be -built as a static library or it would inflate the wheel size too much because -it's needed by multiple Python extension modules, we recommend trying option -(1) first. - -A typical C or C++ project providing a library to link against tends to provide -(a) one or more ``library()`` targets, which can be built as shared, static, or both, -and (b) headers, pkg-config files, tests and perhaps other development targets -that are needed to use the ``library()`` target(s). One of the challenges to use -such projects as a subproject is that the headers and other installable targets -are targeting system locations (e.g., ``/include/``) which isn't supported -by wheels and hence ``meson-python`` errors out when it encounters such an install -target. This is perhaps the main issue one encounters with subproject usage, -and the following two sections discuss how options (1) and (2) can work around -that. +Static linking tends to be easier, and it is the recommended solution, unless +the library of interest cannot be built as a static library or it would +inflate the wheel size too much because it's needed by multiple Python +extension modules. Static library from subproject ------------------------------ -The major advantage of building a library target as static and folding it directly -into an extension module is that no targets from the subproject need to be installed. -To configure the subproject for this use case, add the following to the -``pyproject.toml`` file of your package: +The major advantage of building a library target as static and folding it +directly into an extension module is that the RPATH or the DLL search path do +not need to be adjusted and no targets from the subproject need to be +installed. To ensures that ``library()`` targets are built as static, and that +no parts of the subprojects are installed, the following configuration can be +added in ``pyproject.toml`` to ensure the relevant options are passed to Meson: .. code-block:: toml @@ -217,8 +208,6 @@ To configure the subproject for this use case, add the following to the setup = ['--default-library=static'] install = ['--skip-subprojects'] -This ensures that ``library`` targets are built as static, and nothing gets installed. - To then link against the static library in the subproject, say for a subproject named ``bar`` with the main library target contained in a ``bar_dep`` dependency, add this to your ``meson.build`` file: @@ -235,30 +224,37 @@ add this to your ``meson.build`` file: install: true, ) -That is all! - Shared library from subproject ------------------------------ -If we can't use the static library approach from the section above and we need -a shared library, then we must have ``install: true`` for that shared library -target. This can only work if we can pass some build option to the subproject -that tells it to *only* install the shared library and not headers or other -targets that we don't need. Install tags don't work per subproject, so -this will look something like: +Sometimes it may be necessary or preferable to use dynamic linking to a shared +library provided in a subproject, for example to avoid inflating the wheel +size having multiple copies of the same object code in different extension +modules using the same library. In this case, the subproject needs to install +the shared library in the usual location in ``libdir``. ``meson-python`` +will automatically include it into the wheel in +``..mesonpy.libs`` just like an internal shared library. + +Most projects, however, install more than the shared library and the extra +components, such as header files or documentation, should not be included in +the Python wheel. Projects may have configuration options to disable building +and installing additional components, in this case, these options can be +passed to the ``subproject()`` call: .. code-block:: meson foo_subproj = subproject('foo', default_options: { - # This is a custom option - if it doesn't exist, can you add it - # upstream or in WrapDB? - 'only_install_main_lib': true, + 'docs': 'disabled', }) foo_dep = foo_subproj.get_variable('foo_dep') -Now we can use ``foo_dep`` like a normal dependency, ``meson-python`` will -include it into the wheel in ``..mesonpy.libs`` just like an -internal shared library that targets ``libdir`` (see -:ref:`internal-shared-libraries`). -*Remember: this method doesn't support Windows (yet)!* +Install tags do not work per subproject, therefore to exclude other parts of +the subproject from being included in the wheel, we need to resort to +``meson-python`` install location filters using the +:option:`tool.meson-python.wheel.exclude` build option: + +.. code-block:: toml + + [tool.meson-python.wheel] + exclude = ['{prefix}/include/*'] diff --git a/docs/reference/pyproject-settings.rst b/docs/reference/pyproject-settings.rst index b7806f569..6a7177c10 100644 --- a/docs/reference/pyproject-settings.rst +++ b/docs/reference/pyproject-settings.rst @@ -29,9 +29,9 @@ use them and examples. .. option:: tool.meson-python.limited-api A boolean indicating whether the extension modules contained in the - Python package target the `Python limited API`__. Extension + Python package target the `Python limited API`_. Extension modules can be compiled for the Python limited API specifying the - ``limited_api`` argument to the |extension_module()|__ function + ``limited_api`` argument to the |extension_module()|_ function in the Meson Python module. When this setting is set to true, the value ``abi3`` is used for the Python wheel filename ABI tag. @@ -63,8 +63,36 @@ use them and examples. Extra arguments to be passed to the ``meson install`` command. - -__ https://docs.python.org/3/c-api/stable.html?highlight=limited%20api#stable-application-binary-interface -__ https://mesonbuild.com/Python-module.html#extension_module +.. option:: tool.meson-python.wheel.exclude + + List of glob patterns matching paths of files that must be excluded from + the Python wheel. The accepted glob patterns are the ones implemented by + the Python :mod:`fnmatch` with case sensitive matching. The paths to be + matched are as they appear in the Meson introspection data, namely they are + rooted in one of the Meson install locations: ``{bindir}``, ``{datadir}``, + ``{includedir}``, ``{libdir_shared}``, ``{libdir_static}``, et cetera. + + Inspecting the `Meson introspection data`_ may be useful to craft the exclude + patterns. It is accessible as the ``meson-info/intro-install_plan.json`` JSON + document in the build directory. + + This configuration setting is measure of last resort to exclude installed + files from a Python wheel. It is to be used when the project includes + subprojects that do not allow fine control on the installed files. Better + solutions include the use of Meson install tags and excluding subprojects + to be installed via :option:`tool.meson-python.args.install`. + +.. option:: tool.meson-python.wheel.include + + List of glob patterns matching paths of files that must not be excluded + from the Python wheel. All files recorded for installation in the Meson + project are included in the Python wheel unless matching an exclude glob + pattern specified in :option:`tool.meson-python.wheel.exclude`. An include + glob pattern is useful exclusively to limit the effect of an exclude + pattern that matches too many files. + +.. _python limited api: https://docs.python.org/3/c-api/stable.html?highlight=limited%20api#stable-application-binary-interface +.. _extension_module(): `https://mesonbuild.com/Python-module.html#extension_module +.. _meson introspection data: https://mesonbuild.com/IDE-integration.html#install-plan .. |extension_module()| replace:: ``extension_module()`` diff --git a/mesonpy/__init__.py b/mesonpy/__init__.py index 8192970a9..77b0be327 100644 --- a/mesonpy/__init__.py +++ b/mesonpy/__init__.py @@ -16,6 +16,7 @@ import contextlib import copy import difflib +import fnmatch import functools import importlib.machinery import io @@ -112,14 +113,32 @@ class InvalidLicenseExpression(Exception): # type: ignore[no-redef] } -def _map_to_wheel(sources: Dict[str, Dict[str, Any]]) -> DefaultDict[str, List[Tuple[pathlib.Path, str]]]: +def _compile_patterns(patterns: List[str]) -> Callable[[str], bool]: + if not patterns: + return lambda x: False + func = re.compile('|'.join(fnmatch.translate(os.path.normpath(p)) for p in patterns)).match + return typing.cast('Callable[[str], bool]', func) + + +def _map_to_wheel( + sources: Dict[str, Dict[str, Any]], + exclude: List[str], + include: List[str], +) -> DefaultDict[str, List[Tuple[pathlib.Path, str]]]: """Map files to the wheel, organized by wheel installation directory.""" wheel_files: DefaultDict[str, List[Tuple[pathlib.Path, str]]] = collections.defaultdict(list) packages: Dict[str, str] = {} + excluded = _compile_patterns(exclude) + included = _compile_patterns(include) for key, group in sources.items(): for src, target in group.items(): - destination = pathlib.Path(target['destination']) + target_destination = os.path.normpath(target['destination']) + + if excluded(target_destination) and not included(target_destination): + continue + + destination = pathlib.Path(target_destination) anchor = destination.parts[0] dst = pathlib.Path(*destination.parts[1:]) @@ -580,6 +599,10 @@ def _string_or_path(value: Any, name: str) -> str: 'limited-api': _bool, 'allow-windows-internal-shared-libs': _bool, 'args': _table(dict.fromkeys(_MESON_ARGS_KEYS, _strings)), + 'wheel': _table({ + 'exclude': _strings, + 'include': _strings, + }), }) table = pyproject.get('tool', {}).get('meson-python', {}) @@ -828,6 +851,10 @@ def __init__( # from the package, make sure the developers acknowledge this. self._allow_windows_shared_libs = pyproject_config.get('allow-windows-internal-shared-libs', False) + # Files to be excluded from the wheel + self._excluded_files = pyproject_config.get('wheel', {}).get('exclude', []) + self._included_files = pyproject_config.get('wheel', {}).get('include', []) + def _run(self, cmd: Sequence[str]) -> None: """Invoke a subprocess.""" # Flush the line to ensure that the log line with the executed @@ -914,7 +941,7 @@ def _manifest(self) -> DefaultDict[str, List[Tuple[pathlib.Path, str]]]: sources[key][target] = details # Map Meson installation locations to wheel paths. - return _map_to_wheel(sources) + return _map_to_wheel(sources, self._excluded_files, self._included_files) @property def _meson_name(self) -> str: diff --git a/tests/packages/subproject/pyproject.toml b/tests/packages/subproject/pyproject.toml index 79b6ea07f..d339a1bb7 100644 --- a/tests/packages/subproject/pyproject.toml +++ b/tests/packages/subproject/pyproject.toml @@ -5,3 +5,15 @@ [build-system] build-backend = 'mesonpy' requires = ['meson-python'] + +[tool.meson-python.wheel] +exclude = [ + # Meson before version 1.3.0 install data files in + # ``{datadir}/{project name}/``, later versions install + # in the more correct ``{datadir}/{subproject name}/``. + '{datadir}/*/data.txt', + '{py_purelib}/dep.*', +] +include = [ + '{py_purelib}/dep.py', +] diff --git a/tests/packages/subproject/subprojects/dep/data.txt b/tests/packages/subproject/subprojects/dep/data.txt new file mode 100644 index 000000000..cfa9c9661 --- /dev/null +++ b/tests/packages/subproject/subprojects/dep/data.txt @@ -0,0 +1 @@ +excluded via tool.meson-python.wheel.exclude diff --git a/tests/packages/subproject/subprojects/dep/meson.build b/tests/packages/subproject/subprojects/dep/meson.build index a7567f449..34c6b7508 100644 --- a/tests/packages/subproject/subprojects/dep/meson.build +++ b/tests/packages/subproject/subprojects/dep/meson.build @@ -7,3 +7,5 @@ project('dep') py = import('python').find_installation() py.install_sources('dep.py') + +install_data('data.txt')