Merge pull request #378 from oscarbenjamin/pr_msvc_ext

Use MSVC to build python-flint's extension modules

Author: oscarbenjamin

.github/workflows/buildwheel.yml

@@ -31,7 +31,10 @@ jobs:
 
         # We have to set this here rather than in the cibuildwheel config
         # This is probably something to do with \ vs / in paths...
-      - run: echo "PKG_CONFIG_PATH=${{ github.workspace }}/.local/lib/pkgconfig" >> $env:GITHUB_ENV
+      - run: |
+          $pkgConfigPath = "${{ github.workspace }}/.local/lib/pkgconfig"
+          $pkgConfigPath = $pkgConfigPath.Replace('\', '/')
+          echo "PKG_CONFIG_PATH=$pkgConfigPath" >> $env:GITHUB_ENV
         if: ${{ startsWith( matrix.os , 'windows' ) }}
 
       - name: Build wheels

bin/cibw_before_all_windows.sh

@@ -18,3 +18,36 @@ bin/build_dependencies_unix.sh \
     --use-gmp-github-mirror\
     --patch-C23\
     #
+
+# Assumes the standard GitHub Actions Windows 2022 runner layout.
+PATH="$PATH:$(find "/c/Program Files/Microsoft Visual Studio/2022/" -name "Hostx86")/x64/"
+
+if [ "${RUNNER_ARCH}" = "ARM64" ]
+then
+  msvc_machine=arm64
+else
+  msvc_machine=x64
+fi
+
+mkdir -p .local/lib
+cd .local/bin
+for dll_file in libgmp-*.dll libmpfr-*.dll libflint*.dll
+do
+  lib_name=$(basename -s .dll ${dll_file})
+  exports_file=${lib_name}-exports.txt
+  def_file=${lib_name}.def
+  lib_file=${lib_name}.lib
+  name=$(echo ${lib_name}|sed 's/^lib//;s/[-.][0-9].*$//')
+
+  dumpbin //exports ${dll_file} > ${exports_file}
+
+  echo LIBRARY ${lib_name} > ${def_file}
+  echo EXPORTS >> ${def_file}
+  awk 'NR>19 && $4 != "" {print $4 " @"$1}' ${exports_file} >> ${def_file}
+  sed -i 's/$/\r/' ${def_file}
+
+  lib //def:${def_file} //out:${lib_file} //machine:${msvc_machine}
+  rm ${exports_file} ${def_file} ${lib_name}.exp
+  mv ${lib_file} ../lib/${name}.lib
+done
+cd ../..

doc/source/build.rst

@@ -356,23 +356,17 @@ The git repo for ``python-flint`` has a script `bin/cibw_before_all_windows.sh
 <https://github.com/flintlib/python-flint/blob/master/bin/cibw_before_all_windows.sh>`_
 that installs the dependencies under MSYS2 and builds ``GMP``, ``MPFR``,
 ``FLINT``. This script is used for building the Windows binaries for PyPI. We
-use the ``MinGW64`` (``mingw-w64-x86_64``) toolchain for building on Windows
-rather than MSVC because it makes it possible to have a fat build of ``GMP``
+use the ``UCRT64`` (``mingw-w64-ucrt-x86_64``) toolchain under MSYS2 to build
+``GMP``, ``MPFR`` and ``FLINT`` because it makes it possible to have a fat
+build of ``GMP``
 (``--enable-fat``) which bundles micro-architecture specific optimisations for
 ``x86_64`` in a redistributable shared library. This is important for
 performance on modern ``x86_64`` CPUs and is not possible if building ``GMP``
-with MSVC. Since we need to use ``MinGW64`` for building ``GMP`` it is simplest
-to use it for building ``MPFR``, ``FLINT`` and ``python-flint`` as well and
-means that the same Unix-style build scripts can be used for all platforms.
-
-The ``python-flint`` project does not have much experience using MSVC. Possibly
-it would be better to build ``GMP`` using ``MinGW64`` and then build ``MPFR``,
-``FLINT`` and ``python-flint`` using MSVC. It is also possible that it would be
-better to build ``GMP``, ``MPFR``, ``FLINT`` using MinGW64 and then build
-``python-flint`` using MSVC. Someone with more experience with MSVC would need
-to help with this. We would welcome contributions that explain how to build
-``python-flint`` and its dependencies using MSVC and/or that improve the build
-process for distributed binaries on Windows.
+with MSVC. The Python extension modules themselves are then built with MSVC via
+``meson --vsenv`` while linking against the MSYS2-built ``GMP``, ``MPFR`` and
+``FLINT`` libraries through ``pkg-config``. This mixed-toolchain arrangement
+keeps the ``GMP`` fat build while using the standard Windows compiler for the
+extension modules.
 
 
 .. _non_standard_location:

meson.build

@@ -3,7 +3,10 @@ project(
   'cython',
   'c',
   meson_version : '>=1.3.0',
-  default_options: ['python.allow_limited_api=false'],
+  default_options: [
+    'python.allow_limited_api=false',
+    'c_std=c11',
+  ],
 )
 #
 # The minimum versions are because we know that it will not work with earlier

pyproject.toml

@@ -156,6 +156,7 @@ repair-wheel-command = [
 [tool.cibuildwheel.windows]
 before-all = "C:\\msys64\\usr\\bin\\bash bin/cibw_before_all_windows.sh"
 before-build = "pip install wheel delvewheel"
+config-settings = {setup-args = ["--vsenv"], build-dir = "build"}
 repair-wheel-command = [
 """python bin/cibw_repair_wheel_licenses.py {wheel} \
     --license LGPL-3.0-or-later \

setup.py

@@ -24,7 +24,8 @@
 
 if sys.platform == 'win32':
     #
-    # This is used in CI to build wheels with mingw64
+    # This can be used for local Windows builds against a .local install
+    # produced by the MSYS2 UCRT64 dependency build.
     #
     if os.getenv('PYTHON_FLINT_MINGW64'):
         includedir = os.path.join(os.path.dirname(__file__), '.local', 'include')
@@ -35,7 +36,7 @@
         default_lib_dirs += librarydirs
         # Add gcc to the PATH in GitHub Actions when this setup.py is called by
         # cibuildwheel.
-        os.environ['PATH'] += r';C:\msys64\mingw64\bin'
+        os.environ['PATH'] += r';C:\msys64\ucrt64\bin'
         libraries += ["mpfr", "gmp"]
     elif os.getenv('PYTHON_FLINT_MINGW64_TMP'):
         # This would be used to build under Windows against these libraries if