Fix segfault with -W error on macOS (built with swig 4.3.1)

[ysheikh2]

Problem

On macOS, import pymupdf crashes with SIGSEGV when warnings are errors, for example python -W error or pytest -W error. Without -W error it prints three warnings.

$ python -W default -c "import pymupdf"
<frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute
<frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute
<frozen importlib._bootstrap>:488: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

$ python -X faulthandler -W error -c "import pymupdf"
Fatal Python error: Segmentation fault
  ...
  File ".../pymupdf/mupdf.py", line 10 in <module>
  ...
  File ".../pymupdf/__init__.py", line 395 in <module>
$ echo $?
139

Reproduced with the 1.28.2 wheel from PyPI (cp310-abi3-macosx_11_0_arm64, CPython 3.12.10), and with a local build of main (30d72fc, MuPDF 1.28.5).

This is #4392. On Linux and Windows the 1.28.2 wheels are built with swig 4.4.1 and don't crash. The macOS wheels (arm64 and x86_64) are built with swig 4.3.1, because setup.py pins swig==4.3.1 on macOS with Python < 3.13 to avoid the director problem in swig/swig#3279. I believe this is why @cedral still saw the crash in #4392 with recent releases.

Root cause

With swig < 4.4, SWIG's runtime creates SwigPyPacked, SwigPyObject and swigvarlink with PyType_FromSpec() and names without a module, so CPython 3.12 warns. With -W error the warning becomes an exception and PyType_FromSpec() returns NULL. SWIG ignores the NULL and then calls PyObject_New(swig_varlinkobject, NULL). This is swig/swig#3061. It happens in 2 places.

1. Module init of _mupdf (and _extra, which is imported first).

   * frame #0: libpython3.12.dylib`_PyObject_New + 12      (EXC_BAD_ACCESS, address=0x20)
     frame #1: _mupdf.so`PyInit__mupdf + 3276
   

2. At exit. SWIG_Python_DestroyModule() in _extra calls SWIG_globals(). extra has no global variables, so this creates the globals object and a new swigvarlink type for the first time, during Py_FinalizeEx(). This is the sys:1: warning above. Whether it crashes depends on the warning filters left at exit. For example, python -W error -c "import pymupdf; open('some.file')" exits 139, as does pytest -W error on a test that leaves a file open.

   frame #1: libpython3.12.dylib`_PyType_FromMetaclass_impl + 3332
   frame #2: _extra.so`SWIG_Python_DestroyModule(_object*) + 308
   frame #3: libpython3.12.dylib`capsule_dealloc + 64
   ...
   frame #7: libpython3.12.dylib`finalize_modules + 1416
   frame #8: libpython3.12.dylib`Py_FinalizeEx + 240
   

swig 4.4.0 fixed the warning, which removes both crashes, but on macOS the swig version can't be raised until swig/swig#3279 is fixed.

Fix

• src/__init__.py imports extra and mupdf inside warnings.catch_warnings(), ignoring only DeprecationWarning with the message builtin type ... has no __module__ attribute. Other warnings, and these warnings anywhere else, are unchanged.
• src/extra.i calls SWIG_globals() in %init, so the globals object (and its type) is created during import, where the warning is ignored, instead of at exit.

With swig >= 4.4 the types are named swig_runtime_data5.SwigPyObject etc. and the warning is never raised. There, the filter has no effect and SWIG_globals() only creates one small unused object.

tests/test_general.py:test_4392() now expects pytest -Werror and python -Werror -c "import pymupdf" to succeed with all swig versions, instead of expecting a SEGV with swig < 4.4. It also checks python -Werror -c "import pymupdf; open(path)", and that python -W default -c "import pymupdf" prints none of these warnings.

Testing

Built locally with pip wheel ./PyMuPDF on macOS 27.0.1 arm64, CPython 3.12.10. setup.py chose swig 4.3.1 as it does for the release wheels.

	main	this branch
python -W default -c "import pymupdf"	4 warnings	no warnings
python -W error -c "import pymupdf"	exit 139	exit 0
python -W error -c "import pymupdf; open(path)"	exit 139	exit 0
pytest -Werror on a file with import pymupdf	exit -11	exit 5 (no tests collected)
test_4392	fails (e2=-11 e3=-11 e4=-11)	passes
whole test suite	516 passed, test_4392 failed	517 passed

My local build is pymupdf-lite without layout, so I ran tests/test_tables.py and tests/test_configurations.py with PYMUPDF_TEST_USE_LAYOUT=0. Without it, 11 of their tests fail on both main and this branch.

With only the src/__init__.py change, test_4392 still fails with e4=-11, which is why src/extra.i is changed too.

I haven't tested a build with swig >= 4.4 on macOS, or on Linux or Windows.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[ysheikh2]

I have read the CLA Document and I hereby sign the CLA

[ysheikh2]

recheck