Segfault when printing `MemoryError` raised through `PyErr_NoMemory()` from subinterpreter

[chgnrdv]

Crash report

What happened?

Repro:

import _testcapi
_testcapi.run_in_subinterp("[0]*100000000000")

Backtrace:

BaseException_str (self=<optimized out>) at Objects/exceptions.c:120
120	    switch (PyTuple_GET_SIZE(self->args)) {
(gdb) bt
#0  BaseException_str (self=<optimized out>) at Objects/exceptions.c:120
#1  0x00005555556ed00f in PyObject_Str (v=v@entry=0x7ffff75988f0) at Objects/object.c:606
#2  0x0000555555872d45 in print_exception_message (ctx=ctx@entry=0x7fffffffd870, type=type@entry=0x555555b54f20 <_PyExc_MemoryError>, value=0x7ffff75988f0)
    at Python/pythonrun.c:1079
#3  0x0000555555873c59 in print_exception (ctx=ctx@entry=0x7fffffffd870, value=value@entry=0x7ffff75988f0) at Python/pythonrun.c:1226
#4  0x000055555587443a in print_exception_recursive (ctx=ctx@entry=0x7fffffffd870, value=value@entry=0x7ffff75988f0) at Python/pythonrun.c:1507
#5  0x0000555555874af2 in _PyErr_Display (file=file@entry=0x7ffff743fe30, unused=unused@entry=0x0, value=value@entry=0x7ffff75988f0, tb=tb@entry=0x7ffff74f39d0)
    at Python/pythonrun.c:1557
#6  0x0000555555874c91 in PyErr_Display (unused=unused@entry=0x0, value=0x7ffff75988f0, tb=0x7ffff74f39d0) at Python/pythonrun.c:1585
#7  0x000055555588456f in sys_excepthook_impl (module=module@entry=0x7ffff7631370, exctype=<optimized out>, value=<optimized out>, traceback=<optimized out>)
    at ./Python/sysmodule.c:783
#8  0x00005555558845bb in sys_excepthook (module=0x7ffff7631370, args=0x7fffffffd9c0, nargs=<optimized out>) at ./Python/clinic/sysmodule.c.h:101
#9  0x00005555556e691b in cfunction_vectorcall_FASTCALL (func=0x7ffff7631850, args=0x7fffffffd9c0, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/methodobject.c:425
#10 0x00005555556794b8 in _PyObject_VectorcallTstate (tstate=0x7ffff7598938, callable=0x7ffff7631850, args=0x7fffffffd9c0, nargsf=3, kwnames=0x0)
    at ./Include/internal/pycore_call.h:187
#11 0x00005555556795d3 in PyObject_Vectorcall (callable=callable@entry=0x7ffff7631850, args=args@entry=0x7fffffffd9c0, nargsf=nargsf@entry=3, kwnames=kwnames@entry=0x0)
    at Objects/call.c:327
#12 0x0000555555874e20 in _PyErr_PrintEx (tstate=0x7ffff7598938, set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:840
#13 0x0000555555875199 in PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:878
#14 0x00005555558751a9 in PyErr_Print () at Python/pythonrun.c:884
#15 0x0000555555875b77 in PyRun_SimpleStringFlags (command=<optimized out>, flags=flags@entry=0x7fffffffda70) at Python/pythonrun.c:517
#16 0x00007ffff76ad1a4 in run_in_subinterp (self=<optimized out>, args=args@entry=0x7ffff779a210) at ./Modules/_testcapimodule.c:1405
#17 0x00005555556e65c8 in cfunction_call (func=func@entry=0x7ffff7810f50, args=args@entry=0x7ffff779a210, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:551
#18 0x000055555567916b in _PyObject_MakeTpCall (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, callable=callable@entry=0x7ffff7810f50, 
    args=args@entry=0x7ffff7fc1078, nargs=<optimized out>, keywords=keywords@entry=0x0) at Objects/call.c:242
#19 0x00005555556795ab in _PyObject_VectorcallTstate (tstate=0x555555c02338 <_PyRuntime+508728>, callable=0x7ffff7810f50, args=0x7ffff7fc1078, nargsf=<optimized out>, 
    kwnames=0x0) at ./Include/internal/pycore_call.h:185
#20 0x00005555556795d3 in PyObject_Vectorcall (callable=callable@entry=0x7ffff7810f50, args=args@entry=0x7ffff7fc1078, nargsf=<optimized out>, 
    kwnames=kwnames@entry=0x0) at Objects/call.c:327
#21 0x00005555557e3ad6 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, frame=0x7ffff7fc1020, throwflag=throwflag@entry=0)
    at Python/generated_cases.c.h:3761
#22 0x00005555557ed5e4 in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555c02338 <_PyRuntime+508728>) at ./Include/internal/pycore_ceval.h:107
#23 _PyEval_Vector (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, func=func@entry=0x7ffff777b050, locals=locals@entry=0x7ffff778e690, args=args@entry=0x0, 
    argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:1630
#24 0x00005555557ed68b in PyEval_EvalCode (co=co@entry=0x7ffff7767520, globals=globals@entry=0x7ffff778e690, locals=locals@entry=0x7ffff778e690) at Python/ceval.c:582
#25 0x00005555558718cf in run_eval_code_obj (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, co=co@entry=0x7ffff7767520, globals=globals@entry=0x7ffff778e690, 
--Type <RET> for more, q to quit, c to continue without paging--
    locals=locals@entry=0x7ffff778e690) at Python/pythonrun.c:1720
#26 0x00005555558722b7 in run_mod (mod=mod@entry=0x555555c55608, filename=filename@entry=0x7ffff77b5940, globals=globals@entry=0x7ffff778e690, 
    locals=locals@entry=0x7ffff778e690, flags=flags@entry=0x7fffffffdf98, arena=arena@entry=0x7ffff77de320) at Python/pythonrun.c:1741
#27 0x00005555558723cf in pyrun_file (fp=fp@entry=0x555555c2f3b0, filename=filename@entry=0x7ffff77b5940, start=start@entry=257, globals=globals@entry=0x7ffff778e690, 
    locals=locals@entry=0x7ffff778e690, closeit=closeit@entry=1, flags=0x7fffffffdf98) at Python/pythonrun.c:1641
#28 0x00005555558756bb in _PyRun_SimpleFileObject (fp=fp@entry=0x555555c2f3b0, filename=filename@entry=0x7ffff77b5940, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdf98) at Python/pythonrun.c:464
#29 0x0000555555875938 in _PyRun_AnyFileObject (fp=fp@entry=0x555555c2f3b0, filename=filename@entry=0x7ffff77b5940, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdf98) at Python/pythonrun.c:79
#30 0x000055555589dfda in pymain_run_file_obj (program_name=program_name@entry=0x7ffff79790e0, filename=filename@entry=0x7ffff77b5940, skip_source_first_line=0)
    at Modules/main.c:361
#31 0x000055555589e2b7 in pymain_run_file (config=config@entry=0x555555b9cf20 <_PyRuntime+93984>) at Modules/main.c:380
#32 0x000055555589f3fc in pymain_run_python (exitcode=exitcode@entry=0x7fffffffe10c) at Modules/main.c:611
#33 0x000055555589f454 in Py_RunMain () at Modules/main.c:689
#34 0x000055555589f4a8 in pymain_main (args=args@entry=0x7fffffffe150) at Modules/main.c:719
#35 0x000055555589f51d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:743
#36 0x00005555555cf74e in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15

I can see two simultaneous reasons for this behaviour:

• unlike for main interpreter that preallocates memory errors, for subinterpreter interp.exc_state.memerrors_freelist remains equal to NULL, which means that PyErr_NoMemory() always returns interp.static_objects.last_resort_memory_error.
• interp.static_objects.last_resort_memory_error has args field equal to NULL, thus attempt to print it segfaults.

Possible fix:

• assign interp.static_objects.last_resort_memory_error.args to empty tuple:

diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h
index 2deba02a89..e22ffcd1fb 100644
--- a/Include/internal/pycore_runtime_init.h
+++ b/Include/internal/pycore_runtime_init.h
@@ -177,6 +177,7 @@ extern PyTypeObject _PyExc_MemoryError;
                 }, \
                 .last_resort_memory_error = { \
                     _PyObject_HEAD_INIT(&_PyExc_MemoryError) \
+                    .args = (PyObject*)&_Py_SINGLETON(tuple_empty), \
                 }, \
             }, \
         }, \

• initialize memory errors on subinterpreter creation in a similar manner as in the main interpreter, through _PyExc_InitGlobalObjects() or whatever

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (heads/main:e81bd3fa16, Sep 26 2023, 12:14:51) [GCC 10.2.1 20210110]

Linked PRs

• gh-109894: Fix initialization of static MemoryError in subinterpreter #110911
• [3.12] gh-109894: Fix initialization of static MemoryError in subinterpreter (gh-110911) #111238

[AA-Turner]

cc @ericsnowcurrently

[AA-Turner]

@chgnrdv are you able to reproduce on any earlier versions of Python, or is this just 3.13?

A

[chgnrdv]

@AA-Turner , I've bisected this crash to feec49c, so 3.12 is affected too.