[Python][CI] test_dlpack is failing on some nightly jobs allocating unexpected memory

Describe the bug, including details regarding any error messages, version, and platform.

Several jobs:

are failing for the test_dlpack test with some variants of:

 ________________________ test_dlpack[value_type6-int32] ________________________

args = (), kwargs = {'np_type_str': 'int32', 'value_type': DataType(int32)}
allocated_bytes = 7440
@py_assert1 = <cyfunction total_allocated_bytes at 0x7fc798db2b70>
@py_assert3 = 3728, @py_assert5 = False
@py_format7 = '3728\n{3728 = <cyfunction total_allocated_bytes at 0x7fc798db2b70>\n{<cyfunction total_allocated_bytes at 0x7fc798db2b70> = pa.total_allocated_bytes\n}()\n} == 7440'
@py_format9 = 'assert 3728\n{3728 = <cyfunction total_allocated_bytes at 0x7fc798db2b70>\n{<cyfunction total_allocated_bytes at 0x7fc798db2b70> = pa.total_allocated_bytes\n}()\n} == 7440'

    @wraps(f)
    def wrapper(*args, **kwargs):
        allocated_bytes = pa.total_allocated_bytes()
        try:
            return f(*args, **kwargs)
        finally:
>           assert pa.total_allocated_bytes() == allocated_bytes
E           assert 3728 == 7440
E            +  where 3728 = <cyfunction total_allocated_bytes at 0x7fc798db2b70>()
E            +    where <cyfunction total_allocated_bytes at 0x7fc798db2b70> = pa.total_allocated_bytes

Component(s)

Continuous Integration, Python

This started to fail on the main CI as well (eg #44720).

I don't directly see any difference in the conda env between the failing build and the last build on main (which is still green) ..

I can't reproduce locally with:

$ PYTHON=3.11 PANDAS=latest NUMPY=latest docker compose build conda-python-pandas
$ PYTHON=3.11 PANDAS=latest NUMPY=latest docker compose run conda-python-pandas

The test is successful:

opt/conda/envs/arrow/lib/python3.11/site-packages/pyarrow/tests/test_dlpack.py ............s
...
7539 passed, 231 skipped, 23 xfailed, 2 xpassed, 15 warnings in 202.04s (0:03:22)

The only skipped test is the cuda requiring one, the same skipped in CI:

SKIPPED [1] opt/conda/envs/arrow/lib/python3.11/site-packages/pyarrow/tests/test_dlpack.py:133: could not import 'pyarrow.cuda': No module named 'pyarrow._cuda'

apacheGH-44728: [Python] Investigate failing test_dlpack test on pand…

GH-44728: [Python] Trigger manual Garbage collection before checking …

Issue resolved by pull request 44793
#44793

Navigation Menu

Repository navigation

[Python][CI] test_dlpack is failing on some nightly jobs allocating unexpected memory #44728

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Python][CI] test_dlpack is failing on some nightly jobs allocating unexpected memory #44728

Description

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Activity

jorisvandenbossche commented on Nov 14, 2024

raulcd commented on Nov 20, 2024

raulcd commented on Nov 21, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions