Skip to content

Fix array_map optimization with non-literal function or non-literal args - #23254

Merged
arnaud-lb merged 3 commits into
php:PHP-8.6from
arnaud-lb:pfa-bug-3
Sep 30, 2026
Merged

arnaud-lb merged 3 commits into
php:PHP-8.6from
arnaud-lb:pfa-bug-3

Conversation

@arnaud-lb

Copy link
Copy Markdown
Member

Non-literal expressions must be evaluated once and memoized to maintain semantics:

$callback = function ($value) {
    global $callback;
    $callback = function () { return 'changed'; };
    return $value + 1;
};

array_map($callback, [1,2]);

// Expected result: [2,3]
// Actual result: [2,'changed']

Unfortunately, we can't memoize pre-bound PFA arguments without breaking pass-by-reference, so we disable the optimization if the callback is a PFA with non-literal arguments. We could improve this when the function is known and we can determine that the argument is not passed by ref.

Bug found by Ryan @ Calif.io.

Comment thread Zend/zend_compile.c Outdated
@arnaud-lb
arnaud-lb changed the base branch from master to PHP-8.6 September 25, 2026 09:01
@arnaud-lb
arnaud-lb marked this pull request as ready for review September 25, 2026 09:02
@arnaud-lb
arnaud-lb requested a review from dstogov as a code owner September 25, 2026 09:02
Non-literal expressions must be evaluated once and memoized to maintain
semantics.
@@ -0,0 +1,174 @@
--TEST--
array_map(): foreach optimization - PFAs with non-literal args are not optimizable

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unfortunate, since it kills many PFA use cases of this optimization. Would it be possible to translate this to:

$pfa = plusn(?, $obj->value);
$result = [];
foreach (range(1, 2) as $key => $val) {
    $result[$key] = $pfa($val);
}

instead of

$result = [];
foreach (range(1, 2) as $key => $val) {
    $result[$key] = plusn($val, $obj->value);
}

? Potentially an OPcache pass could later inline $pfa if it knows this is safe to keep compiler complexity low?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried moving the entire array_map optimization to the optimizer with the help of an LLM, in a separate branch: master...arnaud-lb:php-src:array-map-inline-optimizer.

Moving everything to the optimizer allows to inline array_map only when we know that the PFA is going to be desugared.

One difficulty was that it's the first time we increase the number of opcode in the optimizer, so we needed a non-trivial zend_optimizer_insert_oplines() helper.

This is a lot of code, so I wouldn't merge such a change in RC. Doing only the PFA sinking/desugaring part in the optimizer would be similarly complex.

Maybe we can merge this PR as-is, and pursue the optimizer part in master only?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can merge this PR as-is, and pursue the optimizer part in master only?

Yes, I didn't mean to say that the Optimizer part should ship in PHP 8.6. My suggestion was to “define another temporary for otherwise unoptimizable PFA” rather than disabling the optimization entirely (and then a follow-up in master could inline the temporary where possible). Not sure if that would end up simpler or more complicated than this fix.

Overall the complexity of the optimization in the compiler has already grown (much) further than I initially anticipated and this is another 80 lines to fix a bug, that's why I wanted to make sure that the solution we've selected here is the best one given the constraints.

If it works and you are confident that it is correct and stable, then I'm happy.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have mixed feelings about inlining with an unoptimized PFA. In theory that's still beneficial as we avoid the expensive VM reentry, but on the other hand this increases opcode size for less benefits. I wasn't able to measure a benefit or regression in symfony and phpstan benchmarks (with all array_map inlined, including non-PFA ones since these projects do not use PFAs yet).

It adds a few more lines to this PR, too, so I would prefer merging this as-is for now.

Regarding the optimizer, sinking a PFA is likely at least as complex as generating the whole loop, but I haven't explored this enough to be sure, so I will come back to this later. We can still decide to inline array maps with unoptimizable PFAs when we implement the optimizer side.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It adds a few more lines to this PR, too, so I would prefer merging this as-is for now.

I'm okay with that.

* PHP-8.6: (61 commits)
  zend_hrtime: use CLOCK_MONOTONIC instead of CLOCK_MONOTONIC_RAW (php#23790)
  sapi/cli: Fix built-in server truncating responses after a partial write
  Fix phpGH-24006: Skip gh18431.phpt when libzip lacks progress callbacks (php#24009)
  Updated to version 2026.5 (2026e)
  Fix phpGH-23896: Assertion failure in zend_call_function() after a throwing deprecation
  NEWS
  ext/gd: fix undefined behavior with GIFs with problematic LZW compression data
  Document missing deprecation entries for PHP 8.6 (php#23972)
  ext/tidy: Reject tidyNode use after the document is reparsed
  ext/bcmath: Clear the sign of BcMath\Number results that truncate to zero
  Fix phpGH-23980: ZEND_ASSERT violation @ ZEND_INCLUDE_OR_EVAL (include/eval run with a pending exception)
  Fix phpGH-23842: skipLazyInitialization() copies unresolved constant defaults
  Fall back to epoll_wait when epoll_pwait2 is unavailable at runtime (php#23825)
  openssl: Fix memory leak by doing early salt validation
  ext/libxml: Keep SimpleXML children alive across reconstruction
  Fix phpGH-23741: pdo_dblib use-after-free of statement error state
  NEWS
  Fix property hook escape analysis causing misoptimization
  Fix __isset escape analysis causing misoptimization
  Fix memory leak when closing a statement on a killed connection
  ...
@arnaud-lb
arnaud-lb merged commit 40a8468 into php:PHP-8.6 Sep 30, 2026
1 of 2 checks passed
arnaud-lb added a commit that referenced this pull request Sep 30, 2026
* PHP-8.6:
  Fix array_map optimization with non-literal function or non-literal args (#23254)
bukka added a commit to bukka/php-src that referenced this pull request Sep 30, 2026
* upstream/master: (34 commits)
  Remove the mysqlnd_reverse_api feature (php#21409)
  ext/intl: Fix grapheme_strstr() and grapheme_stristr() match offsets after supplementary characters (php#24012)
  pdo: refactor pdo_stmt_construct() to use zend_object* rather than zval* (php#24019)
  Zend: Skip frameless registration for temporary modules
  ext/curl/config.m4: include string.h for test using strncmp()
  ext/soap/tests/bugs/bug62900.phpt: make use of TEST_PHP_ vars
  sapi/cli/tests/php_cli_server_ipv6_error_message.phpt: use TEST_PHP_ vars
  Zend/tests: organize some tests with sub directories (11) (php#22731)
  zend_hrtime: remove zend_hrtime_posix_clock_id (php#24016)
  Zend/lazy_objects: add const qualifiers
  Zend: use zend_get_gc_buffer_add_fcc for lazy objects
  Zend: use uint32_t type for lazy_properties_count
  reflection: mark reflection_class_new_lazy as static always inline
  reflection: use ZEND_FCC_INITIALIZED macro for lazy objects
  Zend/lazy_objects: drop variable which is used once
  Zend: use zend_object* parameter type instead of zval* for closure API (php#24015)
  Fix HashTable UAF when rebound from a parameter __toString()
  Fix phpGH-22051: report errors from SQLite3Result reset and finalize
  Update IR (php#24013)
  Fix array_map optimization with non-literal function or non-literal args (php#23254)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants