Fix array_map optimization with non-literal function or non-literal args - #23254
Conversation
ea294a8 to
9597e62
Compare
Non-literal expressions must be evaluated once and memoized to maintain semantics.
9597e62 to
9f56716
Compare
| @@ -0,0 +1,174 @@ | |||
| --TEST-- | |||
| array_map(): foreach optimization - PFAs with non-literal args are not optimizable | |||
There was a problem hiding this comment.
This is unfortunate, since it kills many PFA use cases of this optimization. Would it be possible to translate this to:
$pfa = plusn(?, $obj->value);
$result = [];
foreach (range(1, 2) as $key => $val) {
$result[$key] = $pfa($val);
}
instead of
$result = [];
foreach (range(1, 2) as $key => $val) {
$result[$key] = plusn($val, $obj->value);
}
? Potentially an OPcache pass could later inline $pfa if it knows this is safe to keep compiler complexity low?
There was a problem hiding this comment.
I've tried moving the entire array_map optimization to the optimizer with the help of an LLM, in a separate branch: master...arnaud-lb:php-src:array-map-inline-optimizer.
Moving everything to the optimizer allows to inline array_map only when we know that the PFA is going to be desugared.
One difficulty was that it's the first time we increase the number of opcode in the optimizer, so we needed a non-trivial zend_optimizer_insert_oplines() helper.
This is a lot of code, so I wouldn't merge such a change in RC. Doing only the PFA sinking/desugaring part in the optimizer would be similarly complex.
Maybe we can merge this PR as-is, and pursue the optimizer part in master only?
There was a problem hiding this comment.
Maybe we can merge this PR as-is, and pursue the optimizer part in master only?
Yes, I didn't mean to say that the Optimizer part should ship in PHP 8.6. My suggestion was to “define another temporary for otherwise unoptimizable PFA” rather than disabling the optimization entirely (and then a follow-up in master could inline the temporary where possible). Not sure if that would end up simpler or more complicated than this fix.
Overall the complexity of the optimization in the compiler has already grown (much) further than I initially anticipated and this is another 80 lines to fix a bug, that's why I wanted to make sure that the solution we've selected here is the best one given the constraints.
If it works and you are confident that it is correct and stable, then I'm happy.
There was a problem hiding this comment.
I have mixed feelings about inlining with an unoptimized PFA. In theory that's still beneficial as we avoid the expensive VM reentry, but on the other hand this increases opcode size for less benefits. I wasn't able to measure a benefit or regression in symfony and phpstan benchmarks (with all array_map inlined, including non-PFA ones since these projects do not use PFAs yet).
It adds a few more lines to this PR, too, so I would prefer merging this as-is for now.
Regarding the optimizer, sinking a PFA is likely at least as complex as generating the whole loop, but I haven't explored this enough to be sure, so I will come back to this later. We can still decide to inline array maps with unoptimizable PFAs when we implement the optimizer side.
There was a problem hiding this comment.
It adds a few more lines to this PR, too, so I would prefer merging this as-is for now.
I'm okay with that.
* PHP-8.6: (61 commits) zend_hrtime: use CLOCK_MONOTONIC instead of CLOCK_MONOTONIC_RAW (php#23790) sapi/cli: Fix built-in server truncating responses after a partial write Fix phpGH-24006: Skip gh18431.phpt when libzip lacks progress callbacks (php#24009) Updated to version 2026.5 (2026e) Fix phpGH-23896: Assertion failure in zend_call_function() after a throwing deprecation NEWS ext/gd: fix undefined behavior with GIFs with problematic LZW compression data Document missing deprecation entries for PHP 8.6 (php#23972) ext/tidy: Reject tidyNode use after the document is reparsed ext/bcmath: Clear the sign of BcMath\Number results that truncate to zero Fix phpGH-23980: ZEND_ASSERT violation @ ZEND_INCLUDE_OR_EVAL (include/eval run with a pending exception) Fix phpGH-23842: skipLazyInitialization() copies unresolved constant defaults Fall back to epoll_wait when epoll_pwait2 is unavailable at runtime (php#23825) openssl: Fix memory leak by doing early salt validation ext/libxml: Keep SimpleXML children alive across reconstruction Fix phpGH-23741: pdo_dblib use-after-free of statement error state NEWS Fix property hook escape analysis causing misoptimization Fix __isset escape analysis causing misoptimization Fix memory leak when closing a statement on a killed connection ...
* PHP-8.6: Fix array_map optimization with non-literal function or non-literal args (#23254)
* upstream/master: (34 commits) Remove the mysqlnd_reverse_api feature (php#21409) ext/intl: Fix grapheme_strstr() and grapheme_stristr() match offsets after supplementary characters (php#24012) pdo: refactor pdo_stmt_construct() to use zend_object* rather than zval* (php#24019) Zend: Skip frameless registration for temporary modules ext/curl/config.m4: include string.h for test using strncmp() ext/soap/tests/bugs/bug62900.phpt: make use of TEST_PHP_ vars sapi/cli/tests/php_cli_server_ipv6_error_message.phpt: use TEST_PHP_ vars Zend/tests: organize some tests with sub directories (11) (php#22731) zend_hrtime: remove zend_hrtime_posix_clock_id (php#24016) Zend/lazy_objects: add const qualifiers Zend: use zend_get_gc_buffer_add_fcc for lazy objects Zend: use uint32_t type for lazy_properties_count reflection: mark reflection_class_new_lazy as static always inline reflection: use ZEND_FCC_INITIALIZED macro for lazy objects Zend/lazy_objects: drop variable which is used once Zend: use zend_object* parameter type instead of zval* for closure API (php#24015) Fix HashTable UAF when rebound from a parameter __toString() Fix phpGH-22051: report errors from SQLite3Result reset and finalize Update IR (php#24013) Fix array_map optimization with non-literal function or non-literal args (php#23254) ...
Non-literal expressions must be evaluated once and memoized to maintain semantics:
Unfortunately, we can't memoize pre-bound PFA arguments without breaking pass-by-reference, so we disable the optimization if the callback is a PFA with non-literal arguments. We could improve this when the function is known and we can determine that the argument is not passed by ref.
Bug found by Ryan @ Calif.io.