Repository navigation
[in_app_purchase_storekit} Fix more test flakes - #13279
Conversation
There was a problem hiding this comment.
Code Review
This pull request removes the testGetProductsWithStoreKitError and testFailedNetworkErrorPurchase tests from InAppPurchaseStoreKit2PluginTests.swift, adds a line to clear simulated purchase errors, and resets the test session to its default state in StoreKit2TranslatorTests.swift. Review feedback recommends moving the simulated error cleanup and session resetting to tearDown methods in both test classes to prevent memory leaks and state pollution across tests.
Does setting https://developer.apple.com/videos/play/wwdc2023/10140/?time=1337
|
So what I think (pls correct if im wrong) what youre saying is in regards to how the old tests turned the fake error on, but never turned it off with The issue is like if ANY test like Next, when This means the test is order-dependent: it will pass if it runs before a product is cached, and it will fail if another test cached that product first. |
flutter/packages@a8549ae...13d38e0 2026-10-28 mdebbar@google.com [material_ui] Unskip long press drag test on web (flutter/packages#13358) 2026-10-28 louisehsu@google.com [google_maps_flutter_ios, pigeon, webview_flutter_wkwebview] Migrate remaining example apps to uiscene (flutter/packages#13280) 2026-10-27 tarrinneal@gmail.com manually roll flutter and fix flaky video player test that is preventing roll from landing (flutter/packages#13465) 2026-10-27 1961493+harryterkelsen@users.noreply.github.com [material_ui] Unskip web tests that pass on CanvasKit and Skwasm: data_table_test, floating_action_button_test, text_field_cursor_test (flutter/packages#13460) 2026-10-27 kevmoo@users.noreply.github.com [material_ui] Decouple disabled CheckboxListTile state in a11y_assessments (flutter/packages#13177) 2026-10-27 oleg.toplionkin@gmail.com [camera_web] Skip video input devices that cannot be started in availableCameras (flutter/packages#13196) 2026-10-27 happytoday83@naver.com [camera] Add support for zero-shutter-lag capture (flutter/packages#12374) 2026-10-27 louisehsu@google.com [in_app_purchase_storekit} Fix more test flakes (flutter/packages#13279) 2026-10-27 21270878+elliette@users.noreply.github.com [material_ui] Refactor `typography.dart` and `text_theme.dart` in preparation for M3E (flutter/packages#13371) If this roll has caused a breakage, revert this CL and set the roller to dry run mode using the controls here: https://autoroll.skia.org/r/flutter-packages-flutter-autoroll Please CC flutter-ecosystem@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
flutter/packages@a8549ae...13d38e0 2026-10-28 mdebbar@google.com [material_ui] Unskip long press drag test on web (flutter/packages#13358) 2026-10-28 louisehsu@google.com [google_maps_flutter_ios, pigeon, webview_flutter_wkwebview] Migrate remaining example apps to uiscene (flutter/packages#13280) 2026-10-27 tarrinneal@gmail.com manually roll flutter and fix flaky video player test that is preventing roll from landing (flutter/packages#13465) 2026-10-27 1961493+harryterkelsen@users.noreply.github.com [material_ui] Unskip web tests that pass on CanvasKit and Skwasm: data_table_test, floating_action_button_test, text_field_cursor_test (flutter/packages#13460) 2026-10-27 kevmoo@users.noreply.github.com [material_ui] Decouple disabled CheckboxListTile state in a11y_assessments (flutter/packages#13177) 2026-10-27 oleg.toplionkin@gmail.com [camera_web] Skip video input devices that cannot be started in availableCameras (flutter/packages#13196) 2026-10-27 happytoday83@naver.com [camera] Add support for zero-shutter-lag capture (flutter/packages#12374) 2026-10-27 louisehsu@google.com [in_app_purchase_storekit} Fix more test flakes (flutter/packages#13279) 2026-10-27 21270878+elliette@users.noreply.github.com [material_ui] Refactor `typography.dart` and `text_theme.dart` in preparation for M3E (flutter/packages#13371) If this roll has caused a breakage, revert this CL and set the roller to dry run mode using the controls here: https://autoroll.skia.org/r/flutter-packages-flutter-autoroll Please CC flutter-ecosystem@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Fixes flutter/flutter#194285 (hopefully)
The main cause of flutter/flutter#194285 (where
Product.products(for:)returned[]across StoreKit 2 tests on the iOS 27 simulator) was fixed in #13213 by preventing the example app from calling StoreKit APIs beforeSKTestSessioninitialized. However,testGetProductsWithStoreKitErrorandtestFailedNetworkErrorPurchaseremains fragile in iOS 26.2+This is still because product caching in
StoreKitTestis still 1. Opaque 2. unable to be manually cleared - the product cache sticks around even after callingresetToDefaultState(). Thus, the state of the cache depends on the order that the tests will run. :/ So, if a test that purchases a product succeeds, if we later hitsetSimulatedError(..., forAPI: .loadProducts), the cached product will be returned, causing the test to fail.This is an issue that RevenueCat also ran into as shown here: RevenueCat/purchases-ios#6088, where they skip tests using
.loadProductfor anything above ios 26. Instead of keeping dead tests around, I think its fair to just remove them and avoid just keeping dead code around till the end of time. Additionally, the code paths tested by the removed tests are already being hit bytestFailedProductUnavilablePurchase, so we arent really losing coverage.Additionally in this PR I fix up some leaky states between tests.
Pre-Review Checklist
[shared_preferences]///).