Skip to content

Outline lift helpers - #1

Open
vigoo wants to merge 33 commits into
mainfrom
golem-outline-lift-v0.58.0
Open

vigoo wants to merge 33 commits into
mainfrom
golem-outline-lift-v0.58.0

Conversation

@vigoo

@vigoo vigoo commented Jun 22, 2026

Copy link
Copy Markdown

No description provided.

…ancelling (bytecodealliance#1638)

`cancel_inter_task_stream_read` issued a synchronous `stream.cancel-read` on
the inter-task wakeup stream while that stream was still a member of the task's
waitable set, only calling `remove_waitable` afterwards.

Per the Component Model (component-model#647), a synchronous
`{stream,future}.cancel-{read,write}` traps if the waitable is still in a
waitable set, for the same reason synchronous reads/writes do. Runtimes that
enforce this trap (e.g. recent Wasmtime) therefore fault here during ordinary
async operation.

Reorder so the stream leaves the waitable set before the synchronous cancel,
matching the unregister-then-cancel ordering already used by the general
`WaitableOperation::cancel` path. The cancel result is discarded as before, so
behavior is otherwise unchanged.
@vigoo
vigoo force-pushed the golem-outline-lift-v0.58.0 branch from 03966d4 to 29a94e3 Compare June 25, 2026 15:01
alexcrichton and others added 22 commits June 29, 2026 14:06
* Evolve the wasip3 async C ABI for tasks

This commit is an evolution of the C ABI used to managed task-related
infrastructure in WASIp3 with the goal of solving bytecodealliance#1618. The basic
problem of bytecodealliance#1618 is that waitables in Rust aren't guaranteed to be
polled within the context of the original task. For example by mixing an
`async` Rust export and `block_on` it's possible to "cross the wires"
and poll in one context while dropping/completing in another context.
This can lead to buggy situations where a waitable is left in a set, not
added to an appropriate set, or generally mis-managed.

The solution here is to enhance the current C ABI of task management
with clone/drop operations. Notably this enables waitables to retain a
strong reference to the task state as opposed to always consulting what
the current task in. This fixes a few situations such as:

* When dropping a half-finished waitable it no longer needs to be
  dropped in the context of the original task. Dropping will unregister
  the waitable from a task that it was originally registered with.
* When a waitable is moved from one task to another it needs to
  implicitly de-register with the previous task, and this was not
  previously done. Now with a retained strong reference it's able to
  clear out previous state upon re-registering with a new task.

This change requires some finesse as this needs to be ABI-stable to work
with previous versions of the `wit-bindgen` crate. The runtime support
additionally can't assume that the new ABI bits are available and
instead needs to handle the previous ABI as well. Not too too bad, in
the end, though.

This additionally did some refactoring of the state associated with
async tasks to juggle things around and better represent the raw
pointers/`Arc`/etc from before.

Closes bytecodealliance#1618

* Document the C ABI

* Fix disabled compile

* Just one version marker
* feat(cpp): add map type support

* fix(cpp): add operator< to wit::string for std::map key support and fix rustfmt

wit::string lacked comparison operators, causing compilation failures
when used as a std::map key. Also fixes rustfmt formatting issues.

* fix(cpp): const_cast map keys during lowering for ownership transfer

std::map keys are const, but the ABI lowering needs to call leak() on
string keys to transfer ownership to the flat buffer. Use const_cast
since the map is consumed during the lowering operation.

* fix(cpp): copy map keys by value instead of const_cast during lowering

const_cast fails when the map key type differs between contexts (e.g.
std::string_view in imports vs wit::string in exports). Copying by
value works universally and is safe since the map is consumed.

* fix(cpp): remove map runtime test until wasi-sdk supports map types

wasm-component-ld bundled with wasi-sdk 30 doesn't support the map
type encoding (0x63) in the component model binary format. The C++
map codegen is still validated by the codegen test. The runtime test
can be re-added when wasi-sdk ships a compatible wasm-component-ld.

* refactor(cpp): use wit::map instead of std::map for map types

std::map requires per-entry rb-tree allocation during lift; wit::map
mirrors the canonical ABI layout (flat pair array) so lift is a single
allocation, matching the existing wit::vector / wit::string pattern.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): drop redundant size_t cast in MapLower

wit::map::size() and std::span::size() already return size_t, so the
C-style cast was a no-op.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): skip guest-dealloc loop when body is empty

Avoids an unused `base` local and the `(void) base;` suppression when
the element has no nested heap-owned fields (e.g. list<u32>, map<char, u32>).

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): use static_cast for MapLower realloc result

The cast converts void* (from cabi_realloc) to uint8_t*, which is a
well-defined static_cast and doesn't need a C-style cast.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* fix(cpp): zero length when moving wit::map

Leaves the moved-from map in a fully consistent state so any later
inspection of size()/empty() reflects the empty invariant, not just
the destructor's short-circuit on a null data_ pointer.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* style(cpp): break long single-line methods in wit::map

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): drop wit::map operator[]

Index-based subscript on a map is a footgun: integral keys would
silently compile while doing positional access. Iteration via
range-for and pointer access via data() are sufficient for codegen
and don't carry the same expectation mismatch.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): drop std::span accessors from wit::map

A span of pairs is a vector-shaped view that doesn't fit a map
abstraction; codegen sites that need a flat pair span build it
explicitly from data() and size() at the call site.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* feat(cpp): introduce wit::map_view for borrowed map arguments

Borrowed map arguments were surfacing as std::span<std::pair<K,V> const>,
which carries vector-shaped affordances (positional indexing, span
conversions) on what is conceptually a map. wit::map_view is a borrow-
only counterpart to wit::map that mimics map semantics rather than
vector semantics.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): rename wit::map to wit::unordered_map

Component-model map<K,V> carries no ordering or hashing guarantee, and
the type's API surface deliberately mirrors std::unordered_map (size,
empty, begin/end) plus bindings-construction primitives, so the name
should reflect the unordered semantics rather than std::map's ordered
ones. Companion borrow type renamed to wit::unordered_map_view.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* fix(cpp): align map iteration variable with renamed _base

Adjacent block bodies (string/list/option dealloc) now reference _base
after main's rename, so MapLower / MapLift / GuestDeallocateMap must
declare the per-entry pointer under the same name to compile. Adds the
matching (void) _base; suppression so loops still compile when the
inner body doesn't reference it.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

* refactor(cpp): remove unordered_map_view and update map handling

The `unordered_map_view` class has been removed to streamline the API, as it was a borrow-only handle that mimicked map semantics. The code now directly utilizes `std::span` for borrowed map arguments, ensuring a clearer distinction between map and vector semantics. Additionally, the handling of dependencies has been updated to include `<span>` where necessary.

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

---------

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
…e#1649)

* Fix exception safety violation in AbiBuffer::advance

* Add more explanation
Signed-off-by: Andrew Steurer <94206073+asteurer@users.noreply.github.com>
* Move all async tests to general folder

WASIp3 and component-model-async have shipped and are stable, so these
are no longer expected to fail.

* Try to fix CI failure

* Try to fix Go in CI

* Fix codegen test expectation

* Another round of attempting ignores

* Disable broken tests

I'm tired of trying to work around them, just disable them.

* Tweak CI configuration
* Update wasm-tools dependencies

Additionally update C intrinsics for threading as they've been renamed
slightly.

* Remove unused import
[automatically-tag-and-release-this-commit]

Co-authored-by: Auto Release Process <auto-release-process@users.noreply.github.com>
* Add support for named implements in Go

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>

* clippy: use package.clear()

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>

* support implements for cpp

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>

* linter fix

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>

* update to go_package_name

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>

* linter update

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>

---------

Signed-off-by: Jeremy Fleitz <jeremy@cosmonic.com>
@vigoo
vigoo force-pushed the golem-outline-lift-v0.58.0 branch from 78a7464 to 4407232 Compare July 21, 2026 17:29
vigoo and others added 5 commits August 11, 2026 16:24
Close materialized local stream sinks without masking producer errors, add runtime coverage for normal-return EOF and producer traps, and teach the runtime harness to validate expected failures by message.

Amp-Thread-ID: https://ampcode.com/threads/T-01a061e4-35fe-778e-863e-5f99777b0290
Co-authored-by: Daniel Vigovszky <me@vigoo.dev>
ampagent and others added 3 commits September 22, 2026 15:06
Apply the generated canonical byte budget before retaining a caller view. Keep nested endpoints serialized and bound other payloads by both element count and canonical size. Cover the WASI filesystem import direction and full/partial component transfers.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0c98e-3797-728e-b811-76f4ada72d7b
Co-authored-by: Daniel Vigovszky <me@vigoo.dev>
Share canonical memory lowering of plain record and variant results, including nested metadata graphs. Keep async task.return borrowing, resource aggregates, import arguments and nested payload vtables inline. Test memory layout, post-return, aliases and mixed sync/async exports.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0c98e-3797-728e-b811-76f4ada72d7b
Co-authored-by: Daniel Vigovszky <me@vigoo.dev>
ampagent and others added 2 commits September 24, 2026 08:35
Outline async export lifts and borrow-aware result lowering while retaining scratch buffers through task.return. Share list-only post-return cleanup without changing resource transfer or cancellation ownership.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0d24c-3562-702a-afc6-ab09e639a67a
Co-authored-by: Daniel Vigovszky <me@vigoo.dev>
Register memory-lowered children of flat task returns as well as indirect results. Keep nested and outer canonical buffers alive through task.return, then release each buffer exactly once.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0d24c-3562-702a-afc6-ab09e639a67a
Co-authored-by: Daniel Vigovszky <me@vigoo.dev>
vigoo added a commit that referenced this pull request Sep 24, 2026
Revert 55421c4 and its dependent
owning-method-chaining follow-up 389bd60.

Restore exactly the pre-merge d1d1637 tree, retaining all changes from
fork PRs #1 and #2, including the ABI and resource cleanup optimizations.
Defer the upstream update and MoonBit stable-toolchain/cancellation
migration to GOL-662.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0d330-9bd5-7748-b23a-cc862e50bf25
Co-authored-by: Amp <amp@ampcode.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants