Skip to content

rust: bounds-check scalar access (panic instead of UB) and fix VTable::get_field OOB read - #9304

Open
just-s1lence wants to merge 1 commit into
google:masterfrom
just-s1lence:rust-bounds-safety-hardening
Open

just-s1lence wants to merge 1 commit into
google:masterfrom
just-s1lence:rust-bounds-safety-hardening

Conversation

@just-s1lence

Copy link
Copy Markdown

Summary

The Rust runtime's byte-level scalar access layer (read_scalar, read_scalar_at,
emplace_scalar, emplace_scalar_array) used raw copy_nonoverlapping calls with
no length checks. Out-of-range accesses were undefined behavior in release builds
(the only guard was a debug_assert!, stripped with debug assertions; two of the four
functions had no guard at all).

This PR makes the whole scalar access layer bounds-checked by construction:

  • All raw pointer reads/writes in the access layer were replaced with slice
    indexing/copy_from_slice behind a single sealed trait
    (endian_scalar::private::TriviallyTransmutable::load_ne / store_ne), which is
    now the only place in the runtime where raw bytes become scalar values.
  • Out-of-range input now panics instead of causing UB, in every build profile.
  • unsafe fn signatures are kept as-is so that existing callers — including
    flatc-generated code that wraps these calls in unsafe {} — remain source
    compatible; their # Safety contracts were updated to document the new behavior.
  • Pointer-based byte access outside the (trusted) builder dropped from 8 sites to 3;
    the remaining 3 are genuinely unavoidable (follow_cast_ref, Array::as_ptr,
    array_init's MaybeUninit loop) and each carries a safety comment.

Additionally, two bugs in VTable:

  1. Soundness: VTable::get_field out-of-bounds read. The guard was
    idx > num_fields() but field indices are zero-based, so idx == num_fields()
    read two bytes past the end of the vtable — reachable from safe code via
    Table::vtable().get_field(n) (e.g. past the end of the allocation). Fixed to
    idx >= num_fields().
  2. num_fields underflow. (num_bytes / SIZE_VOFFSET) - 2 underflowed for
    malformed (0- or 2-byte) vtables — a debug panic and a release-mode wrap to
    usize::MAX. Now saturates. (The verifier can currently accept such vtables.)

The /facepalm comment on Vector::default — which exists solely because the old
read_scalar_at was UB on a zero-length buffer — was updated accordingly.

Evidence: this is UB today

New test file tests/rust_usage_test/tests/safety_hardening_test.rs (11 tests) fails
on current master and passes with this PR. Under Miri (-C debug-assertions=off,
i.e. release-like) the pristine tree reports undefined behavior for every vector:

Test Miri on master With this PR
read_scalar_on_short_slice_panics_instead_of_ub Undefined Behavior: attempting to access 4 bytes, ... only 3 bytes from the end panics (test passes)
read_scalar_at_truncated_tail_panics_instead_of_ub UB: 4 bytes requested, 1 available panics
read_scalar_past_end_of_buffer_panics_instead_of_ub UB: 8 bytes requested, 4 available panics
emplace_scalar_on_short_destination_panics_instead_of_ub UB: 4-byte write into 2-byte slice panics
emplace_scalar_array_past_end_of_buffer_panics_instead_of_ub UB: write past 6-byte stack allocation (array.rs:108) panics
emplace_scalar_array_with_out_of_range_offset_panics_instead_of_ub UB: write past 10-byte allocation panics
vtable_get_field_on_exact_sized_buffer_is_not_ub UB: read 2 bytes past a 16-byte allocation (safe API!) returns default 0
vtable_num_fields_does_not_underflow_on_short_vtable debug: attempt to subtract with overflow saturates to 0
scalar_roundtrip_little_endian, vtable_get_field_never_reads_past_the_vtable, default_vector_stays_usable regression guards pass

The emplace_scalar_array case is notable: the OOB write is silent even in debug
builds on master (no assert, no panic — the test fails with "did not panic as
expected").

Testing (all run locally)

  • tests/rust_usage_test: cargo test — all green (276 integration + 21 + 13 + 11 + …)
  • cargo test --no-default-features (no_std runtime) — all green
  • cargo clippy in rust_usage_test — no warnings; runtime crate: 14 → 13 warnings
    (this PR removes one, adds none)
  • MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri test (the CI recipe) — all green, including the new tests
  • tests/rust_serialize_test (cargo run), tests/rust_reflection_test (112 tests) — green
  • cargo +nightly build for thumbv7m-none-eabi (rust_no_std_compilation_test) — green
  • flatbuffers_alloc_check / flexbuffers_alloc_check bins — green
  • In-crate cargo test -p flatbuffers: 6/7 green; builder::tests::with_internal_capacity_preallocates_vecs
    fails identically on pristine master (b8431fb) and is unrelated: the std
    strings_pool is a HashMap<String, …> (since feat: use HashMap for create_shared_string to fix O(N²) performance #8958), and inserting String keys
    allocates by design, so the "0 allocations" expectation no longer holds. Not touched here.

Performance

cargo bench before/after (same machine, debug off):

bench master this PR
read_monsters 53,410 ns 40,355 ns
serialize_monsters 47,860 ns 45,564 ns
traverse_canonical_buffer 33 ns 36 ns
create_many_tables 95,313 ns 93,219 ns

No regression; the bounds checks are on size_of-derived constant lengths and are
elided by the optimizer in the hot paths.

Compatibility notes

  • No public signatures changed. read_scalar / read_scalar_at / emplace_scalar /
    emplace_scalar_array remain unsafe fn (their bodies are now safe), so existing
    flatc-generated code and downstream unsafe {} call sites compile unchanged, and
    no unused_unsafe warnings are introduced.
  • Behavior change: inputs that previously caused silent UB (release) or a
    debug_assert! panic (debug) now consistently panic. This only affects inputs that
    violated the documented (old) contracts, i.e. buggy callers or malformed-data
    handling that previously "worked" by reading adjacent memory.
  • no_std is preserved (only core slice ops are used).

@google-cla

google-cla Bot commented Oct 5, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@github-actions github-actions Bot added the rust label Oct 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant