Rust Serverless Data Engineering

Companion repo for Course 6 of the Rust for Data Engineering Coursera specialization. The headline crate is recho/ — a cargo lambda-deployable echo handler that demonstrates idempotent, contract-checked serverless data pipelines in Rust. Side-by-side ports in scripting-demos/ (Python, JavaScript, Ruby) show the same handler shape across languages so students can compare ergonomics, performance, and failure modes.

Provable contract

For every payload e accepted by recho::function_handler(e, rng):

1. e.payload.message == Some(m) and m.trim().to_lowercase() == "echo" → Ok(Response { message }) with message ∈ {"🌑 dark ECHO", "💡 light ECHO"}, chosen deterministically from rng.
2. e.payload.message == Some(m) and the normalized form is not "echo" → Ok(Response { message: format!("You said: {}. Say 'echo' to play the tunnel game!", normalized) }).
3. e.payload.message == None → Err(_) whose Display contains the literal substring Missing 'message' field.

Every clause is exercised on every push by the CI coverage gate (cargo llvm-cov --fail-under-{lines,functions,regions} 100 on recho/src/lib.rs).

Install

The commands below are the same toolchain steps CI runs (stable Rust 1.95.0 with clippy, rustfmt and llvm-tools-preview; the workspace MSRV is 1.75). cargo-lambda is only needed to deploy.

Linux (Ubuntu 24.04)

curl -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.95.0
rustup component add clippy rustfmt llvm-tools-preview
cargo install cargo-llvm-cov --version 0.8.5 --locked
cargo install cargo-lambda --version 1.9.2 --locked   # deployment only

macOS

curl -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.95.0
rustup component add clippy rustfmt llvm-tools-preview
cargo install cargo-llvm-cov --version 0.8.5 --locked
cargo install cargo-lambda --version 1.9.2 --locked   # deployment only

Then clone:

git clone https://github.com/paiml/rust-serverless-data-engineering
cd rust-serverless-data-engineering

Usage

# Build, test, and run the 100% coverage gate (matches CI)
cd recho
cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
cargo llvm-cov --all-features \
  --ignore-filename-regex 'main\.rs' \
  --fail-under-lines 100 \
  --fail-under-functions 100 \
  --fail-under-regions 100 \
  --summary-only

# Deploy to AWS Lambda
cargo lambda build --release
cargo lambda deploy

# Invoke via the included shell wrapper
../tunnel.sh "echo"        # dark or light ECHO
../tunnel.sh "hello"       # tunnel-game prompt

Layout

Path	What
recho/	The Rust workspace member. lib.rs holds all logic and is gated at 100% coverage; main.rs is a 9-line tokio::main shim that wires the handler into lambda_runtime::run.
scripting-demos/tunnel.{js,py,rb}	Side-by-side ports of the same handler shape — pedagogy, not production.
tunnel.sh	cargo lambda invoke wrapper for the deployed handler.
docs/hero.svg	The hand-authored hero banner above.
.github/workflows/ci.yml	Build + clippy + test + 100% coverage gate.
CLAUDE.md	Agent collaboration contract — read before editing.

Companion to

• Specialization umbrella: paiml/rust-de-specialization
• Course catalog (course-studio): paiml/course-studio
• Course slot: c6 — Rust Serverless Data Engineering (replaces "DuckDB with Rust")

License

AGPL-3.0-or-later. See the LICENSE file for the full text.