You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Tool results and evidence can end on a lone surrogate half when clipped #207
Mail and page text reaches tool results, recorded evidence and notifications through String.prototype.slice(0, limit) on the UTF-16 code units (for example the mail snippet at apps/server/src/engine/conversation.ts (snippet: body.slice(0, 240), read_mail_thread's body.slice(0, 12000)), the search/web/page clips in apps/server/src/engine/model.ts and the monitor/evidence clips in apps/server/src/engine/service.ts, plus browser.ts and jev/service.ts). When the cut index lands between the two halves of a surrogate pair (an astral character such as an emoji), the clipped value ends on a lone high surrogate:
it is not a character: UI renderers show U+FFFD in its place, and
consumers that encode the value to UTF-8 (for example any Python service that reads the same data) raise UnicodeEncodeError on the lone half.
Two places in the codebase already guard against exactly this: clipCalendarText in conversation.ts (added with the calendar tool) and sanitizeFileName in files.ts ("Slice code points, not UTF-16 units, so a surrogate pair is never split in half"), and #183 fixed the related case of a surrogate numeric character reference failing the mail list. The mail and page clips are the same class but do not use the guard.
Steps to reproduce
Give a mail body or watched page an astral character (for example 👍, U+1F44D, the pair D83D DC4D) straddling index 240 (or 12000/30000/500 for the other surfaces) — or in code: "a".repeat(239) + "\u{1F44D}".repeat(3).
const snippet = body.slice(0, 240) → the result ends with the lone half \uD83D: /[\uD800-\uDBFF]$/.test(snippet) is true.
The same value passed through JSON.stringify keeps a \ud83d escape, and encoding it to UTF-8 outside JS replaces or rejects the lone half.
Expected: clip at the same limit, but drop the high half with its low half when the cut would split a pair, the way clipCalendarText already does for calendar text.
Pure string handling; no provider or platform dependency. Reproduced with Node: ("a".repeat(239) + "\u{1F44D}").slice(0, 240).charCodeAt(239).toString(16) → d83d.
Activity