Conversation
Three things, which together are the other half of attachments. An arriving message no longer has to have text. _on_message_received dropped anything with an empty body, so a message of nothing but files was discarded outright -- it never became a message at all. What makes something not history is having no content of *either* kind, which is also how session-ios spells the same rule. Everything the sender listed gets a row, including a pointer too malformed to fetch. Dropping the unusable ones would make a message of three files look like a message of two, and the count and ordering are what a reader is shown; a row that cannot be saved is a smaller lie than a row that is not there. It also keeps the indices meaning what the sender meant by them. save_attachment fetches one and writes it to a caller-chosen path. Nothing is downloaded until asked: an arriving message records where its files are and stops, because whether a file is worth the bandwidth is the application's decision and sometimes the user's. Which of the two encryptions applies is read from the url, so a caller neither chooses nor needs to know. The file lands at a temporary name and is renamed only once decrypted and verified, so an interrupted save leaves no half-file that looks finished, and the size ceiling is enforced against bytes actually arriving rather than against anything claimed. Progress reports with the same signature as send_message's on_upload, at the cli agent's request: a display built for sending then works unchanged in the other direction.
A message had no identity on the wire, and the content hash that stood in for one could never have been it. A one-to-one message is stored on both participants' swarms and the two copies do not have the same bytes: the copy we keep for our own devices carries syncTarget, naming who the message was for, and that field is inside the Content protobuf, which is what the hash was taken over. So the sender and the recipient know the same message by different hashes, and neither can derive the other's -- protobuf has no canonical serialisation, so parsing, dropping the field and re-serialising is not guaranteed to reproduce another implementation's bytes. The column's comment claimed the opposite, that the hash was "identical for every copy of a message"; that is true of encryption and false of syncTarget. So Content gains msgId: 8 random bytes, set before the copies diverge, so every copy of a message carries the same one. A message is identified by (conversation, timestamp, msgId) -- always both, never the id alone, which is sized only to separate messages within one sender-millisecond. The alternative of identifying by timestamp is what the rest of Session does, and it is already broken in the field: pasting several messages stamps them identically, and clients keying on that treat them as one and silently drop the rest. sfixed64 rather than bytes: 8 bytes on the wire with no length prefix, and signed, which is what every implementation can hold losslessly -- SQLite has no unsigned 64-bit integer, and our binding layer deletes the uint64_t overload rather than let one be silently clamped. That also replaces what the hash was doing locally, which was dedupe, and does it better: being the same on both copies, our own message returning from our swarm is recognised directly rather than by two copies happening to hash alike, which they did not. What is lost meanwhile is narrow: a sender who retried a store that had actually succeeded shows twice, but only from clients that set no msgId, and those clients drop same-millisecond messages themselves -- so working around it here buys correctness their own other devices and other recipients will not share. A visible duplicate is the failure to prefer over a lost message.
Every message in the protocol that refers to another message did so by the referenced message's timestamp, which does not identify it: a client sending several messages at once stamps them identically, and matching on that picks one of them arbitrarily. Now that Content carries a msgId, each of those references can carry one too. Four are changed; each keeps its timestamp field and gains an optional msgId beside it, so a client that sets neither still resolves the way it always did. Reaction and Quote also have their timestamp field renamed from `id`, which was never an id -- the comment on Reaction's said "// Message timestamp" and it was the only thing that said so. Leaving it would have put a field called `id` that is a timestamp next to a field called msgId that is the identifier, which is worse than the original naming: a reader picks the wrong one and nothing catches it. UnsendRequest is the one that matters most. It is the same defect, but resolving it wrongly deletes a message the user did not ask to delete, and the messages most likely to share a millisecond -- several pasted at once -- are exactly the ones someone then thinks better of. Its comment says to leave a request unresolved rather than guess. DataExtractionNotification's `timestamp` is deprecated outright rather than kept as a fallback. It was meant to be the referenced message's timestamp, but nothing said so and the three clients disagreed: session-ios set the message's, session-android set the current time, and session-desktop set the message's, the server's, the received-at time or zero depending on the path. A receiver cannot tell which it was given, so the value cannot be interpreted at all -- not even to test against messages it holds. It also gains attIndex, for which attachment. ReceiptMessage is deliberately untouched: it references a *batch* of messages, so it needs a shape decision rather than another optional field, and that is still open.
ReceiptMessage acknowledges a batch, so unlike the other references it gets a parallel array rather than a single optional field: msgId entry i belongs to timestamp i. Two rules make that safe. A batch can mix messages that carry a msgId with ones that do not, so entries are never skipped and 0 means "this one has none" -- a value a real msgId takes with probability 2^-64. And a receiver finding the two arrays of different lengths ignores the msgIds entirely rather than pairing up the prefix, since a sender that got the alignment wrong has no correct prefix to salvage. That rule also covers the ordinary case of a sender too old to set any: zero against N is a mismatch, so it falls back to timestamps without needing to be a separate case. Packed, since nothing parses it yet and a batch of them is otherwise a tag per entry. attIndex says what it means rather than leaving it to absence: -1 is all of a message's attachments saved together, which is what saving from a gallery view does, and absence now means only "not stated" -- a SCREENSHOT, or a sender predating the field. Neither is "the first one". sint32 because -1 is a common value here and proto2 sign-extends a negative int32 to ten bytes, where zigzag takes one. KeyPair goes. It carried a group's private key to its members back when closed groups had a single shared keypair, and has been unreferenced since closedGroupControlMessage was retired -- reserved a few lines further down in this same file. It survived as generated code nobody called, in our copy and in session-ios's, reading like we broadcast private keys.
save_attachment gains notify_sender, defaulting true because that is what Session's other clients do and so what a sender expects. Passing false keeps the save private -- worth a setting in a UI, since whether fetching someone's photo should be reported back to them is a privacy question rather than a technical one. Sent only after the file is down, decrypted and renamed into place: what it claims is that we have their file, which is not true until we do. And only for a message we received -- saving from our own notifies nobody, which covers a note to self, since that is outgoing. The notification carries the pair that identifies a message, msgTimestamp and msgId, plus attIndex for which file. msgId is left unset when the saved message carries none rather than sent as some placeholder: half of an identifier is not a weaker match, it is an ambiguous one, and the receiving side is meant to leave a reference it cannot resolve alone. Nothing waits on the send, so its id goes in _quiet_sends and its statuses are dropped as they arrive. Without that they would collect in _early_status against an id no message will ever claim -- the same leak _sync_sends exists to avoid.
save_attachment had no test at all: the decryptors have known-answer vectors, but nothing exercised download, decrypt, write and notify together, which is where a wiring mistake would live. Network::download becomes virtual so MockNetwork can serve one, matching get_swarm and send_request, which are virtual for exactly this. It was not non-virtual by intent -- a method meant not to be overridden would be private -- only because nothing had needed it. The stubbed download hands the file over in chunks rather than in one piece, since a decryptor that only works when given the whole thing is one of the bugs this is for. What it serves is a real attachment, encrypted by attachment::encrypt, so the bytes are the ones a sender would actually produce, padding and chunk boundaries included. Covered: the file arrives byte-identical with its padding gone, nothing is fetched until asked, progress reports open with 0/0 and end with exactly one result, and the sender is told. Then the cases worth more than the happy path -- asked not to notify, we do not, and the file still lands; and a file that fails to authenticate leaves neither a corrupt file at the destination nor a .part beside it, and tells nobody a save happened. Both halves were checked by breaking them: suppressing the notification fails the first, and renaming into place on failure fails the second.
Two gaps, both closed offline. The first is that nothing exercised sending and saving *together*. Each half was tested against fixtures of its own making, so the two could disagree about the url, its `d` fragment, the key or the size and both still pass. MockNetwork grows the part of a file server that matters here: upload_file encrypts as the routers do and keeps the bytes, and download serves them back by the id in the url. A message sent with a file can then be saved back through the same object, and a disagreement between the halves shows up as bytes that do not match. upload_file joins download in being virtual -- a method meant not to be overridden would be private; these were merely never needed. The second is that legacy decryption had never run through the download plumbing. It has known-answer vectors, but those test the decryptor, not the path that feeds it -- picking the scheme from the url, checking the key length, holding the whole ciphertext because the MAC covers all of it, trimming to the sender's size. The blob is hard-coded rather than generated: libsession has no legacy encryptor, deliberately, so producing one here would mean writing the thing we chose not to have and then testing it against itself. Both were checked by breaking what they cover: generating the url without its fragment fails the round trip, and ignoring the sender's size fails the legacy save. An older test asserted attachments were *not* uploaded, which was only true because nothing could upload them. With a file server present that state is unreachable, so it now asserts what it was really about -- that the recorded size is the file's own, not the padded ciphertext's.
…d field A ProProof is version 0 by its type, not by a value it carries. Scope ProProofVersion (enum class : uint8_t) and rename ProProof -> ProProof_v0 with `using ProProof = ProProof_v0` (a note explains the alias becomes a variant or virtual base when a v1 arrives). The version field is dropped from the C++ and C proof structs and from operator==, and config no longer sets it. fill_proof also stops reading a JSON `version` off the generate_pro_proof response: the endpoint fixes the format and the proof's version is bound into its signature via the personalisation, so there is nothing for the client to read or gate on -- a future format is a new endpoint returning a new ProProof_vN, not a version bump on this response. The wire version survives only where it must: the protobuf-embedded proof a peer decodes with no endpoint context, where it selects the layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…the message A received Pro proof whose wire version we don't recognize used to fail the parse, discarding the entire message -- the recipient silently never saw it. That's backwards for a forward-compat field: a newer proof format should cost the sender their Pro affordances on old clients, not the whole message. parse_pro_message now treats an unknown (or missing) version as a graceful degrade: it flags the proof ProStatus::UnsupportedVersion (C SESSION_PROTOCOL_PRO_STATUS_UNSUPPORTED_VERSION) and returns a non-pro message, so the caller skips signature evaluation and delivers it as an ordinary message. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
session::json::require<T>/maybe<T> now accept a scoped enum T, validated as its underlying integer (nlohmann converts through the underlying type in extract), so an enum-typed field can be requested directly rather than cast at the call site. No current caller needs it -- it is a zero-cost `if constexpr` branch that only instantiates when asked -- but it rounds out the generic helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inner connection's UDP payload size was derived from the tunnel's suggested_mtu, which cannot be made to mean what it needs to. That suggestion is the outer *endpoint's* configured max_udp_payload minus a fixed overhead -- a policy knob rather than a measurement, unset in the default configuration, so the subtraction does not even run there. Where it does run, on a client that pinned the outer, it produces an inner MTU derived from someone's cap rather than from the path. The two values that are real are no better suited. A connection's max_datagram_size is deliberately the size reachable *by splitting a packet in two* -- get_max_datagram_size() is literally get_max_datagram_piece() * (splitting ? 2 : 1) -- so sizing the inner from it guarantees every inner packet splits. The per-piece size that would avoid splitting belongs to a connection this layer does not hold, and knowing it once would not be enough anyway: it moves when the first hop changes, and nothing here would hear about that. So the inner is pinned to the one size QUIC guarantees every path carries. libquic splits datagrams that do not fit, so this is a throughput question rather than a correctness one: a larger value would win when the outer path is roomy and cost a split per datagram when it is not, and we cannot tell which we have. The outer connection still discovers its own path MTU, which is where the gain actually is, and an application whose discovery misbehaves can still pin it through opt::quic_max_udp_payload.
An application that can save an attachment then wants to show that it did: without it, "save" is offered forever and saving twice writes two copies. Every client faces that, so it belongs in the data model rather than in each of them -- and Client is the only thing that knows a save completed, since save_attachment is where it happens. One nullable timestamp, not a path. Where a file went is the application's business: it chose the location and can move the file afterwards, so a path recorded here would be wrong as soon as it did. What is actually being asked is whether offering "save" again is pointless, and a timestamp answers that. It records an event rather than describing the filesystem, so "this was saved, then" stays true after the file is deleted. The column means the same thing in both directions: when the *recipient* of the message last saved this attachment. For one we received that is us; for one we sent it is them, from their notification. Nothing has to consult `outgoing` to read it, which is the trap the content-hash split just cost us an afternoon of. That second half needed the receive side of DataExtractionNotification, which did not exist -- _on_message_received returned early on anything without a DataMessage. It resolves the message by (timestamp, msgId), both required: the timestamp alone cannot identify a message sent in the same millisecond as another, and msgId alone is far too small. It also requires the message to be one of ours in the conversation with the sender, so a peer cannot make claims about a message they were not sent. attIndex names one attachment, -1 means all of them saved together, and absence means neither -- never "the first one". The saved time is the notification's own timestamp, not msgTimestamp, which identifies the message being talked about and is generally older; the test fails if the two are confused, which is how the mistake was caught while writing it. Unset means "not known to have been saved" and never "not saved": the outgoing half depends entirely on the other end volunteering a notification, which no released client does in a form we can read.
A node that does not participate in session routing has no relay contact, so nothing can be carried to it. Until now the request that discovered that simply failed: _fail_tunnel reported it to the caller, and the node was struck out of the pool only *afterwards*. So each such node cost one dead request -- a send surfaced as a failed message, a poll as a round that quietly returned nothing. The swarm is not in question when this happens, only one member of it, so recovery is to keep the swarm and move to the next member. That is the opposite of the 421 path, which fires when the swarm information itself is wrong and recovers by throwing it away and re-resolving; the two rhyme but must not be shared. Hence a set of failed nodes rather than another counter: "once per node" cannot be written as a number, since choosing the next one has to know which are already spent. Selection walks the swarm in order and stops when nothing is left, reporting the original failure rather than one of its own. Request::retry_count becomes retry_421_count. It only ever counted 421 redirects -- bounded by a config knob actually named redirect_retry_count -- and a generically named field with one specific meaning is how the next person makes the mistake this commit nearly made. Timeouts come down to match: 10s per attempt where it was 20s, with a 60s budget across the whole operation. A node that has not answered in ten seconds is better abandoned than waited on, which is the same conclusion session-ios reaches from the other direction -- it spends its budget on many short attempts rather than one patient one. Each attempt gets the per-request timeout or whatever is left of the overall budget, whichever is shorter, so walking a swarm of unreachable members cannot outlive what the caller asked for; below a two second remainder it gives up rather than start something that cannot finish. The upload path also gets the tunnel callbacks the download path already had. Without them an upload took the local port and sent into a mapping whose session might not be up, so the handshake was dropped and retried on QUIC's own timer, and an unreachable relay was discovered by waiting out the request rather than being told.
Network's own logic sat above routing and below MockNetwork, which overrides send_request wholesale -- so nothing exercised it. Neither the node walk nor _handle_421_retry, which has never had a test at all. Two seams open it up, both following the pattern already used for Core: TestHelper becomes a friend of Network and SnodePool. Substituting the router lets a test script how each node answers; seeding the swarm cache lets get_swarm resolve at all, which it otherwise cannot in a test -- a pool with no seed nodes has no swarms, which is what those "Cannot refresh cache" lines in every run have been saying. FakeRouter is otherwise trivial: eight no-ops and one method that answers from a table. Writing it corrected the implementation. The comment claimed the swarm comes back in a fixed order, so the walk would try members in the same order every time; get_swarm actually shuffles before partitioning by strike count, so what it really gives is least-struck-first in a random order among equals. That is the better preference anyway, and the walk only needs a member never to be chosen twice -- but the comment asserted something untrue and the tests now assert what is. Covered: reaching a working member without spending any twice, trying every member exactly once before giving up, reporting the original failure rather than an invented one, not walking on a failure that is the request's fault rather than the node's, not walking a request that names no swarm, and the budget shortening retries and refusing to start one it cannot finish. The order-sensitive parts are asserted as distinctness rather than sequence, and the suite was run five times over, since a shuffle makes a flaky test easy to write by accident. Verified by breaking the selection to ignore failed nodes, which fails the two tests that exist to catch exactly that.
The user configs carry two different kinds of state and the schema already had the seam for it: what an account *is* goes on accounts, and what a conversation with it is like goes on conversations. contact_info mixes both, and base_group_info carries the conversation half again for groups and communities -- the same fields meaning the same things -- so those become one set of columns here rather than three. Contact-ness becomes a row in `contacts` rather than a column on accounts. A flag would restate something the database already knows with nothing keeping the two in agreement; existence cannot disagree with itself. It also puts each field where it means something: a name and a picture belong to the account, because we learn those for people who are not contacts at all, while a nickname and the approval pair need a relationship before they mean anything. A contact removed on another device is then a row that goes away while the account stays, which is what we want -- we have still seen that person. The Pro proof lands on accounts even though ConvoInfoVolatile carries it per conversation. It describes the peer rather than the conversation, and session-ios already resolves it into its per-account Profile record at both ends -- nobody wants it per conversation, that is just where the sync mechanism puts it. Per-account also generalises where per-conversation cannot: convo::group has no pro_base, so a group member's proof has nowhere to go in the config at all, and can still be cached locally. Time units are per column now, matching whatever the config carries, which is not uniform: ConvoInfoVolatile's last_read is milliseconds while contact_info's created, profile_updated and mute_until are seconds. One rule produces both -- a value compared against a message timestamp is milliseconds, everything else is seconds. last_read qualifies because messages.timestamp is the other side of that comparison, and at second resolution reading a message stamped mid-second forces a choice between leaving it unread and marking the rest of that second read. Nothing else needs to know the sub-second moment a conversation was created or a mute expires, so conversations.created moves to seconds. Every mirrored column then round-trips with no conversion at all, so none of them can dirty a config by being written back slightly differently. The migrations go with it. No released version has a client schema -- v1.8.0 has no src/client/schema at all -- so all six existed to upgrade databases that cannot exist, which is why src/core/schema has been full_schema alone all along. Their comments were checked before deleting: everything in 001-004 and 006 is already reproduced in the column and table comments, and what is only in 005 argues about a column that no longer exists. That leaves the history check nothing to walk, since a schema change without a migration is exactly the break it looks for. SCHEMA_FLOOR names the last revision behind that break; everything after it is still walked, so consecutive commits must still upgrade cleanly and only the cutoff itself is exempt. Not done here: the conversation list still reads accounts.name rather than preferring a nickname, because nothing populates contacts until the config reconciliation lands. Communities and blinded identities wait on the same work -- both need servers split from rooms, which is a change to tables that have no rows yet.
Core gains a Configs component holding the five config objects, and the poll gains the four namespaces they live in. Nothing yet interprets what they contain -- that is Client's work -- but a config now arrives, merges against what we hold, and survives a restart. Core owns them because everything mechanical about a config is Core's work already: retrieving from a namespace, merging, keeping the dump, pushing what changed. None of it needs to know what a contact is. That does mean core.hpp names config types, so core links config publicly rather than privately now. The dumps table is keyed by encryption_domain() rather than by storage namespace. The two coincide for four of the five configs and diverge exactly where it matters: Local has no namespace of its own and reports UserProfile's as a stand-in, so a namespace key would collide. A domain is already required to be unique per config type and can never change, since changing it would break decryption of everything written under it. for_namespace() maps by hand for the same reason -- searching over storage_namespace() would find two configs for namespace 2 and answer according to whatever order it looked in. The configs are built on first use rather than in init(), because they are encrypted to the account key and a Core opened with defer_account has none yet. Nothing can reach them before an account exists in any case: a network cannot be attached without one, so neither polling nor pushing runs. Batch is the other half of the dumping story. A config is dumped when it changes, which is right when nobody knows any better -- but a caller working through a batch of messages does know better, and dumping between them writes intermediate states nobody reads. One poll carrying four config namespaces now dumps once rather than four times. It nests, so merge() can hold one itself without defeating a caller that already has. It also fixes what a debounce timer can only approximate: debouncing is a guess at where a batch ended, and the guess is only needed where nothing knows. Local is left out of the pushable set even though it already reports needs_push() as false unconditionally. That override is not what should be keeping it off a swarm: the push path asks which configs have a destination, and answering by trusting each config to decline makes "has nowhere to go" indistinguishable from "has nothing to say". The poll tests were positional -- three namespaces, Devices at index 1 -- so they broke on contact with a longer list and would break again on the next one. They now find a subrequest by the namespace it asks for, and the mock response is built from the request so that it lines up by construction, which is what the storage server does anyway. Not here: pushing. needs_push() reports it and nothing acts on it, so a local change is held and dumped but never leaves. The per-pubkey debounce and the sequence request are the next piece.
A dirty config now reaches the swarm. Everything owed goes in a single sequence: a store per config message, then one delete naming every hash those stores obsolete. The delete goes last because a sequence stops at its first failure, so nothing is removed before what replaces it has landed -- which is the whole reason this is a sequence and not a batch. Only this account's configs go in that request. A group's live under a different pubkey and are pushed separately even when the swarm turns out to be the same one: a request carrying both would tell that swarm the account and the group belong to the same person, and swarms are assigned per pubkey, so the collision is ordinary rather than unlikely. The component is per-account, so its timer is already per-pubkey and a group would get its own instance rather than a shared map. Confirmation is per config and all-or-nothing across the messages it split into. Confirming a partial push would drop the parts that did land from the obsolete list while leaving the config believing it is clean, so the part that failed would never be sent again. The debounce is the one from the design discussion: no sooner than two seconds after the last change, no later than ten after the first. Both bounds are recomputed from timestamps on each firing rather than tracked, so a re-arm cannot drift past the deadline the first change set. A steady trickle of changes therefore cannot defer a push indefinitely. What arms it is releasing the outermost Batch, unconditionally rather than only after a merge -- so the batch a poll already holds doubles as a sweep, and a config changed locally without one is noticed within a poll interval instead of sitting unpushed forever. Deferred work is handed to the event loop, which outlives this component and cannot be told to forget a call already scheduled. Callbacks hold a weak reference and do nothing once it has expired, which is what stops a pending push firing into a destroyed Core. swarm_request.hpp is the namespace rules, the timeouts, and the signed values, moved out of core.cpp because more than one part of Core now needs them and transcribing them twice is how one copy ends up disagreeing with the storage server. delete is the odd one out and gains its own helper: it signs the hashes it names and carries neither a namespace nor a timestamp. The debounce is tested by driving the decision rather than waiting out real intervals -- the timer fires on the event loop while a test reads from its own thread, so a test that waited would be both slow and racy. Verified by removing the cap and the in-flight guard, each of which fails exactly the test written for it.
Merging updated the configs and dumped them, and nothing above Core was told, so nothing could reconcile. configs_changed now fires with the namespaces a merge altered. It carries namespaces rather than a diff. A config diff describes a transition between config *states*, and the reconciling layer's currency is not a config state and is not recorded anywhere: a merge routinely lands several updates forward at once, `<` only reaches back config_lags, and a crash between merging and reconciling leaves the database behind by an amount nothing measured. Comparing against its own stored rows is what makes reconciliation self-correcting, and a diff would silently skip whatever those cases left behind -- permanently, since nothing would revisit it. Namespace is an unambiguous identifier *here* specifically because only merging configs can appear, and Local never merges -- so the collision with UserProfile's namespace, which forced for_namespace() to map by hand, cannot arise on this path. Firing is once per batch, after the dumps, so a handler never reads state that is not yet on disk and never sees a poll half-applied. Only merges are reported: a config the application changed itself is not news to it, and reporting it would invite reconciling its own write back over itself. ConfigBase gains a seqno() accessor, which is what makes "did that merge do anything?" answerable. The hash set merge() returns does not answer it -- a hash counts as parsed whether or not it turned out to be useful, so a stale config that changed nothing still comes back in it. The seqno moves on a local change, on adopting a higher config, and on resolving a conflict, and stays put when the merge was a no-op. Verified by reporting unconditionally, which fails the case that merges the same message twice.
The watermarks decide when the producer thread -- the loop reading and encrypting the file -- is paused and resumed against the stream's unsent buffer. Resuming at 128KB left very little to send while that thread was waking up and doing its next read, and that read is blocking file I/O, so the stream could run dry waiting on the disk rather than on the network. Raising the pair to 1MB/512KB leaves half a megabyte still queued at the moment the producer is asked for more. The comment on the loop records why that matters, since `next()` looking like a plain iterator call is exactly what makes the blocking easy to miss. Also drops an <fstream> include nothing used.
configs_changed decides "did this merge change anything" by comparing the seqno either side of it, which is only sound if a merge cannot alter the data while leaving the seqno alone. That was argued rather than checked, and the argument had a hole in it. The worry is a conflict at our *own* seqno: if resolving two same-numbered configs could adopt one of them in place, the seqno would sit still while the contents changed, and the comparison would miss it silently and permanently -- nothing revisits a config it decided was unchanged. Four combinations, because two things vary independently. Our own config is either Dirty, meaning the change has not been serialised into a message and nothing else can have built on it, or Waiting, meaning a message exists and has gone to the swarm where another device may have seen it -- and _merge takes a different path for each, since it passes `_state == ConfigState::Dirty` to make_config_message. Crossed with the incoming config either diverging from ours or containing ours outright, the latter being where "adopt the superset" would be tempting if the superset were chosen on data rather than on the diff chain. It is not: two distinct messages at one seqno are a conflict whatever their contents, and a conflict resolves to one past the highest. All four report. ConfigBase::seqno() being public is what makes the precondition assertable at all -- the test checks the two configs really are at the same seqno before merging, so it cannot pass by failing to set up the case it claims to cover.
Two devices making the same change from the same starting point is the case where a naive merge could leave us dirty with nothing to say, and push an update carrying no changes. It does not: the result settles clean against the other device's message, and owes no push. That is what the test pins down. It deliberately does not pin the seqno, which advances despite the data not changing -- merging while dirty builds a MutableConfigMessage, and that constructor increments unconditionally by design, while _merge's unwind for the spurious increment only applies when the winning config is our own. Here it is the incoming one, deliberately: a hash-less local copy is placed last so that an identical incoming message wins and we adopt its hash rather than having none. So we end clean at a seqno the swarm has never seen, and that is a real cost rather than a curiosity. The "within N" rule drops any config more than config_lags behind the highest seqno, and that window is counted in seqno values rather than in actual updates -- so a seqno consumed without publishing anything shortens it in real terms, and a device holding an unpushed change has it discarded sooner than it should be. Two devices making the same change is ordinary enough (marking the same conversation read, approving the same contact) for this to add up. Not asserted, then, because it is a wart to be fixed rather than a contract: pinning it would make the fix look like a regression. It also does not assert whether this reported to the application. It currently does, since the seqno moved, so the reconciling layer walks and finds nothing to do: idempotent, and the safe direction to be wrong in. Were the merge taught to recognise an identical config, this would stop reporting and nothing here would need to change. The direction that must never happen -- data changing without a report -- is covered separately.
The spurious increment from merging an identical config does not stop at one wasted number. Two devices agree at seqno 1; the identical merge leaves us clean at 2 while the swarm's newest is still 1. The other device, correctly at 1, then makes an ordinary change of its own -- which lands on 2, the value we already consumed. What should have been a clean adoption is a conflict against our phantom, so it resolves to 3 and leaves us dirty, owing a push we would otherwise not have made. One real change, two seqnos, and an extra round trip. It settles once the other device adopts ours, so it does not run away, but the cost falls on the "within N" window: two of its five spent carrying one change means a device holding an unpushed change is discarded that much sooner. The test asserts only that no data is lost through the collision -- both changes survive -- since the seqno arithmetic here is the defect and pinning it would make a fix look like a regression. The reasoning is recorded alongside so the fix does not have to rediscover it.
Ported from the dev-side change (Audric Ackermann, f0b9ab2) that better aligns the proof with the protobuf. Rather than the proof carrying a version selector, its format is bound entirely by the signing domain prefix (the `_v0` in BUILD_PROOF_DOMAIN is part of the signed bytes), and a future format arrives as its own protobuf field/message -- not a version bump on this one. - Drop the `version` field from the protobuf ProProof (regenerated pb.cc/pb.h), renumbering so field 1 stays reserved. - Remove the ProProofVersion enum; rename ProProof_v0 back to ProProof and drop the alias (a new format is a new type, not a vN of this one). - Replace ProStatus::UnsupportedVersion with the general ProStatus::Invalid: a proof we cannot read (sender attached none, or it is in a format we don't know, so to us it simply isn't present) is unusable, with no reason-specific status for callers to learn. Such a message is still delivered as non-pro rather than dropped. - Detect that via `!pro_msg.has_proof()` instead of a version check. - The json scoped-enum helper support stays (now caller-less), noted as such. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first config that does anything. A merged UserProfile now reaches the tables that answer queries: our own name and picture land on our accounts row, and note to self gets its priority and disappearing timer. It runs the opposite way round to the ones that follow. UserProfile is authoritative and the accounts row is a projection of it, because the config holds structure the row does not model: a second picture slot carrying the same image at a fresh URL, so a linked device that already has those bytes can skip fetching them again. Re-deriving the config from the row would flatten the two and destroy another device's reupload. What decides whether a conversation exists is the config, not whether there are messages in it. Emptying a conversation does not end it, and an account reimported onto a new client has nothing but its configs to rebuild the conversation list from -- so anything that existed only where messages had been would quietly disappear. For a contact that statement is the presence of an entry in Contacts, with priority saying separately whether it is shown. Note to self has no entry to be present or absent, because UserProfile exists from the moment the account does: it holds the account's own name. So priority carries both there, and a negative value is the only way that config can say "there is no note-to-self conversation". Hence a new account writes one -- an unset value reads back as 0, which would otherwise be indistinguishable from someone deliberately choosing to show it, and would make the conversation appear on every device on the account. And hence writing a note moves it back, since that is how "there is one now" gets recorded. Restoring an account is left alone: its real values are about to arrive from the swarm, and a local one would compete with them. Revealing it reconciles the whole config rather than just the priority. A timer set on another device has been waiting with nowhere to attach, and the moment the conversation exists is when it can land; nothing else would deliver it, because a local change is not a merge and reports nothing back to us. Creating the row is guarded by find_conversation rather than calling ensure_conversation outright: that helper bumps last_activity on a row that already exists, so unguarded it would shove note to self back to the top of the list on every config merge. nts_expiry carries a duration and no mode, because only one mode means anything when the reader is also the writer; it is stored as after-send. The test helpers build the other device from *our* dump rather than from nothing, since a device invented alongside us lands on the same seqno as the account defaults and has to be merged with them, which is a different scenario.
A change notification only arrives for state that changes after someone is listening, which leaves three ways for the tables to fall behind with nothing to announce it: a config merged by a version that did not yet know how to reconcile it, a crash between merging and reconciling, and a dump restored from an older state. In each the database is behind by an amount nothing recorded, and no further notification is owed -- a contact list that has stopped changing would stay unreconciled indefinitely. Core already polls and merges all four config namespaces, so this is not hypothetical: everything arriving for the three that have no reconciler yet is held in the configs and dumped, and would sit there unread even after a reconciler was written for it. Running the walk at startup makes all three self-healing rather than merely self-correcting, and it is safe to run when nothing has changed because reconciliation compares rather than replays: it costs a pass and writes nothing. Guarded on have_account(), since a Core opened with defer_account has no key to decrypt the configs with -- and nothing is missed, because an account that does not exist has no configs to have fallen behind, and whatever arrives once it does comes through a merge, which reports itself.
A safety catch for working against a real account: with push_enabled false, nothing this device does can reach the account's other devices or overwrite what they hold. Checked at the point of sending rather than at scheduling, so everything short of the wire still happens -- changes are held, dumped, and reported by needs_push(). It is deliberately not a pretence that the push succeeded: the state reads as unpublished rather than as settled, and switching it back on lets the next push carry everything accumulated since.
Groups have had `delete_before` and `delete_attach_before` since they existed: an instruction that messages (or just attachments) older than a timestamp are to be deleted, and arriving ones older than it dropped. One-to-one conversations never got them, which is why clearing a 1:1 is local-only while clearing a group as admin propagates, and why hiding one has to be faked from config *timestamps* instead. That fake is worth spelling out, because it is what this replaces. With nothing recording when a conversation was cleared, a client compares an incoming message's timestamp against the newest config message it has processed -- minus two minutes of slack for clock skew -- and drops the message if the config looks newer. So a conversation cleared at noon and some unrelated setting changed at five o'clock is treated as having been cleared at five: every message in between is discarded, permanently, because the swarm cursor moves past them. The imprecision is in the format rather than in any client -- no config records when a field changed -- so recording the instruction is the only fix. `d` and `D` in the contact dict, deliberately the same letters the group info config uses, and seconds to match it: these two will be read side by side forever by code asking "what is the delete-before for this conversation", and one unit across both is worth more than the second of precision milliseconds would buy. Note to self gets its own pair in UserProfile rather than a contacts entry. It has no contacts entry to carry them: UserProfile exists from the moment the account does, so there is nothing there to be present or absent. Keying one under our own pubkey was considered and rejected -- every client already carries a "self is not a contact" carve-out, and android has five contact iterations with no such filter, one of which would put us in the contact picker for group creation. `delete_attach_before` is only stored while it says something `delete_before` does not. Deleting a message takes its attachments with it, so setting either of the pair drops the attachment value when the message value already covers it, rather than leaving an instruction that means nothing.
`path` records where a file went or came from -- closer to a symlink than a handle. The file at the far end is the user's in both directions: theirs to attach, or theirs to save wherever they chose. So nothing here may unlink it, and the comment now says so rather than only hinting at it by mentioning that the file might vanish. Worth stating now because a delete-attachments-before instruction is coming, and the obvious reading of it -- delete the files -- would be deleting the user's own originals on the instruction of a config merge. What it will actually be entitled to remove is a cache of our own: attachments kept encrypted so they outlive the file server. That is a third thing at a path we picked, and it wants its own column; sharing this one would leave the delete path unable to tell our copy from the user's, and the safe implementation would then be the useless one. Comment-only: the schema fingerprint strips comments, so nothing to migrate.
Both directions at once, because they are not separable: the destructive rule that a contact absent from the merged config was deleted elsewhere is only safe while a locally-created contact is written into the config as it is created. Ship the reading half alone and the first merge eats every conversation we made ourselves. Each entry lands on three tables, because it carries three kinds of fact. Who someone is goes on accounts, which anyone we have merely *seen* also has. The relationship goes on contacts, where the row existing is what being a contact means. How the conversation behaves goes on conversations, which is also what brings that conversation into being: a conversation is not defined by having messages in it, or emptying one would lose it and reimporting an account would lose every empty one. Whose name wins is decided by profile_updated rather than by who spoke last -- the config's name is applied only when its stamp is at least as new as what we hold -- so a profile carried on a message that arrived out of order cannot displace a newer one. Out of order is the normal case, since a profile is observed from group messages as readily as from a DM. The reverse direction re-derives an entry from the rows rather than being applied alongside each change. The mapping then lives in one place and cannot drift from the tables it describes; a caller has to remember to call it, but cannot remember to call it wrongly. Presence follows the row too: no contacts row means no entry, so removing a contact is "delete the row and sync" rather than a second place that must remember to erase. Which is worth nothing unless the two directions agree exactly, so that is asserted: apply a config to the tables, derive one back, and the config must still be *clean*. Anything lost, rounded or defaulted in transit shows up as a dirty config -- and a mapping that dirties on every pass would push a pointless update after every merge, forever. Verified by perturbing one field by one second, which fails it. A nickname now wins over the name a contact chose for themselves, which is what the accounts/contacts split was for and had no effect until something populated it. Not here: deletion. A contact absent from the config is not yet acted on, so this pass only ever adds and updates. That lands next, on top of the round-trip property rather than before it.
A contact we hold that the merged config does not mention was removed elsewhere, and removing a contact takes the conversation and its history with it -- deliberately: someone who deletes a conversation means it deleted, not hidden on one device. Merely hiding is a negative priority and arrives as an ordinary settings change, so an absent entry can only mean the stronger thing. The account row stays. We may have seen that person in a group or a community and need their profile to render it; what is deleted is the relationship, not the person. Messages, raw content and attachments go with the conversation by cascade, and the files those attachments name are the user's -- theirs to attach or theirs to have saved -- so nothing is unlinked. _reconcile_all now derives outward before reconciling inward, and the order is load-bearing. Creating a contact commits the row and updates the config in memory, but the dump is written later, so a crash in between leaves a row the config has never heard of -- indistinguishable, inward, from one deleted elsewhere, and destroyed with its history. Publishing what we hold first tells those two apart. It is safe to run because assigning a config field its existing value does not dirty it, so the outward pass writes only where the dump was actually stale; a contact deleted elsewhere is still in our config at that point, so it is not resurrected and the deletion arrives and wins on its own merits. Both are tested, and the removal test had to be fixed before it tested anything: built from a fresh config, the second device is a rival at the same seqno and merges to the *union* of the two, so nothing is ever absent. It now descends from what we published, which is what a device that has seen our config actually is. Recorded but not built: once attachments are cached, freeing them cannot be done by deleting files alongside the rows that name them, because cascade takes those rows with no code of ours running. It wants the same treatment as everything else here -- list the paths still referenced, list the directory, unlink the difference -- which survives a cascade and also collects what a crash mid-download left behind.
Once availability is part of the message, a change to it is a change to the message: an application holding one now holds something we know is wrong, and `message_updated` is how everything else in here says so. Fired on the three transitions a transfer makes -- `absent` to `fetching` when one starts, and back to `cached` or `absent` when it finishes or fails -- and for every message showing that file, since more than one routinely does. Not display pictures: one belongs to a conversation rather than to a message, and has its own progress handler to report itself with. The completion emit goes before the waiters are told, so that a waiter which reads a message in its callback finds the settled state rather than the one it was called about.
A cached file was named `blake2b(base_url)` -- unkeyed, unpersonalised, and so a function of information the adversary already has. That makes the cache directory a membership oracle: hash a url you are curious about, look for that filename, and you know whether this account downloaded it. Encrypting the contents does nothing about it, because the question is answered by the name before any file is opened. It applied to display pictures equally, so "does this account have a copy of that contact's avatar" was testable the same way. Which defeats the reason the cache is encrypted at all, stated in its own header: the disk is not a trusted place, it ends up in backups, in disk images, and in whoever's hands the machine does. So the name is a MAC rather than a digest, keyed on `client:cache_key` -- the secret the files are already encrypted under, which lives only in the encrypted database -- and personalised, so that one key serving two purposes is kept apart by construction rather than by the primitives happening not to meet. A listing now says how many files there are and nothing else. The key is applied in `Client::_cache_path`/`_cache_name` rather than at each call site: a caller that forgot it would silently get the guessable naming back, so the way to get a name is to ask the object that has the key. Every existing name changes, so the migration drops the cache rows: the files they name are then referenced by nothing, which is what a sweep collects. A cache is the one thing that costs nothing to lose.
Every question the attachment cache asks of this table is "which messages show this file": which ones to report as fetching when a transfer starts, as cached when it finishes, and as absent again when the file is evicted. More than one message routinely quotes one file, since an attachment url is a hash of the encrypted body and so the same file sent twice by the same account lands at the same url. Without an index each of those answers is a scan of every attachment row in the account to touch a handful of them. Partial on url because a row without one has nothing to fetch and is never the subject of any of it.
A download that fails because the file server does not hold it, or because what arrived did not authenticate, says something about the file rather than about the attempt. Until now that was reported once, to whoever asked, and forgotten: the next thing to look at the message offered the same fetch again, and the only way to find out it could not work was to run it. Record it on the attachment instead, as `Attachment::unavailable`, so a display can say the file cannot be had rather than invite an attempt that will fail. The value says which of the two it was, because they are different things to tell a user: 404 -- which is how an expired upload answers -- means asking the sender for it again will work, and ATTACHMENT_UNREADABLE means it will not, since an attachment url is a hash of the encrypted body and a resend of the same file reproduces the same bytes. Nothing transient is recorded: a timeout or a server error says nothing about the file and the next attempt may well succeed. Set and cleared across every row naming the url, not just the row that was being fetched, since it is one file and a transcript must not show one message's copy as broken and another's as fine. Clearing happens when a new attachment row quoting that url arrives, which is the resend the user was told to ask for; the verdict has to be a cached answer rather than a permanent one, or it would block exactly the action that repairs it. Both directions emit message_updated for the messages they change.
`availability` says a download is running but not where it has reached, which leaves a conversation opened mid-transfer with a bar it cannot position: the progress reports went to whoever started the fetch, and a display that was not listening has no other way back to the figures. Carry them on the attachment alongside the state they qualify, counting the same encrypted bytes `AttachmentProgress` does, so a bar seeded from the message and then fed by reports continues rather than jumping. Also covers the availability states themselves, which went in untested: absent before anyone asks, fetching from the moment the transfer starts, cached once it finishes, and absent again when the cache gives that copy up for another file.
A message we just sent drew as something to press, and pressing it fetched back a file that had come off this disk a moment earlier. Keep what we upload in the attachment cache so an outgoing attachment answers "is this here" the same way an incoming one does. Under the rule that would have applied to the same file arriving -- shared with the auto-download path rather than restated -- plus anything on a gallery-viewable message whatever that rule says: a gallery is drawn as its pictures, so those bytes have to be here or the conversation shows placeholders over files it could read directly. The gallery rule decides that, the same rule the display applies, rather than an attachment happening to be an image: a message mixing a picture with a document draws as a list like any other. The source is re-read, since the upload streams the file rather than holding it, and only when its length still matches what was uploaded: a url names one particular encrypted body, and storing anything else under it would leave the cache answering for that url with the wrong bytes. Everything else about it is best effort -- a file that has since moved just means a download if the message is ever drawn again.
A cached file is named by a keyed hash of its url, and that hash was how
everything found it: "is this file here" hashed the url and looked the
result up. Two things follow from that, and both are worth being rid
of.
The hash becomes load-bearing for every file already downloaded rather
than only for the ones being written, so changing it cannot be done
without invalidating the lot -- which is exactly what 005 had to do.
And it does not run backwards, so eviction, which holds a file name and
needs the messages drawing that file, could not get to them at all: it
deleted the row and the file and said nothing, leaving conversations
showing pictures whose files had gone. `AttachmentAvailability` already
promises those changes are reported.
Give `attachment_cache` a surrogate key and have an attachment row
reference it, `ON DELETE SET NULL`. Nothing is duplicated -- the row
stores an integer, not a second copy of the name or the url -- and the
reference runs the way the questions do:
- availability comes back with the row, so reading a page no longer
hashes a url or looks anything up per attachment;
- finding an existing copy goes through the reference and uses the
name the entry was stored with, so the hash applies only to a file
being created and can change freely;
- eviction reads the messages off an index before the delete, and the
foreign key clears their reference on the way out.
The reconcile sweep drops its stale rows through the same path, so a
file that vanished behind our back is reported like one we deleted on
purpose. `_in_flight` moves to being keyed by url at the same time: the
hashed name is about what a directory listing reveals, which is not a
question a map in this process has.
Caching a file said nothing. On the download path that was hidden: the fetch's completion emitted right after storing, so the messages showing the file were told. An upload has no such completion, so keeping a copy of a file we sent -- which makes it drawable for every message quoting that url, and an attachment url is a hash of the encrypted body, so somebody having sent us the same file is the ordinary case -- told nobody but the message being sent. Move the emit to `_cache_attachment`, which is where a file becomes cached and is reached both ways. `store` then says whether it got that far, and the fetch reports the transfer ending only when it did not: a failure, a file that could not be written, and no cache at all are the same event to a reader, and all three leave nothing behind. The entry also has to leave `_in_flight` before any of that runs, or the report of a file arriving says it is still arriving. The test drives one file, shown by two messages, through every transition -- absent to fetching, back to absent for a transient failure, to cached, out again by eviction, to unavailable on a 404 and clear again on a resend -- and asserts which messages were told and what they were told the state was, rather than counting events. A second covers the upload case across conversations.
One cause routinely changes several messages, and reporting them one call at a time made the application reassemble what libsession had just taken apart. A file is the clearest case: it is one file, so evicting it, fetching it, or finding it unfetchable changes every message showing it at once -- and reporting a message also reports its replies, since a reply draws a preview of its target. So `message_added` and `message_updated` become `messages_added` and `messages_updated`, taking a vector. Two things are now promised that could not be before: - **ordered oldest first**, by the timestamp history is ordered on and then by id, so a handler applying a batch in order lands where a reload would put it. That is the reverse of `messages()`, which pages backwards from the newest; - **a batch may span conversations**, which is what the fan-out cases need. Nothing is lost by allowing it: `Message` already carries its own `conversation`, which is why the callbacks no longer take one. Eviction and the reconcile sweep now collect across the whole pass, so a message showing two of the files being dropped is told once rather than twice, and the ids are resolved to Messages only at the end -- one build per message, in its settled state rather than in whatever state it was in partway through the pass.
`FileTransferRequest::cancelled` was consulted on every upload path and on none of the streaming download ones, so cancelling a download only stopped us from using what arrived: the rest of the file came down the stream anyway and was dropped a chunk at a time. That is the common case rather than an unusual one. The stream scheme authenticates each chunk as it arrives, so a file that has been tampered with, or one that runs past the length its sender claimed, is known to be unusable from the bad chunk onwards -- and a large attachment can be most of the way through when that happens. Shutdown was the same: the routers cancel their active downloads and then close, with the transfers still running. Check the flag as each chunk arrives, which is the only moment this end of a transfer is given -- a download is driven by the server -- and abort the stream when it is set. A stalled stream still notices nothing, and the caller's timeouts remain what end those. The layer above needed one correction to suit it: a download we abandoned now completes with the cancellation we asked for, so the reason we abandoned it has to be preferred over the status when there is one. Otherwise a decryption failure would report itself as "download failed with status -10200", and the code that says a retry is pointless would be lost.
There was a getter for the cache limit and none for what is in it, so a client could show the ceiling but not how close to it the cache was. Summed from the cache index, in bytes on disk, which is the measure the limit is in and the one eviction compares against.
Eviction and `attachment_cache_size` were the same query written twice, which is also two copies of the answer to "why is this an optional" -- SQL sums an empty set to NULL, so an empty cache came back as no answer rather than as no bytes. Answer it once, in SQL, where the NULL is produced: `coalesce` makes the query incapable of returning one, so the C++ type stops advertising a state that cannot occur. That is how the rest of this file defaults a NULL already. The connection-taking overload is for eviction, which holds one and is about to write through it.
The schema grew over four migrations as this branch was built, one per commit that needed it, which is right while the commits have to stand on their own and wrong once they are merged: a migration is permanent, numbered and ordered, and these four describe intermediate states that have never existed anywhere but here. One migration per merged change instead. 005 now does what all four did, and the commits still show how it was arrived at. Its predecessors are exactly the databases that cannot take it -- they have some of the four recorded already -- so the branch goes on the schema-history skiplist: every commit of it but this one, since no state of it before this is something anything can have started from.
Nothing consumes a caption. It arrived on an incoming protobuf pointer, went into a column, came back out as a struct field, and went back out on the wire unchanged: no client renders it, nothing searches it, no index or constraint refers to it, and the conversation-preview query already skips it deliberately. A field that only round-trips is one more column to keep positionally correct in four statements for no behaviour. Field 11 is reserved rather than deleted: clients on the network keep sending it, and the number must never be handed to anything else. The checked-in protobuf output and the generated debug printer are regenerated to match, with protoc 3.21.12 - the version that produced the existing files.
One migration per merged change: dropping the caption column is part of what this branch does to message_attachments, so it belongs in 005 with the rest rather than in a 006 of its own.
`unavailable` was recorded across every row naming the url, on the reasoning that a url is a hash of the encrypted body and so names one file read one way. That holds for an honest sender and no one else: nothing ties a pointer's key, digest or size to its url, so anyone can send a pointer to a file they have seen with a key of their own making. Fetching it fails to authenticate, and that failure was then written to every copy of the file in every conversation -- permanently, and with the message that asking for a resend would not help. A 404 genuinely is the server's answer about the url, and stays url-wide. An authentication or size failure is about the claim, so it now marks only rows making the same claim; and a resend, which is evidence the server holds the file again, clears only the former, since the resent bytes would reproduce the latter. The same poisoning was reachable without the database, through joining: transfers were keyed by url, so a display asking for a file while a bad pointer's transfer ran would take that transfer's answer. They are now keyed by the whole claim, which honest copies share, so those still share one download; whether anything is fetching a file is still asked of the url. The struct carrying url, key, digest and size is renamed RemoteFile while it is being threaded through more places: it was called a pointer after the protobuf message it comes from, which is not what the word means in C++.
A fetch that landed marked every row naming the url as cached and left `unavailable` as it was, so a row could say both at once -- the file is here, and the file cannot be had. Reachable whenever a not_found was the server's passing trouble and a later attempt worked, or when one row's key failed and another's did not. A file in the cache is served to every row naming it, whatever key that row carries, so none of them can honestly keep a verdict that it is unobtainable. Clear it in the same statement that marks them cached.
`_download_decrypted` throws before sending anything when there is no network, the url is not a download url, or the key or digest is the wrong length. `_fetch_cached` had already registered the transfer and reported it as `fetching`, and the completion that would have removed it never runs -- so the file said `fetching` for the life of the process, and every later request for it joined a transfer that had never been sent and waited for ever. Register, try to start, and report `fetching` only once it has; on a throw, remove the entry and tell its waiters. Told here rather than rethrown, because not every caller still holds its handler by then: `profile_picture` moves it in. Two more of the same shape. An auto-download that throws -- a pointer with no url is enough -- escaped the arrival path before the message was announced, so it was stored and never reported; it is now caught per attachment. And a save that cannot start left its temporary file beside the destination.
Both callers already held one and spelled out all four of its fields to make the call; the function now takes it whole.
`Attachment` said two things about its file: `availability` -- cached, fetching or absent -- and `unavailable`, what the last attempt found. A display has one thing to decide, what to draw, and had to rank the two itself with nothing in the header saying how. `absent` in particular meant "go and look at the other field": fetchable if that was unset, not worth offering if it was. There is a ranking, and it is total: a cached file is what to draw whatever an earlier attempt found, a transfer running is about to settle it, and only with neither does the verdict matter. So say it once, as `not_found` and `unreadable` alongside the other three, and `absent` goes back to meaning only that the decision is the user's. The stored codes keep their own private type. The numbers are what is on disk, and the public enum's values are not meant to be anything in particular; tying the two together would let renumbering one silently change the meaning of the other. A verdict is also a fact about the last attempt rather than about the disk, so it is reported with no cache configured, where the other states all read as absent.
Its preamble described `name` as the hashed base url that a row could be matched against a message_attachments.url by, which is what keying the hash and adding the `cached` reference abolished: the hash does not run backwards, and nothing finds an entry by recomputing it. It also said the sweep adopts files that have no row, where it deletes them. message_raw_content's comment is put back above its own table; folding 002 into this file had left the cache's block between them, with a stray marker where the join was.
Every other migration here is bare SQL, with the explanation living in full_schema.sql beside the thing it explains; 005 had grown a copy of most of that prose, plus an account of how the branch arrived at it. Cut it to the statements and the one thing only a migration needs to say, that it throws the cache away.
A few this branch added describe what something used to be, or argue against an arrangement nobody would now propose, instead of saying why it is the way it is.
Keeping a copy of a file we sent reads it back and encrypts it whole, on the loop -- the same work caching a download does. A download is refused past MAX_REGULAR_SIZE, so that work is bounded there; an upload is not, and the only other check was the auto-download size limit, which is optional. With none set, sending a large file stalled the loop for as long as reading and encrypting it took. Apply the download's own ceiling. The cost is that a file past it cannot be drawn from the sender's own transcript -- but no recipient using this library can fetch it either, so that is where such a file already stands everywhere else.
An image attachment is a grey rectangle until its bytes arrive, and on a slow link that is most of the time the user spends looking at it. A ThumbHash rides on the pointer instead: 25 bytes at most, decodable into a blurred approximation of the picture, so the recipient draws something that resembles the image from the moment the message is listed. Carried opaquely. libsession stores the bytes, hands them back and puts them on the wire without decoding them or checking them against the file -- a client that wants a preview decodes it, and one that does not pays nothing. Like width and height beside it, encoding one needs the pixels, so it is the sender's to supply and an unset thumbhash stays unset. The 25-byte bound is the format's own ceiling: a 7x7 luminance DCT, 3x3 for each of P and Q, and 5x5 alpha. Over-long values are treated asymmetrically, as the surrounding code already treats everything else: thrown on the way out, where the value is our caller's and the mistake is theirs to see, and dropped on the way in, where it came from a remote peer and losing a placeholder must not cost the attachment it describes. Field 12, since 11 is reserved. The checked-in protobuf output and the generated debug printer are regenerated with protoc 3.21.12, the version that produced the existing files.
Brings liboxenquic 1.9.0, whose JobQueue owns the timers it runs, and the router's per-path stats. Loop::call_every is deprecated there, so every timer moves to JobQueue::add_timer: Core's poll, renew and probe timers onto Core's queue, and QuicFileClient's idle and upload progress timers onto a queue of its own, along with the jobs that reach the client. A queue cancels its timers when it stops, so Core no longer relies on declaring its poll timer last, and dropping a subscription no longer has to defer freeing a ticker it may be running inside. The progress timer is still removed on every exit from the upload, since its callback borrows the request. Stream::get_stats() gains a retained count, and a session's path is now reported with its stats rather than as a bare list of hops.
A subdirectory whose cmake_minimum_required predates 3.9 has CMP0069 OLD, so CMake ignores CMAKE_INTERPROCEDURAL_OPTIMIZATION there. protobuf's is 3.5, which made protobuf-lite the one library in an LTO build compiled without -flto, and a link that pulls its archive members in late -- testLogging's -- then fails on the inline std:: functions they reference. Default the policy to NEW beside the other settings that keep USE_LTO the single switch.
A failed download told its requester download_failed whatever went wrong, told its progress listener an integer status, and told every message quoting the file a separate verdict -- three vocabularies for one classification, and the one the requester got said nothing. Every failure now carries an Error code named after the object that failed: message.not_found and attachment.not_found for a request naming nothing, file.not_found and file.unreadable for the two verdicts AttachmentAvailability also shows, file.download_failed for an attempt that says nothing about the file, file.upload_failed, attachment.file_missing and file.save_failed for the rest, and network.unavailable when there is no network. AttachmentProgress and the other progress callbacks end with the same Expected<void> the requester gets, in place of a status integer, which retires the public ATTACHMENT_* constants; the verdict's on-disk values are Unavailable's own. Codes nothing emitted are gone. Inside, _download_decrypted's completion takes a DownloadResult and a reason, from which both the public Error and the recorded verdict are derived. An exception from the consumer of the bytes is failed rather than unreadable: it is our side failing, and recording it against the file would tell every message quoting it not to try again. Failures that used to escape as internal.exception now say what they are. A claim no download could satisfy -- no url, a url that is not one, a key or digest of the wrong length -- is unreadable, and one that reaches the download is recorded as a verdict without anything being sent; an incoming attachment with no url shows as unreadable. A request for a file that already has a verdict answers from it without downloading, since only a resend changes it.
Both want a transfer that fans each chunk out to its consumers and a cache writer that encrypts as it goes, which is a change of its own.
Attachment availability: say what we hold of a file, and tell everyone showing it
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This draft PR tracks the full client support being developed in libsession, to serve as the logic core of future versions of Session. This builds on top of PFS+PQ support (PR #103), and adds a ton of new features and capabilities needed to build a full Session client. The most notable starting point here is the
Clientclass which is the entry point for an active programmable Session client.This branch is not intended for review, but rather merely tracks the progress of the ongoing
clientbranch (which will eventually become thedevbranch).