SIGN IN SIGN UP

fix(droid): attribute session usage to the replies that spent it, and bound it server-side (#1120)

* fix(droid): anchor session usage at last write instead of provider lock

A Droid `*.settings.json` holds one cumulative `tokenUsage` total for the whole session and is rewritten in place every time the agent spends tokens. The parser emits a single record for it and was anchoring that record on `providerLockTimestamp`, which records when the provider was *selected* rather than when the tokens were spent. Droid never updates that field as the totals climb.

For a session that stays open across days — a `/loop`, a long autonomous run — every token it ever spends is therefore attributed to the instant it started. The session reads as completely silent in `--today` and `--yesterday` while it is actively burning tokens, and its usage lands in a bucket days in the past. On a local install with three Droid sessions still running, `--today` reported 0 messages and 0 tokens against ~516M tokens actually spent that day.

Anchor on the settings file's mtime instead: that is when the totals being read were written, so it is the closest available marker for when they were last accrued. The lock timestamp becomes a floor rather than the answer — usage cannot predate provider selection, so an mtime rewound by a copy or restore cannot drag the record earlier than the session could possibly have run. With neither anchor available the record falls back to now() rather than being dropped or bucketed pre-epoch, since it still carries real token usage.

`file_modified_timestamp_ms_opt` exposes the mtime as an `Option` so this parser can distinguish "no mtime" from a substituted value; the existing `file_modified_timestamp_ms` keeps its now() fallback and is now written in terms of it. `modified` is the one timestamp every tier-1 target reports, unlike `created`, so the anchor behaves the same on Linux, macOS, and Windows.

Bumps the Droid parser version to 2. A session that has since ended never changes its bytes again, so its fingerprint keeps matching forever and only the version bump discards the v1 lock-timestamp anchor from warm caches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(droid): treat a pre-epoch provider lock as an unusable anchor

The lock timestamp was filtered with `ts != 0`, which rejects Droid's unset sentinel but lets a negative value through. The mtime anchor has no such gap: `file_modified_timestamp_ms_opt` reports `None` for a pre-epoch mtime because `duration_since(UNIX_EPOCH)` fails. That asymmetry meant a negative lock survived as the resolved anchor whenever mtime was unavailable, so the record was written with a pre-epoch timestamp and bucketed into a 1969 day key that no date filter reaches — the outcome the now() fallback exists to prevent.

Filter on `ts > 0` instead, which keeps the original zero-rejection and adds the negative case, so both anchors share one validity rule and an unusable lock reaches the resolver as absent. Parsing moves into `parse_lock_timestamp` so the rule is unit-testable on its own rather than only through a file whose mtime the test cannot suppress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(droid): attribute session usage to the replies that spent it

Anchoring a session's cumulative `tokenUsage` at a single instant cannot describe a session that ran for days. Whichever instant is chosen, one day absorbs every token the session ever spent. Against a provider-side dashboard, a machine with three multi-day sessions running reported 517M tokens for a day that had actually spent 24M — a 21x overstatement, where the previous lock-timestamp anchor had instead reported zero.

Droid records no token counts in the transcript, so there is nothing to read per call, but the transcript does say when each assistant reply happened and how much conversation was live at the time. Cost per call tracks that closely: nearly every token in these sessions is a cache read of the conversation so far. Weighting each assistant reply by the bytes accumulated since the last compaction therefore recovers what one reply cost relative to another, which is all that apportioning the recorded total needs. Compaction resets the running weight because it discards the transcript the following calls would otherwise re-read.

The session's cumulative total is split across those replies and emitted as one record each, so a multi-day session now reports against the days it actually ran, and hourly reports become meaningful for a client that previously had a single timestamp per session. Splitting allocates on the running total rather than rounding each share independently, so the parts sum back to exactly the total Droid recorded and daily figures cannot drift from it.

Measured against the same provider dashboard: a day that truly spent 24,226,571 tokens now reports 24,547,118, within 1.3%, against 517,527,976 before. The remaining gap over a multi-day window is Droid's own under-recording — errored turns cost provider-side tokens that never reach `tokenUsage` — and is outside what any parser of these files can see.

Sessions whose transcript is missing, unreadable, or has no assistant reply yet keep the previous single-record behavior, so usage is never dropped for want of a transcript.

Bumps the Droid parser version to 3: cached entries hold one record per session under the old shape and cannot be reconciled with the new one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(droid): weight a reply by the context it read, not its own output

The apportioning weight was captured after the reply's own line had already been folded into the running context, so a long answer charged itself for the tokens it produced. Those bytes are output: the call's input and cache-read cost was fixed by the conversation standing before it ran. A verbose reply therefore claimed a larger share of the session's input totals than it earned, and the share came out of the other replies — including replies on other days, which is what makes it an attribution error rather than a rounding one.

Capture the context before folding the line in, and fold it in afterwards so later calls still read it. Two replies that read the same conversation now receive the same weight no matter how much either of them wrote.

The skew is bounded by how large a reply is next to the conversation it reads, so it stays small on long sessions and grows on short ones: across the sessions on hand it moved a day's figure by 0.07%, and a day of short sessions by 4.7%.

Bumps the Droid parser version to 4. Versions 2 and 3 only ever existed in pre-release builds of this change; the bump past them keeps anyone who ran one from holding the superseded weighting.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(droid): correct and bound the transcript-weighted usage split

Three defects in the apportioning added by #1090, all in the transcript
path that splits a session's cumulative tokenUsage across its replies.

A pre-epoch transcript timestamp was accepted as an attribution anchor.
The lock anchor already rejects one (parse_lock_timestamp filters ts > 0)
because a non-positive timestamp is the parsers' "no usable time"
sentinel, which rebucket_date refuses to re-key; parse_timestamp_value
applies that floor only to its numeric branch, so an RFC3339 value like
"1969-07-20T20:17:00Z" reached the turn list and anchored a share of the
session in a 1969 bucket no date filter reaches. Skip such a turn: the
tokens stay in the session because apportioning runs over the turns that
remain, and a transcript with no other usable reply falls back to the
mtime-anchored single record.

Output and reasoning were apportioned on the context each reply read.
That is the right weight for input, cache reads and cache writes, which
pay for the conversation the call sends, but output is what the call
produced. A verbose answer on a short context was credited the output of
the replies around it, including replies on other days. Carry two
weights per turn -- the context standing before the reply, and the
reply's own bytes -- and split the read-side and write-side totals on
their own weight.

The transcript read was unbounded. Every line of the sibling *.jsonl was
deserialized on every parse and every assistant reply was retained as a
cached message, with cost linear in transcript size and paid again on
each scan while the session is live, since a growing transcript keeps
invalidating the cached parse. Measured on a 104.7 MB transcript with
10,000 replies: 148 ms per parse in release, 1.28 s in debug (~1.4
ms/MB), yielding 10,000 retained records for one session. Cap the read
at 32 MiB -- past it the session takes the single-record path, which
reports the identical total and loses only the intra-session detail --
and coalesce the turn list to at most 1024 runs, summing weights so the
apportioned total is unchanged and only the resolution drops.

Bumps the Droid parser version to 5, covering all three: cached rows
hold a split this parser would no longer produce.

* fix(frontend): bound every re-attributing parser by the device high-water

#1090 makes the Droid parser split a session's cumulative tokenUsage across
the replies that spent it, so a session that ran for days reports against
the days it actually ran. That moves tokens between days without changing
the lifetime total, and the hosted merge cannot read it as a move. The
per-(day, client) regression guard in mergeClientBreakdownsWithRegressionGuard
refuses a decrease, so the days that fall keep their old figure while the
days that rise are written: the device's stored total gains exactly what
moved, permanently, and again on every resubmit. A single 321,000-token
session re-attributed across three days stores 621,000.

The device/client lifetime high-water added for Copilot already answers
this -- credit only growth bounded by both the per-cell delta and the
cumulative aggregate, so a pure reshuffle contributes nothing while real
new usage still lands -- but the submit route hardcoded it to one client.
Drive the loop from SUPPORTED_VERSIONED_PARSERS and register Droid there.
The per-client plan, freeze, increment, provenance and state-advance steps
become a map keyed by client; Copilot's behaviour is unchanged, including
the baseline-new case that still merges its snapshot directly.

Droid is registered at generation 1: the generation every installed CLI
already declares for it (SUBMISSION_PARSER_VERSION), not the message-cache
parser_version this branch bumps to 5. The two Droid shapes differ in where
a token lands, never in the lifetime total, so no installed generation has
to be frozen out, and registering anything above 1 would freeze every CLI
in the field instead.

Two consequences follow from joining the high-water path, both shared with
Copilot and both intended. A CLI old enough to send no scanScope has its
Droid rows ignored once the device holds a Droid high-water, because an
undeclared generation cannot be trusted to advance one. A date-filtered
scan (--since/--until/--year) cannot establish or advance the high-water
either, so it freezes Droid rather than crediting a partial snapshot; the
next unfiltered submit re-credits everything it covered.

* test(cli): pin the Droid submission generation to the server registry

The submission parser generation is a cross-language contract: the hosted
high-water registry accepts exactly one generation per client and freezes
every other, so a client whose declared generation drifts from
SUPPORTED_VERSIONED_PARSERS stops being credited server-side rather than
failing loudly. Droid now sits in that registry at generation 1, and it
takes the value from the shared SUBMISSION_PARSER_VERSION default, where a
bump made for any other reason would silently freeze it.

Assert the declared value directly so the coupling breaks a test instead of
a user's history.

* fix(droid): keep the reply count, and fail closed on an unreadable transcript

Three findings from review of the bounded transcript read.

A coalesced run reported one message. Past 1024 replies the turn list folds
into runs so the record count stays bounded, but each run was emitted as a
single UnifiedMessage, so a 1200-call session reported 1024 calls and a
10,000-call session reported 1024. Weight folding was already correct -- the
apportioned tokens never moved -- but the message count is its own metric.
Carry the number of replies each run stands for and emit it as the record's
message_count: the cap costs attribution resolution, not the count of calls.

An unreadable file size waived the ceiling. `metadata()` failing mapped to a
length of 0, which compares below any ceiling, so the one case where the size
is unknown was the case that authorized an unbounded read. Fail closed: no
size, no split, and the session reports its identical total through the
single-record fallback.

A partial read was treated as a whole transcript. `lossy_lines` ends silently
on a hard I/O failure (vanished network mount, EIO), so a prefix is
indistinguishable from the whole file by the turn list alone, and apportioning
the session's cumulative total over that prefix moves every token the unread
tail earned onto the days the prefix covers. The parse is then cached under a
fingerprint a finished session never invalidates again, so the misattribution
sticks. Compare the bytes consumed against the length the file declared and
fall back when the read stopped early. Reading past that length is normal --
a live session appends while the parse runs -- so only a short read
disqualifies the split.

Bumps the Droid parser version to 6: cached rows hold a message count this
parser would no longer produce.

* fix(droid): count one session, not one per apportioned record

The transcript split emits one record per assistant reply (or per coalesced
run), and each carried its own message count. sessionize opens a new interval
whenever two records sit more than the idle gap apart and counts every interval
that reports messages, so a session whose replies are minutes apart became one
session per record: 40 replies ten minutes apart reported sessionCount 40. The
frontend computes timeMetrics.sessionCount before parserPlans and ratchets it
with GREATEST, so the inflated figure bypasses the device high-water and sticks
on a public profile.

Carry the session's authoritative reply total on exactly one record and make
the rest count-neutral, the same split copilot_desktop applies to its shutdown
fragments. The number of calls stays exact; the session count returns to one.

Also drop the unreachable `total_context > 0 && total_output > 0` guard: both
weights are `.max(1)`, so a non-empty turn list always has positive weight.

* fix(droid): invalidate caches holding a count on every apportioned record

A v6 entry carries the reply count on every apportioned record, which is what
sessionize reads as one session per record. A finished Droid session never
rewrites its settings.json, so its fingerprint keeps matching forever and only
a parser-version bump discards the superseded split. Droid parser version to 7.

---------

Co-authored-by: Jackson57279 <otdoges@proton.me>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
J
Junho Yeo committed
151ecff23042503db07da5e4bef91618291da161
Parent: 0daf378
Committed by GitHub <noreply@github.com> on 8/14/2026, 3:28:58 AM