BAN3200 Module 6 — Assessment SPEC

Step sequence

#	Step ID	Component	Type	CLO	Anti-delegation
1	scenario	ScenarioOverview	Display	—	Copy blocked on instructions
2	model	ModelIdentification	CMP-011 + hidden MCQ	CLO-3	Decision Tracing: coefficient interpretation tail
3	audit	DataQualityAudit	CMP-006	CLO-1/CLO-6	Decision Tracing: email opt-out % tail FIB
4	metrics	MetricsAnalysis	CMP-002 × 3	CLO-3/CLO-6	MCQ cap (2 attempts, BBA)
5	ai-audit	AIOutputAudit	CMP-008	CLO-6	Written identification, paste blocked
6	threshold	ThresholdApplication	Custom + hidden FIBs × 2	CLO-3/CLO-5	Decision Tracing: FN count + FP rate tails
7	escalation	BudgetEscalation	CMP-010 timed (3 min)	CLO-5	Timer prevents AI consultation; stale-closure-safe refs
8	report	PreDeploymentReport	CMP-009	CLO-3/CLO-5/CLO-6	Paste blocked; injects all committed answers

Seeded data (fixed, all learners — variants planned)

• Confusion matrix at threshold 0.55: TP=600, FP=600, FN=600, TN=6200
• Disaggregated <2yr: TP=150, FP=150, FN=450, TN=1250 (recall 25%)
• Disaggregated ≥2yr: TP=450, FP=450, FN=150, TN=4950 (recall 75%, F1=0.60)
• Threshold options: 0.40 (1,800 flagged), 0.55 (1,200), 0.65 (840)
• Escalation budget: 1,200 → 900; additional missed at 0.65 = 120

Decision Tracing tail answers

Step	FIB 1 correct	FIB 2 correct
ModelIdentification	MCQ: "a" (log-odds interpretation)	—
DataQualityAudit	12 (email opt-out %)	—
ThresholdApplication (0.55)	600 (FN count)	50 (FP rate %)
BudgetEscalation	120 (additional missed members)	—

AI Output Audit errors

• Statement 2: "85% correctly identifies at-risk members" — accuracy/recall conflation
• Statement 4: "members who joined 0.28 months ago or less" — SHAP importance treated as threshold

Grading

• MCQ/FIBs: server-side answer keys in getCorrectAnswers("BAN3200")
• Written sections (ai-audit + report): Nexus grading against Domain/Reasoning/Contribution rubric
• Rubric descriptors: seeded via scripts/seed-rubrics.ts