The failure mode this audit hunts
Course platforms love difficulty badges. The dishonest version: an exercise tagged "advanced ML" whose solution is a for-loop and two additions. So we built a classifier that reads every solution's actual API surface and tags it by what it really exercises:
- arithmetic: plain JS math, no tensor ops.
- data: tensor primitives (create, add, matMul, read back).
- model: layer composition and prediction.
- training: gradients and fitting.
- evaluation: metrics and drift checks.
- deployment: serialization, runtimes, quantization.
- systems: incident scenarios and production architecture.
The numbers (audit of 2026-04-26)
At audit time, 315 authored exercises across all five themes classified as 73.7% arithmetic, 13.3% data, and the rest spread over model, training, evaluation, and deployment. Difficulty badges contradicting the classification: zero after the fix pass. Where an intermediate badge sat on arithmetic content, either the badge came down or the exercise was rebuilt on real TFJS primitives; a follow-up sprint rewrote twelve arithmetic exercises into tensor-native versions.
Arithmetic dominating is not a scandal by itself: hand-computing a z-score or a gradient step teaches the mechanism before the library hides it. The scandal would be labeling that "advanced." We do not.
The part that keeps it honest today
The classification is not a one-time report. The mlContent tag lives in each exercise's metadata, the live Deep Orbit corpus (85 graded exercises) runs against real TFJS in CI, every solution must pass its own tests, and every starter must fail them. When we change an exercise, the gates rerun.
Found an exercise whose badge oversells it? Email support@tensorcraft.app and we will reclassify it in public.