What this system is actually solving
Educational games often optimize time-on-site instead of learning. This project treats mastery and transfer as the primary objectives.
The project is designed around an observable outcome and explicit constraints. A production decision is only considered successful when the model or algorithm improves a baseline without creating unacceptable cost, latency, reliability or interpretability problems.
How the system is decomposed
- 01Skill graph
A separately testable stage with defined inputs, outputs, observability and failure handling. - 02Challenge library
A separately testable stage with defined inputs, outputs, observability and failure handling. - 03Difficulty controller
A separately testable stage with defined inputs, outputs, observability and failure handling. - 04Scoring service
A separately testable stage with defined inputs, outputs, observability and failure handling. - 05Mastery estimator
A separately testable stage with defined inputs, outputs, observability and failure handling. - 06Achievement system
A separately testable stage with defined inputs, outputs, observability and failure handling. - 07Replay analytics
A separately testable stage with defined inputs, outputs, observability and failure handling.
Why each algorithm exists
Dynamic difficulty adjustment
Role. Dynamic difficulty adjustment is included because it addresses a specific measurable part of the system rather than being added as decoration.
Trade-off. The component must be compared with a simpler baseline and removed if it adds complexity without measurable value.
Evaluate with. Task-specific quality metric, latency, reliability and failure-case analysis.
Mastery scoring
Role. Mastery scoring is included because it addresses a specific measurable part of the system rather than being added as decoration.
Trade-off. The component must be compared with a simpler baseline and removed if it adds complexity without measurable value.
Evaluate with. Task-specific quality metric, latency, reliability and failure-case analysis.
Spaced challenge scheduling
Role. Spaced challenge scheduling is included because it addresses a specific measurable part of the system rather than being added as decoration.
Trade-off. The component must be compared with a simpler baseline and removed if it adds complexity without measurable value.
Evaluate with. Task-specific quality metric, latency, reliability and failure-case analysis.
Bandit content selection
Role. Uses feedback to allocate more traffic to promising choices while reserving some exploration for alternatives.
Trade-off. Biased or sparse rewards can push the policy toward a locally attractive but globally poor choice.
Evaluate with. Reward lift, regret, exploration coverage and stability over time.
Signals entering the system
Every useful model depends on the quality, timing and provenance of its inputs. CortexLab treats feature definitions and leakage checks as part of model engineering, not preprocessing trivia.
- Accuracy — captured with validation, lineage and monitoring so training and production meaning stay aligned.
- Latency — captured with validation, lineage and monitoring so training and production meaning stay aligned.
- Hints — captured with validation, lineage and monitoring so training and production meaning stay aligned.
- Retries — captured with validation, lineage and monitoring so training and production meaning stay aligned.
- Abandonment — captured with validation, lineage and monitoring so training and production meaning stay aligned.
- Topic mastery — captured with validation, lineage and monitoring so training and production meaning stay aligned.
What has to be measured before calling it successful
- Learning gain
- Retention
- Transfer to interview questions
- Challenge completion
Evaluation is split between offline quality, operational performance and failure analysis. A high headline metric does not override poor calibration, unstable segments, leakage or unusable latency.
Where this project can fail
Addiction is not a learning metric
This risk is tracked through tests, monitoring, explicit thresholds or human review depending on where it appears in the architecture.
Failure needs useful feedback
This risk is tracked through tests, monitoring, explicit thresholds or human review depending on where it appears in the architecture.
Difficulty should stay in the productive struggle zone
This risk is tracked through tests, monitoring, explicit thresholds or human review depending on where it appears in the architecture.
How the project grows without becoming untestable
- 01Multiplayer technical challenges
- 02Code sandbox
- 03Teacher dashboards
- 04Generative level builder from approved templates
Questions an engineer should be able to answer
- Why is this architecture preferable to a simpler baseline?
- Which metric can look good while the product still fails?
- Where can data leakage enter this pipeline?
- What changes when traffic, data volume or latency requirements increase 10×?
- Which part should be rolled back first if production quality drops?