Cortex AI logo placeholderCortexLabAI · learning · engineering
Model Atlas · Bayesian bandits

Thompson Sampling

Samples action quality from posterior beliefs to balance exploration and exploitation.

core mathematical viewθₐ~Posteriorₐ ; choose argmax θₐ
Mental model

Understand it before memorizing it.

Act according to plausible worlds sampled from current evidence.

Best fit

Where this model earns its place

A/B allocation

Start with a simpler baseline then compare this model using the same evaluation split and operational constraints.

CTR experiments

Start with a simpler baseline then compare this model using the same evaluation split and operational constraints.

Low-dimensional bandits

Start with a simpler baseline then compare this model using the same evaluation split and operational constraints.

Strengths and limits

Trade-offs matter more than popularity.

Strengths

✓ Natural uncertainty

✓ Simple implementation

Limitations

△ Model assumptions matter

Evaluation

Metrics to watch

RegretInterpret with the product objective and error cost.
Posterior calibrationInterpret with the product objective and error cost.
RewardInterpret with the product objective and error cost.
Production checklist

Before it reaches real users

  1. 01

    Use priors consciously

    Document the assumption and instrument the condition so regressions can be detected.

  2. 02

    Segment carefully

    Document the assumption and instrument the condition so regressions can be detected.

  3. 03

    Cap risky exploration

    Document the assumption and instrument the condition so regressions can be detected.