Ranking Athletes

How The Rating Works

Every athlete starts at 1500. After each match, the winner's rating goes up and the loser's rating goes down. Beating a higher-rated opponent moves the score more than beating a lower-rated opponent.

The rating gap is the difference between the two athletes' scores. The selected model uses scale = 150, which turns a rating gap into an expected win-confidence range. For example, if your rating is about 26 points higher than your opponent's, the model expects your win probability to land in the 55-60% range.

What Baseline ELO Missed

Baseline ELO was already useful. It picked winners well, but the round 5 test set shows that it was under-confident: favorites won 70.34% of evaluated matches while baseline average confidence was only 68.20%. The final model picked favorites at 71.15% and averaged 71.82% confidence.

The experiments used 1,945,611 completed matches. Older matches built rating history, 2023 matches selected parameters, and 2024+ matches tested the final formula on later data. The model never used a future match to predict an earlier one.

Accuracy is how often the model's favorite won. Average confidence is how strongly it believed in those favorites. Calibration error checks whether that confidence was honest. Log loss is the main selection score because it rewards confident correct predictions and punishes confident wrong ones.

Differences between percentages are shown as percentage points. The final model changed average confidence by +3.62 percentage points and calibration error by -1.46 percentage points versus baseline ELO.

Which Changes Helped

Round 5 reran the article examples on the same train, validation, and test windows. That makes the comparisons apples to apples: each row changes one part of the model while keeping the rest of the experiment setup comparable.

Scale: How Much Confidence A Gap Creates

Scale decides how quickly rating gaps become strong predictions. A lower scale makes the same gap more confident. A higher scale makes the model more cautious. Round 5 selected scale = 150 for the final article formula.

Submissions: Stronger Match Evidence

A submission win moves ratings more than a decision win. The selected multiplier is 1.6. That does not mean a submission is literally that much better than a decision; it means the result teaches the model more.

The round 5 submission sweep compares each multiplier against the same final-model setup. Higher multipliers kept accuracy similar, but they pushed confidence higher and worsened calibration.

Trust Rating: How Much To Believe The Score

Each athlete also has a trust_rating. New athletes start with the fastest-moving rating. As match history grows, K falls and trust rises.

Experience trust uses a diminishing-return curve. With 10 prior matches, the athlete reaches the selected midpoint of the trust curve.

Experience And Inactivity Tuning

The selected experience curve uses a 10-match half-life. Inactivity adds temporary uncertainty after long gaps, then lets trust recover as the athlete records more matches.

Trust Also Changes Prediction Confidence

Trust is not just for rating updates. It also affects prediction confidence. The selected prediction adjustment is Max K, impact 0.25. If either rating is uncertain, the prediction is softened.

The selected prediction-trust setting had the best validation log loss in this sweep. Some alternatives were slightly better on one test metric, but they were not the validation-selected setting.

Final Calibration

A confident prediction should mean what it says. If the model says an athlete is a 75% favorite, similar favorites should win about three out of four times. The final model is close across the test buckets, with only mild optimism in the highest-confidence buckets.