Public Chinese Checkers AI benchmark
CCERL is a new public benchmark for Chinese Checkers engines
CCERL is the first public benchmark for evaluating Chinese Checkers AI engine strength. Based on similar concepts from Chess and Go, CCERL provides fixed rules, audited starting positions, paired side swaps, downloadable game logs, and public baselines. The current CCERL-2P10-v2 ladder ranks every displayed engine across 86 starts, paired color swaps, 171 pairings, 29,412 games, and 3,096 games per engine.
Leaderboard
The official v2 ladder is anchored at random = 0 and uses draw scoring of 0.5. Every displayed row now comes from the same combined all-pairs graph: 171 pairings, 29,412 games, 86 audited starts, and 3,096 games per engine. The latest promoted training checkpoint, JumpStar_82, is included.
| Rank | Engine | Type | Elo | CI95 | Games | W-D-L | Score |
|---|---|---|---|---|---|---|---|
| 1 | JumpStar_82 | JumpStar | 1,064 | 1,028 - 1,101 | 3,096 | 2442-267-387 | 0.832 |
| 2 | JumpStar_63 | JumpStar | 1,058 | 1,021 - 1,094 | 3,096 | 2390-342-364 | 0.827 |
| 3 | JumpStar_64 | JumpStar | 1,033 | 997 - 1,069 | 3,096 | 2314-384-398 | 0.809 |
| 4 | JumpStar_61 | JumpStar | 1,010 | 974 - 1,045 | 3,096 | 2261-384-451 | 0.792 |
| 5 | JumpStar_62 | JumpStar | 984 | 948 - 1,020 | 3,096 | 2201-384-511 | 0.773 |
| 6 | JumpStar_60 | JumpStar | 983 | 948 - 1,019 | 3,096 | 2207-369-520 | 0.772 |
| 7 | JumpStar_57 | JumpStar | 973 | 937 - 1,009 | 3,096 | 2186-362-548 | 0.765 |
| 8 | JumpStar_46 | JumpStar | 756 | 722 - 791 | 3,096 | 1685-284-1127 | 0.590 |
| 9 | JumpStar_29 | JumpStar | 742 | 708 - 776 | 3,096 | 1669-243-1184 | 0.578 |
| 10 | tt-pvs | Baseline | 570 | 536 - 603 | 3,096 | 1263-165-1668 | 0.435 |
| 11 | JumpStar_5 | JumpStar | 518 | 485 - 551 | 3,096 | 1144-129-1823 | 0.390 |
| 12 | JumpStar_9 | JumpStar | 477 | 444 - 509 | 3,096 | 992-215-1889 | 0.355 |
| 13 | converter | Baseline | 438 | 405 - 470 | 3,096 | 959-77-2060 | 0.322 |
| 14 | greedy | Baseline | 402 | 370 - 435 | 3,096 | 886-41-2169 | 0.293 |
| 15 | harryz-rule | Open source | 393 | 361 - 426 | 3,096 | 5-1756-1335 | 0.285 |
| 16 | marblefish-ab | Open source | 341 | 308 - 373 | 3,096 | 568-370-2158 | 0.243 |
| 17 | traffic-greedy | Baseline | 290 | 257 - 323 | 3,096 | 586-98-2412 | 0.205 |
| 18 | zedrichu-minimax | Open source | 256 | 223 - 289 | 3,096 | 340-442-2314 | 0.181 |
| 19 | random | Baseline | 0 | 0 - 0 | 3,096 | 23-270-2803 | 0.051 |
Pairwise Matrix
Each non-diagonal cell is from the row engine's perspective: wins-draws-losses, followed by score calculated as (wins + 0.5 * draws) / games. Every completed non-diagonal cell summarizes 172 games from the audited v2 suite, including color swaps.
| Row vs column | J82 | J63 | J64 | J61 | J62 | J60 | J57 | J46 | J29 | TT | J5 | J9 | Conv | Greedy | Harry | Marble | Traffic | Zed | Rand |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
JumpStar_82 | - | 55-19-980.375 | 85-16-710.541 | 83-16-730.529 | 88-36-480.616 | 90-42-400.645 | 120-15-370.741 | 157-7-80.933 | 157-6-90.930 | 162-8-20.965 | 171-1-00.997 | 168-4-00.988 | 170-1-10.991 | 172-0-01.000 | 90-82-00.762 | 166-6-00.983 | 171-1-00.997 | 167-5-00.985 | 170-2-00.994 |
JumpStar_63 | 98-19-550.625 | - | 76-17-790.491 | 81-42-490.593 | 77-19-760.503 | 110-23-390.706 | 63-68-410.564 | 147-9-160.881 | 161-6-50.953 | 164-6-20.971 | 171-1-00.997 | 170-2-00.994 | 171-0-10.994 | 171-1-00.997 | 67-105-00.695 | 166-5-10.980 | 172-0-01.000 | 169-3-00.991 | 156-16-00.953 |
JumpStar_64 | 71-16-850.459 | 79-17-760.509 | - | 64-39-690.485 | 89-41-420.637 | 66-40-660.500 | 91-48-330.669 | 158-5-90.933 | 165-3-40.968 | 162-3-70.951 | 171-0-10.994 | 166-5-10.980 | 167-1-40.974 | 170-2-00.994 | 26-146-00.576 | 166-6-00.983 | 168-3-10.985 | 172-0-01.000 | 163-9-00.974 |
JumpStar_61 | 73-16-830.471 | 49-42-810.407 | 69-39-640.515 | - | 99-12-610.610 | 58-46-680.471 | 88-19-650.567 | 152-6-140.901 | 158-4-100.930 | 164-5-30.968 | 170-2-00.994 | 169-3-00.991 | 170-1-10.991 | 171-1-00.997 | 35-137-00.602 | 140-31-10.904 | 171-1-00.997 | 171-1-00.997 | 154-18-00.948 |
JumpStar_62 | 48-36-880.384 | 76-19-770.497 | 42-41-890.363 | 61-12-990.390 | - | 92-17-630.584 | 88-16-680.558 | 124-36-120.826 | 151-10-110.907 | 153-17-20.939 | 170-1-10.991 | 170-2-00.994 | 172-0-01.000 | 172-0-01.000 | 64-108-00.686 | 142-29-10.910 | 172-0-01.000 | 144-28-00.919 | 160-12-00.965 |
JumpStar_60 | 40-42-900.355 | 39-23-1100.294 | 66-40-660.500 | 68-46-580.529 | 63-17-920.416 | - | 75-22-750.500 | 149-7-160.887 | 156-8-80.930 | 163-5-40.962 | 170-1-10.991 | 172-0-01.000 | 172-0-01.000 | 172-0-01.000 | 33-139-00.596 | 169-3-00.991 | 172-0-01.000 | 170-2-00.994 | 158-14-00.959 |
JumpStar_57 | 37-15-1200.259 | 41-68-630.436 | 33-48-910.331 | 65-19-880.433 | 68-16-880.442 | 75-22-750.500 | - | 146-15-110.892 | 127-35-100.840 | 166-6-00.983 | 170-1-10.991 | 170-2-00.994 | 172-0-01.000 | 172-0-01.000 | 63-109-00.683 | 168-4-00.988 | 172-0-01.000 | 170-1-10.991 | 171-1-00.997 |
JumpStar_46 | 8-7-1570.067 | 16-9-1470.119 | 9-5-1580.067 | 14-6-1520.099 | 12-36-1240.174 | 16-7-1490.113 | 11-15-1460.108 | - | 94-39-390.660 | 150-9-130.898 | 141-1-300.823 | 168-0-40.977 | 165-1-60.962 | 170-0-20.988 | 44-128-00.628 | 167-5-00.985 | 172-0-01.000 | 166-6-00.983 | 162-10-00.971 |
JumpStar_29 | 9-6-1570.070 | 5-6-1610.047 | 4-3-1650.032 | 10-4-1580.070 | 11-10-1510.093 | 8-8-1560.070 | 10-35-1270.160 | 39-39-940.340 | - | 154-10-80.924 | 170-0-20.988 | 168-4-00.988 | 168-0-40.977 | 171-0-10.994 | 75-97-00.718 | 169-3-00.991 | 172-0-01.000 | 168-4-00.988 | 158-14-00.959 |
tt-pvs | 2-8-1620.035 | 2-6-1640.029 | 7-3-1620.049 | 3-5-1640.032 | 2-17-1530.061 | 4-5-1630.038 | 0-6-1660.017 | 13-9-1500.102 | 8-10-1540.076 | - | 114-2-560.669 | 110-5-570.654 | 142-1-290.828 | 144-4-240.849 | 144-28-00.919 | 100-27-450.660 | 157-6-90.930 | 139-23-100.875 | 172-0-01.000 |
JumpStar_5 | 0-1-1710.003 | 0-1-1710.003 | 1-0-1710.006 | 0-2-1700.006 | 1-1-1700.009 | 1-1-1700.009 | 1-1-1700.009 | 30-1-1410.177 | 2-0-1700.012 | 56-2-1140.331 | - | 86-9-770.526 | 126-2-440.738 | 149-2-210.872 | 105-67-00.805 | 129-6-370.767 | 155-1-160.904 | 130-32-100.849 | 172-0-01.000 |
JumpStar_9 | 0-4-1680.012 | 0-2-1700.006 | 1-5-1660.020 | 0-3-1690.009 | 0-2-1700.006 | 0-0-1720.000 | 0-2-1700.006 | 4-0-1680.023 | 0-4-1680.012 | 57-5-1100.346 | 77-9-860.474 | - | 129-3-400.759 | 131-3-380.770 | 39-133-00.613 | 83-14-750.523 | 155-2-150.907 | 147-21-40.916 | 169-3-00.991 |
converter | 1-1-1700.009 | 1-0-1710.006 | 4-1-1670.026 | 1-1-1700.009 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 6-1-1650.038 | 4-0-1680.023 | 29-1-1420.172 | 44-2-1260.262 | 40-3-1290.241 | - | 106-6-600.634 | 118-54-00.843 | 135-2-350.791 | 133-0-390.773 | 165-5-20.974 | 172-0-01.000 |
greedy | 0-0-1720.000 | 0-1-1710.003 | 0-2-1700.006 | 0-1-1710.003 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 2-0-1700.012 | 1-0-1710.006 | 24-4-1440.151 | 21-2-1490.128 | 38-3-1310.230 | 60-6-1060.366 | - | 172-0-01.000 | 139-1-320.811 | 118-4-500.698 | 139-17-160.858 | 172-0-01.000 |
harryz-rule | 0-82-900.238 | 0-105-670.305 | 0-146-260.424 | 0-137-350.398 | 0-108-640.314 | 0-139-330.404 | 0-109-630.317 | 0-128-440.372 | 0-97-750.282 | 0-28-1440.081 | 0-67-1050.195 | 0-133-390.387 | 0-54-1180.157 | 0-0-1720.000 | - | 0-109-630.317 | 0-23-1490.067 | 0-124-480.360 | 5-167-00.515 |
marblefish-ab | 0-6-1660.017 | 1-5-1660.020 | 0-6-1660.017 | 1-31-1400.096 | 1-29-1420.090 | 0-3-1690.009 | 0-4-1680.012 | 0-5-1670.015 | 0-3-1690.009 | 45-27-1000.340 | 37-6-1290.233 | 75-14-830.477 | 35-2-1350.209 | 32-1-1390.189 | 63-109-00.683 | - | 84-4-840.500 | 46-114-120.599 | 148-1-230.863 |
traffic-greedy | 0-1-1710.003 | 0-0-1720.000 | 1-3-1680.015 | 0-1-1710.003 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 9-6-1570.070 | 16-1-1550.096 | 15-2-1550.093 | 39-0-1330.227 | 50-4-1180.302 | 149-23-00.933 | 84-4-840.500 | - | 51-53-680.451 | 172-0-01.000 |
zedrichu-minimax | 0-5-1670.015 | 0-3-1690.009 | 0-0-1720.000 | 0-1-1710.003 | 0-28-1440.081 | 0-2-1700.006 | 1-1-1700.009 | 0-6-1660.017 | 0-4-1680.012 | 10-23-1390.125 | 10-32-1300.151 | 4-21-1470.084 | 2-5-1650.026 | 16-17-1390.142 | 48-124-00.640 | 12-114-460.401 | 68-53-510.549 | - | 169-3-00.991 |
random | 0-2-1700.006 | 0-16-1560.047 | 0-9-1630.026 | 0-18-1540.052 | 0-12-1600.035 | 0-14-1580.041 | 0-1-1710.003 | 0-10-1620.029 | 0-14-1580.041 | 0-0-1720.000 | 0-0-1720.000 | 0-3-1690.009 | 0-0-1720.000 | 0-0-1720.000 | 0-167-50.485 | 23-1-1480.137 | 0-0-1720.000 | 0-3-1690.009 | - |
Artifacts
The reproducibility materials currently live in the GitHub repository rather than a single download bundle. Start with the v2 ruleset, CCP protocol, audited v2 start schedule, baseline manifests, model notes, and Elo summary. A full JSONL game-log archive is still being packaged for public release.
docs/benchmark/CCERL-2P10-v2.md· v2 rulebookdocs/benchmark/CCP_PROTOCOL.md· engine protocolbenchmarks/ccerl-v1/positions/official_elo_v2.jsonl· audited startsbenchmarks/ccerl-v1/baseline_pack/PACK_MANIFEST.json· baseline packbenchmarks/ccerl-v1/baseline_pack/MODEL_NOTES.md· model checksumsexperiments/public_benchmark/ladder_v2_core_latest_s768/elo.json· Elo summary
Challenge
External engines should implement CCP over stdin/stdout. Official ratings are produced by the benchmark runner; self-reported logs are useful for debugging but do not enter the public list until verified or rerun.
- Implement the CCP protocol.
- Run the local smoke gauntlet.
- Submit a Docker image or executable manifest.
- Use direct model matches for top-band claims.