CCERL is a new public benchmark for Chinese Checkers engines
The first public benchmark for Chinese Checkers engines, built like the rating lists for chess and Go: fixed rules, audited starting positions, paired color swaps, public baselines, and every game log.
- Engines
- 19
- Pairings
- 171
- Games
- 29,412
- Starting positions
- 86
Leaderboard
The official v2 ladder is anchored at random = 0 and uses draw scoring of 0.5. Every displayed row now comes from the same combined all-pairs graph: 171 pairings, 29,412 games, 86 audited starts, and 3,096 games per engine. The latest promoted training checkpoint, JumpStar_82, is included.
| Rank | Engine | Type | Elo | CI95 | W-D-L | Score |
|---|---|---|---|---|---|---|
| 1 | JumpStar_82 | JumpStar | 1,064 | 1,028 - 1,101 | 2442-267-387 | 0.832 |
| 2 | JumpStar_63 | JumpStar | 1,058 | 1,021 - 1,094 | 2390-342-364 | 0.827 |
| 3 | JumpStar_64 | JumpStar | 1,033 | 997 - 1,069 | 2314-384-398 | 0.809 |
| 4 | JumpStar_61 | JumpStar | 1,010 | 974 - 1,045 | 2261-384-451 | 0.792 |
| 5 | JumpStar_62 | JumpStar | 984 | 948 - 1,020 | 2201-384-511 | 0.773 |
| 6 | JumpStar_60 | JumpStar | 983 | 948 - 1,019 | 2207-369-520 | 0.772 |
| 7 | JumpStar_57 | JumpStar | 973 | 937 - 1,009 | 2186-362-548 | 0.765 |
| 8 | JumpStar_46 | JumpStar | 756 | 722 - 791 | 1685-284-1127 | 0.590 |
| 9 | JumpStar_29 | JumpStar | 742 | 708 - 776 | 1669-243-1184 | 0.578 |
| 10 | tt-pvs | Baseline | 570 | 536 - 603 | 1263-165-1668 | 0.435 |
| 11 | JumpStar_5 | JumpStar | 518 | 485 - 551 | 1144-129-1823 | 0.390 |
| 12 | JumpStar_9 | JumpStar | 477 | 444 - 509 | 992-215-1889 | 0.355 |
| 13 | converter | Baseline | 438 | 405 - 470 | 959-77-2060 | 0.322 |
| 14 | greedy | Baseline | 402 | 370 - 435 | 886-41-2169 | 0.293 |
| 15 | harryz-rule | Open source | 393 | 361 - 426 | 5-1756-1335 | 0.285 |
| 16 | marblefish-ab | Open source | 341 | 308 - 373 | 568-370-2158 | 0.243 |
| 17 | traffic-greedy | Baseline | 290 | 257 - 323 | 586-98-2412 | 0.205 |
| 18 | zedrichu-minimax | Open source | 256 | 223 - 289 | 340-442-2314 | 0.181 |
| 19 | random | Baseline | 0 | 0 - 0 | 23-270-2803 | 0.051 |
Pairwise matrix
Each non-diagonal cell is from the row engine's perspective: wins-draws-losses, followed by score calculated as (wins + 0.5 * draws) / games. Every completed non-diagonal cell summarizes 172 games from the audited v2 suite, including color swaps.
| Row vs column | J82 | J63 | J64 | J61 | J62 | J60 | J57 | J46 | J29 | TT | J5 | J9 | Conv | Greedy | Harry | Marble | Traffic | Zed | Rand |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| JumpStar_82 | - | 55-19-980.375 | 85-16-710.541 | 83-16-730.529 | 88-36-480.616 | 90-42-400.645 | 120-15-370.741 | 157-7-80.933 | 157-6-90.930 | 162-8-20.965 | 171-1-00.997 | 168-4-00.988 | 170-1-10.991 | 172-0-01.000 | 90-82-00.762 | 166-6-00.983 | 171-1-00.997 | 167-5-00.985 | 170-2-00.994 |
| JumpStar_63 | 98-19-550.625 | - | 76-17-790.491 | 81-42-490.593 | 77-19-760.503 | 110-23-390.706 | 63-68-410.564 | 147-9-160.881 | 161-6-50.953 | 164-6-20.971 | 171-1-00.997 | 170-2-00.994 | 171-0-10.994 | 171-1-00.997 | 67-105-00.695 | 166-5-10.980 | 172-0-01.000 | 169-3-00.991 | 156-16-00.953 |
| JumpStar_64 | 71-16-850.459 | 79-17-760.509 | - | 64-39-690.485 | 89-41-420.637 | 66-40-660.500 | 91-48-330.669 | 158-5-90.933 | 165-3-40.968 | 162-3-70.951 | 171-0-10.994 | 166-5-10.980 | 167-1-40.974 | 170-2-00.994 | 26-146-00.576 | 166-6-00.983 | 168-3-10.985 | 172-0-01.000 | 163-9-00.974 |
| JumpStar_61 | 73-16-830.471 | 49-42-810.407 | 69-39-640.515 | - | 99-12-610.610 | 58-46-680.471 | 88-19-650.567 | 152-6-140.901 | 158-4-100.930 | 164-5-30.968 | 170-2-00.994 | 169-3-00.991 | 170-1-10.991 | 171-1-00.997 | 35-137-00.602 | 140-31-10.904 | 171-1-00.997 | 171-1-00.997 | 154-18-00.948 |
| JumpStar_62 | 48-36-880.384 | 76-19-770.497 | 42-41-890.363 | 61-12-990.390 | - | 92-17-630.584 | 88-16-680.558 | 124-36-120.826 | 151-10-110.907 | 153-17-20.939 | 170-1-10.991 | 170-2-00.994 | 172-0-01.000 | 172-0-01.000 | 64-108-00.686 | 142-29-10.910 | 172-0-01.000 | 144-28-00.919 | 160-12-00.965 |
| JumpStar_60 | 40-42-900.355 | 39-23-1100.294 | 66-40-660.500 | 68-46-580.529 | 63-17-920.416 | - | 75-22-750.500 | 149-7-160.887 | 156-8-80.930 | 163-5-40.962 | 170-1-10.991 | 172-0-01.000 | 172-0-01.000 | 172-0-01.000 | 33-139-00.596 | 169-3-00.991 | 172-0-01.000 | 170-2-00.994 | 158-14-00.959 |
| JumpStar_57 | 37-15-1200.259 | 41-68-630.436 | 33-48-910.331 | 65-19-880.433 | 68-16-880.442 | 75-22-750.500 | - | 146-15-110.892 | 127-35-100.840 | 166-6-00.983 | 170-1-10.991 | 170-2-00.994 | 172-0-01.000 | 172-0-01.000 | 63-109-00.683 | 168-4-00.988 | 172-0-01.000 | 170-1-10.991 | 171-1-00.997 |
| JumpStar_46 | 8-7-1570.067 | 16-9-1470.119 | 9-5-1580.067 | 14-6-1520.099 | 12-36-1240.174 | 16-7-1490.113 | 11-15-1460.108 | - | 94-39-390.660 | 150-9-130.898 | 141-1-300.823 | 168-0-40.977 | 165-1-60.962 | 170-0-20.988 | 44-128-00.628 | 167-5-00.985 | 172-0-01.000 | 166-6-00.983 | 162-10-00.971 |
| JumpStar_29 | 9-6-1570.070 | 5-6-1610.047 | 4-3-1650.032 | 10-4-1580.070 | 11-10-1510.093 | 8-8-1560.070 | 10-35-1270.160 | 39-39-940.340 | - | 154-10-80.924 | 170-0-20.988 | 168-4-00.988 | 168-0-40.977 | 171-0-10.994 | 75-97-00.718 | 169-3-00.991 | 172-0-01.000 | 168-4-00.988 | 158-14-00.959 |
| tt-pvs | 2-8-1620.035 | 2-6-1640.029 | 7-3-1620.049 | 3-5-1640.032 | 2-17-1530.061 | 4-5-1630.038 | 0-6-1660.017 | 13-9-1500.102 | 8-10-1540.076 | - | 114-2-560.669 | 110-5-570.654 | 142-1-290.828 | 144-4-240.849 | 144-28-00.919 | 100-27-450.660 | 157-6-90.930 | 139-23-100.875 | 172-0-01.000 |
| JumpStar_5 | 0-1-1710.003 | 0-1-1710.003 | 1-0-1710.006 | 0-2-1700.006 | 1-1-1700.009 | 1-1-1700.009 | 1-1-1700.009 | 30-1-1410.177 | 2-0-1700.012 | 56-2-1140.331 | - | 86-9-770.526 | 126-2-440.738 | 149-2-210.872 | 105-67-00.805 | 129-6-370.767 | 155-1-160.904 | 130-32-100.849 | 172-0-01.000 |
| JumpStar_9 | 0-4-1680.012 | 0-2-1700.006 | 1-5-1660.020 | 0-3-1690.009 | 0-2-1700.006 | 0-0-1720.000 | 0-2-1700.006 | 4-0-1680.023 | 0-4-1680.012 | 57-5-1100.346 | 77-9-860.474 | - | 129-3-400.759 | 131-3-380.770 | 39-133-00.613 | 83-14-750.523 | 155-2-150.907 | 147-21-40.916 | 169-3-00.991 |
| converter | 1-1-1700.009 | 1-0-1710.006 | 4-1-1670.026 | 1-1-1700.009 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 6-1-1650.038 | 4-0-1680.023 | 29-1-1420.172 | 44-2-1260.262 | 40-3-1290.241 | - | 106-6-600.634 | 118-54-00.843 | 135-2-350.791 | 133-0-390.773 | 165-5-20.974 | 172-0-01.000 |
| greedy | 0-0-1720.000 | 0-1-1710.003 | 0-2-1700.006 | 0-1-1710.003 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 2-0-1700.012 | 1-0-1710.006 | 24-4-1440.151 | 21-2-1490.128 | 38-3-1310.230 | 60-6-1060.366 | - | 172-0-01.000 | 139-1-320.811 | 118-4-500.698 | 139-17-160.858 | 172-0-01.000 |
| harryz-rule | 0-82-900.238 | 0-105-670.305 | 0-146-260.424 | 0-137-350.398 | 0-108-640.314 | 0-139-330.404 | 0-109-630.317 | 0-128-440.372 | 0-97-750.282 | 0-28-1440.081 | 0-67-1050.195 | 0-133-390.387 | 0-54-1180.157 | 0-0-1720.000 | - | 0-109-630.317 | 0-23-1490.067 | 0-124-480.360 | 5-167-00.515 |
| marblefish-ab | 0-6-1660.017 | 1-5-1660.020 | 0-6-1660.017 | 1-31-1400.096 | 1-29-1420.090 | 0-3-1690.009 | 0-4-1680.012 | 0-5-1670.015 | 0-3-1690.009 | 45-27-1000.340 | 37-6-1290.233 | 75-14-830.477 | 35-2-1350.209 | 32-1-1390.189 | 63-109-00.683 | - | 84-4-840.500 | 46-114-120.599 | 148-1-230.863 |
| traffic-greedy | 0-1-1710.003 | 0-0-1720.000 | 1-3-1680.015 | 0-1-1710.003 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 0-0-1720.000 | 9-6-1570.070 | 16-1-1550.096 | 15-2-1550.093 | 39-0-1330.227 | 50-4-1180.302 | 149-23-00.933 | 84-4-840.500 | - | 51-53-680.451 | 172-0-01.000 |
| zedrichu-minimax | 0-5-1670.015 | 0-3-1690.009 | 0-0-1720.000 | 0-1-1710.003 | 0-28-1440.081 | 0-2-1700.006 | 1-1-1700.009 | 0-6-1660.017 | 0-4-1680.012 | 10-23-1390.125 | 10-32-1300.151 | 4-21-1470.084 | 2-5-1650.026 | 16-17-1390.142 | 48-124-00.640 | 12-114-460.401 | 68-53-510.549 | - | 169-3-00.991 |
| random | 0-2-1700.006 | 0-16-1560.047 | 0-9-1630.026 | 0-18-1540.052 | 0-12-1600.035 | 0-14-1580.041 | 0-1-1710.003 | 0-10-1620.029 | 0-14-1580.041 | 0-0-1720.000 | 0-0-1720.000 | 0-3-1690.009 | 0-0-1720.000 | 0-0-1720.000 | 0-167-50.485 | 23-1-1480.137 | 0-0-1720.000 | 0-3-1690.009 | - |
Artifacts
The reproducibility materials currently live in the GitHub repository rather than a single download bundle. Start with the v2 ruleset, CCP protocol, audited v2 start schedule, baseline manifests, model notes, and Elo summary. A full JSONL game-log archive is still being packaged for public release.
docs/benchmark/CCERL-2P10-v2.md· v2 rulebookdocs/benchmark/CCP_PROTOCOL.md· engine protocolbenchmarks/ccerl-v1/positions/official_elo_v2.jsonl· audited startsbenchmarks/ccerl-v1/baseline_pack/PACK_MANIFEST.json· baseline packbenchmarks/ccerl-v1/baseline_pack/MODEL_NOTES.md· model checksumsexperiments/public_benchmark/ladder_v2_core_latest_s768/elo.json· Elo summary
Challenge
External engines should implement CCP over stdin/stdout. Official ratings are produced by the benchmark runner; self-reported logs are useful for debugging but do not enter the public list until verified or rerun.
- Implement the CCP protocol.
- Run the local smoke gauntlet.
- Submit a Docker image or executable manifest.
- Use direct model matches for top-band claims.