There were two concerns with scoring in finals.
- There were some slow servers at the start of finals which caused extra timeouts for bots that needed a lot of CPU time and memory.
- Ranks were very volatile, right up to the end.
I've taken the raw game data, and rescored it to eliminate these two effects.
Errors
Here, you can see the high error rate at the start of the finals:
To correct this, I removed all games with errors from being scored. I initially tried removing the entire first set of games with the higher error rate, but Trueskill really needs some initial games with players of different skill levels.
Volatility
Secondly, I fixed the volatility to some degree. Here is the estimated skill for the top 20 players, over the finals:
The problem is that the ranking algorithm is tuned by default to work for humans, who change skills level constantly. To handle this, Trueskill is a little less certain about the past after every new game, allowing it to respond to a person who is suddenly better or worse. However for these bots, we need to disable this and let Trueskill build up a better picture of the bot's score. Here's the top 20 over time without added uncertainty:
Way smoother!
Before I give the revised rankings, you can access the full data and code to run this yourself at https://github.com/DanielVF/Halite-Ranking-Viz/blob/master/Unofficial%20Scoring.ipynb . Secondly, I'd like to thank @Janzert for automating and sharing the collection of game results, and the organizers for a fantastic contest.
The (Unofficial, Better) 120 Final Rankings
I'd recommend picking the higher score from your official or unoffical rankings, and using that number to brag to your friends with.
#1 mzotkiew
#2 shummie
#3 erdman
#4 timfoden
#5 cdurbin
#6 nmalaguti
#7 PeppiKokki
#8 DexGroves
#9 KalraA v92
#10 ewirkerman
#11 moonbirth
#12 acouette
#13 MoreGames
#14 Ziemin
#15 jstaker7
#16 veden
#17 tondonia
#18 fohristiwhirl
#19 tmseiler
#20 cdmurray80
#21 Maximophone
#22 bencalderhead
#23 En3rG
#24 djma
#25 davidgratton
#26 daniel-shields
#27 JWGS1
#28 yangle
#29 frabi
#30 david-wu
#31 bouwkast
#32 varak69
#33 GaudyZircon
#34 hmate9
#35 Gullesnuffs
#36 b7500af1
#37 TheDuck314
#38 danielborowski
#39 breeko
#40 BigBallerShotCaller
#41 DanielVF
#42 Daniel-Wang
#43 Oreshnik
#44 schmit
#45 MrTwiggy
#46 other-ai
#47 jediahkatz
#48 Sydriax
#49 hyPiRion
#50 kindanoob
#51 MHeasell
#52 bengo1023
#53 alexhad6
#54 0x0L
#55 brianvanleeuwen
#56 Kaczmarczyck
#57 dbf256
#58 kragbot
#59 kbcole
#60 arjunvis
#61 jheilema-nerdery
#62 henripal
#63 Vlad-Shcherbina
#64 bveber
#65 jiatinglu99
#66 markstev
#67 happypepper
#68 hilkoc
#69 jwcdbd
#70 sam-huang1223
#71 sanjeevtewani
#72 adrienball
#73 Andrew-peng
#74 Tautvis
#75 DaanPosthuma
#76 hetong007
#77 TheoKanning
#78 NGamma
#79 bmansfie
#80 nvengal
#81 jerzhang75
#82 platatat
#83 KLFrost
#84 funrollloops
#85 agarap
#86 cyberferret44
#87 mh6283-halite
#88 zluhcs
#89 AisleCC
#90 Patricksm
#91 mbrezu
#92 Risitop
#93 byronwall
#94 NeonYazzle
#95 shubhamjain0594
#96 bireland
#97 DollarAkshay
#98 neverfox
#99 pepijno
#100 dcdulin
#101 navidmx
#102 rossmacarthur
#103 potatoes-are-salty
#104 HelgiMagg
#105 uber5001
#106 charlesxxxx
#107 simonbevan
#108 gbenedisgrab
#109 adereth
#110 HalfVoxel
#111 vagarwala
#112 matthalbersma
#113 bkchiu0
#114 Rexxy4
#115 ccapo
#116 chadz
#117 ahitsdavid
#118 revoklaw
#119 Bobbadillio
#120 jstanderfer