Something that would be very useful for evaluating how well a our bots do would be to be able to challenge other bots. If you go to another user's profile page and challenge that user's bot a 2 player game would be started between your bot and the other player's bot. I have seen this in a few other AI competitions.
Right now if one submits a new bot one has to wait for a long time before it climbs to a rank which is interesting and you actually start to loose against your opponents. It would be very useful to be able to challenge other bots so that we can get feedback about how our bots do higher up the ladder without having to wait until the score goes up (or just to see just how much the current top bot wins against our own bots ).
Also, wouldn't it be better if a new bot that one submits starts at the same rank as the previous bot (but with a larger uncertainty) to speed up the convergence to the correct rank?