Halite Home

Challenge other bots


#1

Something that would be very useful for evaluating how well a our bots do would be to be able to challenge other bots. If you go to another user's profile page and challenge that user's bot a 2 player game would be started between your bot and the other player's bot. I have seen this in a few other AI competitions.
Right now if one submits a new bot one has to wait for a long time before it climbs to a rank which is interesting and you actually start to loose against your opponents. It would be very useful to be able to challenge other bots so that we can get feedback about how our bots do higher up the ladder without having to wait until the score goes up (or just to see just how much the current top bot wins against our own bots :slight_smile: ).

Also, wouldn't it be better if a new bot that one submits starts at the same rank as the previous bot (but with a larger uncertainty) to speed up the convergence to the correct rank?


#2

Something that would be very useful for evaluating how well a our bots do would be to be able to challenge other bots. If you go to another user's profile page and challenge that user's bot a 2 player game would be started between your bot and the other player's bot. I have seen this in a few other AI competitions.
Right now if one submits a new bot one has to wait for a long time before it climbs to a rank which is interesting and you actually start to loose against your opponents. It would be very useful to be able to challenge other bots so that we can get feedback about how our bots do higher up the ladder without having to wait until the score goes up (or just to see just how much the current top bot wins against our own bots :slight_smile: ).

We'll consider it.

Also, wouldn't it be better if a new bot that one submits starts at the same rank as the previous bot (but with a larger uncertainty) to speed up the convergence to the correct rank?

We'll try this out.

Higher uncertainity does correspond with a lower score (score = expected score - 3*uncertainty).


#3

"Challenge bot" option would be abused relentlessly and would not be to the benefit of the competition. I vote against.

I have also noticed the "long time to climb ranks" issue. My experience with TrueSkill suggests that level-matchmaking should happen faster. If I recall correctly, the Ants competition improved their matchmaking during the run of the competition, so I'd encourage the organizers to take a look at where they landed (if you haven't already).


#4

You would of course add some rate limiting to it and it should not contribute to the ranking.


#5

I strongly agree that it should not affect ranking.


#6

Organizers should look at what is happening with bot "veden v3" ... as of this moment, it has played 73 matches without a single loss (as near as I can see from the loadable history), yet it is only ranked 50th in the competition. It will surely be ranked (much) higher as you read this, but it's an example of the matchmaking algorithm taking too long to adjust. TrueSkill can do a much much better job than that.

If bots are ranked more correctly more quickly, ppl will feel much less need to "challenge" other bots. And more incentive to release new botcode more often. Now, it's sorta painful to push new code and wait for a bot to slog through all the randombots.


#7

Increasing the raw number of games played (more game server instances) would help to mitigate the slow matchmaking adjustments. Especially as the number of bots increases (+100 people over last few days) the number of game servers needs to scale more aggressively (though I do not know how it has been increasing).

Though challenging individuals could be fun, get to see my bot get decimated by djma more often. Direct comparison through challenging individuals could also increase people's desire to bring friends and colleagues to the competition.


#8

Exactly. I have a friend in this competition, but we cannot get to see our bots play against each other (without the other person manually sending the code via some other means) without waiting for a long time.


#9

In the Ants competition, somebody (I think it was a user, not officially affiliated with the organizers of that comp) set-up a separate test server, which was extremely useful for testing new code against other ppl looking to do the same without pushing code to replace your existing "official" bot. If that existed here, friends could also challenge one another there.


#10

yea, hehe, that was fun , unfortunately, neither time, nor resources to repeat that now.

also have a look here

(it should be possible, imho, to run halite with a couple of socket bots, connected to ppl elsewhere, and a server, that acts as a proxy, and does the matchmaking, db and such)


#11

I have started hosting some of my bots in the cloud on AWS Lambda.