Halite Home

Server hiccups in final stage

#1

Hi, we have some hiccups in the final stage. My final bot , which was not-too-bad (#36 after 185 games pre the final stage) has errored out twice due to timing issues.
This is odd as i dont do any precompute and this wasnt really a problem pre-final-stage. Are the servers running slower? do people see timeout errors where they had none before? All inputs gratefully received (not that i can do much, but might help @truell20 and @Sydriax make the final stage seamless. )

For example, my last error is :
--- Init ---
bb4
--- Bot used 16659 milliseconds ---

--- Frame #1 ---

ERRORED!
No response received.

#2

I've also had the same problem. I only timed out once out of 32 games so far but it still never happened before today, so it seems like it could be a server thing.

#3

I've timed out 5 times today, often with several of the other players also timing out
https://2016.halite.io/game.php?replay=ar1486971424-2206873538.hlt

I never really had any timeout issues before

#4

If there are 1600 bots that have now played around 30 games each in 10 hours, that is about 80 games a minute. I think these servers are working pretty hard.

#5

I'm having the same problem. I timed out once in nearly 500 games before the final stage. And now I've timed out 4 times in 30 games. It would be too bad if the rankings were skewed by unforeseen server challenges. I definitely would have spent more time optimizing runtime if I knew this have been a problem.

#6

I timed out in 4 of my first 30 games. I too hadn't been seeing any issues with init timeouts since the init timeout was bumped up to 30 seconds.

Hopefully it will get better as some of the players that consistently timeout and presumably use a lot of resources stop seeding.

#7

**Edit: it looks like even @nmalaguti is suffering timeouts. That's erroneously affecting the lb at the top 5 level. ** Agreed, it would be too bad if rankings were skewed. Let's see how things go. Given that @truell20 and @Sydriax are extremely responsive round such issues, I'm guessing they've already asked two sigma to shake out a few more pennies for servers. (?)

#8

We're really looking into it. It's not clear why this would happen.

My best guess is that something (either us or someone else) is making our AWS racks much hotter than we're used to, and that's responsible, but we haven't yet identified what could have happened on our side to cause this.

Our first priority is that the final rankings are fair. Please be patient with us while we work to fix this issue.

#9

No worries @Sydriax and @truell20 youve done an exemplary job so far.

I'm not sure what the solution is though- since TrueSkill is both path-dependent and sticky, playing 10-20 more games will not solve the problem , you'll probably need around 400 ish games per user . And of course you can't simply scrub the timeout games for every user and update scores, as the matches were all path-dependent .
Maybe doing both ie scrub the timeout games and let everyone run for lots more games is the easiest. You guys have sorted every bug so far, so i have no doubt you'll figure out this one too...

#10

I am also concerned about this. My bot is 10-15 places lower than expected, based on the previous rankings. I hope that a fair solution will be found.

#11

Update: It seems that the newest servers we opened (in an effort to get more games played more quickly) were slower than our old servers. We've since closed them down, and are evaluating how to deal with TrueSkill's stickiness.

#13

Trueskill is made to handle changing skill levels. That's actually one reason I'm not sure it is the best for a game like Halite where submissions are immutable. But in this case, it will catch up better with reality than most alternatives.

#14

I guess one benefit of TrueSkill here is that eventually you'll hopefully end up in a good spot. Some of the whole history rankings would unfairly penalize players for losses in some of the early games to lower ranked players due to timeouts. I guess you could go through and try to identify games that were unfairly timed out, but that's not a simple exercise either since there may be games which were fairly timed out.

#15

I think you are right, that with lots of games TrueSkill should eventually get us to a good spot. It took my bot awhile to start moving up the rankings with some of the timeouts in the early games.

I see right now I'm a little more than 5 points below you, and I think our bots generally flip-flop each other in the rankings (unless you made a big improvement in the last couple of hours on Sunday).

It might take quite a few games to close a 5 point gap - maybe they'll let us all play 500 games :slight_smile:

#16

Yeah. I got lucky and was able to wrestle my way out of the timeout issues fairly quickly. I do feel that there are a few others that aren't quite where they should be yet. It's also hard to tell where everyone should be with all the new bots being submitted on Sunday and server slowdowns because everyone was resubmitting.

I trust that Truell and Sydriax will figure out the right solution. They've done phenomenal so far and they stated they don't want server issues to impede proper final rankings. If they really go through and slowly stop seeding lower ranked players, then all the server power could eventually be used to seed diamond level bots only and then you'd really have enough games to make a determination such that the early timeout issues should by then be a non-issue.

#17

I think digging ones way out of timeout issues could take a very long time ,I ran a couple of trueskill simulatons with 1600 players, amidst them 2 identical players (call them @cdurbin and @shummie, one of whom is hit 4 times in the first 20 games with an enforced loss. seems to take around 100 extra games to equalise. (so that isnt 100 games absolute, it is indeed around 500)

Organisers, a plea: look inside your humanity and use something different to trueskill the next time round :smiley:

#18

That's a very interesting analysis. I'm curious to see how that may play out in my current ratings. I've gotten back to where I think I would be, which is fighting for 3rd behind erdman and mzotkiew. I figured there'd be several contenders for 3rd place, but if what you say is true, I could still see my trueskill go up for a while longer since I'm only at 82 games played right now.

Granted, variance has a large part in it, and this also means that I'm "picking" on those who are lower ranked instead of fighting those who might be near me like cdurbin.

#19

That is an interesting analysis. Can you publish the code you used? How long does it take the two players to come within 0.1 ish of one another?

#20

Also, we should note that we plan on giving the top players many hundres of games, by slowly stopping lower rank players from seeding.

#21

What are the details of this process?