Per Team Skills

moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
«134

Comments

  • SquishpokePOOPFACESquishpokePOOPFACE -21,248 posts (ignore below) Join Date: 2012-10-31 Member: 165262Members, Reinforced - Shadow
    Even when players are assigned randomly, long stretches of playing a team they are better or worse at adds noise to their skill value.

    I disagree with this premise. Given random shuffle (which is the norm in most servers), a player's ELO score will be correct. Example: a primary marine player's ELO at 3000 goes through many shuffled games, and by losing many alien games the ELO will drop naturally, even if the shuffle algorithm was not aware of an inflated ELO in the first place.
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    Even when players are assigned randomly, long stretches of playing a team they are better or worse at adds noise to their skill value.

    I disagree with this premise. Given random shuffle (which is the norm in most servers), a player's ELO score will be correct. Example: a primary marine player's ELO at 3000 goes through many shuffled games, and by losing many alien games the ELO will drop naturally, even if the shuffle algorithm was not aware of an inflated ELO in the first place.

    There's two different notions of accuracy here: Low bias, and low variance. Low bias means that the prediction will on average be centered around the true value. Low variance means the prediction will be close to the true value. You are right that random team assignment makes the bias 0, but it doesn't make the variance 0. The variance never goes to 0, and this is one of many factors that add to it.

    (Also, shuffle tries to keep players on their initial team, so even if shuffle is used every round, players may still play more often on their preferred team.)
  • SantaClawsSantaClaws Denmark Join Date: 2012-07-31 Member: 154491Members, Reinforced - Shadow
    I like it. :thumbs_up:
  • VetinariVetinari Join Date: 2013-07-23 Member: 186325Members, Squad Five Blue, Reinforced - Shadow, WC 2013 - Silver
    Moultano, I really appreciate your brain.
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    edited October 2017
    This is really exciting!
  • pSyk0mAnpSyk0mAn Nerdish by Nature Germany Join Date: 2003-08-07 Member: 19166Members, NS2 Playtester, Squad Five Silver, NS2 Community Developer
    Clean, simple yet smart, and doesn't mess with what's already been there and established. Good job!
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    Nordic wrote: »
    This is really exciting!

    :) Agreed. Aside from the experience improvements that I hope it brings, I'm really interested to see what the skill discrepancies in the player base look like.
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    I wager my internet cool points that that 10%+-10% players will have a significant skill difference of greater than 50%.
  • VetinariVetinari Join Date: 2013-07-23 Member: 186325Members, Squad Five Blue, Reinforced - Shadow, WC 2013 - Silver
    Nordic wrote: »
    I wager my internet cool points that that 10%+-10% players will have a significant skill difference of greater than 50%.

    I'll do my part by taking as many awesomes back from you as necessary if you lose.
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    Any predictions for the direction of the biggest outliers? Will there be more crack shot/terrible skulk players or vice versa?
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    I was double checking the math because I didn't quite believe the gradient, when I realized a cute interpretation of the offset: "How much more likely are the marines to win when you join a balanced game?"
  • SantaClawsSantaClaws Denmark Join Date: 2012-07-31 Member: 154491Members, Reinforced - Shadow
    Now I'm confused with what you mean by balanced game. If you include the offsets, why would you not expect even chances?
  • .trixX..trixX. Budapest Join Date: 2007-10-11 Member: 62605Members
    SantaClaws wrote: »
    Now I'm confused with what you mean by balanced game. If you include the offsets, why would you not expect even chances?

    QED x)
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    edited October 2017
    SantaClaws wrote: »
    Now I'm confused with what you mean by balanced game. If you include the offsets, why would you not expect even chances?

    He means balanced given the hive system assuming average team skill is equal and team standard deviation of skill.

    Although this hypothetical game is "balanced" it is not perfect. There is the assumption that different players are better or worse at different teams to varying degrees. Separate skill values, even with offsets, could improve the accuracy of the skill value by reducing variance.

    Aliens also have historically had a higher chance of winning. Theoretically if someone played aliens only they could inflate their hive skill. This could also create some variance.
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    What @Nordic said. I was surprised that it doesn't matter which team you are on. If you are in the game at all, on either team, and marines win, your offset goes up.
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    edited October 2017
    moultano wrote: »
    Any predictions for the direction of the biggest outliers? Will there be more crack shot/terrible skulk players or vice versa?

    I think it depends on which player population. It would make sense that newer players will always be better at marines. Marine play is somewhat familiar compared to the entirely unfamiliar aliens. Rookies may be the largest population but I think they also have the lowest variance because of their inexperience. This hypothesis is further supported by rookies playing marines more often than aliens. I am assuming players will generally play the team they are better with.

    Middle skilled players tend to be better at aliens. At least this is what I have observed while playing and in the data. Middle skilled players do play aliens more than marines which follows my assumption. Servers that have a high quantity of middle skilled games have very high alien win rates. Diamond gamers and TGNS are good examples of this.

    High skill player tend to play marines significantly more than aliens. High skill games tend to have lower alien win rates. The variance may be quite high in these players but their alien skill level is rarely bad. It is that their marine skill is just that far above average.

    I once tried to estimate the variance by creating separate skill values by muliplying their hive skill by the percent of time played by team. This experiment produced results like I have hypothesized here. I think that the percent variance will be similar to the variance in the percent time spent on each team.
    moultano wrote: »
    I was surprised that it doesn't matter which team you are on. If you are in the game at all, on either team, and marines win, your offset goes up.
    What suprised you? Did you do an experiment?

  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    I've updated my original blog post on hive to contain all the more recent additions, Adagrad, and now per-team skills. https://moultano.wordpress.com/2014/08/04/a-skill-ranking-system-for-natural-selection-2/ So you can now use that as a pretty comprehensive reference for how everything works (with the caveat that some things like commander skills, and solving for full convergence aren't actually in the game.)
    Rereading it, I found two things amazing.
    1. I published it over 3 years ago. Holy cow.
    2. I know waaaay more now than I knew then!
  • NovoReiNovoRei US Join Date: 2014-11-18 Member: 199718Members
    The shuffle algorithm will look at the player marine/alien skill when assigning and not the skill, correct?

    By coupling race skills you are assuming a good player is generally good at both teams. In other words you are assuming a certain offset magnitude which will be driven by the rate difference between adagrad rate and regular rate. It may need a coefficient to adjust it's gain.

  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    edited October 2017
    NovoRei wrote: »
    The shuffle algorithm will look at the player marine/alien skill when assigning and not the skill, correct?
    From his blog post:
    Moultano wrote:
    We could fix these by just separating skill into two separate values, but we’d also like our solution to have the following properties.

    1) It should converge in as few games as the current system. Separating skills naively would not.
    2) Shuffle should not force players to play on their better or worse team. If a player is generally better than average, a naive shuffle would tend to put them on their worse team.

    The proposal that solves the majority of these issues, rather than learning two separate skills, is to learn an average and a difference. In addition to the common skill value used above, we’ll learn an offset that turns the average into their skill for each team when added or subtracted.
  • NovoReiNovoRei US Join Date: 2014-11-18 Member: 199718Members
    edited October 2017
    Nordic wrote: »
    NovoRei wrote: »
    The shuffle algorithm will look at the player marine/alien skill when assigning and not the skill, correct?
    From his blog post:
    Moultano wrote:
    We could fix these by just separating skill into two separate values, but we’d also like our solution to have the following properties.

    1) It should converge in as few games as the current system. Separating skills naively would not.
    2) Shuffle should not force players to play on their better or worse team. If a player is generally better than average, a naive shuffle would tend to put them on their worse team.

    The proposal that solves the majority of these issues, rather than learning two separate skills, is to learn an average and a difference. In addition to the common skill value used above, we’ll learn an offset that turns the average into their skill for each team when added or subtracted.

    Average and offset is just a mathematical representation. It's the same content as having two separate skills. Moultano cleverness is coupling the two skills by using different convergence rates on the two components of that representation.

    I just want to know if the shuffle algorithm (last phrase of your quote does not make it clear) will look at the (average+offset) or just (average) when distributing players. I believe it should look at the (average+offset) otherwise the team imbalance will be the same or worse.
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    NovoRei wrote: »
    Nordic wrote: »
    NovoRei wrote: »
    The shuffle algorithm will look at the player marine/alien skill when assigning and not the skill, correct?
    From his blog post:
    Moultano wrote:
    We could fix these by just separating skill into two separate values, but we’d also like our solution to have the following properties.

    1) It should converge in as few games as the current system. Separating skills naively would not.
    2) Shuffle should not force players to play on their better or worse team. If a player is generally better than average, a naive shuffle would tend to put them on their worse team.

    The proposal that solves the majority of these issues, rather than learning two separate skills, is to learn an average and a difference. In addition to the common skill value used above, we’ll learn an offset that turns the average into their skill for each team when added or subtracted.

    Average and offset is just a mathematical representation. It's the same content as having two separate skills. Moultano cleverness is coupling the two skills by using different convergence rates on the two components of that representation.

    I just want to know if the shuffle algorithm (last phrase of your quote does not make it clear) will look at the (average+offset) or just (average) when distributing players. I believe it should look at the (average+offset) otherwise the team imbalance will be the same or worse.

    The plan for now is to use the average only. I believe this will still yield benefits. Once it is out there for a while, we (or modders) can experiment with other criteria. I'd really like to guarantee that a player won't be biased towards one team or another by shuffle, and it seems hard to guarantee that using the per team skills to shuffle.
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    I don't understand. Seems to me like this is just making the average (your skill number) more accurate.

    If I'm a 1200 hivescore, how does it know that my 1200 is really a 200 on marine and a 1500 on alien?

    Is there a way to explain it without the mathematical mumbo jumbo for the simple folk like me?
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    MoFo1 wrote: »
    I don't understand. Seems to me like this is just making the average (your skill number) more accurate.

    If I'm a 1200 hivescore, how does it know that my 1200 is really a 200 on marine and a 1500 on alien?

    Is there a way to explain it without the mathematical mumbo jumbo for the simple folk like me?

    It is essentially taking your 200 and 1500 separate team skills and averaging them to make it more accurate. This reduces variance from your skill from going up or down a lot by being better at one team.

    The problem with using actually separate skill values is what I have been telling you for years.
    1) Separate skill values would converge slower.
    2) Shuffle would force players to play on their better or worse team. If a player is generally better than average, a naive shuffle would tend to put them on their worse team. If a player is generally worse than average, a naive shuffle would tend to put them on their better team.

    The cool thing is that even without actually using separate hive skill values is that we would actually have the data. It would prove or disprove that your marine skill is closer to 200 and your alien skill closer to 1500. It would allow server operators or UWE to enable separate skill value shuffling in the future. Some servers could choose to live with those problems I mentioned, and others could choose to not live with them.
  • Dusanking123Dusanking123 Serbia,Belgrade Join Date: 2017-10-29 Member: 233762Members
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    Ok I know we haven't gotten off on the right foot, and a lot of you think I'm a troll (I'm really not) I'm just trying to understand... So let me try to clarify.
    Nordic wrote: »

    It is essentially taking your 200 and 1500 separate team skills and averaging them to make it more accurate. This reduces variance from your skill from going up or down a lot by being better at one team.

    So it is exactly what I thought then... Simply making the average more accurate.
    Nordic wrote: »
    The problem with using actually separate skill values is what I have been telling you for years.
    1) Separate skill values would converge slower.
    2) Shuffle would force players to play on their better or worse team. If a player is generally better than average, a naive shuffle would tend to put them on their worse team. If a player is generally worse than average, a naive shuffle would tend to put them on their better team.

    Not sure why you even mentioned this? I didn't mention wanting seperate hive scores. (gave up on that dream) I'm asking how will this new algorithm (or whatever you want to call it) know that I'm awful at Marine and decent at Alien? (or rather if you'd prefer, how will it "learn" that information)

    I ended by asking if there's a way to explain it without the mathematical mumbo jumbo, in the hopes that I might be able to understand better... If that's not possible then fine.




    Nordic wrote: »
    The cool thing is that even without actually using separate hive skill values is that we would actually have the data. It would prove or disprove that your marine skill is closer to 200 and your alien skill closer to 1500.

    This is one of the reasons I'm very interested in what the data will reveal. If it doesn't show that my Marine skill is way way waaaaay below my Alien skill, then it will be concrete proof beyond any shadow of doubt that the system is flawed. I can actually contribute and help with wins on Alien, whereas I'm just completely and totally useless on Marine.
  • SantaClawsSantaClaws Denmark Join Date: 2012-07-31 Member: 154491Members, Reinforced - Shadow
    edited November 2017
    MoFo1 wrote: »
    Ok I know we haven't gotten off on the right foot, and a lot of you think I'm a troll (I'm really not) I'm just trying to understand... So let me try to clarify.
    Nordic wrote: »

    It is essentially taking your 200 and 1500 separate team skills and averaging them to make it more accurate. This reduces variance from your skill from going up or down a lot by being better at one team.

    So it is exactly what I thought then... Simply making the average more accurate.
    That's the wrong way to look at it. This new Elo will show you your average between your factions - where the old essentially show you the average, but weighted proportional to your bias towards your most common faction. It's not more or less "accurate", you could just as well argue that the current one would be the "accurate" representation.
    MoFo1 wrote: »
    how will this new algorithm (or whatever you want to call it) know that I'm awful at Marine and decent at Alien? (or rather if you'd prefer, how will it "learn" that information)

    I ended by asking if there's a way to explain it without the mathematical mumbo jumbo, in the hopes that I might be able to understand better... If that's not possible then fine.
    This is addressed in the google doc:
    The learning rate of the skill value is the sum of two components, the adagrad learning rate, and the constant learning rate. To learn the correct value for this offset, while ensuring the system converges at the same rate, we will not apply the adagrad learning rate to the offset, only the constant learning rate. This will cause a player's initial games to affect both skill values equally, and later games to only affect the skill for the team they played on.
    MoFo1 wrote: »
    Nordic wrote: »
    The cool thing is that even without actually using separate hive skill values is that we would actually have the data. It would prove or disprove that your marine skill is closer to 200 and your alien skill closer to 1500.

    This is one of the reasons I'm very interested in what the data will reveal. If it doesn't show that my Marine skill is way way waaaaay below my Alien skill, then it will be concrete proof beyond any shadow of doubt that the system is flawed. I can actually contribute and help with wins on Alien, whereas I'm just completely and totally useless on Marine.

    Yeaa it might be flawed OR, the truth might be that you are so lacking at both that you suffer from a Dunning-Krueger effect and you in reality don't contribute as much as you think as alien. Or you could be performing poorly subconsciously as a sort of self-fulfilling prophecy in order to prove your point. In short, I don't think it would be proof of anything at all.
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    MoFo1 wrote: »
    Ok I know we haven't gotten off on the right foot, and a lot of you think I'm a troll (I'm really not) I'm just trying to understand... So let me try to clarify.
    ...
    I ended by asking if there's a way to explain it without the mathematical mumbo jumbo, in the hopes that I might be able to understand better... If that's not possible then fine.
    I am trying to explain it to you as best I can. I always struggle to communicate with you. I am trying to use your own language to explain it.
    MoFo1 wrote: »
    Nordic wrote: »
    The problem with using actually separate skill values is what I have been telling you for years.
    1) Separate skill values would converge slower.
    2) Shuffle would force players to play on their better or worse team. If a player is generally better than average, a naive shuffle would tend to put them on their worse team. If a player is generally worse than average, a naive shuffle would tend to put them on their better team.

    Not sure why you even mentioned this? I didn't mention wanting seperate hive scores. (gave up on that dream)

    I know you want separate hive skills. I do too. This is almost the same thing. It could potentially be used that way.
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    Ok well I give up...

    I just hope we get more balanced games and I get put on Alien more often.
  • SantaClawsSantaClaws Denmark Join Date: 2012-07-31 Member: 154491Members, Reinforced - Shadow
    Assuming this idea pans out. Do you think it can be expanded to have a commander offset in the future as well?
Sign In or Register to comment.