How does Hive Skill work?

124»

Comments

  • Person8880Person8880 Join Date: 2013-01-02 Member: 177167Members, Squad Five Blue
    Aeglos wrote: »
    Is it zero or close to zero? Rookies can luck into a win just by joining the correct team and won't always be at zero.
    Only absolutely 0, which yes, not all rookies fall under. That's what the rest of the sorting algorithm is for, making sure that teams are as evenly spread across the skill spectrum as possible by comparing the standard deviations of the team's skills. There's no need for it to play favourites with rookies because the skill data already tells it who's low skill and who's just left rookie mode on for funsies.
    Deck_ wrote: »
    Thats actually a lie, pretty sure
    That's exactly how the code is written to work, and tested to work. Have a look for yourself if you don't believe me. There's also a set of tests around the shuffling that confirm it works as intended (and is deterministic). If you can reliably reproduce a situation where this is not the case, then by all means do so and I'll see why it happens and fix it. Otherwise, I have no reason to believe otherwise that, as far as the raw skill data is concerned, the shuffling is balancing teams correctly.
  • AeglosAeglos Join Date: 2010-04-06 Member: 71189Members
    Person8880 wrote: »
    Aeglos wrote: »
    Is it zero or close to zero? Rookies can luck into a win just by joining the correct team and won't always be at zero.
    Only absolutely 0, which yes, not all rookies fall under. That's what the rest of the sorting algorithm is for, making sure that teams are as evenly spread across the skill spectrum as possible by comparing the standard deviations of the team's skills. There's no need for it to play favourites with rookies because the skill data already tells it who's low skill and who's just left rookie mode on for funsies.
    Deck_ wrote: »
    Thats actually a lie, pretty sure
    That's exactly how the code is written to work, and tested to work. Have a look for yourself if you don't believe me. There's also a set of tests around the shuffling that confirm it works as intended (and is deterministic). If you can reliably reproduce a situation where this is not the case, then by all means do so and I'll see why it happens and fix it. Otherwise, I have no reason to believe otherwise that, as far as the raw skill data is concerned, the shuffling is balancing teams correctly.

    Well, I haven't been paying attention so I don't know if people use vote force even or shuffle when it happens, but multiple rookies on only one team has happened and is rather common. When I read that I thought if a rookie is not considered a rookie for sorting purposes at say, fifty hive skill, it still lumps them on the same team which is all sorts of terrible. If you say the standard deviation takes care of it then fine, must have been the other sorting system then. What information do you need if I do encounter such a situation though? Hive skill is not visible on many servers and I can't possibly be going through every player's hive page.
  • Deck_Deck_ Join Date: 2014-07-20 Member: 197526Members
    edited September 2015
    Person8880 wrote: »
    Aeglos wrote: »
    Is it zero or close to zero? Rookies can luck into a win just by joining the correct team and won't always be at zero.
    Only absolutely 0, which yes, not all rookies fall under. That's what the rest of the sorting algorithm is for, making sure that teams are as evenly spread across the skill spectrum as possible by comparing the standard deviations of the team's skills. There's no need for it to play favourites with rookies because the skill data already tells it who's low skill and who's just left rookie mode on for funsies.
    Deck_ wrote: »
    Thats actually a lie, pretty sure
    That's exactly how the code is written to work, and tested to work. Have a look for yourself if you don't believe me. There's also a set of tests around the shuffling that confirm it works as intended (and is deterministic). If you can reliably reproduce a situation where this is not the case, then by all means do so and I'll see why it happens and fix it. Otherwise, I have no reason to believe otherwise that, as far as the raw skill data is concerned, the shuffling is balancing teams correctly.

    your first point says it needs to be absolute 0 to force all the absolute 0 players on different teams evenly. So based on that, there is nothing to stop all 5 rookies on a server from being on the same team as long as they have slightly above a 0 score and players on their team with much higher scores make up the difference. Again, I think it needs to be for all rookies, or the threshold has to be above absolute 0.

    I'm not saying this because I don't understand what you're trying to say or how it makes sense mathematically. The point is, it doesn't work in how this game is played. If you have 5 guys on a team (and none on the other team) that have barely played the game, even if they are linked up with a couple great players- it usually makes for a terrible game. It's what I've seen in pubs. I wouldn't be surprised if many other people have seen the same thing. It has to be a terrible experience for rookie players too. The thing is, if this gets fixed, it could be great. A lot of people vote for the shuffle wanting a better/more even game, but I think the system is broken as is. If this get fixed, I think it will be way less frustrating for new players and everyone else.
  • Person8880Person8880 Join Date: 2013-01-02 Member: 177167Members, Squad Five Blue
    edited September 2015
    Deck_ wrote: »
    your first point says it needs to be absolute 0 to force all the absolute 0 players on different teams evenly. So based on that, there is nothing to stop all 5 rookies on a server from being on the same team as long as they have slightly above a 0 score and players on their team with much higher scores make up the difference. Again, I think it needs to be for all rookies, or the threshold has to be above absolute 0.
    You're still not understanding what the standard deviation is and what balancing based on it is doing.

    The standard deviation indicates the distance of each value from the average of the values. By making sure both team's standard deviations are as close as possible, it is making sure that there is not extreme values of high and low skill on a single team. If there were, then that team would have a massively high standard deviation, while the other would likely have a very low, which is what the sorting is preventing.

    For an example, take a team of players with skills:
    2500, 2000, 1800, 500, 400, 50
    The average skill is: 1208.33.
    The standard deviation of skill for this team is: 925.75.

    For a much more evenly distributed team, with skills such as:
    2000, 1800, 1500, 1200, 1000, 700
    The average skill is: 1366.67
    The standard deviation of skill is: 449.69

    So by ensuring that teams are not massively far apart in standard deviation, it's making sure that players are evenly spread (as far as is possible). There cannot be teams of extremes unless the entire server is extremes at which point it's impossible to sort the teams fairly in the way you'd like anyway.

    The only reason 0 skill value players are treated differently is because there is no data on their ability, so using the last few rounds of score/KDR provides a poor but somewhat meaningful substitute until their skill value is populated at a later time.
    Aeglos wrote: »
    What information do you need if I do encounter such a situation though? Hive skill is not visible on many servers and I can't possibly be going through every player's hive page.
    This is admittedly a problem. However, the shuffling will output the averages for both teams and I can change it to also output the standard deviation of both teams too. That way, you'd only have to look out for values that seem very far apart and not have to check individual player's skills.

    Edit: Thinking about it further, I'm going to remove the public notification and add a command (sh_teamstats) that those who are interested can use to view the skills of both teams (anonymously, there's no player names linking to skill) and see the average and standard deviation. That way you can see exactly how it distributed skills and easily spot if there really is a problem or not.
  • Deck_Deck_ Join Date: 2014-07-20 Member: 197526Members
    edited September 2015
    Person8880 wrote: »
    Deck_ wrote: »
    your first point says it needs to be absolute 0 to force all the absolute 0 players on different teams evenly. So based on that, there is nothing to stop all 5 rookies on a server from being on the same team as long as they have slightly above a 0 score and players on their team with much higher scores make up the difference. Again, I think it needs to be for all rookies, or the threshold has to be above absolute 0.
    You're still not understanding what the standard deviation is and what balancing based on it is doing.

    The standard deviation indicates the distance of each value from the average of the values. By making sure both team's standard deviations are as close as possible, it is making sure that there is not extreme values of high and low skill on a single team. If there were, then that team would have a massively high standard deviation, while the other would likely have a very low, which is what the sorting is preventing.

    For an example, take a team of players with skills:
    2500, 2000, 1800, 500, 400, 50
    The average skill is: 1208.33.
    The standard deviation of skill for this team is: 925.75.

    For a much more evenly distributed team, with skills such as:
    2000, 1800, 1500, 1200, 1000, 700
    The average skill is: 1366.67
    The standard deviation of skill is: 449.69

    So by ensuring that teams are not massively far apart in standard deviation, it's making sure that players are evenly spread (as far as is possible). There cannot be teams of extremes unless the entire server is extremes at which point it's impossible to sort the teams fairly in the way you'd like anyway.

    The only reason 0 skill value players are treated differently is because there is no data on their ability, so using the last few rounds of score/KDR provides a poor but somewhat meaningful substitute until their skill value is populated at a later time.

    Ok, I wasn't using a good example. I was trying to mention an extreme example, but I didn't do the research if that was possible or not. I think I understand what you are saying, the standard deviation wouldn't work if one team had a couple really high skills and a bunch of really low skills and the other team was more evenly distributed - it wouldn't happen because of the standard deviation. The problem is, you left open the possibility that there can be teams of extremes, "unless the entire server is extremes at which point it's impossible to sort the team fairly in the way you'd like anyway." - I disagree with this mindset or what you think I'm looking for. My point is about new players/rookies. If you have 3 rookies on a server, and the shuffle makes them all on the same team - I don't care what the standard deviation says, I don't care if there is extremes on both teams...the point is it doesn't play well for ns2. This is the difference from a mathematical argument and a practical one. I play in the pubs, and I see this happen. If you have all the new players on one team, it makes for a terrible game. What I'm asking is, is it possible to have a threshold of a certain low skill level (decided through testing) or hour level ceiling where those players below that line are sorted evenly and then you can do the standard deviation/sorting with the rest of the players excluding the rookies.

    All I'm asking for is to try it. I think it could work a lot better. If you disagree because you don't like the math, fair enough. I cede that you understand the math better, but I know it isn't working and my idea is something I hope could have practical benefits. I could be totally wrong too.
  • YojimboYojimbo England Join Date: 2009-03-19 Member: 66806Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    This is becoming one of those threads that would even give an aspirin a headache.
  • Person8880Person8880 Join Date: 2013-01-02 Member: 177167Members, Squad Five Blue
    Deck_ wrote: »
    All I'm asking for is to try it. I think it could work a lot better. If you disagree because you don't like the math, fair enough. I cede that you understand the math better, but I know it isn't working and my idea is something I hope could have practical benefits. I could be totally wrong too.
    I don't understand what you think the difference is between how it works now, and what you propose. A true rookie is going to be a player with a low skill value. Low skill value players will be distributed evenly between both teams, because if they're not then one team will have a massively higher standard deviation.

    What you seem to be asking for, is to discard the skill values of rookies, then essentially hardcode behaviour that accomplishes sort of the same thing but less precise. Why should a rookie's skill value be dismissed?

    The skill system is telling the shuffling system the same thing you want it to know, that is, they're low skill. If, for some reason, the Hive skill system is giving rookie players a higher skill value than they're worth, then that is something that should be fixed in the Hive system and not worked around in the shuffling system. It can (and should) only work with the skill data it is given.

    Also, to clear up what I meant by "unless the entire server is extremes", I mean if you had a server of skill values such as 8 2000+ and 8 less than 500. That would be a server of extremes where there's no way to not have teams of half high skill and half low skill. Most of the time I'd imagine skill distribution on a server is nowhere near that bad.
  • FrozenFrozen New York, NY Join Date: 2010-07-02 Member: 72228Members, Constellation
    The only way you're going to improve upon the algorithm is by realizing that you can't use the standard deviation as it is. Deck is correct in saying that you should group rookies together based on a grouping, not the number 0. Doing that means non-rookies are included in that, also.

    What you need to do is use grouping, such 0-499, 500-999, 1000-1499 and so on. So someone with 2800 skill is equivalent to someone with 2600 skill in the calculation, etc.

    I was talking to a programmer at a bar around the Kentucky Derby. He was saying you could say all horses are equal or some shit, so I mentioned that different horses run different in different track conditions and weather. He said that wasn't true. I said then why the fuck does anyone race horses. Then I stopped talking to him. Anyway some kind of analogy about why similar skills should be considered equal, not account for the precise differences between them.

  • sotanahtsotanaht Join Date: 2013-01-12 Member: 179215Members
    mattji104 wrote: »
    The only way you're going to improve upon the algorithm is by realizing that you can't use the standard deviation as it is. Deck is correct in saying that you should group rookies together based on a grouping, not the number 0. Doing that means non-rookies are included in that, also.

    What you need to do is use grouping, such 0-499, 500-999, 1000-1499 and so on. So someone with 2800 skill is equivalent to someone with 2600 skill in the calculation, etc.

    I was talking to a programmer at a bar around the Kentucky Derby. He was saying you could say all horses are equal or some shit, so I mentioned that different horses run different in different track conditions and weather. He said that wasn't true. I said then why the fuck does anyone race horses. Then I stopped talking to him. Anyway some kind of analogy about why similar skills should be considered equal, not account for the precise differences between them.
    Keep in mind that a '0' has not been ranked yet and therefor could be a smurf with skill equivalent to a 3000 player as a truely oblivious rookie. These people are deliberately split up to avoid accidentally stacking a team with smurfs, or other reasonably skilled and able new players.

    Once a player has played enough games to have a real rating, they can be split up based on that rating. Even if it's a 1, they need to be addressed separately from potential smurfs.

    As far as "grouping", that's a solution for players who are trying to parse and make sense of the data themselves. Averaging already handles that, and the standard-devation control makes certain that people from every point along the skill spectrum are assigned to both skills. Explicit grouping is entirely unnecessary to the algorithm.
  • FrozenFrozen New York, NY Join Date: 2010-07-02 Member: 72228Members, Constellation
    edited September 2015
    Someone with 4 hive skill can still be a smirf after winning 4 points. Someone with 345 hive hive score can still be a smurf after winning 345 points.

    You're saying you're deliberately ignoring that. I'm saying that's not representative of reality.

    I'm also someone who thinks hive score already works really well as it is, by the way
  • Person8880Person8880 Join Date: 2013-01-02 Member: 177167Members, Squad Five Blue
    mattji104 wrote: »
    What you need to do is use grouping, such 0-499, 500-999, 1000-1499 and so on. So someone with 2800 skill is equivalent to someone with 2600 skill in the calculation, etc.
    If you want to discard the precision of the real values, then the entire system is meaningless. Every little difference can and should matter, if it does not, then the underlying skill value is flawed, not the system used to sort players based on it. There has to be a point at which you draw the line, and I see no good reason why data should be discarded based on gut feeling.
    sotanaht wrote: »
    As far as "grouping", that's a solution for players who are trying to parse and make sense of the data themselves. Averaging already handles that, and the standard-devation control makes certain that people from every point along the skill spectrum are assigned to both skills. Explicit grouping is entirely unnecessary to the algorithm.
    This is what I've been trying to explain.
    mattji104 wrote: »
    Someone with 4 hive skill can still be a smirf after winning 4 points. Someone with 345 hive hive score can still be a smurf after winning 345 points.

    You're saying you're deliberately ignoring that. I'm saying that's not representative of reality.

    I'm also someone who thinks hive score already works really well as it is, by the way
    It is impossible to build a sorting algorithm where the basis is that the data may be flawed. It has to work on the assumption that the data is correct because otherwise there's no point in the entire Hive system. The more edge-cases and hardcoded logic you try and throw at it, the less precise, less grounded in reasonable thought and less reliable the sorting will be.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    mattji104 wrote: »
    (...) What you need to do is use grouping, such 0-499, 500-999, 1000-1499 and so on. So someone with 2800 skill is equivalent to someone with 2600 skill in the calculation, etc. (...)

    Any game will be filled with a lot of rookies and few godlike players (real ones and farmers). You know that the higher score they have (farmers) the bigger the nonsense it is.

    In this case how do you cut the pie ? 3, 4, 5 parts ? Where do you place (locate) the rookie group limit in a skill range from 0->1000 and another one with 0->5000 ? A simple division won't be quite realistic.

    As long as you can have an infinite (at least a way to high value) it will be meaningless to try. It needs some ingredients that will ensure you can't go to infinite (whatever the system) to provides the solid grounds you need for what you describe.

    Not that i don't like it or something, it's just you have to make a choice to "group" players. It's not backed up by tangible data other than the hive score.

    Who's to say a 0-500 player is a total rookie ? He could have lost many games but still is a really good players. 0-200 ? 0-1000 ? Who knows ?

  • AeglosAeglos Join Date: 2010-04-06 Member: 71189Members
    Person8880 wrote: »
    Edit: Thinking about it further, I'm going to remove the public notification and add a command (sh_teamstats) that those who are interested can use to view the skills of both teams (anonymously, there's no player names linking to skill) and see the average and standard deviation. That way you can see exactly how it distributed skills and easily spot if there really is a problem or not.

    I tried using the command, but all it gives me is hive balancing not enabled or something. Pretty sure it was shuffle.

  • Person8880Person8880 Join Date: 2013-01-02 Member: 177167Members, Squad Five Blue
    Aeglos wrote: »
    I tried using the command, but all it gives me is hive balancing not enabled or something. Pretty sure it was shuffle.
    That means the server you're on is not using the Hive skill mode for shuffling. There's a few legacy modes alongside Hive skill (KDR, score and pure random) which that server's probably using one of instead. It should say when votes are made and if the vote passes what mode was used to shuffle.
  • FrozenFrozen New York, NY Join Date: 2010-07-02 Member: 72228Members, Constellation
    You can base groupings off of the standard deviation to make the group sizings dymamic.

    I'm not assuming the data is wrong. I'm assuming there's not enough active players in the system to have the shuffle work based on exact values.
  • AeglosAeglos Join Date: 2010-04-06 Member: 71189Members
    edited September 2015
    Person8880 wrote: »
    Aeglos wrote: »
    I tried using the command, but all it gives me is hive balancing not enabled or something. Pretty sure it was shuffle.
    That means the server you're on is not using the Hive skill mode for shuffling. There's a few legacy modes alongside Hive skill (KDR, score and pure random) which that server's probably using one of instead. It should say when votes are made and if the vote passes what mode was used to shuffle.

    Oh. I think it said KDR or something. Not sure how that works but its not very good.

    Edit - Played a server that uses hive skill. Seems fine as long as hive score is reliable. So far at least.
  • ValoValo Join Date: 2005-03-22 Member: 46102Members, Constellation, NS2 Playtester
    the main issue I have with the hive stats is that the average hive score of a player is about 1-1.5k. As it is now becoming common every game begins with a force even teams/shuffle.
    This usually results in me being put on a team with people with low hive scores to balance out the 3k i have. This doesn't make even games IMO! I'm not saying all low hive scored people are bad, but it does make the team-play lacking.

    I know "play on non rookie friendly servers" - that's not always possible due to the small number of live servers and I refuse to sit for 10mins waiting for a slot :D

    Also surely kdr should also be factored into the resultant score to some extent (not a sole means of skill level). If you have a higher kdr you are in effect suppressing the enemy and assisting your team. eg a marine holding a strategic position getting lots of kills outside the hive allowing the rest of the team to cap the rest of the map. If you die lots - you block the spawn queue having a detrimental effect.

    What about a weighted system taking into account accuracy, kdr, ELO, build time, structure damage. Basically account for all the factors shown in the post game stats?
    I'm not even sure if win or lose should be accounted! I mean you can be a skilled player but still be on a bad team.
    Ultimately the hive stats are used to balance games - surely how many times you choose the winning team shouldn't be the sole value used for this!
  • jrgnjrgn Join Date: 2006-11-03 Member: 58289Members
    Well i have tried to understand hive skill lately, i'm not a stats-kind-of-guy. I use it mainly for teambalancing purposes. I tried to understand the math in this thread but it was to hard for my tiny brain. I didn't check the hive very often before then i got more interested in it for avoiding teamstacking and pubstomping. I started to follow my own skill point and it hilariously low. I played the game for even since ns1, i understand the game mechanics pretty well. I'm not a very skilled fps player, probably average or something. I used to be like at 900-1000 hive skill but the last 3 months the skill has been falling pretty quick. I tried to understand why so i searched for clues how to let the hive skill mirror my teamplay skills. Something obviously happened to the algoritm or the skill reporting system because my playing style didn't change much. Am I right in this? Is it bugged? Am i a really lousy player?

    Own skill is always a subjective opinion and therefore hard to judge sometimes.

    And in the end I want to know if you can easily rephrase the math jumbo to something everybody could relatively easily understand. I tried myself but i lack the knowledge of the exact method it is calculated, maybe something like this:

    "Hive skill is a measure of your ability to be on a team that wins a game against the other teams average hive skill"

    So this post is about these 2 questions: why is my hive skill sinking (490 something now), what can i do to make it increase? And how can i easily rephrase what it is all about on one sentence...?

    I tend to agree that "skill" is maybe the wrong word used in this context like some people said above, and maybe it should be hidden from players.
  • ValoValo Join Date: 2005-03-22 Member: 46102Members, Constellation, NS2 Playtester
    Don't worry about it jrgn! Keep it as low as you can!
    ELO drips when you're on the losing team. If the oppositions average hive is higher than your teams it will fall less than if it is equal to your teams and more if it is less than your teams. The opposite is true if you are on the winning team.

    Only way to increase is to be on the winning team!
  • jrgnjrgn Join Date: 2006-11-03 Member: 58289Members
    then i have to be on your team! ;)
  • snacksy13snacksy13 Norway Join Date: 2017-03-10 Member: 228823Members
    My performance greatly depends on my mood/sleepyness so sometimes i overperform, others i am pretty bad. So no one number can define my skill.
Sign In or Register to comment.