Hiveskill convergence rate | Hive 2 better?

barniebarnie Join Date: 2016-07-26 Member: 220695Members
After playing some rather one-sided rounds on a server checked the hiveskill on the player that was carrying.
A player with 3.9 KDR but only 1200hivescore @ 66 hours playtime logged on hive.
That was the explanation for the unbalanced rounds, a player like that should be rated 2500+.
So why did hive rate this player so low?
Maybe because he was a comp player mostly playing other comp players?
Or is hive just too slow in coming up with a proper rating?
But 66 hours of logged playtime should be plenty to come up with a rating.

Hive seems to rate skill relative to the players/teams the one is playing against. [citation needed]
So if you mostly play in a certain skill group of players i.e. Comp/NSL then your skill rating is in relation to that group of players and not the playerbase as whole because it is only based on rounds won/lost.

Conclusion: We need a skill rating system that rates player skill relative to the whole playerbase and not only the player you happen to be playing against.
Based on accuracy, KDR, points/sec and wins/losses.

@Nordic i require you to go full explanation mode
«1

Comments

  • MephillesMephilles Germany Join Date: 2013-08-07 Member: 186634Members, NS2 Map Tester, NS2 Community Developer
    I could have sworn that I read atleast 2 or 3 times in a specific way how hive 2 will deal with exactly that problem
  • KasharicKasharic Hull, England Join Date: 2013-03-27 Member: 184473Members, Forum Admins, NS2 Playtester, NS2 Map Tester, NS2 Community Developer
    hive 2 will more accurately find your skill level and it will do it way faster... so anyone pulling that kind of KDR won't be unbalancing teams for long. maybe a 3 - 4 rounds ish.
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    3.9 kdr should automatically put you at over 4k hive skill (or at the very least 3k)

    Seriously with kdr that high you can wipe out entire packs of aliens/marines and easily win 9/10 games simply by rushing and cutting off the other teams res..
  • .trixX..trixX. Budapest Join Date: 2007-10-11 Member: 62605Members
    hive2 is still being stress-tested atm and is only gathering data, the score you see is still the old version (afaik)
  • barniebarnie Join Date: 2016-07-26 Member: 220695Members
    edited September 2016
    So hive2 is going to solve all that? - good!
    Because that guy being still extremely underrated after ~300 logged rounds is ridiculous!

    What's the status of Hive 2, I read there was a test and it got removed it again.
  • VetinariVetinari Join Date: 2013-07-23 Member: 186325Members, Squad Five Blue, Reinforced - Shadow, WC 2013 - Silver
    barnie wrote: »
    So hive2 is going to solve all that? - good!
    Because that guy being still extremely underrated after ~300 logged rounds is ridiculous!

    What's the status of Hive 2, I read there was a test and it got removed it again.

    Well maybe he just got himself an aimbot recently.
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    MoFo1 wrote: »
    3.9 kdr should automatically put you at over 4k hive skill (or at the very least 3k)
    It is posts like these that hit the nerve that compels me to go into full explanation mode. I think I can do this one short.
    There are times of day where every server I join has an average hive skill of less than 1000. There are no higher skilled servers I can join. In these servers I have the potential to go 50/5 KDR if I tried. If I played there often I could potentially get >3.5 KDR most games. Does this mean I am a 3000 hive skill player? No. It is reletive. I can go play in a gather full of 3000 hive skill players and <1 KDR.

    A players ability to get kills and win engagement is the most important skill in the game. A high KDR is generally a good indicator of a skilled player, but please do not think that is always true. Things are not that black and white.
    barnie wrote: »
    Nordic i require you to go full explanation mode
    @Foxy is a moderator and told me not to go full explanation mode. Because I am so kind I will do as you request but I fear I may receive a PM.
    barnie wrote: »
    After playing some rather one-sided rounds on a server checked the hiveskill on the player that was carrying.
    A player with 3.9 KDR but only 1200hivescore @ 66 hours playtime logged on hive.
    That was the explanation for the unbalanced rounds, a player like that should be rated 2500+.

    Or is hive just too slow in coming up with a proper rating?
    But 66 hours of logged playtime should be plenty to come up with a rating.
    Yes, Hive 1 is that slow to accertain a players skill. There are also cases where hive severely undervalues a player for some reason regardless of how much they have played. You may have found one of these players. They do exist.If he went from 0 to 1200 hive skill in 66 hours he is moving up quickly. I have also seen players who are magnitudes more skilled than myself yet have significantly less hive skill. It is hard to know why.

    If you don't mind, send me that players hive profile or steam id. I wouldn't mind looking at it, but this is only for my own curiosity.

    One of the most anticipated features of Hive 2 is Adagrad. If you are interested here is a scientific paper about it. I am no where near qualified enough to explain it in detail. I can describe how it will effect hive. Adagrad will introduce a variable learning rate into hive such that it learns faster in the beginning and learns slower over time. In the spoiler you will see the results of a simulation comparing Hive 1 + Adagrad in yellow, Hive 1 in red, and the players underlying skill in blue. The X axis is number of rounds played, and the Y axis is hive skill. For reference, the median round length in NS2 is about 15 minutes.
    441xoIy.png
    If this picture is still not working you can use this link to see where I got it from.
    barnie wrote: »
    Hive seems to rate skill relative to the players/teams the one is playing against. [citation needed]
    Based on my own observations I believe this to be true. Hive skill only measures your skill against people you play with. Even though everybody has a number, not all of them are directly comparable. This is mostly a problem stemming from the server browser and community servers. I do not think there is enough random mixing amoung server communiteis for many hive skills to be very comparable. Take this cherry picked example. A player who only plays on Woozas and has developed a hive skill of 2000 is not very comparable to a player who only plays gathers and has developed a 2000 hive skill.
    barnie wrote: »
    We need a skill rating system that rates player skill relative to the whole playerbase and not only the player you happen to be playing against.
    Based on accuracy, KDR, points/sec and wins/losses.
    I do agree that it would be better if hive was less relative to who you play against. This is not exactly a solvable problem. It is not that hive does this poorly, but it is how all skill rating systems work. They learn your skill by comparing you to who you play against. Using accuracy, KDR, points/sec and wins/losses won't change this.

    I will also comment that using Win/loss alone does work. You can dig through this post/thread if you would like to see my evidence showing this.
    barnie wrote: »
    So hive2 is going to solve all that? - good!
    barnie wrote: »
    solve
    No. Hive 2 is mostly a backend upgrade to the system. It will add in a few new features to the hive algorithm such as Adagrad which I briefly explained earlier in this post. Adagrad will make it so hive finds a players true skill much much quicker. This may be true but don't expect Hive 2 to solve all the problems, make every game balanced, and solve world hunger. Hive 2 will only make it better but there is only so much it can do.
    barnie wrote: »
    What's the status of Hive 2, I read there was a test and it got removed it again.
    Hive 2 is currently deployed as a mod via the UWE extension system. It is collecting data alongside Hive 1 but is unused in any other way. The hive website only shows information from hive 1 and will not work with hive 2. We may not have a front end website for some time.
  • 2cough2cough Rocky Mountain High Join Date: 2013-03-14 Member: 183952Members, Reinforced - Shadow, WC 2013 - Supporter
    I will say I've started playing again here and there since i was sick at home 2 wks ago. I have noticed several former prem div players who i have not seen since before hive was implemented. Perhaps this is part of the same people you are referring to.
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    edited October 2016
    @Nordic

    Then maybe we need separate hive skill for casual/pub and comp/gather servers?

    Because a 3.9 kdr player is going to dominate and destroy balance on the vast majority of pub games they're in... Especially if they have a lower hive score than they should due to playing gathers.
  • 2cough2cough Rocky Mountain High Join Date: 2013-03-14 Member: 183952Members, Reinforced - Shadow, WC 2013 - Supporter
    edited October 2016
    Gotta start somewhere i guess, but really disappointing indeed to hear there will be no separation of factions, as that seems to be one of the MOST requested features to be added, and rightly so, I'd say. Using @MoFo1 as a prime example of someone who has probably got quite a disparity in skills between alien and rine, given his preference. A carry rine may not be a carry alien.
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    I'm a prime example all right...

    On marine there is literally never ever any time where i contribute to helping the team win..

    On alien I couldn't even begin to count how many games I've played a major role in our victory..

    My marine score should be 200-500 at the highest.. While my alien score should be closer to 1500-2000..

    Separation of marine/alien skill was by far the #1 best thing about hive2.0.. To hear it's not a part of it is saddening.. One skill score for everything is the #1 biggest flaw with the current hive, and now hive 2.0 is going to have the same flaw...
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    MoFo1 wrote: »
    Then maybe we need separate hive skill for casual/pub and comp/gather servers?
    Because a 3.9 kdr player is going to dominate and destroy balance on the vast majority of pub games they're in... Especially if they have a lower hive score than they should due to playing gathers.
    You still are not understanding what I mean by relative. It does not matter if we have separate hive skill for casual/pub and comp/gather servers. That would not solve the problem you are looking to solve.
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    2cough wrote: »
    Gotta start somewhere i guess, but really disappointing indeed to hear there will be no separation of factions, as that seems to be one of the MOST requested features to be added, and rightly so, I'd say. Using @MoFo1 as a prime example of someone who has probably got quite a disparity in skills between alien and rine, given his preference. A carry rine may not be a carry alien.

    I think we can figure out a good way to learn it, but we'd also have to figure out how to rewrite the balancing algorithms to take per team skills into account, and I'm not sure what the right strategy is. Suppose everyone on the server is better at Marines than Aliens. What criteria do you use to balance the teams in that situation?
  • McGlaspieMcGlaspie www.team156.com Join Date: 2010-07-26 Member: 73044Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Onos, WC 2013 - Gold, Subnautica Playtester
    MoFo1 wrote: »
    Separation of marine/alien skill was by far the #1 best thing about hive2.0.. To hear it's not a part of it is saddening..
    You're not the first person I've heard make this comment. It was never officially stated that it would be separated, so I'm not sure how this got started.

    As Moultano mentioned, it's not a simple as just recording two different values. There is a LOT of context that would be missing, which is _required_ in order for separated values to function in a useful manner. My sole focus with rewriting Hive (from scratch) is to resolve its two biggest issues: Instability and Inconsistency. As I mentioned before, enhancements can and will be added later.

  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    McGlaspie wrote: »
    MoFo1 wrote: »
    Separation of marine/alien skill was by far the #1 best thing about hive2.0.. To hear it's not a part of it is saddening..
    You're not the first person I've heard make this comment. It was never officially stated that it would be separated, so I'm not sure how this got started.

    As Moultano mentioned, it's not a simple as just recording two different values. There is a LOT of context that would be missing, which is _required_ in order for separated values to function in a useful manner. My sole focus with rewriting Hive (from scratch) is to resolve its two biggest issues: Instability and Inconsistency. As I mentioned before, enhancements can and will be added later.

    Back when Matso was working on hive I believe he mentioned he was working on it.
  • barniebarnie Join Date: 2016-07-26 Member: 220695Members
    edited October 2016
    Thank you McGlaspie for taking the time to get us up to speed with the current hive development.
    It is good to hear that Hive 2.0 will be a better platform for more sophisticated scoring algorithms.
    McGlaspie wrote: »
    As an example, Hive1 would require approximately 200-400 rounds of gameplay in order to "find" a player's skill.

    The user i was talking about here has >300 logged rounds and is still far from where his skill should be.
    It is an extreme case of a veteran from before hive coming back starting @ 0 Skill. or a smurf

    Maybe players should be put in a skill group a after a few logged rounds by KDR/Accuracy.
    For example:
    Player has played 12-15 rounds? -> do skillgroup sort. he's got 2.0 > KDR & 30% > accuracy? -> skill is now instantly 2000 -> normal hive proceedings continue from there.
    Basically an convergence accelerator for the hive against smurfs.

    Balanced team strengths are very important for for enjoyable gameplay these days.
    With the shrunk playerbase there is a large disparity between individual players skill and not enough population for every skill level to have their own servers/communities thus the hive balancing has become more important then ever.
  • HandschuhHandschuh Join Date: 2005-03-08 Member: 44338Members, NS2 Playtester, NS2 Community Developer
    So, all your saying is Hiveskill will still be a load of crap for balancing a full server since there is still no separate skill for the alien/marineteam?
    Good job for bringing any hopes up for balanced teams and getting the disappointment going..(since you announced that hive2.0 will be there soon)

    From my point of view this is the most important issue why hiveskill is so terrible and leads to bad games. And this issue was brought up literally thousands of times.

    So, we really have to wait for hive 3.0 which will never come?
  • MoFo1MoFo1 United States Join Date: 2014-07-25 Member: 197612Members
    It's not really entitlement when we were led to believe that hive2.0 would seperate alien/marine skill...

    I wish I could find the post that said it would be included...
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    As much as I want separate skill rating myself there is one clear issue. Right now when you are placed on a team with shuffle it is random. I know Mofo likes to complain that it is not and puts him on marines 5 times in a row, but it is random. With separate marine and alien skills it would not be random. For the sake of balance shuffle would force certain players to play one team more often than another.

    Take a look at this hypothetical balance I created with separate marine and alien skills. You can make a copy and try to balance teams yourself. Is there a way in which none of those players would be forced onto one team or the other more often than not?
  • greenhorsegreenhorse Join Date: 2014-11-11 Member: 199605Members
    MoFo1 wrote: »
    My marine score should be 200-500 at the highest.. While my alien score should be closer to 1500-2000..

    Mofo, how would you shuffle a group of 20 players where each player has this kind of skill break down?
  • barniebarnie Join Date: 2016-07-26 Member: 220695Members
    edited October 2016
    No seperate skillrating - ok that sucks i would have hoped for that to be in hive 2.
    But i take faster convergence and better quality over no improvements.
    And he said it will be a more versatile platform for new algorithms so separate ratings could be introduced later.
    I do think UWE has recognized how important team strength balancing is, why else would they bother to work on it.
    greenhorse wrote: »
    Mofo, how would you shuffle a group of 20 players where each player has this kind of skill break down?
    Unlikely scenario!
    There are quite a few players with exactly the opposite skill distribution, they can shoot quite accurately but they know jacksh*t about alien movement.

  • greenhorsegreenhorse Join Date: 2014-11-11 Member: 199605Members
    barnie wrote: »
    greenhorse wrote: »
    Mofo, how would you shuffle a group of 20 players where each player has this kind of skill break down?
    Unlikely scenario!
    There are quite a few players with exactly the opposite skill distribution, they can shoot quite accurately but they know jacksh*t about alien movement.

    If you have for example four players

    A (1000m 2000a) (marine skill 1000 alien skill 2000)
    B (1000m 2000a)
    C (2000m 1000a)
    D (2000m 1000a)

    There are lots of different ways to shuffle the teams
    to achieve equal average skill:

    AB marine (average 1000)
    CD alens (average 1000)

    AC marine (average 1500)
    BD alens (average 1500)

    CD marine (average 2000)
    AB alens (average 2000)

    [etc]


    If you make shuffle try to maximize average team skill then players will
    mostly play on teams where they are at their best (what Mofo is asking for).

    The downside would be that you will never be able to play on a team
    where your skill is lacking.



  • barniebarnie Join Date: 2016-07-26 Member: 220695Members
    edited October 2016
    greenhorse wrote: »
    If you make shuffle try to maximize average team skill then players will
    mostly play on teams where they are at their best (what Mofo is asking for).

    Wasn't smallest skill difference between teams with smallest variance the target for the skill solver? [i might be wrong here]
    You wouldn't have to make maximum skill on both teams the target.
    As i look at it two separate skill values would make for a much larger solution space enabling better quality of shuffle.
    (at the expense of more computational resources.)
  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    While annoying at first, wouldn't that be a good thing in the long run?
    Wouldn't additional play time increase their capabilities and eventually lead to a more equal skill level between the two teams for that player?
  • barniebarnie Join Date: 2016-07-26 Member: 220695Members
    I see the problem with the drag towards the average skill level, and there are no simple solutions to that problem, but its not impossible.

    Forcing players to play on their weaker side most of the times would lead to frustration.

    I would just go with a preference setting, giving flexibility to the players.
    Vote shuffle went through -> menu pops up: "team preference: [Aliens | dontcare | Marines]"
    Note that its preference and not guarantee that you going to end up in that team.

    Other then that you could come up with complicated schemes, for example shuffling them to roughly 50/50 playtime in both factions etc.

    Yes doing any of the above will likely increase the standard deviation of the shuffle but by how much?
    Gamebreakingly much?
    Played some really high std-dev games with a prem div comp player + a bunch of noobs vs. a team of average joes.
    These rounds endet up surprisingly not terrible.
    Its all fun and games until a noob takes the chair.
  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    Games with high standard deviations are not guaranteed bad, and neither are games with low standard deviation guaranteed good. Lower standard deviation of hive skill just means teams are nearer in skill. Near skilled games are likely to be more balanced. Balance is never a guarantee.
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    I'm thinking the most fair method is to compute the average marine skill and average alien skill for the server, then try to match each team against those averages. That prevents the weird behavior when everyone is better at one team than the other.
Sign In or Register to comment.