Concern: Skill ranking for commanded games is always zero

AdambeanAdambean Cardiff, South Wales Join Date: 2005-06-03 Member: 53038Members, Constellation, Reinforced - Shadow
Discussed already? Don't know, this was a hard one to search for... what with "hive" and "commander" being used for alien commanders in general.

I have a concern about the Hive rankings for games played as commander. If you command a game whether you win or lose, you will be given a skill ranking of zero for these games. This will quickly cause your skill rating to plummet if you command often, giving the appearance of an inexperienced or bad player when in fact you may be experienced / good. (in my case just experienced -- certainly not good)

I guess the skill for each round is calculated by your score, kill count, death count, and kill:death ratio. Maybe not all of those. Obviously it would be ridiculous to make some sort of prediction of what skill rating a commander should get, or take some form of average from the players in the team. It doesn't help that commanders get zero score throughout all games (apart from sudden base defence kills and sentry/whip kills), not even a score for structures placed (then completed building) or ARC kill points. What may be better is to exclude all commanded games from a player's overall skill rating all together. Just display a hyphen '-' against those rounds instead of zero.

There is also the issue of how do we determine the commander as you can just switch. NS2 Stats seems to track this fine. It may be fair to assume that if you've been a commander for at least 33% of the round time, you're marked as a commander for that round. Even if there were more than one of you. In most games the commander doesn't leave the chair/hive for 10% of the game time, let alone a third of it.

Why would anyone care about the original point? The problem lies in a couple of places.

1) The server browser: This displays an average skill rating of those players present (after completing connection and being "in the world") -- this display is impaired as good/experienced appear as bad/inexperienced prompting more newer people into a game they think they are suited for.

2) Balance modifications: I've seen a couple of servers now using a skill balance modification using the Hive ranking data. This is in an effort to stop stacking by balancing teams based on accumulative skill rating. Not perfect, but better than just randomizing players. This is again impaired as I appear to have low skill then get "balanced" onto a team with higher skill people, and these plug-ins prevent you from switching teams manually. Build 263 now uses Hive ranking data in the same way for its' new balance vote.

3) Inaccurate representation: The least concerned part. You simply look bad on the Hive rankings, and makes all of the data less meaningful. Stats are rarely perfect, but this is still something that can be considered to make them slightly more respectable.

Also I'm not sure how true this is but I've read somewhere here that the current Hive ranking system is incomplete and going to be wiped clean/restarted once finalized? True or not the original point here should still be considered, especially as the data is tied in with the game on multiple occasions now.



Here come the flames.

155283579.jpg

Comments

  • AsranielAsraniel Join Date: 2002-06-03 Member: 724Members, Playtest Lead, Forum Moderators, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Shadow, Subnautica Playtester, Retired Community Developer
    Ignore the skill at the moment, it is being reworked. Only level works correctly.
  • AdambeanAdambean Cardiff, South Wales Join Date: 2005-06-03 Member: 53038Members, Constellation, Reinforced - Shadow
    That's what I suspected, thanks. :)
    Is the level rating simply an accumulation of score? (nothing to do with kills/deaths directly)
  • AsranielAsraniel Join Date: 2002-06-03 Member: 724Members, Playtest Lead, Forum Moderators, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Shadow, Subnautica Playtester, Retired Community Developer
    Level is simple playtime. It gives a first idea of player "skill". The skill rating, once the formula is reworked, will then give a more precise idea of player skill, to make it easier to create balanced teams (for example with the new "force balanced teams" vote introduced in 263)
  • _INTER__INTER_ Join Date: 2009-08-08 Member: 68392Members, NS2 Playtester, Reinforced - Shadow
    The only way this gonna work if the skill is based on win:loose ratio. Nothing else, KDR means next to nothing in NS2.
    => Proposal for abuse-proof skill system
  • CyberKunCyberKun Join Date: 2013-02-02 Member: 182733Members, Reinforced - Shadow
    Skill ranking is horribly screwed still.

    I command a ton and I have noticed this.
  • AdambeanAdambean Cardiff, South Wales Join Date: 2005-06-03 Member: 53038Members, Constellation, Reinforced - Shadow
    I concur Inter, Win:Lose is a better ratio to work with than K:D.
  • dusterduster michigan, US Join Date: 2014-01-19 Member: 193329Members
    there are like 3 devs working on ns2 and they are working on stuff other than hive stats so it's pretty much irrelevent now.
  • CyberKunCyberKun Join Date: 2013-02-02 Member: 182733Members, Reinforced - Shadow
    Meh. Just make anyone that commands for the majority of the game gets the same skill ranking as the highest 75% skill a teammate will get.
  • SUPER_SARSSUPER_SARS Join Date: 2013-02-13 Member: 183039Members
    CyberKun wrote: »
    Meh. Just make anyone that commands for the majority of the game gets the same skill ranking as the highest 75% skill a teammate will get.

    Or make it an average of what the team received during that round. Since the commander is in charge of their team...
  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    duster wrote: »
    there are like 3 devs working on ns2 and they are working on stuff other than hive stats so it's pretty much irrelevent now.
    You severely underestimate @Acedude and his ability to rock it.
  • RoobubbaRoobubba Who you gonna call? Join Date: 2003-01-06 Member: 11930Members, Reinforced - Shadow, WC 2013 - Shadow
    edited February 2014
    @ironhorse if @acedude has read and is implementing the suggestion from @moultano, then we may be okay. Anything else is, unfortunately, going to be short of the mark.
  • CCTEECCTEE Join Date: 2013-06-20 Member: 185634Members, Reinforced - Shadow
    _INTER_ wrote: »
    The only way this gonna work if the skill is based on win:loose ratio. Nothing else, KDR means next to nothing in NS2.
    => Proposal for abuse-proof skill system

    Ok consider this:
    Proplayer (win/lose = 7:3) joins game.
    Noobplayer (win / lose 3:7) joins game.
    ->
    Shuffle based on win/lose ratio
    ->
    Proplayer's chance of winning the game: 50% (fair teams)
    Noobplayer's chance of winning the game: 50% (fair teams)
    ->
    Game ends, data send to Hive.
    ->
    Repeat x 100 games.
    ->
    Proplayer win/lose: 5:5
    Noobplayer win/lose: 5:5

    So the trouble with this pure win/lose ratio as a base for the system is that the system will fuck up itself (its own data) by making the teams fair.
    Or is my logic flawed?
  • moultanomoultano Creator of ns_shiva. Join Date: 2002-12-14 Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
    CCTEE wrote: »
    So the trouble with this pure win/lose ratio as a base for the system is that the system will fuck up itself (its own data) by making the teams fair.
    Or is my logic flawed?
    Your description is missing one key bit. The skill levels change proportionate to how much the prediction of the outcome of the game differs from reality. So if the teams are perfectly even, and the outcomes are also perfectly even, everyone ends up with the same skill they had at the beginning.
  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    Weighting systems seem like a nightmare to design
  • halfofaheavenhalfofaheaven Join Date: 2012-11-09 Member: 168660Members, Reinforced - Supporter, Reinforced - Gold
    You can't automatically put a number on skill in this game, because awareness and communication is everything. Basing it on win/lose ratio is BS as well, because lots of good players will anti-stack for a challenge and naturally lose a lot of their games. The only way this skill system would ever stand any chance to work is if players were able to rate other players non-anonymously. At the very least in addition to a semi-reliable automatic system.
  • TheriusTherius Join Date: 2009-03-06 Member: 66642Members, Reinforced - Shadow, WC 2013 - Supporter
    I do not understand how a W/L based score would work any better than the current one in NS2.

    I do respect statistics, but I refuse to believe that your W/L stats will even out to your 'true level' after enough observations simply because the player always gets to choose whether to stack or anti-stack. In effect, the player gets to choose whether to handicap themselves or the opposing team. A good player always anti-stacking would end up with a lower W/L ratio than a player of equal skill who always stacks.

    How would you make it work? An Elo-type scoring with your personal score compared to the average of the opposing team, or your team's average score against that of the opponents?
  • ZinkeyZinkey Join Date: 2013-06-25 Member: 185694Members, NS2 Playtester, Reinforced - Onos, WC 2013 - Shadow, Subnautica Playtester
    Therius wrote: »
    I do not understand how a W/L based score would work any better than the current one in NS2.

    I do respect statistics, but I refuse to believe that your W/L stats will even out to your 'true level' after enough observations simply because the player always gets to choose whether to stack or anti-stack. In effect, the player gets to choose whether to handicap themselves or the opposing team. A good player always anti-stacking would end up with a lower W/L ratio than a player of equal skill who always stacks.

    How would you make it work? An Elo-type scoring with your personal score compared to the average of the opposing team, or your team's average score against that of the opponents?

    Because the skill gained is weighted based on the teams and the expected outcome. I.e losing against a team with higher average skill and in turn a higher chance of winning results in a lower/negligible skill loss as they were expected to lose. By the same argument winning a game you are expected to win will result in less/negligible skill gain as you were expected to win. So stacking/anti stacking is irrelevant.



  • RoobubbaRoobubba Who you gonna call? Join Date: 2003-01-06 Member: 11930Members, Reinforced - Shadow, WC 2013 - Shadow
    @halfofaheaven and @Therius

    You should read Moultano's original thread to answer those points.

    Really, the theory works beautifully. So much better than any other implementation I've seen. Now we just need to test it on a live system. It does need a bit of tweaking as regards the commander role, but I think Moultano said he was thinking about that already.
  • TheriusTherius Join Date: 2009-03-06 Member: 66642Members, Reinforced - Shadow, WC 2013 - Supporter
    Seems solid enough. I'll take your word for it.
  • AdambeanAdambean Cardiff, South Wales Join Date: 2005-06-03 Member: 53038Members, Constellation, Reinforced - Shadow
    The description of Moultano's formula sounds rather good, but the AFK player can be a slight concern. This is why NS2 needs to automatically ready-room players after idle time, then kick for further idle time as a vanilla feature, not with a mod.
  • meatmachinemeatmachine South England Join Date: 2013-01-06 Member: 177858Members, NS2 Playtester, NS2 Map Tester, Reinforced - Shadow, WC 2013 - Supporter
    If team is victorious
    commander.playerlevel = commander.playerlevel + 1
    else
    if team loses
    commander.playerlevel = 0
Sign In or Register to comment.