In the end… what is skill in NS2 ?

2»

Comments

  • TheriusTherius Join Date: 2009-03-06 Member: 66642Members, Reinforced - Shadow, WC 2013 - Supporter
    krOoze wrote: »
    Therius wrote: »
    It's already been proved. Look at the goddamn paper that's even been linked in this thread for your convenience.
    I did. It contains definitions, claims and relatively dangerous assumptions, no proofs. I mean its still nice work. It is slightly better than random and way better than people assigning themselves to teams in practice. But it simply is not the thing you present it to be.

    You say it does, yet you continuously refuse to point out what these dangerous assumptions are or where the paper fails. You say you own a sports car but when asked to show for it you refuse to open the garage. Or better yet, you claim that you see rust in someone else's sports car, and when asked where the rust is because no one familiar with the sports car sees it, refuse to point it out, instead saying "your sports car isn't as unrusty as you think it is".

  • krOozekrOoze Join Date: 2014-04-24 Member: 195593Members
    edited March 2015
    @Nordic Sorry 'bout that, I think I am partly at fault turning this healthy thread into philosophical nonsense. I can't stop myself, when I see some of those extraordinary claims and them being said with such cofidence. Help meeee! :)
    SantaClaws wrote: »
    The truth claims made by these systems however, are all true and false with or without man.
    How would you know since there IS a man. But for the sake of fighting anthropocentrism (and preventing someone verifying it with nukes), lets assume thats true.
    Nordic wrote: »
    Ad hominem. HA! That makes me laugh and if the discussion here wasn't so serious I would think it was trolling. The best trolls are the ones who seem 100% serious though.
    Well he DID bring out the formal education level into the discussion. Whats worse, he missed. But I was not that serious, was I? - if anything my retort to that one was rather merry, no?
    SantaClaws wrote: »
    In a universe with no minds, the truth remains that a rock is still a rock.
    "rock" is but a symbol for symbolic analysis (read as "word in english language people communicate rock-like ideas with"). As for reality, you can make an object that 50% people would classify as "rock" and other 50% would not. So is it a rock still or not then? :P Ohh, I kinda like formal language theory and analytical philosophy :)

    PS people look up what the word "Mathematics" mean - it's greek. Saying something like "universe is made of mathematics" or "there was mathematics before man" is linguistical nonsense.
    Nordic wrote: »
    Decibel is a non linear measurement, where 80 decibels is not double the loudness 40 decibels.
    Human hearing and sight sensitivity are not linear either. By your table "ordinary conversation" would be 30x louder than "quiet conversation".
    I agree with you though, that the SPs need to be hidden - esspecially if you agree thay are undecodable for end user. Showing them only leads to bad behavior like avoiding FET, doing multiple FET in row untill eb is happy, F4ing and stat hording and being otherwise obsesed by ones stats.
    dragonmith wrote: »
    But does 0.99 recurring equal to 1?
    Yes, for real numbers? I think that one even has its own wikipedia page. :) Counter question: does 1+2+3+4+5+... = -1/12

    @Therius I did. You even said it's being fixed. You claimed before, it is perfect and unbiased and unfallible. I proved you wrong, even when burden of proof was not on my side. You addmited it had an error. I am perfectly happy with that. I may do line by line proof-reading later, with exhaustive list of counterexamples to your claim, but I am also limited by time and actually need some to PLAY NS2 too :) . Also I am not necessarily the best man for the job, but I assume every set of eyes help.
  • [AwE]Sentinel[AwE]Sentinel Join Date: 2012-06-05 Member: 152949Members
    'What is life? What is skill? The head of a dead cat.' - ARHAT, fictive shaolin monk; adapted quote.
  • krOozekrOoze Join Date: 2014-04-24 Member: 195593Members
    edited March 2015
    ^^^ That insults Hanumán, the monkey-god, who thought the sun is a ripe mango! - fictive ind shopkeeper
  • TheriusTherius Join Date: 2009-03-06 Member: 66642Members, Reinforced - Shadow, WC 2013 - Supporter
    krOoze wrote: »
    @Therius I did. You even said it's being fixed. You claimed before, it is perfect and unbiased and unfallible. I proved you wrong, even when burden of proof was not on my side. You addmited it had an error. I am perfectly happy with that. I may do line by line proof-reading later, with exhaustive list of counterexamples to your claim, but I am also limited by time and actually need some to PLAY NS2 too :) . Also I am not necessarily the best man for the job, but I assume every set of eyes help.

    This has devolved into a test of reading comprehension. I have never said the system is perfect, I am well aware of the problems within it, as I have stated multiple times in my posts in this and the other thread. My main argument so far has been that the problems within the system are not related to the metric that is used to determine the skill level, i.e. the win-lose ratio. This is something I have said close to a dozen times so far.

    Problems like no separate skill score for marines and aliens and the distribution of player skill within a team not being taken into account (i.e. is the arithmetic mean a good statistic to determine the skill of a team) are something we should discuss and create solutions for. My main peeve is what many people are advocating for and what the original post of this thread was fully dedicated for; that the win-lose ratio is a bad metric to base the skill score of a player on and that we should instead adopt a system where red players are better than blue or whatever. This is simply not true and stems from the psychological bias of players not being able to accept that their skill level is not going up even though they keep farming the scoreboard.

    As for mathematics, without getting too philosophical, my main problem is with people who say things like "you can do your maths while I live in the real world", and then proclaiming that their experience and opinion is more valid than mathematics or statistics. Ridiculous.

  • krOozekrOoze Join Date: 2014-04-24 Member: 195593Members
    Allrighty. You like the w/l metric, we get it. It should have stopped there, only if you haven't started making some of those dubious claims, for which it is hard not to offer some opposing opinion. Also I don't remember ever bashin usage of w and l for that purpose, but be warned I might in the future :P
    You understandably confuse it with that other thread (we have too many skill and balance threads). Before we derailed it, this thread was actually about some visible ranking, by which commander or other skilled teammembers could lead other people in battle, or perhaps even pick people into the team. Which is interesting idea. It's something one would find in classical RTS (unit has strength, speed, visibility distance, skills and so on on which the general builds his strategy).


  • NordicNordic Long term camping in Kodiak Join Date: 2012-05-13 Member: 151995Members, NS2 Playtester, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, Reinforced - Shadow
    edited March 2015
    The OP is the one who was primarily saying that W/L is not a good enough parameter. This thread I believe was him saying what he thinks a skill system should look like instead. I am not sure what you think is wrong with it krooze. You just seem to like to argue but I might of missed something.

    Some seem to generally imply that the math does not match with reality. How does it not match with reality?
  • krOozekrOoze Join Date: 2014-04-24 Member: 195593Members
    edited March 2015
    I suppose he can have ulterior motives, but I haven't seen anything about W/L in the OP.
    Yeah, I mostly like to argue when someone makes dubious claims, yet is so confident about it. I have no problem with people that like the system. I did not press any of my ideas in this particular thread, just oposing to some myths so far.

    But alright, up to date sum-up of what is wrong with it and some suggestions:
    1) social problems:
    • People still avoid FET, force FET multiple times, do not vote for FET, F4, swich teams, bitch about it, stack by any and all possible means and prolong pregame to a length almost equal to the game itself (based on my Hive and Steam stats). And don't tell me again it is the fault of the people. It is, but they are not gonna change, it is who we are -> the system must budge.
    • On the other hand some people claim the system is perfect, unbiassed, correct, proved, mathy, verified, undeniable, innocent untill proven guilty - that sort of thing, only making the people in the first group do their thing even more to spite them. The author himself does not make such claims in his article. He uses the word "useful" to describe the system, nothing more.

    2) user facing interfaces problems:
    • On some servers the avg skill and personal SP nonsense value is shown making 1) even worse. -> Maybe show the % probability of win, the algoritghm is making, if anything. Everyone knows what that one means.
    • The Skill Points are not user decodable, making showing it on Hive server useless to most people, other than cause more complaints (regardless of them being right or wrong).
    • The people are detered to use FET (because it's opt-in), and it is often hard to get that vote passed (guilty myself - I am alt-tabed or on toilet, I don't need that RR wait).
    • Some people are afraid of FET, because it kept the old name. They perhaps didn't even catch there was a algorithm change (arguably for the better).
    • The game does not inform the player he's making a preference by joining a team. I assume going Random still counts as preference for the side you end up with, which would be outright error - to save me a search: is the system exposed in the readable lua scripts, so I can see what is truly done?
    • The game does not allow player to make preference because of the join limit -> the preference should be perhaps made by other means (options?, simply asking the player in RR?)
    • FET does not consider AFKs + The vote is short and perhaps has low visibility considering its importance -> if we wanna keep the vote, perhaps improve visibility, kick anyone who does not vote f1 nor f2 (with due warning of course).
    • The algorithm according to the article does something separately with commander skill. It would be nice to have some commander balancing system, or at least some way to know we have comparable comms. Some way to expedite comm choosing would be nice too.
    • (Also I hate Random not being random, but that's only loosely tied to this ranking+balancing subsystem.)

    3) statistical problems -
    • Skill for marines and aliens is not separate. If you do good on one side and suck at the other(common thing - people know some FPS but suck at alien or vice versa or do not like one of the sides at all and are not playing it), then your Skill has high error for you for both sides you play.
    • The system assigns unreasonable value to new players. The system reset assigned unreasonable value.
    • The system essentialy rewards picking the right team - it is a lottery at best, Speculation warning: or at worst the alleged people uncontrollably gaining/losing SPs potentialy (subconsciously) found a way to predict it better than the system. Your Skill is how often you happen to be on the winning side + calculated odds you face (irregardles if the team won despite you or because of you). I wish the article was a little more clear as to when to whom the Skill Points are being given. The probability as computed is sort of a "time spent in game"-weighted average of all players' Skill that joined and exited the game. Do you get reduced SP if you F4 before losing? If I folow it correctly, you are actually rewarded for exiting lost game before anybody else.
    • It is ELO based - it essentialy ranks teams. But the teams change on NS2 pub games radically and the SPs are distributed equally among team-members irrespective of their merit. At this point I need solid proof the algorithm converge to players real skill. It may as well also work like this: new players get nonsense value -> are assigned to team by probability based on the nonsense value -> get SP based on the nonsense probability -> repeat.
    • It seems to be a sort of a zero sum game, by the skill update function. For one that makes SPs untransferable from server to server(every server has a different SP capital, if it has too many regular players). And secondly SPs are actually made from nothing when you win against 0 SP player. Won't the SPs experience inflation because of that? Won't they experience deflation when new players come to the game with 0 SP and start earning them.
    • As I said before, there is many unverified assumptions in the article. There are also some implications that seems non-sequiter to me at the moment, but I assume the author knows well what he is doing, untill I do a serious read of that. So one can believe in the system or not.
  • TheriusTherius Join Date: 2009-03-06 Member: 66642Members, Reinforced - Shadow, WC 2013 - Supporter
    From where I stand, I haven't made a single dubious claim and you haven't said a single thing to counter them. But that conversation bears no fruit, and I will stop here.

    You make valid points. Some points in your "social" and "interface" lists I disagree with, but those are matters of opinion and I do not really know any better so I'm not going to concentrate on those. For example, from my experience and opinion, the FET is used almost every single game, and if someone switches to debalance it, people vote it again. And I would hate to see people being kicked if they do not vote in the FET vote, that's just way too harsh.

    Anyway, I do have some comments on your "statistical problems":
    krOoze wrote: »
    Skill for marines and aliens is not separate. If you do good on one side and suck at the other(common thing - people know some FPS but suck at alien or vice versa or do not like one of the sides at all and are not playing it), then your Skill has high error for you for both sides you play.

    Agreed. Even though the model still statistically speaking predicts your chances of winning correctly ON AVERAGE because your tendency to join either faction is factored into your skill rating, in effect it forces you to play imbalanced games. Every other game is a cakewalk (you join your stronger team while the system thinks you're much weaker) and every other game is a hopeless uphill battle (you join your weaker team and the system thinks you're much stronger). Giving separate scores for marines and aliens would do much to solve this problem.

    krOoze wrote: »
    The system assigns unreasonable value to new players. The system reset assigned unreasonable value.

    Why do you think it's unreasonable? The system has no idea what the skill level of new players is, so they have to start somewhere. Of course they will be a problem for the first games they play, since their skill rating is not yet accurate, but it will start converging on their actual skill level rather quickly. How would you go around this? There is absolutely no way for a quantitative system like this to give newcomers any other value than some predetermined one. An obligatory interview before your first game?

    krOoze wrote: »
    The system essentialy rewards picking the right team - it is a lottery at best, Speculation warning: or at worst the alleged people uncontrollably gaining/losing SPs potentialy (subconsciously) found a way to predict it better than the system. Your Skill is how often you happen to be on the winning side + calculated odds you face (irregardles if the team won despite you or because of you).

    The system does not reward picking the right team and is not a lottery. Your second sentence is the entire point of the skill system; the system calculates your score based on how many times you win and the odds of you winning those matches. If you have a greater chance of winning, you gain less points from winning and lose more points from losing, so a player always picking the 'right' team and having a 90% chance of winning will still converge towards the same skill number as an equally-skilled player who always picks the wrong team and has a 10% chance of winning. Because the first player will only gain 1 point per win and lose 10 points per defeat, while the other player will only lose 1 point per defeat and gain 10 points per win. The net result is the same.

    Now comes the beauty of it: if your skill rating is wrong, the system will automatically balance it. Because if your score suggests that you should win 50% of your matches, but it thinks you're weaker than you actually are and, in reality, you win 60% of your matches, then in the spirit of the example above, the points you lose from defeats no longer offset the points you gain from the victories. Your skill level will go up. It will continue going up until you reach a level where the skill system predicts that you win 60% of your matches, the prediction will be correct, but your skill level has gone slightly up to represent your actual skill level.

    krOoze wrote: »
    I wish the article was a little more clear as to when to whom the Skill Points are being given. The probability as computed is sort of a "time spent in game"-weighted average of all players' Skill that joined and exited the game. Do you get reduced SP if you F4 before losing? If I folow it correctly, you are actually rewarded for exiting lost game before anybody else.

    What do you mean 'to whom'? It's all there. Points are being given to everyone who wins, according to the probability the system calculated for the odds of them winning that match. Likewise, everyone who lost will lose points according to the odds of them losing that match. The specific formulas to see how many points are being added/deducted are there in the article.

    The time-spent in-game is one feature that could be problematic, I agree. I can see the reasoning behind it, but it's one thing that's based on opinion. Who says that the first minutes of the game has more impact than the rest? Sounds reasonable, but this is just something that has been decided by one person, not really a statistical fact.
  • krOozekrOoze Join Date: 2014-04-24 Member: 195593Members
    edited March 2015
    Therius wrote: »
    But that conversation bears no fruit, and I will stop here.
    Agreed, I said my peace anyway. There's no point in repeating ourselves on points that are in written form recorded here for eternity, if no one but the two of us actually cares.
    Therius wrote: »
    And I would hate to see people being kicked if they do not vote in the FET vote, that's just way too harsh.
    They will be kicked anyway as AFK (unfortunatelly after 5 mins which is somewhere in middle of the game). Of course there would have to be big ugly red writing of what happens when you do not vote on that. And its not harsh, it's not like they are being banned or something, they can join again as they become available again. But for the sake of conversation, what about assigning them to Spectator instead? And of course making FET the default mode would remove the need for such harsh treatment (though AFKs still need to be handled well in some way).


    Therius wrote: »
    Even though the model still statistically speaking predicts your chances of winning correctly ON AVERAGE
    Yes. We discussed it before. Average is not good enough. On average every person on earth have half a weenie and half a VG.
    Therius wrote: »
    krOoze wrote: »
    The system assigns unreasonable value to new players. The system reset assigned unreasonable value.

    Why do you think it's unreasonable? The system has no idea what the skill level of new players is, so they have to start somewhere. Of course they will be a problem for the first games they play, since their skill rating is not yet accurate, but it will start converging on their actual skill level rather quickly. How would you go around this? There is absolutely no way for a quantitative system like this to give newcomers any other value than some predetermined one. An obligatory interview before your first game?
    I assume they get 0 SP or 1000 SP, based on what has been told. I failed to find it in the article and haven't yet dug through the lua scripts. Those does not strike me as some estimate of average of newcommers (while the system can get an empirical value based on the statistics it collects). Those slow down convergence, and if large number of population had them - well I simply would need more explanation how the system is able to converge, if everyone has wrong Skill.

    Therius wrote: »
    krOoze wrote: »
    The system essentialy rewards picking the right team - it is a lottery at best, Speculation warning: or at worst the alleged people uncontrollably gaining/losing SPs potentialy (subconsciously) found a way to predict it better than the system. Your Skill is how often you happen to be on the winning side + calculated odds you face (irregardles if the team won despite you or because of you).
    the system calculates your score based on how many times you win and the odds of you winning those matches.
    It does not. It calculates based on how many times your team wins. You change team after the round, making the previous calculation invalid for the new case. At this point I need explanation of realtion of one indidual person skill and team wins and why would the system converge instead of diverge, when people are rewarded as a large group and started with unsensible SP value when the system were put in place.
    Therius wrote: »
    krOoze wrote: »
    I wish the article was a little more clear as to when to whom the Skill Points are being given. The probability as computed is sort of a "time spent in game"-weighted average of all players' Skill that joined and exited the game. Do you get reduced SP if you F4 before losing? If I folow it correctly, you are actually rewarded for exiting lost game before anybody else.

    What do you mean 'to whom'? It's all there. Points are being given to everyone who wins, according to the probability the system calculated for the odds of them winning that match. Likewise, everyone who lost will lose points according to the odds of them losing that match. The specific formulas to see how many points are being added/deducted are there in the article.
    For example F4 person is explained unclearly there for me. Is he still computed into the probability (seems yes). Does he still win or lose when he left before the victory? Is he ment to get reduced penalty for leaving lost game early (it is computed as "his time"/"sum of all players time in game untill victory" - his penalty will quickly be reduced to zero by that ratio or is the sum stopped too when he leaves?). Is his SP calculated from probability of win when he left or the probability at the end of the game? There also comes the dangerous assumption that the time at beginning counts more that at the middle of the game and at the end (Oftentimes time at beginning is not and time at middle is, or when the game "restarts"=people destroyed their main hive and cc, it behaves like the beginning).
    I too can see the reasoning, but it conflicts with you constantly claiming the algorithm is unbiased. That decision is simply the authors bias of how things work in the game.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Therius wrote: »
    ... My main peeve is what many people are advocating for and what the original post of this thread was fully dedicated for; that the win-lose ratio is a bad metric to base the skill score of a player on and that we should instead adopt a system where red players are better than blue or whatever. This is simply not true and stems from the psychological bias of players not being able to accept that their skill level is not going up even though they keep farming the scoreboard.
    ...
    I'm afraid you have the wrong picture. It isn't binary at all. A tip: Paper, Rock, Scissors.

    It's not about a skill score.

    The assumption that the Skill number used by FET is right, perfect blah blah. Maybe yes. But i would restrict this to the CS or COD kind of games. In these games, it's more or less the same units (in both teams BTW), same weapons and it's relying on the same set of behavior. Basically a "Drone" game.

    Those games in which people organize to a point they play like robots executing a set of instructions (move left, wait 1 secs, throw a grenade if you hear noise, move forward...). Kind of predictable strong moves. You can even talk about Nash.

    Then the ultimate tactic against that is A/ to accept this and do the same way (in the end flipping a coin) or B/ to provide a random behavior that breaks the so precisely tailored and optimized script (flipping a bigger coin). A gambit or some kind. This tends to bring entropy and lots of fun from the logical point of view... Building statistics... predictions... hahaha... good times...

    But this game is not a standard shooter game. It's N-S-II (straight from the Hybrid FPS/RTS ghetto! (haha)). Even if you see some routines (i see the trolls coming); it's far more dense than any other game i could lay my eyes on. But not on the player base number unfortunately.

    By being so extensively providing "options" (see other games); no RED player is better than the rest of them (more or equally Green or Blue inclined... whatever). I just need a tunnel to perfectly align 12 eggs and their faces. That's the beauty of NS2. Isn't it ? ... What are the odds ?? Hmm, Yes... What are the odds ??
  • krOozekrOoze Join Date: 2014-04-24 Member: 195593Members
    edited March 2015
    It isn't binary at all.
    Everyrthing is binary. :P Sorry, my occupation caused demetia...
    It's not about a skill score.
    That's what I thought. Again sorry for stealing your thread.

    The assumption that the Skill number used by FET is right, perfect blah blah. Maybe yes. But i would restrict this to the CS or COD kind of games. In these games, it's more or less the same units (in both teams BTW), same weapons and it's relying on the same set of behavior. Basically a "Drone" game.
    The Skill number is still necessary and IMHO unless SPs get human-readable form should be used exclusively for: balanced team making. Evolution unfortunately prefers stackers, so we can't make do without that. That is unless you propose some way to create balanced(or fair) teams using your system (e.g. comms picking which players want based on that).
    NS2 is partly guilty of being that kind of game (you do have same weapons). But at least you fight over structure, you know why and not some flag or stupid building you don't even care about.
    Then the ultimate tactic against that is A/ to accept this and do the same way (in the end flipping a coin) or B/ to provide a random behavior that breaks the so precisely tailored and optimized script (flipping a bigger coin). A gambit or some kind. This tends to bring entropy and lots of fun from the logical point of view... Building statistics... predictions... hahaha... good times...
    You do not increase unpredictability by going against weaker opponent, but the opposite. The trick is to make so perfectly balanced teams, that you have to often defeat your opponent by surprising and cunning tactics -> fun. The system is not really predicting - it's attempting to distribute the risk fairly among the two teams. But I have my doubts, that it's actually succeeding as it is implemented - I am tempted to write a simple simulator to test a bit the mathematical part of the problem... But you should get some respect for statistics. It can predict well e.g. how cities will grow. With some effort it can model NS2 game too with sufficient precision (or trick us into thinking it does).
    By being so extensively providing "options" (see other games); no RED player is better than the rest of them (more or equally Green or Blue inclined... whatever). I just need a tunnel to perfectly align 12 eggs and their faces. That's the beauty of NS2. Isn't it ? ... What are the odds ?? Hmm, Yes... What are the odds ??
    That's what I thought you ment. The RED, BLUE and GREEN players are not comparable. In classical RTS they would be different units for all intents and purposes.
    The odds are dependent on how inteligent and experienced your players are to even think of making that sneaky tunnel and group-up and gorge-up?

    PS: I entertained your idea, but your system pretty much make decisions on KDR and play-hours. In practice, that would be disaster (BTW, isn't that how the old system worked? ).
Sign In or Register to comment.