Quantifying skill!
draktok
Join Date: 2013-02-18 Member: 183156Members
Join Date: 2013-02-18 Member: 183156Members
This is something that was briefly touched on another thread so I think it deserves its own thread because it's an interesting topic.
So, lets get down to what this thread is going to mostly be about: Can you put a number on skill?
The short answer, is yes. You can. That's just the truth. Many people might say it's never going to be quite accurate and you can't quantify something in such a complex team game, but the point is there is better players than others, and you actually can give a number of how much better one person is, than another.
Lets look at a basic basic example. Chess uses ELO. The Elo rating system is a method for calculating the relative skill levels of players in such competitor-versus-competitor games as chess. (Wikipedia definition). At its core, when you win, you take points from the loser. Someone who is much higher rated than the opponent will take less points in victory, and lose more in defeat. This ELO rating has been adopted in many ways, most recently at a large scale in League of legends. League of legends is probably the most well known esport team game to date, and although it's a slightly different rating system now, it is still a play off of ELO. Despite criticisms that rewards bad players when they have good enough team mates, due to the law of large numbers it mostly serves its purpose at a fairly (although not perfect) rating system on players. Obviously this system will be off in shorter samples, and due to pure luck can also place someone lower or higher than they truly are, but it is still a valid way to rate skill. NS2 has the great advantage of NOT queueing people in skill match ups, and if someone is given a lower or higher skill rating than their true skill isn't going to actually effect them as a player like it can in League of legends, or starcraft 2.
Ns2 is a little more complex, of course. A rating system that works in League, or starcraft 2, albeit both team games wouldn't exactly work in NS2 since NS2 has a much different style of play. The game is only an rts for one person on each team...and that person usually doesn't exactly have a strong score due to playing an rts, not a shooter. Due to not having skill match ups and instead server drop in, it's even more difficult to track skill efficiently. I don't believe this means it's impossible. Assuming that it wouldn't work because the common way to track someones skill isn't necessarily valid in NS2 doesn't excuse NS2 from trying. And they have tried, to a point. The game does officially track skill now, however halfheartedly it does so.
One thing that needs to happen in order to allow people a quantifiable number in skill is the number of servers allowed to track skill needs to be increased, probably ten fold. Right now there is VERY few servers that actually track the current implementation, and half those servers are seldom played due to not having a number of convenience mods installed to help the server (by the way, map voting should be a thing on vanilla servers). Besides that, I think people need to realize that score is honestly a great way to tell how effective someone is being at helping the team. People also need to realize that some things reward too little, and too much in some instances which can throw off someone's score immensely. One person that goes 1 kill, 8 assists and 27 deaths but still maintains a 3rd place score by running around and being a MAC isn't actually helping as much as someone like to pretend it does. Because believe it or not, in smaller scale matches, you DO need to be able to aim your gun. On that same note, someone going 30/1/1 isn't helping as much as people pretend when they are playing Team Death Match, and not helping the tide of the battle by killing the same skulk in hub all game.
I see no reason for score to not be cleaned up a little bit to properly reward players for how effective they are in game, and to allow that to directly correlate to skill. This game is played at a much LARGER per match scale than league of legends and starcraft 2 is, and I don't believe basing your overall skill based on your W/L ratio would even come close to where we need to be for NS2. I also don't believe that half assing the skill system serves any purpose, and there is no reason why it can't be cleaned up to allow for fair teams to be made when necessary based on that number. On top of that, each player needs TWO skill ratings. Even if competitively teams play aliens and marines, in pubs, a lot of players only play one race. There's no reason to not "skill track" Marines and Aliens play. Allowing for skill to truly be measured couldn't hurt the game in any way. We don't have a ladder, so if players want to "pad" their numbers, who cares? Competitive is a total 180 in terms of true skill since the game is actually team based, so that wouldn't be impacted either by allowing skill tracking in singular / pub play.
I've never seen a real reason posted by anyone to not allow true skill tracking besides "it isn't measured correctly!!1!11one!". Due to law of large numbers and skill not directly impacting match making, if we were to properly quantify skill based through an average score it would actually eventually show you your true skill. So, that's just a bad complaint. Obviously any skill measurement system, even at its basic core of ELO being used in chess can be flawed. Did you know ELO was also created to be a slightly more accurate way to show your true rating because the previous one was slightly inaccurate, and most major chess leagues use a "type" of elo rating, not true elo rating, because things can be adjusted to fit other games?
Does anyone else have an opinion on skill rating besides "INACCURATE AHHHH" and "JUST MEASURE YOUR **** WHILE YOU'RE AT IT!"? I would like to have a real discussion.
all edits are spelling / grammar errors cause ingles is hard. I think i fixed a lot of its problems for people to understand the paragraph since ingles is hard for me and others
So, lets get down to what this thread is going to mostly be about: Can you put a number on skill?
The short answer, is yes. You can. That's just the truth. Many people might say it's never going to be quite accurate and you can't quantify something in such a complex team game, but the point is there is better players than others, and you actually can give a number of how much better one person is, than another.
Lets look at a basic basic example. Chess uses ELO. The Elo rating system is a method for calculating the relative skill levels of players in such competitor-versus-competitor games as chess. (Wikipedia definition). At its core, when you win, you take points from the loser. Someone who is much higher rated than the opponent will take less points in victory, and lose more in defeat. This ELO rating has been adopted in many ways, most recently at a large scale in League of legends. League of legends is probably the most well known esport team game to date, and although it's a slightly different rating system now, it is still a play off of ELO. Despite criticisms that rewards bad players when they have good enough team mates, due to the law of large numbers it mostly serves its purpose at a fairly (although not perfect) rating system on players. Obviously this system will be off in shorter samples, and due to pure luck can also place someone lower or higher than they truly are, but it is still a valid way to rate skill. NS2 has the great advantage of NOT queueing people in skill match ups, and if someone is given a lower or higher skill rating than their true skill isn't going to actually effect them as a player like it can in League of legends, or starcraft 2.
Ns2 is a little more complex, of course. A rating system that works in League, or starcraft 2, albeit both team games wouldn't exactly work in NS2 since NS2 has a much different style of play. The game is only an rts for one person on each team...and that person usually doesn't exactly have a strong score due to playing an rts, not a shooter. Due to not having skill match ups and instead server drop in, it's even more difficult to track skill efficiently. I don't believe this means it's impossible. Assuming that it wouldn't work because the common way to track someones skill isn't necessarily valid in NS2 doesn't excuse NS2 from trying. And they have tried, to a point. The game does officially track skill now, however halfheartedly it does so.
One thing that needs to happen in order to allow people a quantifiable number in skill is the number of servers allowed to track skill needs to be increased, probably ten fold. Right now there is VERY few servers that actually track the current implementation, and half those servers are seldom played due to not having a number of convenience mods installed to help the server (by the way, map voting should be a thing on vanilla servers). Besides that, I think people need to realize that score is honestly a great way to tell how effective someone is being at helping the team. People also need to realize that some things reward too little, and too much in some instances which can throw off someone's score immensely. One person that goes 1 kill, 8 assists and 27 deaths but still maintains a 3rd place score by running around and being a MAC isn't actually helping as much as someone like to pretend it does. Because believe it or not, in smaller scale matches, you DO need to be able to aim your gun. On that same note, someone going 30/1/1 isn't helping as much as people pretend when they are playing Team Death Match, and not helping the tide of the battle by killing the same skulk in hub all game.
I see no reason for score to not be cleaned up a little bit to properly reward players for how effective they are in game, and to allow that to directly correlate to skill. This game is played at a much LARGER per match scale than league of legends and starcraft 2 is, and I don't believe basing your overall skill based on your W/L ratio would even come close to where we need to be for NS2. I also don't believe that half assing the skill system serves any purpose, and there is no reason why it can't be cleaned up to allow for fair teams to be made when necessary based on that number. On top of that, each player needs TWO skill ratings. Even if competitively teams play aliens and marines, in pubs, a lot of players only play one race. There's no reason to not "skill track" Marines and Aliens play. Allowing for skill to truly be measured couldn't hurt the game in any way. We don't have a ladder, so if players want to "pad" their numbers, who cares? Competitive is a total 180 in terms of true skill since the game is actually team based, so that wouldn't be impacted either by allowing skill tracking in singular / pub play.
I've never seen a real reason posted by anyone to not allow true skill tracking besides "it isn't measured correctly!!1!11one!". Due to law of large numbers and skill not directly impacting match making, if we were to properly quantify skill based through an average score it would actually eventually show you your true skill. So, that's just a bad complaint. Obviously any skill measurement system, even at its basic core of ELO being used in chess can be flawed. Did you know ELO was also created to be a slightly more accurate way to show your true rating because the previous one was slightly inaccurate, and most major chess leagues use a "type" of elo rating, not true elo rating, because things can be adjusted to fit other games?
Does anyone else have an opinion on skill rating besides "INACCURATE AHHHH" and "JUST MEASURE YOUR **** WHILE YOU'RE AT IT!"? I would like to have a real discussion.
all edits are spelling / grammar errors cause ingles is hard. I think i fixed a lot of its problems for people to understand the paragraph since ingles is hard for me and others
Comments
E: didn't know auto merge wasn't implemented
and if necessary you can "vote" for "sort by skill" rather than random since random doesn't always help the problem, if do anything at all
Here's a few things they do not currently consider:
1. I get no points for using umbra, which is something I do very often
2. win:loss ratio is so weakly correlated with skill (I'll admit I tend to just go marine-alien-marine etc) but if someone were to try and fight the stack they'd do less well.
3. There needs to be an opt-out feature. What if for one round you just want to mess about with a welder? Or maybe we're trying a weird strategy. So perhaps every x number of rounds you get an opt out.
Obviously no system can be perfect (how do you quantify field commanding?) but it has some room for improvement.
The trouble is that any measurement can be abused once people know the calculation used behind it. Want to artificially decrease your skill to avoid being auto'd from your mate all the time? Get some losses in! Skill measured by k/d? Ignore objectives to farm kills, or commit suicide a lot.
Any kind of statistic used imo should be completely hidden from the public to minimise the abuse that WILL happen.
Yeah W/L with HUGE amounts of data is the way to go. Unfortunately people are too impatient for that data so they need to try and find that 'ideal formula' to calculate it from game one.
1. Not many server support it
2. NS2 lacks any kind of public league system
The combination of Elo rating and league system is what works so well in other games. 2. will be hopefully fixed soon for ns2. 1. can only be changed by the server admins.
We already had a thread which covers up the main points of this thread: http://forums.unknownworlds.com/discussion/133155/me-my-elo-and-i/p1
@Roobubba:
I totally disagree to hide the stats system, because ppl can only trust in a system when they know how it works. Otherwise there would be some magical numbers but noone could be sure if they are correct.
Maybe English isn't his first language?
That mod is curretly in Beta stage and is not completely finished. Although I wasn't programming the mod and I don't know much of its details...
The intention behind this is good, but in practice I found it immensely frustrating to play on your server with this running. I am happy to anti stack all day long, but essentially it would NOT let me play marines ever. I was always forced to play aliens, even if they had 2 more players I had to join alien to make it 3 more players (and then the marines got roflstomped). I just wanted to get a mix of games, some alien, some marine. Always having to play one side is really very annoying indeed. More work needed, and welcome - as I said it is a good idea in principle.
The thing with the uneven player number was a bug that was fixed (was a bug in early stage). You should be able to go in every team when they are too much players in the other team. Even though this mod does not eradicate stacked teams. You can't balance teams when players are leaving...
Perhaps percentage of your teams overall score and brute force accuracy per weapon/lifeform.
Like in League, it doesnt do any good if everyone calls adc, a team matched with 5 fades isnt going to cooperste as well as 2 lerks 2 fades and 1 onos each dedicated to their role
Iapologize I didn't mean to offend anyone with my grammar I type this on my tablet and allow it to take care of spelling. Its very hard and time consuming to dot my i's and cross my t's... And everyone else seemed to understand what I was saying
I can break it down in simpler terms if its is too hard friend
lets look at successful games that track "skill".
Call of duty series
Battlefield Series
Counter strike
League of legends / dota
starcraft 2 / warcraft 3
Various starcraft two MODS: Squadcraft td ... Probes V zealots some popular ones
To some extent, payday 2
The only one that brands itself as a competitive game and fails to bring some meter to track your skill is ns2. Nevermind that we don't have a ladder, we don't even have a basic skill tracker because everyone is too busy arguing how it wouldn't be perfect (Which doesn't totally matter since we DON'T have a ladder, mind you).
Also the difference for most of those games is the much lower player count per match. As more variables are added it becomes harder to track skill. So tracking skill with elo in 1v1 games like SC2 is much easier than tracking skill in LoL or CS as there are four other people on your team you must rely on (can still win if you're actually good enough though..my god the level of whinging..). Which is then still easier than 16v16/32v32 games of BF, or 12v12 games of NS2.
Well since BF and CoD push the huge play counter boundries, they mostly have NS2 beat, and they track K/D ratio (on all servers, even modded... christ ns2) as well as W/L, accuracy, and other stats similar.
Obviously it's a little bit different, you're right, ns2 is a unique game. I don't think it excuses the game from trying - and it does try, we have score. That score just needs to correlate to a visible skill / elo for people to be proud of or want to strive for a higher number. People like numbers. I know there is people, in fact I KNOW people who grew bored of the game due to a lack of progression. Progression doesn't always have to be epeen unlocks or new guns, a simple rating is enough for sc2 players and dota players. It's just something this game lacks for no reason other than to complain it's too hard to do right... or something
Stats like KD and Accuracy aren't actually a "skill" rating. A skill rating is any single value that can be compared to another players to say you are better or worse than them without even having to play them.
But I suppose I hsould say I haven't played a call of duty since Mw2, and bf since bf 2 BC, so I may be behind the times, but the games still allowed progression for time played and if someone was running around with a lvl 10 elite w/e on CoD, they were usually pretty good. Of course that's not skill, but at least they had something that showed time played hehe
One of the many things I love about NS2 is that whatever you do in one round doesn't impact on future rounds (hive tracking bad; NS2stats needs to be opted out of - would prefer opt-in system; servers that track your 'skill' locally... MEH).
I don't mind if you stats wh0res fancy an opt-in stat tracking system, please be my guest. But stop trying to force my stats to be saved somewhere. I don't want them, and you have no right to make me have them.
rofl ns2stats has a opt out since over 1 year. Log into ns2stats.com ->my account -> hide my stats from puplic
To opt out one round either disable ns2stats for that round or use cheats 1 -> cheats 0 in console done. NS2stats doesnt track rounds after cheats has been used once until next mapchange.