moultanoCreator of ns_shiva.Join Date: 2002-12-14Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
Thinking about it more though, it should be safe to use KDR as a "prior" for when hive doesn't have enough data about a player. There wouldn't be any incentive to exploit this because it wouldn't be displayed anywhere, and wouldn't contribute to any leaderboard, and this would help the problem that @dePARA is worried about.
The hard part is making the units match up between the KDR values and hive scores so they are comparable. I think I can do this. Let me think about it for a day or so, and if my wife doesn't imminently go into labor I'll write something up.
It would be no different from when the server doesn't shuffle at all.
Even if the teams are completely skewed, it's not a problem for hive. No matter how the teams are formed, the process is the same. It predicts which team is favored, if that team wins the scores are left unchanged.
More interested in Points per Minute or Accuracy than KDR.
KDR is skewed aliens. Your lifeform play will always skew it towards a higher KDR. Onos players will have lower KDRs than Lerk players, fades as well. PPM is more reasonable IMO. PLus this and accuracy (team based accuracy) make a more fair assesment of the mediocre players.
@Calego Glad to see what i was advocating for did produce 1 more supporter. Well... The pace is still slow, but i'm confident it eventually goes up in the next weeks. In the mean time the % can only go up as player leave the game. what are the odds ?
Give it a few weeks and those scores will converge on each other.
That was the basis for not resetting in the first place wasn't it? It hasn't happened since and it isn't going to happen now.
Good point. They probably won't play a few weeks. The main reason it did not converge is because most of them don't play more than a few games. The graph where I showed the divide includes all players, even those who only played 10 minutes. I put it here because it clearly shows that divide.
Looking at my graphs again, I should of used this one. It is more representative of the active player base and better shows the divide.
Never understood why people are so reluctant towards pure KDR/Acc ratings (unless you command). It's not perfect, but hits the nail 90%. Current system hits it 30% at best.
I've been seeing a lot of stomps too, but it's not clear to me that returning rookies with inflated scores is what is driving it. Are you sure this is the cause? Can you point to someone's hive profile who is an old rookie with recent games? It seems like most of the rookies I'm seeing are genuinely brand new and have a hive score of 0.
"Are you sure this is the cause? "
Yes.
Especial on alienside its a huge problem right now if you have more 1000point rookies than the other team.
But i would say as a general rule:
The more 1000point rookies a team have the higher is the chance of losing.
Its simple:
Aliens have two 1000points rookie for example and Marines one 0 point rookie and a 2000 point vet.
Both would end in 1000 hive score.
Well, its easy to predict who is winning, isnt it?
This keeps players with the same playtime sorted by their current skill values. Players with lots of play time retain their current score, players with 0 playtime go to 0 skill, and it smoothly interpolates between those two. Players with 100 hours will get half of their current score.
If we can figure out which players started at 1000, we can do something different, but if we can't, this is the best thing I can think of.
moultanoCreator of ns_shiva.Join Date: 2002-12-14Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
edited December 2015
@Nordic Awesome! We could use this method to choose the half point parameter instead of just 100hrs. We'd compute the correlation between the score and log(kdr), and choose the value that maximizes the correlation.
I'll compute this with the data you sent if I get a chance.
Indeed a problem exist, if a rookie earned 1k skill after +- 50 (assumption) games won, he will be significantly better than the others rookies that come back after the hive reset.
To make a point, let's imagine a server 18 slots.
Composed with:
1 player with 2K skill
2 rookies with 1k skill
and 15 real rookies (0).
Guess the result of a shuffle vote? And the result of the round.
Well OK the 2k vet should not be there, but let's do the same with 2 players with 1k3 (instead of 1 players 2k) and we obtain the same result one game on
two.
Another assumption:
2 players with 1k skill earned
2 rookies with 1k skill
14 real rookies (0)
Let's again image the result of a shuffle, we will have 1 round on 2 with the two no so rookies players in the same team.
Finally and it's only let's imagine that i am a veteran and i have lost my steam account what happens if i come back to the game?
I will have a skill of 0 and all 100-200games following will not be even.
Below you can find some screens with the two types of rookies, one that get a free 1k skill and another that is still blocked to 0 after +-20 games recorded
(yep that's another problem, normally if the algorithm works, his probability of winning is not 0 and thus he should not have a skill of 0 )
If/when the skill system gets improved, how opposed to a total reset would people be?
That would probably make the skill system more accurate faster than allowing current scores to be adjusted.
A complete reset would be chaos at first with the top level being equal to the bottom level. Shuffle would be entirely useless for a few weeks. Why not use peoples current hive skill value as a starting point, which would be further improved after they are adjusted.
If/when the skill system gets improved, how opposed to a total reset would people be?
That would probably make the skill system more accurate faster than allowing current scores to be adjusted.
A complete reset would be chaos at first with the top level being equal to the bottom level. Shuffle would be entirely useless for a few weeks. Why not use peoples current hive skill value as a starting point, which would be further improved after they are adjusted.
Why should we throw out data we already have?
Yeah even if we wanted to do a seasonal "reset" I'd much rather divide all skills by some constant than set them to 0 just so balancing teams never stops working.
DePARA, some people get banned for cheats, some stop playing for >1 year and cannot remember their account password/email rescue password.
Some people like to troll.
The important is not the how, it's the fact that those people exist and the current balance system cannot do anything about it.
Statistically: is it a significant problem? Well I don't know and i don't know if the hive data available can allow us to find the average number of active players concerned on the total average players playing ns2.
Soul_RiderMod BeanJoin Date: 2004-06-19Member: 29388Members, Constellation, Squad Five Blue
edited January 2016
How would changing the 'zero' point work for speeding up balance?
ie - if the rookie score was set to say 250, how would this affect speeding up convergence?
BTW, i am not a 'rookie', but haven't played much since hive was introduced on hive registering servers, so when I came back recently, I had a score which is now nosediving to the new floor.
I don't think that K/D ratio and accuracy reflect the level of a player because those parameters doesn't take account of the level of your enemy.
I think that a better system could be the following:
- I want it to be based on the player skill and not the team skill
- To compare things that are comparable and allow fast convergence i propose to use each kill and death.
- A lot of player roles have advantage and disadvantage regarding the ease of making kills, that's why i propose to only adapt the skill of players when kills occurs between a skulk and a marine.
- if the marine player equips some unsuitable weapon like flame-tower, that data should also be ignored.
- I assume that all players spend enough time in skulk and normal marines (ie: with lmg/pistol).
- Then each time you kill a marine/skulk your earn an amount of skill between a minimum of 0 and a maximum of X. the skill you win should depend on a function based on the difference of skill between you and your enemy. Basically if the difference is too big in your favor you will not win a lot of skill. On the opposite situation you will get the maximum.
- each time you are killed by a marine/skulk you lose an amount of skill following the reverse equation.
- Then, if we take a look on the macro level we will also probably see that skulks die more often than marines. That's why the amount of skill win/loss should be multiplied by a constant "K" (or a function).
- The K can in the simplest world be calculated based on the total kill recorded on the hive between marines/skulk once for all.
Finally the commanders needs also an evaluation of their commanding level. For them i am forced to confess that i don't see any simple way to calculate their skill except the current one.
Comments
The hard part is making the units match up between the KDR values and hive scores so they are comparable. I think I can do this. Let me think about it for a day or so, and if my wife doesn't imminently go into labor I'll write something up.
It would be no different from when the server doesn't shuffle at all.
Even if the teams are completely skewed, it's not a problem for hive. No matter how the teams are formed, the process is the same. It predicts which team is favored, if that team wins the scores are left unchanged.
KDR is skewed aliens. Your lifeform play will always skew it towards a higher KDR. Onos players will have lower KDRs than Lerk players, fades as well. PPM is more reasonable IMO. PLus this and accuracy (team based accuracy) make a more fair assesment of the mediocre players.
That was the basis for not resetting in the first place wasn't it? It hasn't happened since and it isn't going to happen now.
Good point. They probably won't play a few weeks. The main reason it did not converge is because most of them don't play more than a few games. The graph where I showed the divide includes all players, even those who only played 10 minutes. I put it here because it clearly shows that divide.
Looking at my graphs again, I should of used this one. It is more representative of the active player base and better shows the divide.
A sane person
"Are you sure this is the cause? "
Yes.
Especial on alienside its a huge problem right now if you have more 1000point rookies than the other team.
But i would say as a general rule:
The more 1000point rookies a team have the higher is the chance of losing.
Its simple:
Aliens have two 1000points rookie for example and Marines one 0 point rookie and a 2000 point vet.
Both would end in 1000 hive score.
Well, its easy to predict who is winning, isnt it?
new_skill = old_skill * playtime / (playtime + 100hours)
This keeps players with the same playtime sorted by their current skill values. Players with lots of play time retain their current score, players with 0 playtime go to 0 skill, and it smoothly interpolates between those two. Players with 100 hours will get half of their current score.
If we can figure out which players started at 1000, we can do something different, but if we can't, this is the best thing I can think of.
I will go run that formula through my data and report back.
It looks really good. I think this function would work well. Check the spoiler for some basic stats.
Average Skill = 938
Median Skill = 1000
Mode of Skill = 1000
Minimum Skill = 0
Maximum Skill = 3753
After Function:
Average Skill = 150
Median Skill = 65
Mode of Skill = 29
Minimum Skill = 0
Maximum Skill = 3313
I'll compute this with the data you sent if I get a chance.
That would probably make the skill system more accurate faster than allowing current scores to be adjusted.
Indeed a problem exist, if a rookie earned 1k skill after +- 50 (assumption) games won, he will be significantly better than the others rookies that come back after the hive reset.
To make a point, let's imagine a server 18 slots.
Composed with:
1 player with 2K skill
2 rookies with 1k skill
and 15 real rookies (0).
Guess the result of a shuffle vote? And the result of the round.
Well OK the 2k vet should not be there, but let's do the same with 2 players with 1k3 (instead of 1 players 2k) and we obtain the same result one game on
two.
Another assumption:
2 players with 1k skill earned
2 rookies with 1k skill
14 real rookies (0)
Let's again image the result of a shuffle, we will have 1 round on 2 with the two no so rookies players in the same team.
Finally and it's only let's imagine that i am a veteran and i have lost my steam account what happens if i come back to the game?
I will have a skill of 0 and all 100-200games following will not be even.
Below you can find some screens with the two types of rookies, one that get a free 1k skill and another that is still blocked to 0 after +-20 games recorded
(yep that's another problem, normally if the algorithm works, his probability of winning is not 0 and thus he should not have a skill of 0 )
http://postimg.org/image/r9u6z3e43/
http://postimg.org/image/j3277io1f/
http://postimg.org/image/4il4cot2r/
http://postimg.org/image/inqxei243/
http://postimg.org/image/k80uqmzw3
My point of view is that hive skill with is current formula based on the winning probability mostly work well macroscopically but give a lot of unexpected results microscopically.
Why should we throw out data we already have?
Yeah even if we wanted to do a seasonal "reset" I'd much rather divide all skills by some constant than set them to 0 just so balancing teams never stops working.
And here comes the most important question:
How and where can you lose your steam account?
Did you carry it arround in your portemonnaie?
Some people like to troll.
The important is not the how, it's the fact that those people exist and the current balance system cannot do anything about it.
Statistically: is it a significant problem? Well I don't know and i don't know if the hive data available can allow us to find the average number of active players concerned on the total average players playing ns2.
ie - if the rookie score was set to say 250, how would this affect speeding up convergence?
BTW, i am not a 'rookie', but haven't played much since hive was introduced on hive registering servers, so when I came back recently, I had a score which is now nosediving to the new floor.
Maybe some sort of crystal ball formula could help here.
I believe that the convergence speed can be greatly improved. Of course for now it is slow mainly because it is based on win/loss.
We are at a point where everyone can see that the current system as reached is limits.
And even after reading this post https://moultano.wordpress.com/2014/08/04/a-skill-ranking-system-for-natural-selection-2/
describing the algorithm, i still cannot believe in its accuracy on the micro level.
I don't think that K/D ratio and accuracy reflect the level of a player because those parameters doesn't take account of the level of your enemy.
I think that a better system could be the following:
- I want it to be based on the player skill and not the team skill
- To compare things that are comparable and allow fast convergence i propose to use each kill and death.
- A lot of player roles have advantage and disadvantage regarding the ease of making kills, that's why i propose to only adapt the skill of players when kills occurs between a skulk and a marine.
- if the marine player equips some unsuitable weapon like flame-tower, that data should also be ignored.
- I assume that all players spend enough time in skulk and normal marines (ie: with lmg/pistol).
- Then each time you kill a marine/skulk your earn an amount of skill between a minimum of 0 and a maximum of X. the skill you win should depend on a function based on the difference of skill between you and your enemy. Basically if the difference is too big in your favor you will not win a lot of skill. On the opposite situation you will get the maximum.
- each time you are killed by a marine/skulk you lose an amount of skill following the reverse equation.
- Then, if we take a look on the macro level we will also probably see that skulks die more often than marines. That's why the amount of skill win/loss should be multiplied by a constant "K" (or a function).
- The K can in the simplest world be calculated based on the total kill recorded on the hive between marines/skulk once for all.
Finally the commanders needs also an evaluation of their commanding level. For them i am forced to confess that i don't see any simple way to calculate their skill except the current one.