Stats. Win, loss and life.

Uh-OhUh-Oh Members Join Date: 2002-11-04 Member: 6917Posts: 262
edited December 2012 in NS2 General Discussion
Stats are useful when you know how to use them.
Hello. I've seen people misuse stats a lot in arguments about game balance lately.

TL;DR: As of today (december 10, 2012) Both these facts are true: Aliens win 74% of the time. Aliens win 56% of the time. They are true if you explain what data is used to get that number (sample pool origin, date, criteria of selection...). Thus, stats without context are useless.

Please consider the following before listing stats or inferring knowledge from stats.

  • Poster: Be precise and transparent so people know you are using accurate and trustworthy data.
  • Reader: Ask yourself if the data is from a reliable source and if it's relevant.
  • Poster: If you are going to use stats as an argument, make it clear and coherent with the argument.
  • Reader: Are the stats supporting the argument? Do they realistically explain the argument, or is it an incorrec conclusion.

Precision and clarity.
A percentage without context is irrelevant. A percentage without its source is made up.
Where does the info come from? What does it represent exactly?
Most people use ns2stats dot org. Please list your search criteria and the actual numbers that make up the final statistic that you are listing.

Wrong way to use stats: Marines win 55% of the time.
Right way to use stats:
Source: ns2stats dot org.
Search criteria:
- All games played on servers featured on ns2stats dot org.
- Date: December 1st to December 10th
- Important information: 9% of the games were played on unofficial maps.
- Statistic you want to use: Marines have a 37.83% win rate (608 games out of 1607)


Argument:
It is thus shown that Aliens win a lot more, if you look at the overall games played. This percentage could be different if looking at individual maps or servers, or looking at older dates.


This might seem obvious or to be too much work, but if you want to be taken seriously, please give us: context and data information (source, dates, etc..)
Otherwise, you're just like 87% of the people and pull stats out of your behind.


Why context is important:
One issue can have multiple view points depending on what we're looking at. For instance, the perception of a popular issue, balance, is often based on the win/loss ratio of the teams.
Depending on which data you use, your stats can vary greatly.
Are you looking at stats compiled from old data that is irrelevant (previous builds)?
Are you including both competitive and pub play? One could be very different than the other and skew the average.
Are you looking at all the maps, including unofficial ones?
Without knowing what your final stats are compiled from, you cannot know what it represents (a coherent and realistic view of the issue, or a irrelevant number that doesn't match up with reality).
That is why context and data information is so important. It tells us what we are looking at. It's the difference between saying I like beautiful legal-aged woman, rather than I like girls (uh-oh).
See below for an exemple.

Just looking at the ns2stats dot org homepage I see:
November 26 to December 10, All maps, Aliens win 57.96%

Going into filters and:
- Selecting ONLY build 232 (current build), only pub matches and No mods. Dates unchanged: Aliens win 57.62%
- Selecting ONLY build 232 (current build), only competitive matches and No mods. Dates unchanged: Aliens win 66.45%

Now, to show the disparity between maps:
- Important note: I cannot select only pub or competitive matches, and I cannot select only games played without mods. Dates unchanged:
Refinery: Aliens win 74.09%
Mineshaft: Aliens win 59.62%
Docking: Aliens Win 64.2%
Summit: Aliens Win 63.42%
Veil: Aliens win: 61.17%

That's a 14.47% difference in wins for the aliens.


Now to show disparity between map start location:
- Important note: I cannot select only pub or competitive matches, and I cannot select only games played without mods. Dates unchanged:
Summit Overall:
Atrium start for marines: Marine Wins: 28.57%
Data core start for marines: Marine Wins 32.56%
Flight control start for marines: Marine Wins 38.37%
Sub acces start for marines: Marine Wins 41.84%

That's a 13.27% difference in wins for marines.


Now to show disparity between the most and least balanced start location:

Docking - Start in Departures: 55.88% Aliens Win
Refinery - Start in Flow Control (only start location on the map): 74.09% Aliens Win

That's a 18.21% difference in wins for aliens.
Thus, while your average public game sees marines win 42% of the time, your average Mineshaft game sees marines win only 26% of the time. Big difference. This (the discrepancy between maps and start locations) can partly explain why some people feel the game is more or less balanced than others.

Bottom line, saying Marines win 42% of the time (while true), fails to communicate a lot of necessary information to depict a complex issue, such has the perception of balance.

*Please let me know if I made any errors.

On a final note: Please consider what information is not included in statistics.
For instance, the data I used today (wins and losses), does not included any information about player count, games won or lost in less than 5 minutes (or any other amount of time), presence or absence of a commander, time of day the games were played, server region, player region, latency, disconnects, F4s...
Maybe a significant amount alien wins were of the less-than 5 minutes kind. Maybe servers with over 120 pings favor aliens. Maybe players in europe are affected by the plurality of languages, rendering marines less coordinated because they relly more on communication (this is just an exemple for exemple sake).

Thus, a lot of vital information is missing to come to a clear and sure-fire "fix" for this issue (win/loss balance). The fix could be simple, complex, singular or plural.
One thing is certain, it serves little purpose to incorrectly infer information from incomplete, irrelevant or inaccurate stats!
Post edited by Unknown User on

Comments

  • Shrike3OShrike3O Members, Constellation Join Date: 2002-11-03 Member: 6678Posts: 718
    QUOTE (Uh-Oh @ Dec 10 2012, 02:23 AM) »
    Otherwise, you're just like 87% of the people and pull stats out of your behind.

    Well, approximately 50% of people do have sub-median intelligence, what do you expect?
  • ellnicellnic Members, Reinforced - Shadow Join Date: 2010-07-19 Member: 72559Posts: 915 Advanced user
    edited December 2012
    This is a good thread and pretty much coveres about everthing to do with stats. if i get into a balance 'discussion' with someone who clearly hasn't a clue, i'll be sure to send them over here to read this. You make a good point aswell saying

    'Thus, while your average public game sees marines win 42% of the time, your average Mineshaft game sees marines win only 26% of the time. Big difference. This (the discrepancy between maps and start locations) can partly explain why some people feel the game is more or less balanced than others.'

    Because in a lot of cases people come and complain after played a few hours on certian map and they never tell us what maps or where they started, and a lot of the time, they dont see why this infomation matters.
  • BigTracerBigTracer Members Join Date: 2012-12-04 Member: 174169Posts: 235
    A decent research indeed.


    I wish UWE did random starting points for both teams, not just a choice between two. It would gave a lot new game experience.
  • gnoarchgnoarch Members, Reinforced - Gold Join Date: 2012-08-29 Member: 156802Posts: 259 Advanced user
    QUOTE (BigTracer @ Dec 10 2012, 12:30 PM) »
    A decent research indeed.


    I wish UWE did random starting points for both teams, not just a choice between two. It would gave a lot new game experience.


    I think random spawn locations are one big problem of the current balance situation. Because of the asymmetric design of the races it is nearly impossible to balance a map for random start locations. If you balance a start location properly for one race, the chance of it being better/worse for the other race is quite high.
  • ellnicellnic Members, Reinforced - Shadow Join Date: 2010-07-19 Member: 72559Posts: 915 Advanced user
    They tried random befor, It didn't work as well as they hoped
  • SehzadeSehzade Members Join Date: 2010-12-29 Member: 76024Posts: 204
    QUOTE (Uh-Oh @ Dec 10 2012, 10:23 AM) »
    Docking - Start in Departures: 55.88% Aliens Win
    Mineshaft - Start in Flow Control (only start location on the map): 74.09% Aliens Win


    there is no flow control in mineshaft.
    and mineshaft definetely does not have just one starting location. Neither for aliens nor for marines
  • Uh-OhUh-Oh Members Join Date: 2002-11-04 Member: 6917Posts: 262
    QUOTE (Sehzade @ Dec 10 2012, 08:35 AM) »
    there is no flow control in mineshaft.
    and mineshaft definetely does not have just one starting location. Neither for aliens nor for marines


    Thank you, I meant Refinery. It has been fixed.
  • PsympleJesterPsympleJester Members Join Date: 2008-04-06 Member: 64024Posts: 719
    Its unclear on the "Now to show disparity between map start location:" section whether the win %'s are marines or aliens.
    Othewise a good read and reminds me of my Stats modules in maths...

    Its a shame you didn't show the % win when there are long/close spawns and % wins depending on the length of games.
    I imagine the longer the game the more likely aliens win and the closer the spawns the more likely marines win.
    QUOTE (Mkilbride @ Nov 5 2012, 09:27 PM) »
    In no circumstances, should it take more than an entire SMG Clip to kill a fade. I did several tests on this, even standing still, with next to 0 ping, it takes more than a clip from the SMG to kill a Fade.

    Marine Starting Gun should ###### up the Aliens 50 res life form yo... Seems Legit.

    NS2_Siege007 Remake.
  • MisterNubsMisterNubs Members Join Date: 2012-03-01 Member: 147912Posts: 568
    Now get the time on wins and let's see where the win percentage sits.

    You still have only half the information.
  • Champlo0Champlo0 Members Join Date: 2012-04-17 Member: 150617Posts: 465
    Nice to see someone using statistics non idiotically. This obsession with winrate reminds me of kids on LoL and LoLKing.
    In Game: Champloo
    Check out my YouTube
    Steam URL: http://steamcommunity.com/id/champloomusashi/
  • Uh-OhUh-Oh Members Join Date: 2002-11-04 Member: 6917Posts: 262
    edited December 2012
    QUOTE (PsympleJester @ Dec 10 2012, 11:19 AM) »
    Its unclear on the "Now to show disparity between map start location:" section whether the win %'s are marines or aliens.
    Othewise a good read and reminds me of my Stats modules in maths...

    Its a shame you didn't show the % win when there are long/close spawns and % wins depending on the length of games.
    I imagine the longer the game the more likely aliens win and the closer the spawns the more likely marines win.


    Thanks, fixed to make it more clear. The information for duration of games is partially available on the ns2stats site. It's actually very interesting how it fluctuates between both sides. Look at this:

    image


    QUOTE (MisterNubs @ Dec 10 2012, 11:25 AM) »
    Now get the time on wins and let's see where the win percentage sits.

    You still have only half the information.


    I've got very little information. That's the message I'm trying to get through with this thread. Stats do not draw a complete picture.


    QUOTE (Champlo0 @ Dec 10 2012, 12:13 PM) »
    Nice to see someone using statistics non idiotically. This obsession with winrate reminds me of kids on LoL and LoLKing.


    I believe win / rate is important up to a point. My problem was that people were claiming stats without source or thought about what they represented, and thus seemed highly inaccurate to me.
    Anyone can make numbers seem to indicate something, but it's only when considered carefully that they really mean anything. I loathed stats without context.
  • includeinclude aka RpTheHotrod Dallas, TXMembers, Constellation, Reinforced - Shadow, WC 2013 - Shadow Join Date: 2003-01-08 Member: 12027Posts: 658 Fully active user
    Thank you for putting effort into this.
  • Uh-OhUh-Oh Members Join Date: 2002-11-04 Member: 6917Posts: 262
    edited December 2012
    At the moment my intent is not even to try to share accurate stats.
    It's primarily to encourage people to think about what a number means, how it came to be, and if it's worth believing and sharing.
    I've seen claims of marines or aliens (or protoss) winning or losing or (calling a truce) anywhere from 5% to 95% of matches, based on stats gathered from the interwebs.

    It's lead to counter constructive arguments and has wildly exaggerated or undermined the real issues (depending on the viewpoint and the stat used to prove it).
    There is a lot of "THE SKY IS FALLING" or "Stop whining its super balanced" claims, whereas reality is somewhere in between.
  • ScardyBobScardyBob ScardyBob Forum Admins, Forum Moderators, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Shadow Join Date: 2009-11-25 Member: 69528Posts: 4,983 mod
    I went a bit further than this during the beta in this thread. We're actually close to being able to make a decent comparisons, since NS2stats does track player skill (Elo rating or total score). I'm hoping Toastie's metagame analysis project can either do this or provide the relevant data in a way possible to do proper analysis of balance.
  • MaximumSquidMaximumSquid Members Join Date: 2010-07-20 Member: 72593Posts: 627
    edited December 2012
    Uh-Oh:

    The time info is probably the most interesting

    I'd really would like to see how effect certain weapons or lifeforms are on certain maps

    Jetpacks dominate on Mineshaft more than any other map; as an example
    Knowing what tech is good where will probably have a far greater impact on helping to balance the game down the road than just looking at raw win rates

    Also when ARCs aren't broken and actually shoot when you scan and stop getting stuck we'll probably see that alien win % go down at least a little

    If everyone was getting 70+ frames you'd see a lot more marine wins as well

    Stuff that UWE already is aware of I'm sure, but it's still interesting to think about how they might handle the next few updates

    -
  • ZekZek Members, NS1 Playtester, Constellation, Reinforced - Shadow Join Date: 2002-11-10 Member: 7962Posts: 4,690 Advanced user
    edited December 2012
    I'm also curious how team size affects the results. First chamber choice would be useful info as well.
  • SoulfighterSoulfighter Members Join Date: 2012-11-05 Member: 167432Posts: 80
    edited December 2012
    QUOTE (Shrike3O @ Dec 10 2012, 10:39 AM) »
    Well, approximately 50% of people do have sub-median intelligence, what do you expect?


    90% of people think they're not in that sub-median part
    Post edited by Unknown User on
    image
  • SteveRockSteveRock Members, NS2 Developer, Subnautica Developer Join Date: 2012-10-01 Member: 161215Posts: 486 Advanced user
    This is just good advice for life in general. Any time any one mentions stats in any context, whether it's science or politics or sociology or business, remember this thread.
Sign In or Register to comment.