ServerPerformanceData and (proposed) 269 ServerBrowser

matsomatso Master of Patches Join Date: 2002-11-05 Member: 7000Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer
During 268, the new ServerPerformanceData kit will be phased in.

This kit is going to be used for 3 things
1. Let server admins accurately scale their server.
2. Rating servers on the Server Browser
3. Warning players on a server that the server is overloaded

There are two console commands available straight out of the box, "perfmon" and "perfbrowser". Also, the "net_stats" console command has a new line using the ServerPerformanceData to display server info.

"perfmon" controls dumping out data to the log file. It cycles through various detail levels, with the most detailed dumping out the per-second snapshot.

"perfbrowser" toggles the server browser to show the proposed 269 version of the ServerBrowser.

The intent is that server admin use these tools during 268 to ensure that theirs servers are properly scaled and scores well on the 269 ServerBrowser. Do checkout the json docs for the ServerPerformanceData, and the ServerPerformanceData.lua file

ServerBrowser and scoring
The 269 ServerBrowser will group servers into The Good and the Bad.

The browser will ALWAYS list the Good servers on top and the Bad at the bottom, below all Good servers.

The score is calculated as "%idle - BadThingsHappeningToYourPlayers".

%idle is simple the percent of time the server is fiddling its thumbs and not doing anything. To avoid running into BadThings territory, a good rule of thumb is to have the server at about 20% idle, even in the endgame. Ideally. But as long as the server don't BadThings to its players, it will still be counted among the Good.

The only way to get listed as a Bad server is if the server consistently does BadThings to its players. And the only bad thing that counts is causing players to get an interpretation buffer overrun. Players experience an interpretation buffer overrun as teleporting, rubberbanding and loss of avatar control - basically, the game becomes impossible to enjoy.

Not a desirable situation.

In normal settings, the interp is set to 0.1 or 100ms, and normally the server sends out a network update every 50ms. So it is only if the server ever gets so busy that it is unable to send an update inside 100ms that it gets a really bad score (we give a small negative score for exceeding 75% of the interp buffer as well).

The server inner loop tries hard to maintain the sendrate, and it takes a concerted effort (or rather, very optimistic settings) to make it break it.

And with averaging and worst-case discarding, a server will only be in the Bad group if it consistently gives its players a bad game.

The server will maintain a performance history - basically, the worst 30-second periods in recent history - and use this to update the master server performance information.

Note: THIS IS EXPERIMENTAL - DO provide feedback.

PS. And no, the ServerBrowser will not look like that in the final version; the perf-line is just there to provide feedback. DS
«1

Comments

  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    edited August 2014
    Currently this is a joke. My combat server for example is constant 30tick and never dips yet does not show up as good even.

    Is this going to come with some actual improvements to server performance? Currently I do not see how this is really even needed. Is there still such a major issue with shit performance servers? All this seems to be trying to do is to slit hairs and favor a select group of servers.

    Also 20% my ass I checked and it gives me not good even when at 30tick and <50% on my highest core. (# in game was -3 to -7)
  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    edited August 2014
    What I did notice for combat was that when the number of players went down it then went to good even though the actual performance remained exactly the same b/c it is already at 100%...

    I could see this if maybe we were still having really bad performing servers running the game after actually doing some updates to focus on server performance for a chance.

    Also right now a good number of servers do not even list as good even with 0 players on them. I guess since the match up to mine in this level when they are empty and mine is full they will all be listed as bad when full but still this can not be right.
  • dePARAdePARA Join Date: 2011-04-29 Member: 96321Members, Squad Five Blue
    Finally something like this is implemented.
    Its more transparent for admins now how there servers perform and they can do what they want with this informations.
    Most players didnt care anyway or didnt notice the constant rubberbanding.

    Your 22 slot server @ZEROibis cant handle this slotammount btw. I dont need these new tools to see that.
    Wasnt that a problem in the past? no
    Is this a problem in the furure? no, cause people dont care

    For me as an admin who want the best possible perfomance, these are great changes.
  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    edited August 2014
    Actually now it is ok and it was good before as well all while still having 30tick the entire time. So how can I literally change between 3 tires of this shit all at the same tick rate it does not even logical.

    Also your 18 slot server when full just now was pulling in with the same rating as mine (the one at 89.163.177.130 even though it was full 30 tick as well. So unless your trying to say that your own server has just as bad performance as you alleged mine does you might want to reconsider if you really like the koolaid or not.
  • dePARAdePARA Join Date: 2011-04-29 Member: 96321Members, Squad Five Blue
    edited August 2014
    @ZEROibis‌
    Your server has problems in endgame wich result in rubberbanding.
    Its still on an relative payable level during endgame but the issues are there.

    You can use the new tools optimize things (lower playercount to 20 maybe) or let the server as it is.
    Your choice

    Dont know what your problem is with the new performancerating.

    And btw. the tickrate in Ns2 is used for AI. There more important rates wich affect the players.
    These more important rates are part of the new performancerating now, wich is great.
  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    Also with regards to this kit will it be used in the webui I guess or what?
  • GhoulofGSG9GhoulofGSG9 Join Date: 2013-03-31 Member: 184566Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Reinforced - Supporter, WC 2013 - Supporter, Pistachionauts
    edited August 2015
    And there is the logic failure this system tries to fix a stable tickrate means not as much as pll asume at the moment for the the server performance.

    Ways more important is that the time between ticks stays constant that their is enought cpu load buffer for lategame, that there is enough bandwidth avaible so you don't run into choke issues and so on.

    The tickrate is therefore only one of the many values that determ if your server performance is okay. If you look into how the spark engine and its networking works you will understand that this new algorithm is ways better to say anything about the server performance.

    You can compare reading the server performance from the tickrate to guessing tomorrows weather from looking at the clouds at the evening.

    Yes sometimes it works like if there is a horrible storm coming but in most cases its not really possible to do :D

    And for the end have in mind that this is not anywhere near being done. That's why matso shows it here.
    ZEROibis wrote: »
    Also with regards to this kit will it be used in the webui I guess or what?

    Yes that is kinda planned.
  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    edited August 2014
    Enough bandwidth I can assume you I am not running out of 100MBs Tier 1...

    Or are we now punished for high ping players?
  • GhoulofGSG9GhoulofGSG9 Join Date: 2013-03-31 Member: 184566Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Reinforced - Supporter, WC 2013 - Supporter, Pistachionauts
    edited August 2014
    ZEROibis wrote: »
    Enough bandwidth I can assume you I am not running out of 100MBs Tier 1...

    Or are we now punished for high ping players?

    There is still a bwlimit of 25 kb/s by default so even if you have 100mbit/s ns2 won't use them unless you increase the bwlimit ... .

    So your server could still have choke issues due to that and that could influence the rating .
  • ATFATF Join Date: 2014-05-09 Member: 195944Members
    Could you tell us who (by name) suddenly decided that -server performance- is the only and most prominent value by which to decide an unnecessary server browser ranking of each server?

    The most important value of a server is the -level of fun- users perceive.

    Whether a community server is popular or not is currently self-regulated by the level of fun people have playing there. The result is: un-fun servers are empty.
    The level of fun factors in a lot of different things including for example, how nice the regular players are.

    You are now making plans to regulate something that so far has been just fine by itself. Could you elaborate on the reasoning behind this decision?
    Forcing small empty servers with annoying/restrictive settings and adolescent admins on everyone is not how you save NS2.
    2. Rating servers on the Server Browser
    The browser will ALWAYS list the Good servers on top and the Bad at the bottom, below all Good servers.
    Your server performance measurement and information endeavour will work without this. It will actually have more impact without it.

    You might try to argue that server performance affects fun. It does no doubt. But it's way down the bottom of priorities.

    If performance were such an issue, Wooza's would be empty.
  • dePARAdePARA Join Date: 2011-04-29 Member: 96321Members, Squad Five Blue
    edited August 2014
    Its like watching analog Tv on an 1080p digital screen.
    Some peopple didnt see the difference between analog->digital, some dont care about a blurry pixelated interpolated picture and others want a perfect super sharp television picture. Thats why they invest extra money in calibration hardware for example.

    The server rating is a rating about the serverperformance, nothing more.
    And as i wrote above: some people dont care.

    So whats the problem?
    Its not a rating of the regular player base of the server.

    If i am new to a game and see an red warning that the server has problems i would try another one if im rubberbanding.
    This "warning" tells me: this is not the normal case.

    Like in 1000 endless why people prefer large servers threads (only the different way in this case):
    If people want to play on optimal running servers, its there choice.
    And now they can see the real perfomance wich is great.

    Btw.
    Here is a small example of how "funny" rubberbanding shit servers are:
    http://www.twitch.tv/schustor/c/4527004
  • the_tickthe_tick Netherlands Join Date: 2014-01-20 Member: 193352Members
    edited August 2014
    I really don't see the point of this system, other then to bully large server operators

    Go focus your efforts somewhere else, server crashes increased 500% since 267, try to get those memory leaks out.

    Or come up with a solution to the workshop problem, which currently makes custom maps an impossibility

    Or improve client performance, it has gotten a lot worse with 267,

    Let us focus on the player count, we don't need developers to police around.
  • dePARAdePARA Join Date: 2011-04-29 Member: 96321Members, Squad Five Blue
    Its funny that both admins of the >24 slot servers disagree with everything here.
    Coincidence? Dont think so.

    Maybe cause there servers are not on top of the serverlist anymore?
    Maybe cause players finally see the issues with these kind of servers?
    Hmm, who knows.
  • woozawooza Switzerland Join Date: 2013-11-21 Member: 189496Members, Squad Five Blue
    Please do not turn this into another discussion about large Servers. Thank you
  • PelargirPelargir Join Date: 2013-07-02 Member: 185857Members, Forum Moderators, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, WC 2013 - Silver, Forum staff
    Wooza's server is known for his terrible performances, that's right. But if the server is known, that means this server is famous & very often full. I'm not a huge fan of >24 slots servers to be honest but, it doesn't mean I have no fun sometimes on them. When I join it, and same for some 24 slots servers or even some 18 slots, I know I will have some troubles to play (that's why you can sometimes hear me complaining about performances ingame).

    New players are, since b267, aware thanks to a short message when they attempt to connect to a large server NS2 hasn't been designed for a such amount of players. When they read this, they should understand performances aren't guaranteed either.

    So, bad or good for this kind of servers seems a bit out of subject here. In my opinion, and personally, I think this system shall be used for a smaller number of slots & regarding of their performances, etc. Like a 18/20 slots with rubberbanding or whatever and seriously, there are a bunch of bad performances servers...
  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    The issue is that currently when u look at the test browser 95% if not more of the servers are not rated good. What this says is less about people running bad servers and more so bad server performance of the game in general. I think the real issue is that for build after build server performance tanks and now rather than server operators hear about how there is plans to fix something like the workshop or server performance we hear this instead. As a result it just comes across like we are being tossed under the buss.

    If virtually all the servers are going to be listed as sub par performance anyways with this what is the point other than to communicate to the players how bad the games server code really is? That is what I mean when I call this entire thing a joke.
  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    ATF wrote: »
    Could you tell us who (by name) suddenly decided that -server performance- is the only and most prominent value by which to decide an unnecessary server browser ranking of each server?

    The most important value of a server is the -level of fun- users perceive.

    ...........
  • the_tickthe_tick Netherlands Join Date: 2014-01-20 Member: 193352Members

    [s
    dePARA wrote: »
    Its funny that both admins of the >24 slot servers disagree with everything here.
    Coincidence? Dont think so.

    Maybe cause there servers are not on top of the serverlist anymore?
    Maybe cause players finally see the issues with these kind of servers?
    Hmm, who knows.


    What I know is that larger servers are more popular, because of the following reasons.

    A, On a map change, there is less chance of a server dying due to more players connected to it.

    B, There is more action on the map, and more to do.

    C, There is less of a feeling that an individual can fuck up the game for the entire team, means there is less pressure to those players. (for example dying as an onos has less impact to the entire team)

    D, There is more of a feeling of teamspirit (which results in more fun)

    E, Attacks can be organized on different fronts, instead of just one or maybe 2


    This has nothing to do with the quality of the server, if this system would that matso is developing is taking the above in account and would calculate that into the equation aswell only then this system would be transparent as the developer says what the purpose should be of this system.

    And the purpose of this system is to define the GOOD servers from the BAD servers. saying the server is BAD by reading performance statistics is completely ludicrous. That is the same like saying Well this pizza is bad because of the thickness ofthe bottom of the pizza, But totally disregards the toppings that are on it.


    This is why this system is retarded, and completely irrelevant. Like I posted before, go focus on the real problems this community is suffering from. Work on a solution for the workshop. 75% of the time not all mods are loaded in on my server. They are important, they define administration, balance, configuration and custom maps.
  • shonanshonan Join Date: 2013-01-28 Member: 182562Members, Reinforced - Shadow
    Thank you for your efforts, finally we can have a proper server performance indicator.
  • GhoulofGSG9GhoulofGSG9 Join Date: 2013-03-31 Member: 184566Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Reinforced - Supporter, WC 2013 - Supporter, Pistachionauts
    edited August 2014
    I can't get why people argue that this will punish large playercount servers.

    The playercount doesn't matter for the server performance rating. All it takes into account is your cpu load, tickrate, sendrate, your network bandwidth and if that fits to the needs of your server.

    So you can still get a good rating for a 24+ slot server if you have a powerfull enought cpu + enought bandweight.

    Server with cpu load issues and too less bandwidth will be rated by this system as bad no matter what playercount they have.

    So if your server is rated as bad try to find out why it's so and fix the issues your server seems to have.
  • PelargirPelargir Join Date: 2013-07-02 Member: 185857Members, Forum Moderators, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, WC 2013 - Silver, Forum staff
    Just to give my opinion, Workshop is an issue, but Steam's issue as far as I know. Doesn't make sense here.

    There are advantages & disadvantages to join a large server or skip it. Personally, I don't like them but I can understand why some people, contrary to me, enjoy to play on these servers. But, you have to admit, this kind of servers doesn't reflect the initial gameplay made by UWE for some reasons i'm not going to explain now & here.

    Your comparison, isn't really appropriate I think. If the performances are terrible, experienced and old players are aware & they know that when they attempt to join these servers, but new players aren't. You can enjoy some games despite the performances, that's right, but, just imagine reactions of new players if they join these servers for their first try, seems logical if they filter out their servers depending of the amount of players.

    GOOD or BAD should show up regarding of the performances. Why? Because as new player, the community & developers want to show them a great image of the game & their servers (that is not currently true on some servers, even some <24 slots). And what the best indicator except performances? You can have different systems, if the server is often full, performances, skill levels, whatever. These indications should appear on the loading time, i don't know, something like: "This server cannot guarantee optimal performances/the amount of slots can affect performances/this server is popular concerning the last visits".

    It really depends on what criterias this system is based. Blame a server for its performances seems legit (personally, as a new player, the unique condition i'm going to rely is performances) but of course, and that can be easily understandable, large servers, which ones cannot ensure acceptable or optimal performances will be shown as BAD.
  • the_tickthe_tick Netherlands Join Date: 2014-01-20 Member: 193352Members
    I am done talking to this CDT,

    269 marks the end for my server, I will take it offline and host for other games where there is better support and stability.


    Hope other large server operators will do the same.



    Bye
  • dePARAdePARA Join Date: 2011-04-29 Member: 96321Members, Squad Five Blue
    Like Wooza said:
    This is not a discussion about large servers.

    Its a discussion about bad performing servers.
    There 16 slot servers out there wich are bad.
    The chance that a large server is marked as "bad" is just higher cause the nature of NS2.

  • lwflwf Join Date: 2006-11-03 Member: 58311Members, Constellation
    edited August 2014
    The level in here... staggering.


    Despite everything said, there is something that seems like it actually does penalize large servers. Excerpt from the comments on the performance scoring code:
    // this is per failed player, so the more players you have, the faster this will drop you
    // A single failed server tick running > interp will throw a 32 player server down the
    // equivalent of 320 ms worth of idle time.
    ...
    // Getting an interpretation overrun means that the server WILL cause players to tick and hitch, and it is the
    // most serious insult a server can deliver. Therefore we punish it severly.
    // Note that this is measured on a per-player basis.

    So if a server with 32 players gets a "lag spike", noticeable by every player (as it probably is if it's down to server performance which we're interested in now, though players will get data sent to them in sequence so not necessarily...), the score penalty will be twice that of a server with 16 players. I don't think that is neither fair for ranking in a browser nor useful for an operator doing performance analysis. In my opinion, if a server hosting 32 players and a server hosting 16 lags equally (every player lags), the score should be equal since the experience of the individual player is equal.

    The obvious but wrong thing to do would be to penalize based on the average players interp issues. But then if not everyone lags but only a smaller portion of players, the smaller server will be more heavily penalized instead...

    This is feedback is only based on what I feel is right (but I'm not sure what's correct), maybe it's accurate enough, I don't know. :)


    By the way, I really appreciate this was implemented in Lua even though it will be done in the engine, so that it can be read, cheers for that! Now for some questions...

    Where does the GetNumInterpFails/Warns come from? If a client is experiencing interp failures due to something not directly related to the server, for example a bad network at the clients end, does that increase the counters, and thus the performance score penalty?

    When viewing the raw tickrate (as shown as the last value in perf row with perfbrowser enabled), it appears that servers with currently bad performance shows their ticks (lower than 30), but every other server show some value often higher than 100. What's that?

    How are custom rates handled? If a server is performing badly, is would increasing interp/decreasing ticks (of various kinds) be an acceptable adjustment? Perhaps this needs be indicated somehow, as that could cause a different kind of lag, although maybe it's a valid option. On a side note I would really want to see the rate settings on the detailed popup, the custom rates warning was just annoying so I had to disable it. :p

    ---

    @the_tick, I played a game at your server and kept track of how the performance score was changing to see how things were doing. The server was full, and ran fine at the start of the game.

    For reference, this is how the browser interprets the performance score (a value ranging from -99 to 100):
    Higher than 20: Good
    Lower than 20: OK
    Lower than 0: Loaded
    Lower than -20: Bad

    About 8 minutes in I started getting brief rubberbanding in the combat. At this point the score was lowered from around 20-30 to down to 15 but then up again to near 20.

    Next few minutes the rubberbanding got significantly worse and the score was lowered to less than 10, often less than 5 and as low as 0.

    A few minutes before the finale the pauses were long enough that it was no longer just some rubberbanding but instead getting the red plug for a second or so, oddly the score increased to above 10.

    After a few red plugs one other player asked if it lagged and another confirmed, so this was not just me.

    Still, during all of this the game remained playable in a casual way and apparently enough players to fill a large server will accept that tradeoff, so I'm sure they still will even if the performance stats are adjusted to be more accurate! Note that the server never once got close to getting a Bad performance rating, I never even saw it reach Loaded. The server browser remained at OK. "The only way to get listed as a Bad server is if the server consistently does BadThings to its players." holds true and I don't see what's so horrible about it. I think you're grossly overreacting.

    I've checked the perfbrowser a few times and it's only the same few servers that are marked as bad, and it's consistently them, so my assumption is they're consistently lagging, and their tickrate was low - your server is not even one of those. Unfortunately I couldn't join any of them because they were all full and I'm not really dying to get in either :P (and by the way, one of the three servers hosted only 16/16 players).


    Edit: Almost forgot, here's the log from the game: http://pastebin.com/N2bBKs74
  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    edited August 2014
    the_tick wrote: »
    saying the server is BAD by reading performance statistics is completely ludicrous.
    Why?
    What is wrong with ensuring players have a smooth, lag free experience like they assume they would get for their money?


    Kids running servers on their laptops occurred in the first few weeks of launch and this prompted us to include the performance tab right away because of the HORRID first time experience customers were receiving. It's always been an insufficient measurement... but you cannot deny such a gauge has been very much needed.

    Lastly, like others have told you : Large servers have nothing to do with a performance gauge!! Some servers can't handle 16 players... and yet its possible to run 40 just fine if you have the hardware.
    This is about providing the admin an accurate measurement of their server load for once, so that they can adjust as necessary to best provide the expected lag free experience to customers.
    Else, you are potentially just another kid with a laptop who does not care about such a thing...
  • the_tickthe_tick Netherlands Join Date: 2014-01-20 Member: 193352Members
    at 6:41 pm the server crashed, (event viewer) server autorestarts (using ntwrapper) A crash causes the server to "lag" because for some unknown reason threading of the application goes WRONG. This means some tasks which normally are performed by the 4 other cores in the cpu now uses the cpu where the server has it's load on. Which causes queues in that cpu core, which causes freezes, red plugs, severe tick rate drops etc etc.

    The only way for me to resolve this is to cold restart, (shutdown and restart the server completely) I wait till the round ends and then perform a restart. Then the server has te repopulate again, and with my luck in this case, """"""""0""""""" mods loaded in, zero, nill nothing, nada, which means there is absoltely no server configuration loaded in. Which is pissing me off so much.


    So to sum this up, due to crashes, (twice a day average) my server performance gets fucked, which means it is a BAD server, which means it gets blacklisted by the newly implemented server browser,

    For me this is the last straw, and I say to you, goodbye.


    and lwf, accoording to your time index you joined after the server crash.
  • ATFATF Join Date: 2014-05-09 Member: 195944Members
    Are the spikes in server load during round end (deleting stuff, placing players in rr) included in the "worst-case discarding"? Because it definitely shows alerts there.

    We are monitoring the performance of the server closely while we keep pushing it to the limits. The new tools will make that a lot more precise. Awesome!
    If such a thing occurs that it goes below a playability / can have fun level, which, regrettably, it did with the 267 patch, settings will be adapted. In the worst case, it will be forced to drop slots.

    Since then a number of regulars asked when it's going to be back to it's original size. It makes Wooza's babbler sad to disappoint them with "We hope to work something out for 269". Fortunately, matso has been utterly helpful in that regard ( Thanks! ).

    I'm all for the RATING and pointing fingers at terribly performing servers. The suggested system labels a server's performance good, ok or bad. Done.
    Yet, I am asking intently, who thought it would be a good idea to additionally RANK servers by the performance rating in the server browser?

    "Bad things" is a term that is used a lot by matso in his post. He's undoubtedly right. But network issues are not the only "bad things" that players experience.
    I've seen rookies ask questions like "Hey, is this also a server that kicks you because you're below a certain hour count?"
    Most servers do not even allow you to pick your team anymore. What label (or should I go so far as to say -ranking-) do those get?

    Showing an accurate performance rating = yay | Ranking by only performance = nay

    Allow me to rephrase ZEROibis' post into a question, since he appears to be getting ignored:
    Does the majority of the servers not receiving a good rating right now signal that all plans to ever increase dedicated server performance have been scrapped and instead the server operators are forced to adjust their hardware or settings? The timing with the official server shutdown comes to mind.

    Server operators that believe the server performance rating (sans the ranking) specifically targets large servers are mistaken. Large servers are accepted to not always have a perfect performance. The small and medium servers are getting the short end of the stick because they are all suddenly not good, only "ok". Those aren't all laptops now, are they?
  • lwflwf Join Date: 2006-11-03 Member: 58311Members, Constellation
    I don't understand how you're coming to this conclusion as you do realize this is a problem with the server, not the browser, which as you say has real implication on the quality of play ("freezes, red plugs, severe tick rate drops etc etc"). The server browser is merely becoming accurate, as you confirm. It really makes it look like you're upset because the bad performance is about to be uncovered some time in the future rather than worrying about already the bad performance lowering the quality of play for your players, which is where you should be concentrating your efforts.
  • matsomatso Master of Patches Join Date: 2002-11-05 Member: 7000Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer
    lwf wrote: »
    The level in here... staggering.

    Despite everything said, there is something that seems like it actually does penalize large servers. Excerpt from the comments on the performance scoring code:
    // this is per failed player, so the more players you have, the faster this will drop you
    // A single failed server tick running > interp will throw a 32 player server down the
    // equivalent of 320 ms worth of idle time.
    ...
    // Getting an interpretation overrun means that the server WILL cause players to tick and hitch, and it is the
    // most serious insult a server can deliver. Therefore we punish it severly.
    // Note that this is measured on a per-player basis.

    So if a server with 32 players gets a "lag spike", noticeable by every player (as it probably is if it's down to server performance which we're interested in now, though players will get data sent to them in sequence so not necessarily...), the score penalty will be twice that of a server with 16 players. I don't think that is neither fair for ranking in a browser nor useful for an operator doing performance analysis. In my opinion, if a server hosting 32 players and a server hosting 16 lags equally (every player lags), the score should be equal since the experience of the individual player is equal.

    Well, the biggest negative score a single one-second performance data will give you is -99, so it won't matter in practice - if the server starts hickuping enough to trigger it, the number of players won't matter; you are pretty much assured a -99 score for that second no matter how many players you have.
    Where does the GetNumInterpFails/Warns come from? If a client is experiencing interp failures due to something not directly related to the server, for example a bad network at the clients end, does that increase the counters, and thus the performance score penalty?

    It comes from how long the interval was between two network updates, compared to the interp variable. Warns is for > 75% of interp, Fails is for > 100% of interp buffer.

    Network latency or client problem does not figure into it, so internet problems will not affect the server performance score.
    When viewing the raw tickrate (as shown as the last value in perf row with perfbrowser enabled), it appears that servers with currently bad performance shows their ticks (lower than 30), but every other server show some value often higher than 100. What's that?

    It it because what is show as tickrate is actually number of times the server goes through the main loop - a lightly loaded server waits 5ms on the network socket before doing a loop, so it tops out at 200 "ticks" a second. It will only do a "proper" tick, ie update the AI enties every 33 ms.

    When a server becomes loaded, it will spend more and more time processing player moves and loop around fewer times.
    How are custom rates handled? If a server is performing badly, is would increasing interp/decreasing ticks (of various kinds) be an acceptable adjustment? Perhaps this needs be indicated somehow, as that could cause a different kind of lag, although maybe it's a valid option. On a side note I would really want to see the rate settings on the detailed popup, the custom rates warning was just annoying so I had to disable it. :p

    The server performance is not affected by ratings - it just looks at the net result. Lots of idle is good, gives you margins. Only when the server is not able to fulfill its obligations will it get bad results.

    That being said, the best way to save yourself from going bad is to increase the interp variable - it allows the server more margin to deliver updates to the players.
    I've checked the perfbrowser a few times and it's only the same few servers that are marked as bad, and it's consistently them, so my assumption is they're consistently lagging, and their tickrate was low - your server is not even one of those. Unfortunately I couldn't join any of them because they were all full and I'm not really dying to get in either :P (and by the way, one of the three servers hosted only 16/16 players).

    Yea, I have also noticed that Woozas and Ticks servers are actually not having server performance issues (well, they are running really close to full, but from what I have seen, they do manage to stay on the right side of the line).

    Considering the amount of red-plugging you actually experience, compared to how well his servers actually manage to perform, I think that Woozas is probably correct in that his server can drive the amount of players he has... or at least close to it.

    The real problem is probably insufficient upload bandwidth (see my other post about that), and that is much more difficult to catch, as the server actually writes the data for the clients in a timely fashion - the problem is that the packets then get queued up/lost at the upload bottleneck.

    The bandwidth unlocking in 267 probably made things WORSE for Woozas... I think he may have tried to increase it and then got even worse problems.

    Required bandwidth goes up by about the square of the number of players ... at a VERY rough guess, you can probably fit 16 players fits inside a 10Mbit line... 32 players would be about 40Mbit or so, but I would definately recommend a 100Mbit line for anyone running > 24 players.

    All in all, I'm kinda pleasantly surprised by how few servers are actually bad.

  • ZEROibisZEROibis Join Date: 2009-10-30 Member: 69176Members, Constellation
    edited August 2014
    If your saying that rate settings are what is holding us back right now how can we adjust these right now. I would like to be able to play around with the settings more before something like this goes live so we have a better understanding of the ratings. Like I said before I find that my server never gets a good rating even when it is constantly at 30 tick and not having dips. I can even have no players on it and it does not rate as good. So something is wrong there and I suppose it is the rates then.

    Also yes onto my note before if we can all stop for a min crying about our individual servers take a look at the list and see how many servers in your area have good performance the majority of them have ok performance around 0.

    I think the reason so many are upset is that it should be noted that the server operators here are trying to do the best we can to balance between offering players a fun server with the number of slots they want and performance impact. There is only so much we can do we can not force intel to magically make 20Ghz single core xeons to run slow server code. It takes a lot of money and time to run and manage servers and when we see stuff like this it makes us feel betrayed.

    In line with what was said before about servers on laptops, this is the sort of thing that really upsets server operators. We feel that basically the view is oh the game is fine nothing wrong with the code or performance at all. Just these evil server operators ruining the game we need to put a stop to these bad guys. Now I am sure there might have been people running severs on laptops or who knows what when the game launched but how long ago was that? Is there really such a major issue with laptop servers we even need to bring that up b/c it just reinforces to server operators that they are not liked and makes them upset.

    The reality is that most server operators already have gotten themselves the best equipment they can for their servers and this is why build after build after build server operators have been looking for more performance from the game and getting less. Operators have been waiting to hear about simple long standing issues with the dedicated servers like the poor workshop implementation being fixed.

    Instead of being told how server performance is going to get better or that server issues are going to be fixed, operators get told how their server is now going to be marginalized and segregated. Rather than actual improvements to the servers, this is what time is being spent on.

    I understand from the devs perspective they want to be able to inform clients about how server performance impacts the game but doing this before even trying to improve anything with regard to the servers is what has everyone up in arms.


    The real problem is that this being such a big issue more so highlights how badly the actual server software in its current form runs on most of the servers. You never hear of this being such a big deal in other games which have properly optimized server code.


    I guess at the end of the day you could call the entire issue on both sides here the pot trying to call the kettle black.


    (on another note what us server operators are calling bad is anything not rated good, it is those of us in the middle or on the edge that are most impacted by this (which is the vast majority of the servers))


    Also do I understand that this system should punish servers that use trubo mode on a cpu to increase permanence when needed or other sorts of cpu based throttling? If so is there anything that can be recommended for us to run to force the core the process is running in to always be in turbo mode to prevent us from being scored negatively?
Sign In or Register to comment.