Do the "Performance-Jam"!!!

124678

Comments

  • durdenns2durdenns2 Members Join Date: 2013-01-30 Member: 182634Posts: 16
    SteveRock wrote: »
    And since you seem to be fairly technically minded, the major challenge we face with LuaJIT is the Lua->C++ API. For LuaJIT to be performant, you have to architect that API in some pretty specific ways, otherwise you will always break the JIT compiles and get almost no benefit from it. Completely changing the API is not an attractive option (rewrite a lot of code..break all the mods..), so we're currently investigating ways of avoiding this.

    So we're basically boned for a really long time?
    ShinoLocklear
  • RockyMarcRockyMarc Members, Reinforced - Shadow Join Date: 2009-11-24 Member: 69519Posts: 474 Fully active user
    edited May 2013
    So we're basically boned for a really long time?

    :( sad face

  • DestherDesther Members Join Date: 2012-10-31 Member: 165195Posts: 520 Advanced user
    xDragon wrote: »

    The overall truth is that there hasn't been experiments on a stripped down codebase to see what would be possible in Spark with a vastly simplified game logic AFAIK, really that's something I would have expected to see from the combat team - they could remove massive amounts of game logic. I don't see there being notable improvements in frame rates unless you massively simply the game logic at this point, baring underlying engine changes that speed up the LUA/Renderer. I think they have optimized their current logic close to as far as possible.

    Go and play the Fighter mod from the Mod Jam. In normal NS2 I get ~40 average fps and in the fighter mod I get 150 fps.

  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    edited May 2013
    Hell, even Planetside 2 runs far better...
    That's about the most damning comment you can possibly make about performance.

    I didn't know minstrel was on teh forums
    hi minstrel :D



    On that note, I heard about the luaJIT too, and how it's a really fast scripting what not (sorry I don't know much about this)

    But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though :D

    edit: Although I too am not happiest with performance I still find it very playable in low playercounts and especially on the NSL maps, and I really hope it improves in the future because this is one of my favorite and only played games at the moment and also one of my favorite development teams :D
  • DestherDesther Members Join Date: 2012-10-31 Member: 165195Posts: 520 Advanced user
    ezekel wrote: »

    But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though :D


    OpenGL and D3D are the lower level drawing APIs you use to program the graphics hardware. Stuff like loading textures into graphics memory, manipulating the drawing buffers, loading shaders and drawing the triangles. They are fairly similar in concept but with different function names and syntax.

    There shouldn't be any difference in visuals going from DX9 to DX11 as all the art assets and shaders should be the same, unless UWE get a fat cheque from Microsoft and add DoF, motion blur and lens flare and put a DX11 sticker on the box.

  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    edited May 2013
    Desther wrote: »
    ezekel wrote: »

    But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though :D


    OpenGL and D3D are the lower level drawing APIs you use to program the graphics hardware. Stuff like loading textures into graphics memory, manipulating the drawing buffers, loading shaders and drawing the triangles. They are fairly similar in concept but with different function names and syntax.

    There shouldn't be any difference in visuals going from DX9 to DX11 as all the art assets and shaders should be the same, unless UWE get a fat cheque from Microsoft and add DoF, motion blur and lens flare and put a DX11 sticker on the box.

    I don't care about visuals, and from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11 (visual wise of what they're capable of)

    However if there's a performance benefit, then that sounds nice. I play any type of competitive fps game on lowest settings possible so textures are simple and not distracting, the only thing you want to see is your opponent and nothing else; plus you want performance

  • HamletHamlet Members, Reinforced - Shadow Join Date: 2008-08-17 Member: 64837Posts: 373 Advanced user
    edited May 2013
    ezekel wrote: »
    from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11

    Are you currently living under a rock?

    DirectX11 provides many neat new things like better multi-threading, tessellation and GPGPU...
    Hell, check out this list of games with DirectX 11 support.
  • MilaniumMilanium Members Join Date: 2012-12-27 Member: 176388Posts: 96 Advanced user
    edited May 2013
    If they had licensed Valve's Source engine (maybe at a friendship price for an upcoming Indie developer) I bet there would already be a Linux version. :( Would not have minded that the tighter corset of the Source engine meant there would be fewer innovative features and harder modding. The artwork quality of Unknown Worlds is still extraordinarily and would definitely shine bright on Source 1, too. All I want is to play this game on mid-end hardware without choppy performance.

    Does anyone know how I can return games on Steam and get a refund? I could than re-buy Natural-Selection 2 once they fixed their engine.
  • xDragonxDragon Members, NS2 Playtester, Squad Five Gold, NS2 Map Tester, Reinforced - Shadow Join Date: 2012-04-04 Member: 149948Posts: 1,954 Fully active user
    For anyone curious (and ScardyBob) i made a chart with some basic frame time data. I got this data from a listen server with some bots while i just ran around a bit, I am hoping to get some 'real' data later today from pubs. http://www.mediafire.com/download/x02f414sw7n55ji/NS2FrameTimes.xlsx is the spreadsheet.
  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    ezekel wrote: »
    But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though :D

    The primary reason this is being done right now is to be able to port the game over to other platforms, like MAC and Linux. However, we are also hoping there will be some additional performance boost on the GPU side as well, though exactly how much is unknown at the moment. Between this and the LuaJITstuff that is being worked on, which will help out on the CPU side, there should be noticeable performance improvements coming.

    This whole post sounds fantastic, hope it works out
  • SteveRockSteveRock Members, NS2 Developer, Subnautica Developer Join Date: 2012-10-01 Member: 161215Posts: 486 Advanced user
    moultano wrote: »
    SteveRock wrote: »
    And since you seem to be fairly technically minded, the major challenge we face with LuaJIT is the Lua->C++ API. For LuaJIT to be performant, you have to architect that API in some pretty specific ways, otherwise you will always break the JIT compiles and get almost no benefit from it. Completely changing the API is not an attractive option (rewrite a lot of code..break all the mods..), so we're currently investigating ways of avoiding this.

    Tell us more! Reading about things like this gives me a "break out the profiler" itch. It's exciting. Is there a performance sensitive subset you could rewrite, and assume the remainder will fall back to being interpreted?

    One challenge is even identifying the hotspots, so Dushan has been working on a sample-based Lua profiler to help with this. Think of it as Intel VTune for Lua scripts (cuz otherwise, VTune just tells us...Lua is slow). Once that is done, we can better assess if it's possible to address just that subset.
  • CrazyEddieCrazyEddie Members Join Date: 2013-01-08 Member: 178196Posts: 657
    Milanium wrote: »
    Does anyone know how I can return games on Steam and get a refund?
    According to the Google search I just did, Valve's customer service manager is Helen Waite. So if you want a Steam refund you can go to Helen Waite for it.
    LerkinOff0ni
  • CrazyEddieCrazyEddie Members Join Date: 2013-01-08 Member: 178196Posts: 657
    Steve, Cory - as long as you're answering performance and technology questions:

    Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
  • matsomatso Master of Patches Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer Join Date: 2002-11-05 Member: 7000Posts: 1,554 mod
    CrazyEddie wrote: »
    Steve, Cory - as long as you're answering performance and technology questions:

    Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.

    Right now the client uses slightly more than 2 cores at peak load, so further MT would not benefit 2-core machines much. Better to go for LuaJIT, which uses the CPU more efficiently.
    Member of CDT, Senior Spark Engine Hacker
  • CrazyEddieCrazyEddie Members Join Date: 2013-01-08 Member: 178196Posts: 657
    I guess it does make sense to spend modest effort on modest gains for everyone before spending huge effort on huge gains for the best PCs.

    Thanks for the reply.
  • ZEROibisZEROibis Members, Constellation Join Date: 2009-10-30 Member: 69176Posts: 1,017 Advanced user
    edited May 2013
    I have one simple thing you guys could add that is not major but does help in late game performance... file appending. I know this crazy new age technology only recently developed by scientists sounds risky but it can and does effect performance. In addition the lack of file appending also hurts mods in that for example an admin mod can not have a single log file dedicated to each admin as over time the file would kill the server due to lack of file appending.

    To test how the lack of file appending effects NS2 vs a game like cs:s I had a defrag operation run to simulate higher I/O load on the HDD. The cs:s server actually writes much more data to disk due to constant demo recording along with normal txt logging however it was totally unaffected by the increased I/O load. NS2 on the other had turned into a slideshow until logging was disabled at which time the game then ran as smoothly as it should. Given that we know that it was the logging in ns2 causing the lag and that there was no lag in cs:s from logging under the same conditions we can logically assume that the lack of file appending is the cause for this lag.
    image
    Server IP: NS2.IBISGaming.com
    Server IP: NS2CO.IBISGaming.com
  • lwflwf Members, Constellation Join Date: 2006-11-03 Member: 58311Posts: 486 Fully active user
    Half of Steam users CPUs has more than 2 cores. Many games on Steam are not heavy, for a game like NS2 better CPUs should be even more common.

    http://store.steampowered.com/hwsurvey/cpus/
  • SteveRockSteveRock Members, NS2 Developer, Subnautica Developer Join Date: 2012-10-01 Member: 161215Posts: 486 Advanced user
    CrazyEddie wrote: »
    Steve, Cory - as long as you're answering performance and technology questions:

    Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.

    We do this for prediction already, which resulted in huge gains, but that's literally running two separate Lua worlds. Doing it within the same world is more complex, and much more risky (in terms of bugs, code maintainability, whether or not it would even work, etc. etc.). So, it's less likely we'll pursue this avenue.
  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    edited May 2013
    Hamlet wrote: »
    ezekel wrote: »
    from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11

    Are you currently living under a rock?

    DirectX11 provides many neat new things like better multi-threading, tessellation and GPGPU...
    Hell, check out this list of games with DirectX 11 support.

    I don't think you properly read my post, I said visually no game has done what direct x 10 or 11 is capable of doing, all I've seen is some fancy lighting with the suns rays and "tessellation" can you name a game that fully utilizes everything dx10 has to offer? Or dx11?

    Not talking at all about games which "use" them or have one feature and try and market it as a selling point "tressFX, look at laras hair its so real!!1!"

  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    matso wrote: »
    CrazyEddie wrote: »
    Steve, Cory - as long as you're answering performance and technology questions:

    Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.

    Right now the client uses slightly more than 2 cores at peak load, so further MT would not benefit 2-core machines much. Better to go for LuaJIT, which uses the CPU more efficiently.

    According to steamstats hardware server http://store.steampowered.com/hwsurvey

    Majority of users are on direct x 11 capable cards, and majority have 4 cores or more (add up 4 cores + other) it's extremely close to 2 core users though, but will continue to trend only upward
  • herakl3sherakl3s Members, Reinforced - Shadow Join Date: 2010-12-22 Member: 75852Posts: 61 Advanced user
    Huh Ezekiel?
    Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
    +2% every 4 months by the graph so the majority will be on 4 cores in December/january!
    UWE it's ok you got time for the performance boost :>
  • xen32xen32 Members, Reinforced - Supporter Join Date: 2012-10-18 Member: 162676Posts: 1,011 Advanced user
    herakl3s wrote: »
    Huh Ezekiel?
    Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
    +2% every 4 months by the graph so the majority will be on 4 cores in December/january!
    UWE it's ok you got time for the performance boost :>

    extrapolating.png
    TweadleshonanArgosh
  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    edited May 2013
  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    herakl3s wrote: »
    Huh Ezekiel?
    Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
    +2% every 4 months by the graph so the majority will be on 4 cores in December/january!
    UWE it's ok you got time for the performance boost :>

    334432.4.jpg
    AurOn2
  • matsomatso Master of Patches Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer Join Date: 2002-11-05 Member: 7000Posts: 1,554 mod
    SteveRock wrote: »
    CrazyEddie wrote: »
    Steve, Cory - as long as you're answering performance and technology questions:

    Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.

    We do this for prediction already, which resulted in huge gains, but that's literally running two separate Lua worlds. Doing it within the same world is more complex, and much more risky (in terms of bugs, code maintainability, whether or not it would even work, etc. etc.). So, it's less likely we'll pursue this avenue.

    The extra Lua VM running the Predict world is very much a special case; it it responsible for taking the server network updates and merge it with any missing client moves, then deliver that to the main thread. It reduced an O(fps^2) problem in the main thread to an O(network rate) in a secondary thread - a very good thing.

    Also increased fps so much that Max had to add move rate limiters to avoid servers imploding under the load ... oh, fun times, good times :-)

    (http://forums.unknownworlds.com/discussion/122189/224-tech-changes-part-1 for those interested in more details )

    Lua does not support true multithreading (yet), so if you want to split the logic work you would probably go for using multiple Lua VMs running in their own threads, and then have a merge phase in between... doable, but adding lots of complexity.

    So going for LuaJIT first is the right way ... hopefully, it will be fast enough to tide Spark over until Lua supports true multithreading.
    Member of CDT, Senior Spark Engine Hacker
  • moultanomoultano Creator of ns_shiva. Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts Join Date: 2002-12-14 Member: 10806Posts: 4,219 Advanced user
    edited May 2013
    SteveRock wrote: »
    One challenge is even identifying the hotspots, so Dushan has been working on a sample-based Lua profiler to help with this. Think of it as Intel VTune for Lua scripts (cuz otherwise, VTune just tells us...Lua is slow). Once that is done, we can better assess if it's possible to address just that subset.

    Wow, I can't live without my trusty pprof. It wouldn't surprise me if there were an obvious 20% gain that fell out of the first look at the graph.
  • ScatterScatter Members, Squad Five Blue Join Date: 2012-09-02 Member: 157341Posts: 635 Advanced user
    Is LuaJIT something that must be implemented game wide or can it be used in certain critical pathways? I have a basic coding understanding but not enough to really know this off hand.
  • CrazyEddieCrazyEddie Members Join Date: 2013-01-08 Member: 178196Posts: 657
    matso wrote: »
    It reduced an O(fps^2) problem in the main thread to an O(network rate) in a secondary thread
    Oooh you sexy thing.
    Lua does not support true multithreading (yet), so if you want to split the logic work you would probably go for using multiple Lua VMs running in their own threads, and then have a merge phase in between...
    Yeah, that's what I was imagining.
    doable, but adding lots of complexity.
    I was imagining that, too. :)
    So going for LuaJIT first is the right way ... hopefully, it will be fast enough to tide Spark over until Lua supports true multithreading.
    Would in-VM multithreading buy you that much vs. using multiple-VMs? Isn't the hard part making the game logic multithreadable in the first place, i.e. separating out the parts that can run asynchronously from each other?

    My apologies if I'm asking absurd questions here. I know just a little bit about these topics and nothing at all about your code.

  • ezekelezekel Members, NS2 Map Tester Join Date: 2012-11-29 Member: 173589Posts: 1,385 Advanced user
    Scatter wrote: »
    Is LuaJIT something that must be implemented game wide or can it be used in certain critical pathways? I have a basic coding understanding but not enough to really know this off hand.

    I think luaJIT splits the workload, so one part of the game goes through luaJIT, then another part goes through regular lua.. so making the work more evenly split on the cores already supported

    or I'm an idiot and you can just disregard what I said because it's a complete guess
Sign In or Register to comment.