IronHorseDeveloper, QA Manager, Technical Support & contributorJoin Date: 2010-05-08Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
@dictator93
So the buffered frames hitching may actually be due to streaming textures..
I originally thought it wasn't because i thought i was ruling them out by using wireframe mode.. but it turns out wireframe mode doesn't actually disable the textures or texture streaming.
This would make perfect sense, considering this only started occurring exactly when streaming textures was forced on.
Let me know if you find anything new about it.
Soul_RiderMod BeanJoin Date: 2004-06-19Member: 29388Members, Constellation, Squad Five Blue
Wow, it seems this build has led to a great many discoveries of issues and possible solutions, to even long-standing problems. This patch may turn out to be more valuable than first realised..
@dictator93
So the buffered frames hitching may actually be due to streaming textures..
I originally thought it wasn't because i thought i was ruling them out by using wireframe mode.. but it turns out wireframe mode doesn't actually disable the textures or texture streaming.
This would make perfect sense, considering this only started occurring exactly when streaming textures was forced on.
Let me know if you find anything new about it.
For me it occurs exactly at the exact same areas in most maps. When rounding specific corners, etc. I have always assumed it is due to the vis portal gemoetry entities that are manually placed. That that system itself was responsible...
Hrmmm.
I have tested this quite a bit... and always assumed it was the culling portals... grmmm.
I guess a way to test would be to turn down the textures to horrible levels... and see what happens?
Did much more testing concerning texture settings (since we cannot adjust texture streaming anymore). On Low texture settings I get 700-800 mb VRAM usage. On medium texture settings I get 900-1000 mb VRAM usage.
On these settings, running around an empty map like biodome DOES NOT HITCH.
@ High texture settings I get 1200-1240 mb... and when I run around I GET THE STUTTERING.
My cards have 1280 mb VRAM... oddly enough I really thought that would be enough for this game. hrmm... maybe not on high texture settings... Then again, I also do not remember having this problem before texture streaming was force implemented.
So perhaps it is a problem with high textures in combination with the texture streaming?
How much VRAM do your cards have? and does the stutter stop when you turn down the texture quality?
EDIT: More testing including different levels of anisotropic filtering (1-16X w/ or w/o high quality setting in NVCP). It definitely, at least on my rig, is tied to the HIGH TEXTURE settings.
So either it is a problem with high textures in combo with texture streaming, or my card having its VRAM wall breached when rounding corners. I must say I am surprised that it could be this.
This thread poster was most likely nostrodamus level here. A texture cache streaming size option would be great. That is ofcourse, inaddition to the current "textures" cvar. Other games, like Crysis 3 have it, allowing you to have high quality textures but on a smaller streaming pool. Thereby increasing texture popin on at times, but still allowing you to have high res textures.
See, the thing is, if i turn the textures to low, it lessens the VRAM usage ofc but it also lessens the "WaitingonGPU" in r_Stats.. which is how you reproduce this bug: by being GPU bound.
Whats more, it doesn't solve it for me completely.. i mean.. the hitches are barely notable that way, (which is expected, given what i just said previously) but they are definitely still there, just less so. I was seeing 30ms to 60ms hitches instead of god awful 800ms during combat.
Something i haven't tried: The ULTRA LOW texture mod on the workshop.
Ya that old post is definitely interesting.. he noticed it long ago hehe. Funny that i only began to notice it after reinforced when streaming became forced.
Is there no way to force this through nvidia inspector or driver settings?
Agree, the mouse behaviour still doesn't feel right. Maybe it's mostly in my head, but it doesn't feel smooth and responsive with higher fps as you would expect
Yea, I just found a bug in the Spark Engine that causes the game to behave worse at high fps. The latency-change detector is too sensitive and slightly buggy, so the client spends most of its time chasing a false latency change signal. The speed at which it chases is 2ms per frame, so at 60fps it constantly shifts between running 12% (each frame takes 16ms, 2ms change per frame = 2/16 -> ~12%) too fast and 12% too slow, while at 200 fps it shifts between +40% and -40% (5ms/frame, 2 ms change per frame -> ~40%).
As this changes 20 times per second (every time the client receives an update from the server), it averages out and may be too fast to "see" (like how our brains stiches movie frames together); but you just feel that something isn't quite right.
This is probably the source both of the mouse responsiveness not being quite right and a feel that the movement isn't quite as smooth as the fps says it should be - at least I think so from my single-player/empty server testing.
I've submitted a patch (oh, thanks for the old assets @Soul_Rider - this was the bug I was referring to), I have high hopes that it will get fixed (the fix is rougly a 5-liner; just introduce a low-pass filter on the latency detector and the chase amplitude drops to <1%).
Knew there was SOMETHING wrong with high fps, people been complaining about it for years now (wow it's been that long?), good to see you found the possible cause.
@shonan - I was using a c2d E7400 and a HD5770, on lowest settings, getting around 50-60fps but now I have a 4670k@4.4ghz and a 560ti, but I can run on high with around 130-150fps. What other settings are you using, because you should not need to run on lowest settings with your rig...
Yeah, you shouldnt, but you need to in NS2. And FPS doesnt mean anything, as even matso confirms above.
This latest patch did however seem to improve the texture streaming issues, looks like I can now use high texture quality whereas before it was impossible due to constant hitching thanks to texture streaming.
Soul_RiderMod BeanJoin Date: 2004-06-19Member: 29388Members, Constellation, Squad Five Blue
The fix from Matso is about mouse smoothness and general mouse responsiveness. The feeling of the game at high FPS. It won't have any affect on 'hitching'. That is purely related to texture streaming. That will require a separate fix.
See, the thing is, if i turn the textures to low, it lessens the VRAM usage ofc but it also lessens the "WaitingonGPU" in r_Stats.. which is how you reproduce this bug: by being GPU bound.
Whats more, it doesn't solve it for me completely.. i mean.. the hitches are barely notable that way, (which is expected, given what i just said previously) but they are definitely still there, just less so. I was seeing 30ms to 60ms hitches instead of god awful 800ms during combat.
Something i haven't tried: The ULTRA LOW texture mod on the workshop.
Ya that old post is definitely interesting.. he noticed it long ago hehe. Funny that i only began to notice it after reinforced when streaming became forced.
Is there no way to force this through nvidia inspector or driver settings?
Examining my FPS fluctuations more, I do notice hitching at even low and medium (but not very large amounts: the difference in milliseconds per frame is not as large on low and med textures). It is within the human threshhold of not being totally bad. But still Problematic. So right now we do know at least that it is tied to texture streaming, and gets progressively worse the higher your textures are set.
Sadly, there is no way to set a texture streaming cache through NVCP, it is all done in engine. Currently the engine probably does it automatically. If perhaps a possibiliity was opened up for us to try different stream cache sizes (256, 512, 768, 1024, etc...), we could see more as to what is causing the problem.
The best way to check oddly enough if it is a VRAM wall problem in total, would be run the game at a very high resolution, but with lower teyxture res. Your VRAM would not be throttled, thereby disproving that hypothesis, but you would still be heavily pixel fill rate bound on the GPU. BRB, gonna try this.
edit_update: Having a hard time seeing the spiking in the fast moving graph on medium and low textures (@ 1080p and 1440p). There are definite tiny spikes in frame rendering at similar locations on medium and low textures, but they are within such a smaller ms threshold. That they are indeed less less noticable. But still, the textures streaming in due cause a frame drop.
I captured the exact moment of what happens when textures streaming in on high texture settings.... I noticed something very interesting.
The in game moment when I round a corner and new textures stream in, causing a huge stutter. You can see the Milisecond per frame shoot up. This is well documented.
But a less documented thing:
At that exact moment captured in the frame above, the amount of VRAM used temporarily shoots up and goes against my VRAM wall. So at the moment of streaming a new batch, you actually use more VRAM then you have... but while running around... it stays in a threshhold below your VRAM amount.
For example: 1180 mb VRAM when walking around with no new textures streaming. New loading textures spikes up to about 1240 mb (knowing that my desktop and windows takes up about 50-70mb VRAM, that means I am disk scratching for Video storage at that moment, and a huge halt in frame miliseconds). Then it drops down again into a level of VRAM taken up that is not choking the GPU (1180mb again/1280mb).
If we could adjust texture streaming cache size I could guarantee this would not happen most likely, because when it shoots up, it would have more headroom. Similarly, if streaming new textures in did not have such a large differential in VRAM space utilized, something similar would not happen.
matsoMaster of PatchesJoin Date: 2002-11-05Member: 7000Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer
Use p_logall to save the performance profile to a file and then use the utils/PerfAnalyser/perfanalyser.py program to analyze it afterwards.
Much simpler than having to have the profiler up all the time, though you do need to have python64 installed.
IronHorseDeveloper, QA Manager, Technical Support & contributorJoin Date: 2010-05-08Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
@matso
All profiler or a plog shows is the function "WaitingForBufferedFrames" like previously shown: http://i.imgur.com/TzZ6XVp.jpg
Nothing different.
@dictator93
I'm keeping you in the loop in PMs, but just for public records sake for everyone else tracking:
The engine devs are on this as of today!!
In my day we had MHz, and the more you had the better it was. Going from a 100 MHz to 400 MHz was awesome. :ar!
Ever since we hit 2 GHz and we started having these generation things I haven't had a clue about what power someones processor is. Damn youngsters. MHz was good times. ~O)[-(
In my day we had MHz, and the more you had the better it was. Going from a 100 MHz to 400 MHz was awesome. :ar!
Ever since we hit 2 GHz and we started having these generation things I haven't had a clue about what power someones processor is. Damn youngsters. MHz was good times. ~O)[-(
It's the same now as it was then .
Smaller architecture (transistors) means more actual performance per mhz. About 5-10% improvement per clock every intel generation with these 'newer cpus'.
Comments
So the buffered frames hitching may actually be due to streaming textures..
I originally thought it wasn't because i thought i was ruling them out by using wireframe mode.. but it turns out wireframe mode doesn't actually disable the textures or texture streaming.
This would make perfect sense, considering this only started occurring exactly when streaming textures was forced on.
Let me know if you find anything new about it.
For me it occurs exactly at the exact same areas in most maps. When rounding specific corners, etc. I have always assumed it is due to the vis portal gemoetry entities that are manually placed. That that system itself was responsible...
Hrmmm.
I have tested this quite a bit... and always assumed it was the culling portals... grmmm.
I guess a way to test would be to turn down the textures to horrible levels... and see what happens?
Did much more testing concerning texture settings (since we cannot adjust texture streaming anymore). On Low texture settings I get 700-800 mb VRAM usage. On medium texture settings I get 900-1000 mb VRAM usage.
On these settings, running around an empty map like biodome DOES NOT HITCH.
@ High texture settings I get 1200-1240 mb... and when I run around I GET THE STUTTERING.
My cards have 1280 mb VRAM... oddly enough I really thought that would be enough for this game. hrmm... maybe not on high texture settings... Then again, I also do not remember having this problem before texture streaming was force implemented.
So perhaps it is a problem with high textures in combination with the texture streaming?
How much VRAM do your cards have? and does the stutter stop when you turn down the texture quality?
EDIT: More testing including different levels of anisotropic filtering (1-16X w/ or w/o high quality setting in NVCP). It definitely, at least on my rig, is tied to the HIGH TEXTURE settings.
So either it is a problem with high textures in combo with texture streaming, or my card having its VRAM wall breached when rounding corners. I must say I am surprised that it could be this.
EDIT 2: Research into everything leads me to this thread: forums.unknownworlds.com/discussion/123910/sugestion-bugfix-streaming-related-console-command
This thread poster was most likely nostrodamus level here. A texture cache streaming size option would be great. That is ofcourse, inaddition to the current "textures" cvar. Other games, like Crysis 3 have it, allowing you to have high quality textures but on a smaller streaming pool. Thereby increasing texture popin on at times, but still allowing you to have high res textures.
1 gb 570gtx
See, the thing is, if i turn the textures to low, it lessens the VRAM usage ofc but it also lessens the "WaitingonGPU" in r_Stats.. which is how you reproduce this bug: by being GPU bound.
Whats more, it doesn't solve it for me completely.. i mean.. the hitches are barely notable that way, (which is expected, given what i just said previously) but they are definitely still there, just less so. I was seeing 30ms to 60ms hitches instead of god awful 800ms during combat.
Something i haven't tried: The ULTRA LOW texture mod on the workshop.
Ya that old post is definitely interesting.. he noticed it long ago hehe. Funny that i only began to notice it after reinforced when streaming became forced.
Is there no way to force this through nvidia inspector or driver settings?
This latest patch did however seem to improve the texture streaming issues, looks like I can now use high texture quality whereas before it was impossible due to constant hitching thanks to texture streaming.
Examining my FPS fluctuations more, I do notice hitching at even low and medium (but not very large amounts: the difference in milliseconds per frame is not as large on low and med textures). It is within the human threshhold of not being totally bad. But still Problematic. So right now we do know at least that it is tied to texture streaming, and gets progressively worse the higher your textures are set.
Sadly, there is no way to set a texture streaming cache through NVCP, it is all done in engine. Currently the engine probably does it automatically. If perhaps a possibiliity was opened up for us to try different stream cache sizes (256, 512, 768, 1024, etc...), we could see more as to what is causing the problem.
The best way to check oddly enough if it is a VRAM wall problem in total, would be run the game at a very high resolution, but with lower teyxture res. Your VRAM would not be throttled, thereby disproving that hypothesis, but you would still be heavily pixel fill rate bound on the GPU. BRB, gonna try this.
edit_update: Having a hard time seeing the spiking in the fast moving graph on medium and low textures (@ 1080p and 1440p). There are definite tiny spikes in frame rendering at similar locations on medium and low textures, but they are within such a smaller ms threshold. That they are indeed less less noticable. But still, the textures streaming in due cause a frame drop.
I captured the exact moment of what happens when textures streaming in on high texture settings.... I noticed something very interesting.
The in game moment when I round a corner and new textures stream in, causing a huge stutter. You can see the Milisecond per frame shoot up. This is well documented.
But a less documented thing:
At that exact moment captured in the frame above, the amount of VRAM used temporarily shoots up and goes against my VRAM wall. So at the moment of streaming a new batch, you actually use more VRAM then you have... but while running around... it stays in a threshhold below your VRAM amount.
For example: 1180 mb VRAM when walking around with no new textures streaming. New loading textures spikes up to about 1240 mb (knowing that my desktop and windows takes up about 50-70mb VRAM, that means I am disk scratching for Video storage at that moment, and a huge halt in frame miliseconds). Then it drops down again into a level of VRAM taken up that is not choking the GPU (1180mb again/1280mb).
If we could adjust texture streaming cache size I could guarantee this would not happen most likely, because when it shoots up, it would have more headroom. Similarly, if streaming new textures in did not have such a large differential in VRAM space utilized, something similar would not happen.
Much simpler than having to have the profiler up all the time, though you do need to have python64 installed.
All profiler or a plog shows is the function "WaitingForBufferedFrames" like previously shown: http://i.imgur.com/TzZ6XVp.jpg
Nothing different.
@dictator93
I'm keeping you in the loop in PMs, but just for public records sake for everyone else tracking:
The engine devs are on this as of today!!
Ever since we hit 2 GHz and we started having these generation things I haven't had a clue about what power someones processor is.
Smaller architecture (transistors) means more actual performance per mhz. About 5-10% improvement per clock every intel generation with these 'newer cpus'.
Roughly giving something like
5ghz sandy = 4.5ghz ivy = 4.1ghz haswell