Hunting 'Jitter' in Subnautica - Subnautica
System
Join Date: 2013-01-29 Member: 182599Members, Super Administrators, Reinforced - Diamond
Join Date: 2013-01-29 Member: 182599Members, Super Administrators, Reinforced - Diamond
Hunting 'Jitter' in Subnautica - Subnautica
I’ve been filming Subnautica for the purpose of creating Seamoth promo assets for several days now. For some reason, this video has been breaking my balls. I think I’ve discovered...
Comments
I had a large base one from the island one room up two down and a moonpool. From there down to the sea bed at 265 meters. Used a very long down tube and it got very bad frame rates. My rig plays everything else smoothly and quickly. I was getting around 144 frames a second on recommended setting until my base got very large with lots of stuff.
Not only do I *completely* agree with your assessment on the concern for the quality of the product and the impact it can have (looking at you Batman Arkham Knight) as far as reputation and consumer satisfaction goes - but I have been testing and reporting this issue for about a year now, as it very much negatively impacts my enjoyment of Subnautica considering how immersive it is intended to be, and how disrupting these frametimes are to the experience.
You and me are not alone in noticing this issue, either.
Dc darkling has done some impressive legwork regarding this issue in this very forum.
I have created some trello cards in regards to testing out forcing real fullscreen (since Unity defaults to fullscreen windowed) - and since then have abandoned this as the cause as I found a hack-ish video driver way to force SN to go true fullscreen (and combined with utilizing the engine's Vsync for extra effect) still did not see the jitters disappear.
Since then, I've had some talks with Jonas about the origins of this issue.. and from what it sounds like it can be narrowed down to Unity's archaic implementation of garbage handling.
If you google unity and garbage collector you will find many many complaints about it, and the limitations involved due to its age. There are some workarounds (many are already employed in SN from what I understand?) and at this point I am unsure as to what can be done further...
Would adding an additional frame (utilizing Max Frames Rendered Ahead) assist in smoothing it out at the expense of mouse input delay?
While I (and most PC gamers) absolutely loathe mouse input delay, I doubt anyone would mind in a single player game if the alternative was the jittery and spiking frame times.
I cannot tell you how excited I am that this has caught your attention, and truly hope it can be resolved in any way necessary.
I can reproduce this issue on a moment's notice, (circle strafing a vertical object at melee range with high FPS) so if there is any further data required or help reproducing it, I am sure @DC_Darkling and myself are here to assist!
As @IronHorse described, Unity's internal (Mono v2.6 based) garbage collector is utterly terrible. In my recent foray into learning Unity I discovered just how terrible some facets of Unity are (I'm referring to Unity 5.1.x). Not only is the current one quite bad, it is also largely uncontrollable due to the analysis method (https://en.wikipedia.org/wiki/Boehm_garbage_collector) ...which essentially means it _will_ GC at every chance it deems it needs to. Regardless of it "should" clean up memory. Also, if the game is not being fully compiled with Visual Studio, versus Unity itself...it should be. Doing so gains a more optimized CIL (byte code) as a result of VS's superior compiler semantics. Not too mention it goes MUCH deeper than that...Unity doesn't have a "clone X" operation...anytime you want to duplicate something (like a Peeper, for example) you have to run it through Unity's *massively* heavy weight "GameObject.Instantiate()" method...which can trigger GC actions (not just allocations). On, and on, and on...no sense in listing out all the pains, as they can be found with just a little bit of Google-Fu
Back to your original topic, I firmly believe smoothness is vital to delivering excellent player experiences. For singleplayer games, this doesn't even have to be ultra-high frame rates. Consistency is much more important than max-limits. And with a sandbox / adventure title such as Subnautica, imo, immersion is a very large facet of the game's delivered experience. So...anything that breaks immersion, should be eliminated and quickly.
I forgot to mention the worst part...Unity's API is NOT thread-safe. That means calls back into Unity itself (which is damn near everything, or rather anything derived from UnityEngine.Object) must be executed on the program's main thread. An excellent use-case of multi-threading for Subnautica would be a Look-Ahead terrain streamer. In other words, a child thread is loading the terrain outside of the player's FOV, basically in a radius around the player but not drawn on-screen yet. This would allow the world-data to be ready to be used and just sitting in memory, instead of on disc.
However, since Unity is fundamentally not thread-safe (there are some workarounds, but they're limited and sensitive to which OS program is running on), all of the loading has to be done in the main thread. I'd be willing to be this is the primary cause of jitters / hitching in SN right now. Whenever more of the terrain needs to be loaded, everything (creatures, VFX, rendering, etc) has to wait until said loading routines complete. Broadly speaking if the total main-thread execution time is at or above ~33ms then there will be inconsistent frametimes.
It very well might be possible to stream _just_ the geometry data for the terrain in a child-thread, but I doubt the problem of art assets (grass, textures, etc) could be streamed. So, the visual-issue of "pop in" would likely still remain. I'm just spitballing here but I can say this would not be a simple change, and likely require some significant reengineering time.
Improvements, after all, does not equal solution.
I do not think a slight input delay is a problem in subnautica due to the nature of the game. It is relative slow paced.
I do think, as I described before in the topic @Ironhorse linked, that any performance issue, especially with visuals, is a BIG concern.
Simply put, if it was not alpha when I tested the jittering but a 'released' game, I would have refunded with such sub-par performance.
Especially considering the genre, a exploration game needs stuff to explore. Meaning beautiful surroundings for one. Meaning you want to be able to look at those surroundings.
Having it a slideshow would beat the whole purpose of the game. (Ever tried a slideshow exploration? I did, in subnautica, it was brutal.)
damnit, AGAIN world updates... I shall try the newer world soon.
What I did notice is that frametimes currently have a heartbeat spike. Enable frametime graph, and do not move. You can see a bigger spike pop up as a heartbeat.
All the spikes are currently noticable.
Moving does not truly provide a smooth experience. However sitting in a not so interesting area and just rotating while not moving, does produce a more smooth experience. (more, not fully).
This seems to stay even on a gamespeed of 0.01, where 1 is normal speed. Rotation is fairly smooth. (moving instead of rotating on a slower gamespeed does obviously stutter like mad)
I'm by no means expert on Unity or its GC though, it could very well be something in the rendering process or world update that makes the GC trigger.
Old upgraded savegame.
* The heartbeat frametime spike matches the swimming bobbing. You know, when in subnautica we hover in water and our view goes slightly up/down due to us floating?
THATS this smaller frametime which so loyally shows up every x seconds.
* Performance compared to a few patches ago, is down across the board.
* Rotating like Ironhorse described is still far from smooth. Forward motion is better.
* I can still not match the spikes to the big (what was is, 25mb, 35mb) io spikes on the disk requiring new data.
New game:
* performance goes down across the board. I notice compared to my old save, that safe shallow is very much so filled with fish. (I assume I ate most on my old save.
This may be why performance is down. I am not sure. But larger framespikes. And more often.
The only thing I have not tried is to not just ramdisk the saved game, but everything. (SImply because I have no program which does that in such huge GB amounts, for free)
Essentially the GC was considered a proof of concept algorithm when made. (Mono 2.6.5, using Boehm GC)
A simple mark-and-sweep collector is a blocking collector – it would pause the entire program so that it can process the entire heap at once.
The current versions of Mono, Java, .Net, etc, etc use what are called generational algorithms which can collect short-term garbage much more quickly and with less of a pause and are really what is used for real time applications like games.
Unity have stated they will not update their GC for some extremely odd reason..
Supposedly the key to working around this is not creating/destroying objects during game play, but to create a pool of required objects which are accessed as needed?
@Sewlek @LumpN @Hugh Have you guys looked into Unity's IL2CPP? It is currently using Boehm GC but they still plan on updating it with the open-source CoreCLR garbage collector at some point supposedly.
From what I understand, IL2CPP is just a process that converts your C# code in Unity to C++ much faster, basically, and comes with performance boosts. (and hopefully one day a better GC?)
Let us know what we can do to assist / recording / reproducing.
Ok, last time I tested a ramdrive with JUST the saved game. Free versions of such programs usually have enough ram drive to barely fit that in.
But what if we fit MORE into a ramdrive? Considering a computer with a high speed SSD on which subnautica is installed, would it matter?
After obtaining a copy of dimmdrive on steam, I decided to see what I could do.
Now subnautica has grown since last I attempted, and I was worried if it would fit in the ramdrive. A few tests later and I was convinced I could not ramdrive whole subnautica. (There was no memory to reasonably run the game.)
Excluding the game files and certain other files, we got the following setup:
First a benchmark run without a ramdisk. Comparison shots.
Frametime spikes are present and huge. (not shown in picture)
More interesting however are the average frametimes, so ignore the small spike in this one:
Then a run with dimmdrive running.
Spikes are just as huge (shown on pic), but look at the higher frametimes:
Hunting in a screenshot of, roughly the same length, a subnautica run, shows the reason.
Without ramdrive:
With ramdrive
I roughly lack 150MB+ of memory with a dimmdrive to run subnautica 'well' so it has to page more to disk.
Consider you usually want to open up some breathing room, I would need to ramdisk even less. (And I am yet uncertain what to exclude)
I shall edit this in a few min with a smaller ramdisk.. To be continued.
>edit
The good news is that with a more acceptable dimmdrive setup, there are no longer any additional pages going on. It can be compared to running Subnautica without a dimmdrive.
Frametimes averages are also in the ok again. Sadly the same frametime spikes do appear.
Tested with, what earlier test shown were the usual culsprits in harddisk usage
So excluding the disk as a culprit of frametime spikes entirely, and already having excluded hardware, it seems the idea of it being something engine related becomes more solid so far I can see.
If anyone has more test ideas, I am open to suggestions.
> edit 2.
Due to reasons (read issues) with g-sync/ulmb/nvidia, I am doing all these tests in ULMB setting. Subnautica actually runs quite high on fps on average, making it doable.
If the team are instantiating and destroying gameObjects continuously in Unity, then there will be massive spikes as Instantiate() and Destroy() are very expensive computationally.
The common method for solving this is object pooling, where you have a set number of objects in the pool, which is created at load time, and objects are removed from the pool (enabled) and the returned to the pool (disabled).
This does in turn bring it's own problems of once you have the pool created, you always have that memory overhead carried around with you. Also, if you run out of objects in the pool, then you have to resort to the Instantiate() methods, or remove currently active objects to re-use.
I am dealing with these issues in my current game and it is a royal pain in the posterior.
Yes, it is a shame that Unity is not multi-threaded, and despite having multi-core processors since what feels like forever now, we still can't get programs built with multi-threading in mind.
I mean, the biggest loss in time per step so far was a AV scanner each file, at least so far I could measure.
But very small steps can make a big effect and small steps barely register as anything odd as the difference is nominal.
What is understood now?