Texture memory usage and its effect on loading times

Howser

As a hobbyist modeler I've been looking at a lot of the wonderful art for reference on how to make quality models for games. One thing I've noticed is there's quite a few odd choices in terms of the resolution of the textures and the lack of recycling of certain maps. For instance the skulks texture is the same size as the onus and hive, in some instances the shadow textures could (with a little blending) double up for the vanilla, some specular and luminosity maps at 2048* are unnecessarily high res (and would probably have no visual difference if reduced to 512* -like the harvester has). I believe I could reduce the amount of memory used by the core content by 20%+ with no visual difference and that's without researching compression DDS formats for certain textures. the question is Would this have an effect on loading times? and What is texture streaming's role in all this?

I'm not critting the art in any shape or form so please don't think I'm slating any of the talented buggers that made this great content... I just think there's some room to clean up some of the materials and textures to be more memory efficient. If people can confirm it will reduce loading and improve performance then I'll start working on a clean up mod and we can test times but I literally have no idea about the engine stuff and what causes the loading times to be quite so long. It seems unfair that all the hard-work has fallen on the coding team to reduce loading times if there's culling that could be done on the art side of things.

I've also noticed a few things in environment textures, for instance biodome_glass_01 calls up two 1024* textures in the material file which actually have no effect on the material as all its visual properties come from the surface shader it uses. There's quite a few examples of redundant textures being called up which could also be tidied up. But again would this make a difference? If a texture is not being used does the engine know not to load it?

Without including resizing I believe these textures (of the three lifeforms i looked at) could be merged and so halved (correct?) the memory usage that's a saving of 10.9mbs+ , which could reduced even further looking at compression methods and resolution reductions where possible.:

So whats the story? is is worth pursuing?

SamusDroid

As informative as this is, it is a BIT late now. Bit being key term. Maybe something they will know for subnautica. It is probably engine related.

Howser

SILENCE! Its never too late to improve something!... :P besides which I still haven't got my shadow fade and onus. So we need to make more room on the little ram i have!

ZEROibis

I mean if your planning on doing it I see no reason why they would not officially roll it in as to improve game performance. I am sure it would also increase the number of systems the game can run on and help out those on lower end systems get some gains.

Howser

well i went through the aliens folder and managed to save around 200 mbs with some aggressive re-sizing and texture reuse from shadow maps with no editing. For example the skulk goes world textures go down down from 26.6 mbs to 2.83mbs. Spec, illum and normal merged with shadow's. Normal and diffuse Downsized to 1024*. Spec, illum downsized to 512*
Some of the minor details on the normal are visible but that's really minor and given the speed/size of the skulk its not worth considering. I have taken some high-res comparison images:
Original:

Modified:


Now I suspect it'll be a bit trickier with the marine hard surface stuff but I'm certain we can still make some serious savings.

McGlaspie

Personally, I don't think this would be too late at all. If this can be a proven performance gain, why would it Not be added in an official patch?
"Too late" when it comes to improving software, doesn't make sense unless you're speaking about funding.

I don't think we'd see much in the realm of performance gains (fps, etc), but I'm certain this would improve load times. It's just a question of how much.

Excellent work Howser, this is damned interesting to me. *thumbs up*

Howser

Yeah its certainly interesting but to be honest I don't really know what the effects will be... I'll go through all the core stuff I can then do some testing on a bare map to see if it actually does make any difference. Still looking to hear some facts on people who actually know how all the engine stuff works!

IronHorse

I can tell a difference when looking at the closest skulk leg/bone .. but still amazing work non the less.
I'm trying to imagine how much you could do this to in the game..

SamusDroid

I can pretty much guarantee this won't happen, ever. Like I said before it's too late to be doing something like this, everyone is working on other projects, and are not going to come back to change hundreds and hundreds of files. It's never as easy as "switch some textures around".

Ghosthree3

This will likely not affect load times much as the main throttle on that is consistency checking. Could speed up hashing if the files were smaller though I'm not sure. However it would greatly reduce the memory used by the game which would stop crashes for low memory users etc.

Howser

@SamusDroid Its just an experiment to see what can be done to reduce the texture memory load and see what (if any) effects it has on performance and loading times. Its nothing more than me satisfying my own curiosity and hopefully learning something in the process. I've requested nothing from no one. That said I've not seen anything to suggest editing .material files or downsizing existing maps actually have any wider implications to the games functionality out side of consistency checks. And I assume defining what is 'consistent' is based around an automated system, no? I spent about an hour going through the Aliens, that's like 40% of the core content. So its not exactly time consuming work.

Howser

Ghosthree3 wrote: »

This will likely not affect load times much as the main throttle on that is consistency checking. Could speed up hashing if the files were smaller though I'm not sure. However it would greatly reduce the memory used by the game which would stop crashes for low memory users etc.

The longest phase of my loading screen is the precaching.... what exactly is it doing during this stage?

Ghosthree3

It's consistency checking during the precaching, as well as loading all that content into memory. Try going to your %appdata%\Natural Selection 2\ConsistencyConfig.json file and changing it to the following and see the difference when making a local server.

{
	"restrict": [ ],
	"check": [ ],
	"ignore": [ ]
}

Feha

Regarding that skulk, I see a clear difference in the eye. Seems to me as if the emissive term somehow disappeared from the original or something (more specular in the original?) :S
The lower skulks eye is actually the one I like the most since it seems to glow more :P

Soul_Rider

Like @howser said, this is a combination of skins to save texture size, so there are overlaps and little bits creeping through, but it does work very well. With NS2 loading over 500mb of models into the game at start, this could certainly help reduce the memory load very significantly.

Plus this give me great news knowing that I can probably create twice the number of players and models (unique sides for each team in proving grounds) and still potentially use less memory than vanilla ns2 does for the 1 version of the same models. Whether this gets implemented in the main game or not, understanding this makes things a lot better for big customisation modders like myself and McGlaspie.