Crazy idea about precaching

ghabanghaban Join Date: 2004-02-16 Member: 26595Members
edited January 2014 in Technical Support
Hi, im one of those guys with extremely long precaching load times.(this differs from server to server but up to 10 min, probably due to my slow hdd ). Im running 32bit win7 with 4 gb ram, and have noticed that running ns2 maxes out to about 1.3 gb memory.

So i dont really need to go through all specs to tell you its not a great gaming machine, but i have 'decent' performance in-game, its just the precaching that takes for ever.

What i was thinking is that i have at least 1 gb spare mem, which i was thinking to allocate for a ram disk where im going to put the files which are precached, so im mostly just wondering about what files are actually loaded in this step? Does anyone know? Is it possible to get a list of that or find out through the console by some command?

EDIT: Just to clarify - I'm just interested in the precaching phase, the other stages might be slow, but it's not unbearable. In other words the map and a lot of other things should already be loaded before the precaching phase.

G

Comments

  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    edited January 2014
    So i found out i could see this with 'loadtimes' so maybe this thread is useless now.. Im trying to log some of the slow files (+1000ms to load) and see if it would make any difference by putting them on a ram disk.

    I will post my findings here later on.

    (Also maybe it wasnt the best place to put the thread? so any mod can feel free to move it if there is a better place to put it)

    G
  • EvilBotEvilBot Belarus Join Date: 2014-01-06 Member: 192556Members
    edited January 2014
    Hi. I had the same problem with long precaching and micro frizes during the game. The reason was my slow hdd.
    I had 4gb ram (same as you) so i decided to put some NS2 files on RAM disk.
    To find out what files/folders had the most performance impact i used Mark Russinovich Process Monitor.
    Filter by NS2.exe process name, join a game and then examine 'File System Activity' tree.

    I moved some NS2 files on 2 GB ram disk and it helped alot to decrease loading times.
    But at the end i decided to upgrade my notebook to 16 gb ram and just copied all NS2 and cache folders to 9gb ram disk. And now i dont remember what files/folders i moved on my 2gb ram disk.

    The only thing i can suggest you to start with is to is to move ...\AppData\Roaming\Natural Selection 2\cache folder to ram disk.

    To 'move' files i used junction.exe util, again from Mark Russinovich.



    Precaching from hdd > 240 seconds
    Precaching from ram disk ~ 10 seconds and even faster then from some ssds.

    So ram disk is a good idea, especially if you can't upgrade your hdd to ssd.


  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    thanks, that proves its possible anyway :) ssd is a little bit outside of my price range atm so i will try this for now!
  • Racer1Racer1 Join Date: 2002-11-22 Member: 9615Members
    What tool did you use to create the ramdisk?
  • EvilBotEvilBot Belarus Join Date: 2014-01-06 Member: 192556Members
    I'm using SoftPerfect RAM Disct.
  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    Should be noted that consistency checking is what takes the most amount of time.
    If you disable it entirely for yourself and start your own server, you will see how long it takes purely based on your HD speed to access files, instead of how long it takes to check hash files.
    For me its the difference between 30-50 seconds and 10 seconds... and i'm already on an SSD.

    Also.. why 32 bit? 0.o... there's pretty much no reason anymore to hold onto 32 bit OS, even if you work on embedded applications/software development.(worst case scenario dual boot it)
    And NS2 loves 64 bit..
  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    edited January 2014
    well i should go for 64-bit now, i actually just run on a mac mini now, and boot camp only supported 32-bit when i installed win7 but i think its fixed by now so i should upgrade now i guess, just need to get at it. (Well if i had any money i would just buy a comp intended for gaming instead)

    The precaching times are very different, but surely the precaching is what takes the longest time for me, im not kidding when i say i clocked it on 8 min when connecting to servers with certain mods. Usually it takes the longest on servers with player badges and nsl mod on it, servers without mods seems to be 'fine'.

    EDIT
    *fine meaning i can live with it
    Also: just measuring loadtimes i can see that the female rine model takes me 28 seconds to load in the precaching phase
  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    edited January 2014
    Well, since i didnt have a lot of room for a ram drive anyway i went to the extreme and put the most intense part of the precaching (mostly models/marine and models/alien) on a flash drive(usb2) which surprisingly works a lot better. (not an issue of fragmentaded hdd since i have tried to defragment the ntfs partion already).

    The reason is that there are so many small files loaded that my flash drive is faster(2x speedup maybe havent checked exactly and havent put everything on the flashdrive yet, just folders i thought were important),

    (latency should be higher but finding a file should be faster on the flash drive).

    Conclusion: Bundle the files a lot more should improve the precaching a lot for us with slow hdds ?? Most of it are loaded everytime anyway.


    Edit: Now the 'initializing game' phase i longer than the precaching phase, but i still havent tested with the mods on the flash drive.


  • HamletHamlet Join Date: 2008-08-17 Member: 64837Members, Reinforced - Shadow
    edited January 2014
    @IronHorse
    Could you help me (or point me to a tutorial) how I can optimize the ConsistencyConfig.json for my server?
    A little tweak, so that people can join faster.
  • _INTER__INTER_ Join Date: 2009-08-08 Member: 68392Members, NS2 Playtester, Reinforced - Shadow
    @EvilBot I thought this RamDisk idea is worth a try. I created a 2GB Ram Disk with "ArchiCrypt Ultimate Ram Disk". (SoftPerfect RAM Disc did fuck up my system). Could you explain how to exactly proceed with junction.exe?
  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    edited January 2014
    @_INTER_ doesnt a symlink work as well? just use mklink /D dirname R:pathtodir where R: is your ramdisk...
  • _INTER__INTER_ Join Date: 2009-08-08 Member: 68392Members, NS2 Playtester, Reinforced - Shadow
    ghaban wrote: »
    @_INTER_ doesnt a symlink work as well? just use mklink /D dirname R:pathtodir where R: is your ramdisk...
    Alright I've tried with mklink and junction again, both not working (telling, that link already exists eventhough they clearly don't, cmd opend as admin ofc).
    So I installed Link Shell Extension and it works like a charm. I created a symbolic link of the whole "Natural Selection 2" folder in %APPDATA%/Roaming.
    Gonna test it later if that gives me any speedup.
  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    If we are just talking about precaching i suggest you move models/marine models/aliens and perhaps also models/props (but that one might not be always neccessary) instead, those are the one i clocked longest times on. (a female marine model taking 28000+ms to load for me)
  • EvilBotEvilBot Belarus Join Date: 2014-01-06 Member: 192556Members
    _INTER_ wrote: »
    ghaban wrote: »
    @_INTER_ doesnt a symlink work as well? just use mklink /D dirname R:pathtodir where R: is your ramdisk...
    Alright I've tried with mklink and junction again, both not working (telling, that link already exists eventhough they clearly don't, cmd opend as admin ofc).
    So I installed Link Shell Extension and it works like a charm. I created a symbolic link of the whole "Natural Selection 2" folder in %APPDATA%/Roaming.
    Gonna test it later if that gives me any speedup.

    Have you tried to rename the original folder before trying to make a link to it with junction/mklink ?
    Let's say you have your NS2 folder on D:\Natural Selection 2 and want to 'move' it to RAM/Flash drive E. So the new destination will be E:\Natural Selection 2
    So you'll need to copy NS2 folder from D to E and rename/delete old NS2 folder on D drive.

  • IronHorseIronHorse Developer, QA Manager, Technical Support & contributor Join Date: 2010-05-08 Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
    @hamlet
    Had to search for this one :)

    @Ghosthree3 did an awesome job of already singling out the best consistency checking for server owners here
    Scroll up or look around on that page for more options as well.
  • _INTER__INTER_ Join Date: 2009-08-08 Member: 68392Members, NS2 Playtester, Reinforced - Shadow
    EvilBot wrote: »
    Have you tried to rename the original folder before trying to make a link to it with junction/mklink ?
    Let's say you have your NS2 folder on D:\Natural Selection 2 and want to 'move' it to RAM/Flash drive E. So the new destination will be E:\Natural Selection 2
    So you'll need to copy NS2 folder from D to E and rename/delete old NS2 folder on D drive.
    Ehh the last part about deleting it from my hdd bothers me. Wouldn't it then just be recreated by Steam if I'd delete it?
    Well I got symlinks of Roaming and the models on my RamDisk R:\ now. I still need to delete the original?

  • ghabanghaban Join Date: 2004-02-16 Member: 26595Members
    @_INTER_ just move it to backup folder then.. if you have an ntfs junction or symlink it still 'exists' in that path but it points to another direction so to speak. Therefore you need to clear that path to be able to put the junction / symlink there.

  • GhoulofGSG9GhoulofGSG9 Join Date: 2013-03-31 Member: 184566Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Reinforced - Supporter, WC 2013 - Supporter, Pistachionauts
    edited January 2014
    Hamlet wrote: »
    @IronHorse
    Could you help me (or point me to a tutorial) how I can optimize the ConsistencyConfig.json for my server?
    A little tweak, so that people can join faster.

    That's hard to really do as you need to know how mods basically work.

    I'll just give you the ConsistencyConfig.json i use atm.

    https://gist.github.com/BrightPaul/26ca9bab503ef821f8af

    Basically i only check files i'm aware of which were used in the past or could basically be used for cheats etc.

    But be aware that this check allows ppl to use alot client mods like custom marine/weapon models. So if you don't want to allow these, you have to set all models back onto the check list.
Sign In or Register to comment.