Hlrad Using 16x Too Much Memory

JedediahJedediah Join Date: 2003-01-27 Member: 12847Members
in Mapping Forum
I'm compiling a map with ~88000 patches with the -sparse and -nomatrix switches which I believe should minimize memory usage. About half way through the MakeScales phase, the memory reported by the hlrad.exe process (in task manager) goes up to 100MB but the "mem usage" and "commit charge" in the performance tab reports 1.8GB! Indeed, my pagefile.sys is expanding to about that size. At this point, with all the swapping going on, hlrad doesn't even get any CPU time (the CPU usage is reported as 3%) and it eventually aborts with a memory allocation failure.

Without hlrad running, my mem usage is at about 200MB. So if you do the math, 1800MB - 200MB = 1600MB = 100MB * 16. This suggests to me that there is some sort of overhead cost to hlrad memory allocation that is 16x the actual memory it uses. Obviously this can't be right.

Anyone know what's going on here?
Off Topic Disagree Agree Awesome

Comments

  • MerkabaMerkaba Digital Harmony Join Date: 2002-01-24 Member: 22Members, Retired Developer, NS1 Playtester
    Try taking out the -sparse parameter and using -chop instead to bring down your patches. I believe a huge load more memory is used when compiling with the sparse setting, since it has to compress it all down. I could be wrong, but give it a try anyway. Sparse eats time, thats for sure.
    Off Topic Disagree Agree Awesome
  • ImaTargetImaTarget Join Date: 2002-11-01 Member: 3415Members
    -sparse uses more processor but less memory for the vismatrix. -chop needs less processor but more memory. Thats the way it sounds to me in the infos to these parameters.
    what the problem is you are facing i have no idea. have you tried to just let it run? i never encountered a swapfile more then 1 gb big through rad, but you have only 256 mb ram wich could be the cause for this. The amount of memory rad uses can get really big. i had it while compiling a rather "big " map (not in size but enclosed room) where my cpu usage dropped very low since my PC was trying to move the data around and my swapfile grew and grew while my ram was 100% full. took about 40 minutes then the "Normal" calculation resumed, memory usage went down and cpu up. This happen while building VisLeafs. wasnt a problem , it was absolutely normal.
    Off Topic Disagree Agree Awesome
  • AxehandlerAxehandler Join Date: 2003-01-08 Member: 12038Members
    I have VC++ 6.0 did a quick File find and came up with HEAPWALK.EXE is this what your refering too?<!--emo&???--><img src='http://www.unknownworlds.com/forums/html/emoticons/confused.gif' border='0' style='vertical-align:middle' alt='confused.gif'><!--endemo-->

    if so.. let me know I can e-mail it, it's only 46kb.

    Axe
    Off Topic Disagree Agree Awesome
  • OlljOllj our themepark-stalking nightmare Fade Join Date: 2002-12-12 Member: 10696Members
    <!--QuoteBegin--Jedediah+Feb 2 2003, 07:22 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (Jedediah @ Feb 2 2003, 07:22 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> ...At this point, with all the swapping going on, hlrad doesn't even get any CPU time (the CPU usage is reported as 3%) ... <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    I want your PC !!!

    use the -chart command and tell me your
    visdata ... .../2097152 ( ....%)
    please.
    Off Topic Disagree Agree Awesome
  • JedediahJedediah Join Date: 2003-01-27 Member: 12847Members
    Nah you probably don't.. (P3-800, 256MB RAM)

    Is this what you want?
    visdata [variable] 4038/2097152 ( 0.2)

    Also FYI, these are the flags I'm using:

    FLAGS = -estimate -low -verbose
    CSGFLAGS = -hullfile C:\Games\Halflife\ns\nshulls.txt
    BSPFLAGS = -chart
    VISFLAGS = -full -chart
    RADFLAGS = -sparse -extra -bounce 4 -smooth 80 -notexscale -texchop 16 -coring 0.2 -chart

    I'm compiling with a makefile from a cmd.exe prompt on windows 2000
    Off Topic Disagree Agree Awesome
  • JedediahJedediah Join Date: 2003-01-27 Member: 12847Members
    BTW if anyone has a copy of Process Walker (pwalk.exe), it would be really swell if you could send it to me. It's a sample program from VC++ that lists the heap blocks used by a process and it would help me shed some light on this hlrad problem (hyuk hyuk).
    Off Topic Disagree Agree Awesome
  • bluemanblueman Join Date: 2002-04-09 Member: 399Members
    mm strange in my compilations i always get a 100% CPU usage and a 100-150MB ram usage at a maximum. computer normally has 160 used with XP so during compilation i get a maximum of 280-300Ram usage.
    Off Topic Disagree Agree Awesome
  • JedediahJedediah Join Date: 2003-01-27 Member: 12847Members
    edited February 2003
    <!--QuoteBegin--blueman+Feb 2 2003, 09:13 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (blueman @ Feb 2 2003, 09:13 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> mm strange in my compilations i always get a 100% CPU usage and a 100-150MB ram usage at a maximum. computer normally has 160 used with XP so during compilation i get a maximum of 280-300Ram usage. <!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->
    Yes, I'm fairly certain it shouldn't be using 2GB of ram to compile this map. My best guess is this is a problem with the way hlrad is allocating memory. Perhaps somehow it is allocating off the heap in such a way that tons of memory gets wasted. I have no idea why this is only happening to me though. Do you mind if I ask what your command line options are, number of brushes in your map and how you are running the compile tools?

    EDIT: er, I mean number of patches
    Off Topic Disagree Agree Awesome
Sign In or Register to comment.