Revenge of the Failed to allocate ? bytes crash
krOoze
Join Date: 2014-04-24 Member: 195593Members
Join Date: 2014-04-24 Member: 195593Members
What happens: I am in started game (this time I was spectating, crash before that I just evolved Onos). Suddenly red plug appears(often I still hear other people communicating, sometimes I still see other people moving chopily, but I am out of sync - can move but cannot do anything else) and after about half a minute game crashes with the Fatal error: failed to allocate X bytes message. X is I think random number. Happens rather rarely (every ~10 h) and after extended session I think (> 1 h after running game). Somtimes alt-tabing works poorly, and actually shows the error message with invisible text.
My reporter dump (game was still running, showing the fatal error message): https://dropbox.com/s/zfz947sfmlu1wxl/tech_support.zip?dl=0
My reporter dump (game was still running, showing the fatal error message): https://dropbox.com/s/zfz947sfmlu1wxl/tech_support.zip?dl=0
Comments
If the game crashes, run that techsupport.exe with NS2 still open in the background.
p.s.
"The Celeron is a family of microprocessors from Intel targeted at the low-end consumer market" - Intel Wiki
Your CPU is more than 6 times less powerful than CPU's that were released in the same month. It barely passes the minimum requirements for the game!
Which is a shame considering how relatively nice your GPU (video card) is.. Very odd combination of parts, there.
Dont judge me. STOP IT. I can feel you judging me right now
Also: I did run the techsupport.exe while ns2.exe still open with the fatal error message. It clearly stated it detects it and is making dump. But the zip does not contain anything I would consider a memory dump, which I think you need. The log.txt just shows some lua stack trace of some error....
Also: After the fatal error message the automatic error reporter start telling me to check log.txt for private things before send, I guess. But the send takes too long to be just log.txt (several minutes). Is it sending other data?
Also: I noticed the installation folder contains 200 MB of *.pdb files. Does it make sense for me as end user to have theese in my installation.
Are those three bug report worthy?
Yea techsupport.exe should make a folder called dump or minidump inside the zip when it's done, and of course will be a large file size.
If it's not... well.. that makes things slightly more difficult, as you have to run the 32 bit version of Windows Task Manager (it's hidden away in a windows folder) and manually click on the process and create a dump. (which will be larger than the one techsupport.exe makes..)
Dont bother with the automatic error reporter. As the Community Development Team we do not have access to where it is sent, unfortunately. So we can't guarantee anyone is reviewing what you send..
Don't worry about the pdb files , and yes all information is worthy.
Well it didn't. I can certanly make dump file your way, if thats useful to the team. By "hidden" I assume you mean SysWOW64 folder.
Well I am honored to store the games pdb debug symbols.
@krOoze That's the one, thanks
Anyway the reporter lied again about making a dump:
To be exact, I am sure it made a dump (process froze for a few seconds), but I don't know what it has done with it. It's not in the zip...
report: https://dropbox.com/s/f6rfoei3121zpmd/tech_support2.zip?dl=0
windows dump file: https://dropbox.com/s/njx766s51mp8d6e/NS2.7z?dl=0
@Doctik Well thanks for forcing me to read the last thread in entirety - that led nowhere fast. My network is rock stable, thanks for asking. Actually I haven't got any crashes at the time of the old thread and it was many patches ago - so lets not zombie that one, ok? (Hence the "Revenge of". Get a hint, man
-> http://ns2servers.devicenull.org/servers/1530213/85.6.202.192:27015
With 42Players there is definetly more to display / calculate for your gpu/cpu so yes, it may use a bit more memory usage than on other servers i guess.
About your problem, it happened to me too when i started playing the game the first times (But really rarely). I don't know exatlcy what i did to fix this but i remember i had a defective ram at this time. I used the programm MemTest. If you using it, start multiple instances so that almost every megabyte is checked
http://ns2servers.devicenull.org/servers/1530213/85.6.202.192:27016/?date=02/13/2015
You have changed it since then, but not the hacked server executable. (or mods I'm assuming)
Unsurprisingly, your server consistently produces false positives for tech support related issues. Therefore, it cannot be used as a testing environment.
@krOoze please continue to test long sessions on other servers to attempt to reproduce the issue. I've forwarded your dumps to some devs, just waiting on what they found, if anything.
@IronHorse I'm on it!
PS Woozas servers are always full, even when others are not and I had no (other) problems with them, so maybe make them legit in the long run (that is without hack, you speak of)? You already know about them and warn about potential performance issues in game anyway. Sometimes there is not even many alternatives where to play...
Not to mention we don't have 42 users consistently to even begin testing such a thing
We definitely try, and wish to foster all environments, despite any non-preferable change in gameplay or balance, we do test with mods and attempt to support all - but maintaining stability for the mod authors is unfortunately just not realistic - we already have our hands very full with the vanilla version of NS2.
Thanks again for the helpful info, i'll update you when I have more.
It was again preceded by minute long red plug. Again I could hear well other players speaking and observed no performance drop during that time, but could only walk around desynced.
On first Fatal Error message:
TR zip: tech_support3.zip
dump: maxidump3.7z
After recovery and exit:
TR zip: tech_support4.zip
Pointing towards dx9 and network related.. but no promises.
At least we have the data on hand, though, thanks again.
I will update when we have more
Yea, actually I investigated it a bit myself. When the red plug appears, the quick runaway memory leak starts (or vice versa). Alt-tabing seems to slow it down. It crashes with the bad memalloc error when it reaches 4GB (for the obvious reasons). Sometimes it just ends with ingame timeout message or "verify your cache" message (like in the graph above), especially if you alt-tab during the red-plug time. (I verified and nuke reinstalled it several times now.)
Don't hesitate to ask if you need me to test something, or further data.
PS: Oh didn't realize that everybody doesn't speak czech. Red is processor utilization, green is working set of the NS2 process.
Im playing on quite high settings on 1080p, and I purposely attempted to hit 4GB to break it, and never got close.
WHATS happening in your executable that its hitting 4GB, makes me wonder.
My guess would be that the devs would have hard time finding the culprit, since in the dumps they probably would see only the last memory allocation, that tipped it over 4GB, which likely would not be the cause of the error in the first place. They would need to dig through it to see the actual error source, if they cannot reproduce it themselves on their machines. The red plug could be the biggest hint and finding what stalls the netcode could mean a fix soon (fingers crossed
But I kinda wonder how it is the game always uses ~1.5 GB during normal behavior, no matter what I do
Definitely looking into it, though.
The values in the graph still mean nothing to me, but I guess I get the general idea.
You could attempt to keep a eye live on memory usage and try to find out 'what' is eating the memory.
Sadly that would only boil it down to a handle or ddl so I am unsure if that is useful info.
Also debugging that externally can get quite tricky. IF you are unfamiliar with it, im not sure how helpful it is if I try to explain it.
Best I can do is link you to the article explaining it, but note that it will not be of much use if you do not understand the underlying information:
http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx
Way ahead of ya. I had only one two crashes from last time and tried "memory" command in console. For that crashes it seems the program spammed someting it calls NetworkMemory of a ClientWorld, and every memory seems to be allocated by Spark engine, so no point looking which dll does that. So better leave it to the devs, they will see that and much more, since they have both the full dump from me and game source code. I am not quite so desperate yet to try to fix it in asembly language.
I uninstalled Fraps (which I was not using, but it still pushes its meddling dll into everything) and had peace from crashes for now (perhaps statistical fluke, perhaps I did something else. will try to reinstall it in a week or so, if I get none till then). So I sugest to anyone having the same problems to uninstall Fraps and similar software and report back too.
To start the available memory in dxdiag itself is to be ignored. It basicly says how much memory the system has and how much the OS can use, not how much there is left. (I was in doubt as I never check memory by dxdiag but thats how its viewed)
In this regard so far so good. You have 8GB of memory and almost all is used for the OS, meaning no driver is going nuts on reservations for hardware.
The odd part is your page file. Many rumors exist about the size of a pagefile but it truly boils down to 'what you need'.
Your very first post has a dxdiag containing the following:
Page File: 7735MB used, 666MB available.
Thats well over 7GB of paged memory, not accounting for the non paged memory which ideally should be pretty full also. To compare, idle my pc pulls a 2.5GB of paged memory.
While thats impressive memory usage, which makes me wonder where it all goes as ns2 cant use THAT much GB even with large address awareness, the real question is.. why do you have 'only' 666MB left? Was it only 666 during the crash, or less?
I now have a small assumption / theory that you may be hitting either the commit limit or the paged pool limit (which are almost identical).
Does not mean ns2 is not at fault, but this can not be ns2 only.
Start be severely raising your commit limit by setting a bigger pagefile manually, as something in your pc wants all that paged memory. (I know its auto due to its changing max size)
After play NS2 for about exactly 30 min ( or map change one) >> Crash to destop=='
P/s: Using Window7 32bit
First of all dxdiag shows nonsense. What it shows is not "Page File". It is known!
It is most likely backed virtual space(RAM + page file - not acounting for possible page file expansion) and commited memory(=not only used, but reserved and cached too, possibly drivers' adress spaces also).
Windows warns when what you describe happens. Besides I have page file expansion on and free disk space.
Something within NS2 is clearly at fault. It is using 1.5 GB, then something breaks, then it starts allocating memory linearily untill it's stopped by something (e.g. running out of 32bit Large address aware space = 4GB). And my dumps I sent are 4GB so no doubt there.
https://dropbox.com/s/pcnbosxv946mpl6/tech_support7.zip?dl=1
I actually managed to catch it in log using "memory" command (and "verbose 3", so it shows some weird silenced errors too) Thats how I know it bloats NetworkMemory of a ClientWorld and possibly another unaccounted memory not managed by Spark engine e.g. DirectX. Coincidentaly: So we seem to be little behind the devs in our conspiracy theories.
I was just wondering as your virtual memory usage is... quite high.
Yeh if the dumps are 4GB... didnt check their size. I only tried some memory commands which obviously failed as its not a system dump but a program dump.
I should really not try to be helpful when im ill and not fully lucid.
Hmm I see what you mean, network memory is quite high.. (I should really just run verbose 3 myself a few times, been to long)
Which is why I am not sure of the following but damn.. thats a lot of errors on the prediction, with network memory rising fast.
Did you try dx11 or is your amd to old for that? (just curious, I know dx11 has input lag in ns2)
Best ideas usually comes, when a person is not lucid.
Nah, my GPU is new and shiny. I played on dx11 some time ago. It was not exactly stable either and run slightly slower. I think the frame pacing was better though.
I still think the Fraps could possibly trigger that memory leak somehow. I forgot I installed AMD Gaming Evolved/raptr, which does the same thing. (Un)fortunately I do not get the crashes frequently and regularily enough to test it easily.