Half crash repro

babblerbling

Posting this for @matso to see.

We found a way to reliably create the half crash. Please run this in a debugging environment.

Start a server, have it run faded mod 1b0bc710.
enable the 'mysterious_option' for the pistol in the faded config file that is created in the server config path.
Be a marine, use alt fire of pistol to spam a couple hundred mines. Make them explode.
The server will run into hc.

We can spawn a million babblers and the server will throttle properly even to long red plugs but eventually recover and function normally... yet, exploding a few hundred mines -> hc every time.

It's suspected that the hc is related to the fast creation and deletion of entities, but with a repro the exact cause should be within grasp.

This is one of the last remaining server crashes. 
For example this server has remained in this state for 2 months because nobody is restarting it:
http://ns2servers.devicenull.org/servers/1638485/176.57.130.192:27015

I pray this will allow you to drill down to the origin of this crash swiftly.   

EDIT: Credit for the find goes to Katzenfleisch, the one currently behind the Faded mod.

matso

Thanks! A repro for the hc will pop it straight up to the top of my todo list... it has been occupying the top spot on my "I really should be looking into this but I don't know where to start list" for quite a while now.

Katzenfleisch

I send you an invite on steam, if you have some time this weekend we can show you.

babblerbling

matso said:

Tried to repro that, but I must have blown up upwards 100k mines now without getting any hc ... could you take a video showing how you do it so I can be sure we are doing the same thing?

  Linky

matso

Most of the time when that many mines are blown up at the same time, the server tick interval hits into the multi-second range and then recovers... sometimes it takes a minute or two.

But I did manage to actually trigger something that looked like a halfcrash once. Tried to replicate it using the debug mode but it made the game too slow ... but it's promising, so I will keep digging.

The testing for 275 has started though (yay!) which means I have a bunch of bugs that requires fixing before the release, so I'll have to put the halfcrash hunt on the backburner for a while.

babblerbling

When we do that, on the NS2-fr server, on the club, or some testbox on my pc, it happens -every time-, like it says in the video. All run on windows.
We've tried waiting, it doesn't come back.
If you look at the net_stats late in the video, you can tell.
The client times out, reconnection impossible. Query works. It'll stay like this forever, consuming exactly one physical core worth of cpu cycles.

Less important:
Katzen made a tiny mod that writes the time to a text file every second. We use this to detect the hc and auto restart. It works. Except for once today where apparently (unconfirmed) the server kept running at least that part of lua code. All the same symptoms as the hc tho.

Thank you for your efforts. We will patiently wait for you. As always.