Hit Registration Problems
GORGEous
Join Date: 2012-02-19 Member: 146762Members, NS2 Map Tester
Join Date: 2012-02-19 Member: 146762Members, NS2 Map Tester
<div class="IPBDescription">(with pictures!)</div>After seeing Ironhorse post about the "hitreg" command, I figured I'd try it out. Rant and I spent 20 minutes or so testing hit reg on various lifeforms. Here is what we learned. Feel free to correct me if any of my information is wrong as I don't fully understand client -> server hit registration. The blue lines are a result of a hitreg problem. This seems to happen when the client's animation and prediction show a hit but the server actually has the target NOT getting hit. Red are hits on both, yellow are misses. This seems to have two main culprits
<a href="http://imgur.com/a/CLV2G#0" target="_blank">http://imgur.com/a/CLV2G#0</a>
(numbered 0->6, 7 total pictures)
Focusing on the skulk because he showed the greatest amount of lost hits due to his wall walking mechanics, fast speed, and maneuverability. Server tickrate was ~30 consistently, FPS was high, ping was <15. I can only imagine that these problems are greatly magnified under low tickrate, low FPS, and high ping conditions.
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo-->Animations:<!--sizec--></span><!--/sizec-->
Pictures #0, 1, 4 clearly show this problem. You can see the skulk's running animation on the pillar as they transitioned from ground -> wall. I shot the running animation on the wall with the shotgun. The game ate all of my pellets. This happened semi-consistently. I would guess ~50% of the time the game missed a majority of pellets any time I fired exactly at the floor -> ground transition.
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo-->Prediction:<!--sizec--></span><!--/sizec-->
Prediction hitreg problems can be seen in #2, 3, 5, 6. Prediction hitreg problems become extremely common at high speeds and with erratic movements.
One of the main scenarios we saw with prediction related hitreg:
My client predicted you'd continue left, I aimed left. I hit on my screen. You actually moved right a fraction of a second before I fired. Your client and the server saw you move right and saw my shot miss by a foot. This scenario is uncommon at close ranges ( around 1 -> 10 feet). It is a huge problem at distance, though. When firing across crossroads, most of my pellets (of the few that actually landed due to spread) were hitreg problems when the skulk was traveling fast, spamming jump, and zig zagging. It was common to see 1-2 blue lines and 1-2 red lines. I'd estimate that you're losing at least 25% of your hits any time you fire your shotgun at medium to long range against a fast moving target.
Leap was another huge source of prediction related hitreg problems. As seen in screenshot #5, the skulk was leaping and I was falling with my jetpack. 8/8 of my hits were discounted. When both the shooter and the skulk are moving, hitreg problems seem to increase in frequency and severity.
There were also problems at very close distances (<1) as seen in screenshot #6. The skulk was point blank and my entire screen was skulk, but my entire shot was disregarded.
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo-->
Concluding thoughts:<!--sizec--></span><!--/sizec-->
The skulk was the most susceptible to hits getting lost. I think this is because of his animations and also his leap mechanic. We tested other lifeforms briefly. Lerk had some problems with hitreg against the LMG, but seemed to register better against the shotgun. That could have been a discrepency due to short testing of the lerk. At medium to long ranges (10+ feet), there were frequent hit discrepencies with both lmg and shotgun against skulk, lerk, and gorge. The fade and onos seemed to have the fewest hit reg issues, most likely due to the large hit boxes. From the couple minutes we spent on shotty vs fade, I saw only a few stray blue lines despite actively trying to shoot in and immediately around rant's blinks/shadowsteps. I have not tested this with higher ping, server choke, low tickrate, or low fps. I'd bet it would just compounds the problems I've listed, though.
Overall, hit reg performed pretty well at short distances and under 10 or 11 speed targets. I'd love to see some others test out how severely latency or tickrate impacts the hit registration.
<a href="http://imgur.com/a/CLV2G#0" target="_blank">http://imgur.com/a/CLV2G#0</a>
(numbered 0->6, 7 total pictures)
Focusing on the skulk because he showed the greatest amount of lost hits due to his wall walking mechanics, fast speed, and maneuverability. Server tickrate was ~30 consistently, FPS was high, ping was <15. I can only imagine that these problems are greatly magnified under low tickrate, low FPS, and high ping conditions.
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo-->Animations:<!--sizec--></span><!--/sizec-->
Pictures #0, 1, 4 clearly show this problem. You can see the skulk's running animation on the pillar as they transitioned from ground -> wall. I shot the running animation on the wall with the shotgun. The game ate all of my pellets. This happened semi-consistently. I would guess ~50% of the time the game missed a majority of pellets any time I fired exactly at the floor -> ground transition.
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo-->Prediction:<!--sizec--></span><!--/sizec-->
Prediction hitreg problems can be seen in #2, 3, 5, 6. Prediction hitreg problems become extremely common at high speeds and with erratic movements.
One of the main scenarios we saw with prediction related hitreg:
My client predicted you'd continue left, I aimed left. I hit on my screen. You actually moved right a fraction of a second before I fired. Your client and the server saw you move right and saw my shot miss by a foot. This scenario is uncommon at close ranges ( around 1 -> 10 feet). It is a huge problem at distance, though. When firing across crossroads, most of my pellets (of the few that actually landed due to spread) were hitreg problems when the skulk was traveling fast, spamming jump, and zig zagging. It was common to see 1-2 blue lines and 1-2 red lines. I'd estimate that you're losing at least 25% of your hits any time you fire your shotgun at medium to long range against a fast moving target.
Leap was another huge source of prediction related hitreg problems. As seen in screenshot #5, the skulk was leaping and I was falling with my jetpack. 8/8 of my hits were discounted. When both the shooter and the skulk are moving, hitreg problems seem to increase in frequency and severity.
There were also problems at very close distances (<1) as seen in screenshot #6. The skulk was point blank and my entire screen was skulk, but my entire shot was disregarded.
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo-->
Concluding thoughts:<!--sizec--></span><!--/sizec-->
The skulk was the most susceptible to hits getting lost. I think this is because of his animations and also his leap mechanic. We tested other lifeforms briefly. Lerk had some problems with hitreg against the LMG, but seemed to register better against the shotgun. That could have been a discrepency due to short testing of the lerk. At medium to long ranges (10+ feet), there were frequent hit discrepencies with both lmg and shotgun against skulk, lerk, and gorge. The fade and onos seemed to have the fewest hit reg issues, most likely due to the large hit boxes. From the couple minutes we spent on shotty vs fade, I saw only a few stray blue lines despite actively trying to shoot in and immediately around rant's blinks/shadowsteps. I have not tested this with higher ping, server choke, low tickrate, or low fps. I'd bet it would just compounds the problems I've listed, though.
Overall, hit reg performed pretty well at short distances and under 10 or 11 speed targets. I'd love to see some others test out how severely latency or tickrate impacts the hit registration.
Comments
I think if these issues still occur after major performance gains it should be looked at again. Until then it seems like a wild goose chase, imo.
There is probably problems with the skulk model orientation. The skulk model need to be oriented to face the wall on which he is walking, but that's not easy, because the surfaces are so complex, and can change very rapidly due to the high velocity.
Lerks, fades and onos all seem to be hit easy enough. Just those damn skulks.
client predicts a hit and server confirms it -> leads to hit
client predicts a hit and server denies it -> leads to no hit and bullet is "lost"
I have 2 questions:
- If the client predicts a hit, and server denies it, does the server continue the bullet trace on to see if it hits something else?
---this could mean hit another enemy, or it could mean client predicts a hit in the foreleg of an enemy, server denies it because on the server it actually hits the rear leg.
- what about the 3rd case, client does not predict a hit, but the server confirms it. I'd imagine this is still a valid case, but none of your screenshots or descriptions show that actually happening, makes me wonder if the netcode is designed around client predicted hits and verifying them, in which case it would ignore a hit if no client predicts it.
As for the skulks orienting themselves, couldn't the mappers just have a super-simplified invisible geometry that determines which way the skulk should face? Similar to a collision geometry, I guess.
It's why I only use skulks now. Onos and lerks are terrible, fades are decent but still paper towards a few well aimed shotguns.
The game is interpolating player movement most of the time. It only resorts to prediction when it's not getting packets from the server often enough.
The server knows what packets you have recieved(the ones you've ACKed, which should be tacked onto the packet containing what your client is doing); the server tries to rewind time and reconstruct what YOU saw when you fired. This is "lag correction".
Regarding that utility its helpful but IMO needs to show the client hitbox and the server's hitbox to make the comparisons easy.
"Matso on Tuesday, 10th April 2012
yellow are correctly traces missed, red correctly traced hits, blue indicate that there was a mismatch between client and server."
To me it seemed that at times the server did not even receive my shot, when my client (obviously) did. There were edge causes where i died immediately after the shot were its obvious that i was dead before the shot server side, but other times where the shot just gets lost for no reason.
"Matso on Tuesday, 10th April 2012
yellow are correctly traces missed, red correctly traced hits, blue indicate that there was a mismatch between client and server."<!--QuoteEnd--></div><!--QuoteEEnd-->
Though, my understanding is that blue doesn't necessarily indicate a miss, just that the values between the client and server are different. For example, if the client says the value is 1 and the server says its 1.00000000000001, that would register as blue, but still be a hit for all intents and purposes. Which makes me wonder about how many of the blue lines are just a result of small rounding errors between the client and server.
After giving a hitreg log to matso we have found some larger misses than were previously thought to be capable(5meters)
Edit- Even some hits on a Fade that the server said no. :z
Edit- Even some hits on a Fade that the server said no. :z<!--QuoteEnd--></div><!--QuoteEEnd-->
Could it be related to the same bug that causes every 3rd or 4th nade to shoot off at a different angle? If that is happening client side, but the server is predicting the normal angle for shotgun pellets/rifle shots, then maybe that is what's causing the huge mismatch?
Yeah. After retesting some I've come to the conclusion that my initial posting was definitely not correct. I'd like to refine my definitions, feel free to correct them.
--Blue lines are mismatches between the client and server, but can still be hits or could be misses or could be hits on client and misses on server.
--Yellow lines are misses by the server, but could be hits by the client.
--Red lines are hits on server or could be misses on client.
Basically, it's impossible to tell from looking at line colors what worked, didn't work, or what should have hit but didn't count from a player's perspective. It's also impossible to tell when hitreg issues are occuring when playing real games. The RNG spreads can easily RNG hits to misses, regardless of how good your aim is or how perfect your shotgun shot was.
Here's a picture which may demonstrate a problem:
<a href="http://i.imgur.com/S6NcP.jpg" target="_blank">http://i.imgur.com/S6NcP.jpg</a> - You can see two yellow shots going through the fade's leg. Also one or two of the lower shots are counted as hits, but don't seem to match up with the model.
Is it possible that hitreg is decent (with ideal server/client), but models don't line up perfectly. Most blue lines are acceptably small variations, but there are definitely some that were a large enough deviation to indicate something was broken.
It's also possible the hitreg command isn't entirely accurate in its snapshots of player locations.
As I'm not a dev, it's hard to tell for sure what the problem is. Clearly something is at least somewhat off, though maybe not as bad as my initial post claimed.
From the discussions I had about the hitreg tool the dummy model that is created is not 100% accurate, and should not be relied upon for confirming the hits. The real useful information from the hitreg command is dumped in the console, and shows the amount of mismatch in the traces between client/server. The last bits of information I saw showed that they seem to be aware of some problem that was occuring that seemed to be a mismatch between the state of the game client/server side, which leads to the lag compensation being off (well percieved as such since the servers state is ahead of the clients by more than the 150ms).
I've had the same troubleshooting this issue myself. I can tell something is wrong, but I can't tell where and can't seem to capture it clearly in a picture or video.
At the end of the day, I keep reverting to fact that from the numbers a marine with an assault rifle (10 normal damage/bullet, 50 bullets/clip) could theoretically kill 5 vanilla skulks (70hp/10ap) with one clip. However, in practice, I've never seen even the people I consider the most skilled in NS2 to get even close to that. To me, that speaks of a very serious problem that probably, but is not limited to, hitreg.
I rewrote the function in a much simpler way. If the client got a target it draw a green line from the player half-way to the target. If the server has a target it completes the line with a red one. I'm not 100% sure it works, but what I get makes sense. Here I was going up with a jetpack and shooting at a skulk, the first 9 bullets hit, the last one are not confirmed by the server, probably because the skulk was already dead :
<img src="http://i.imgur.com/tbNml.jpg" border="0" class="linked-image" />
Testing locally I had very few misses, here is one with the shotgun:
<img src="http://i.imgur.com/2Zo3f.png" border="0" class="linked-image" />
If someone want to test online, overwrite this function in HitRegEvent.lua :
<!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->function RegisterHitEvent(shooter, sequenceNum, startPoint, trace, damage)
local target = trace.entity
if target then
if Client then
local v = GetNormalizedVector(trace.endPoint-startPoint)*(trace.endPoint-startPoint):GetLength()/2
DebugLine(startPoint,startPoint + v, 10,0,1,0,1)
end
if Server then
local v = GetNormalizedVector(trace.endPoint-startPoint)*(trace.endPoint-startPoint):GetLength()/2
DebugLine(startPoint+v,startPoint + 2*v, 10,1,0,0,1)
end
else
Log("No target found")
end
end<!--c2--></div><!--ec2-->