Whoa. As much as I'd love new patches. I don't think anyone here wants folks working unhealthy hours. Props to the lad but stay on the up side of sleep.
Whoa. As much as I'd love new patches. I don't think anyone here wants folks working unhealthy hours. Props to the lad but stay on the up side of sleep.
Fixing bugs you're obsessed with helps programmers sleep.
on a more serious note (i know you guys are doing your best, please take your time): i am aware that any change can cause a bug in a totally different part, but this seems to happen very often in the case of ns2 (since long before the CDT started).
are there any patterns on how these bugs are interacting? maybe you can identify them and/or add more automated tests/tools/checklists to quickly identify such issues?
In software development, bugs are like commercials. They come unexpectedly, often right when things are just getting good. Also, the more characters (developers), in the plot, the greater the chance for unexpected consequences (bugs).
dePARAJoin Date: 2011-04-29Member: 96321Members, Squad Five Blue
edited July 2014
Coding is like players playing the game:
Some know where to position with awesome aim and gamesense.
Other players missing some shots or making the wrong decisions.
But in the end the game is over after a while with a win or loss.
Developer philosophy. Which do you choose:
1. Bugless game with awesome features - released fall of NEXT year, ad infinitum
2. Game with new features that have been laboriously debugged with minor issues - released quarterly to semi-annually
3. Game with no new major bugs, and a few extra features with some bugs - released monthly
4. Game with all kinds of fun features and countless bugs - released weekly
Corollary question:
A. How many current players will be lost while waiting for the new content/bugfixes to come out?
B. How many players will you loose/gain due to bugs/features in the new release?
Long story short, at some point, you have to make a release, and it WILL contain bugs.
P.S. This is an eternal struggle in every development project.
well, getting my masters in computer science next semester and having worked on projects before (longest was 6 years, non-commercial though) i know how frequent bugs are.
but usually, they affect things in the same (or at least semantically close) "area" of functionality. it seems to be this is different in ns2 (at least that's my impression having only limited information). ok maybe that's more common in commercial projects with many more minds involved.
what i've been told is that one of the major improvements in commercial coding is the employment of sophisticated testing approaches in several stages (e.g. unit testing).
i have no idea how exactly things are working in ns2 and i have no experience on commercial scales, i am merely proposing it might be worth rethinking if there are possibilities to improve the methods of detecting unexpected bugs e.g. by adding more in-code tests that are redundant at the moment but assert a warning/error when something is unexpectedly breaking. question is wether it is worth the investment or not, that is for the CDT to judge.
Soul_RiderMod BeanJoin Date: 2004-06-19Member: 29388Members, Constellation, Squad Five Blue
Charlie was a big proponent of unit testing during the early days of NS2 development and was very happy to talk about the unit tests they were developing and using in the early days. I don't know how that went after time, but they started out that way at least.
We think we've got fixes for both now, but they will need testing tomorrow (Thursday). So it's looking like next week now for release, if the Patch Gods allow it. *Shakes fist at Patch Gods*
How about releasing the current RC via the Steam Beta feature? I'm sure you'll get plenty of volunteers to test-drive it... pretty sure there will be servers, too.
Pros:
- More eyeballs and test time
- More different hardware/driver configurations
- Pacify the rampaging hor... err, the community
Cons:
- A bit of effort to set up the beta channel.. not sure how much?
Developer philosophy. Which do you choose:
1. Bugless game with awesome features - released fall of NEXT year, ad infinitum
2. Game with new features that have been laboriously debugged with minor issues - released quarterly to semi-annually
3. Game with no new major bugs, and a few extra features with some bugs - released monthly
4. Game with all kinds of fun features and countless bugs - released weekly
Corollary question:
A. How many current players will be lost while waiting for the new content/bugfixes to come out?
B. How many players will you loose/gain due to bugs/features in the new release?
Long story short, at some point, you have to make a release, and it WILL contain bugs.
P.S. This is an eternal struggle in every development project.
While certainly true, you dont want to release with a bug that crashes all marine players at the end of the round, or one that crashes the server if anybody is still in the commander mode when changing maps. Both appeared yesterday when the game went into release candidate mode. So yes, bugs will always be there, but certain types of bugs can not be released (but dont worry, both bugs have been fixed, now its just a matter to test the fixes and not releasing on fridays)
what i've been told is that one of the major improvements in commercial coding is the employment of sophisticated testing approaches in several stages (e.g. unit testing).
i have no idea how exactly things are working in ns2 and i have no experience on commercial scales, i am merely proposing it might be worth rethinking if there are possibilities to improve the methods of detecting unexpected bugs e.g. by adding more in-code tests that are redundant at the moment but assert a warning/error when something is unexpectedly breaking. question is wether it is worth the investment or not, that is for the CDT to judge.
The problem with unit tests is that they test small-scale features. They are definitely useful, but they can only test precisely what they've been set up for.
Example: Create some Hydra entities; create some hostile entities (Marines, ARCs, buildings) in different positions (range, line-of-sight etc.); check which Hydra chooses which target to attack. Compare with the expected result.
To test something as complex as a multi-player game, you would have a much better chance to catch bugs with higher-level integration tests.
Example: Set up a server; connect two simulated clients (using as much real code as possible); simulate one client going Fade, the other going Shotgun Marine; move both to defined starting positions; send timed movement and aim commands; send a "pull trigger" command when the Marine should have the Fade in its crosshairs; let simulated time pass. Compare the damage feedback from the server to both clients with the expected result.
Building on this, you can create more complex tests: For example, let the Fade kill the Marine a moment before the shot is registered in game-time, and check that the Fade does not take damage (or vice versa).
As you can imagine, this gets complicated *very* fast. And still, you'd have to do this for a huge number of scenarios. Evolve in various places and check where the new lifeform ends up... Have many Marines use a Phase Gate at the same time... Check how Spores affect Sentries... Check how MACs and Drifters react to movement commands... etc etc.
So in some cases, creating tests can be useful, but for all the complex interactions in the game, you're usually better off testing manually.
As you can imagine, this gets complicated *very* fast. And still, you'd have to do this for a huge number of scenarios.
yes, setting up more demanding tests will probably quite some effort. still, even when it takes an entire week it could be worth it in the long run (even when deployed this late), e.g. if you save 2-3 days debugging/playtesting per patch.
anyway, this is getting quite off topic so i'll shut up now
@Asraniel
I was just trying to point out that it is not easy to either predict or fix all of the bugs that may come up, and that you must weigh the benefit of fixing bugs against the cost, and only work on the ones that tip the scale.
I've been developer for 15 years, so I understand all too well that releases must be delayed when a showstopper comes along. My only problem is that people (especially in projects without hard deadlines) often place such a high bar on the quality of the release that it never happens. (I'm not claiming the CDT is doing this). For example, a popular mod for another game I enjoy spent two years in beta. By the time they finally released, the next version of the game came out (which had many of the features that the mod was adding), and as a result nobody really ever could play the mod.
How about releasing the current RC via the Steam Beta feature? I'm sure you'll get plenty of volunteers to test-drive it... pretty sure there will be servers, too.
We have not done this because the last 2 bugs are so game breaking, that we would not be able to get any feedback.
On the plus side, we think we've found the issue and will be testing a fix later today. Once that's done, stress testing over the next few days, through the weekend will inform us on what day next week we will release.
Can't you at least release the patch on the beta branch?
Then you have from now till next week sometime for random people to test and possibly find bugs before you plan on an actual release.
Coding is like players playing the game:
Some know where to position with awesome aim and gamesense.
Other players missing some shots or making the wrong decisions.
But in the end the game is over after a while with a win or loss.
That is perhaps the worst analogy I have ever seen, but thanks for the effort Para lol
Can't you at least release the patch on the beta branch?
Then you have from now till next week sometime for random people to test and possibly find bugs before you plan on an actual release.
MrFangs literally asked the same question - you cant actually play the game because it's so broken in our beta branch. As in, a round cant be completed. At all due to this bug.
We will be testing the fix for it soon, once we find you can play NS2 again, the patch will be ready to go as everything else has been checked already.
What I don't really understand... this is the community development team. Why not release on a friday? I mean, you are working in your spare time after all, and there's more of that during the weekend.
Not that I don't want you guys to have a weekend, but this might help spread the workload a bit.
What I don't really understand... this is the community development team. Why not release on a friday? I mean, you are working in your spare time after all, and there's more of that during the weekend.
Not that I don't want you guys to have a weekend, but this might help spread the workload a bit.
The weekend is when the playercount is highest - there will be a lot of frustration for people who only play in weekends and who can't play because of a botched patch. It's better to wait till monday or what ever.
What I don't really understand... this is the community development team. Why not release on a friday? I mean, you are working in your spare time after all, and there's more of that during the weekend.
Not that I don't want you guys to have a weekend, but this might help spread the workload a bit.
We can't publish to Steam ourselves, we have to go through UWE, we're not going to force them to work a weekend for us, are we? :P
Comments
They just don't care do they! *shakes fist*
It's okay.. Come.. I shall give you comfort..
Patch? :P
So in conclusion, nanites.
Shout out to @ghoulofGSG9 for working to 4AM last night fixing those newly uncovered ancient bugs!
(I'm kidding)
Good to hear that bug testing is working. QA is one of those things that's rarely appreciated and often hated, but it is so essential.
on a more serious note (i know you guys are doing your best, please take your time): i am aware that any change can cause a bug in a totally different part, but this seems to happen very often in the case of ns2 (since long before the CDT started).
are there any patterns on how these bugs are interacting? maybe you can identify them and/or add more automated tests/tools/checklists to quickly identify such issues?
Some know where to position with awesome aim and gamesense.
Other players missing some shots or making the wrong decisions.
But in the end the game is over after a while with a win or loss.
1. Bugless game with awesome features - released fall of NEXT year, ad infinitum
2. Game with new features that have been laboriously debugged with minor issues - released quarterly to semi-annually
3. Game with no new major bugs, and a few extra features with some bugs - released monthly
4. Game with all kinds of fun features and countless bugs - released weekly
Corollary question:
A. How many current players will be lost while waiting for the new content/bugfixes to come out?
B. How many players will you loose/gain due to bugs/features in the new release?
Long story short, at some point, you have to make a release, and it WILL contain bugs.
P.S. This is an eternal struggle in every development project.
but usually, they affect things in the same (or at least semantically close) "area" of functionality. it seems to be this is different in ns2 (at least that's my impression having only limited information). ok maybe that's more common in commercial projects with many more minds involved.
what i've been told is that one of the major improvements in commercial coding is the employment of sophisticated testing approaches in several stages (e.g. unit testing).
i have no idea how exactly things are working in ns2 and i have no experience on commercial scales, i am merely proposing it might be worth rethinking if there are possibilities to improve the methods of detecting unexpected bugs e.g. by adding more in-code tests that are redundant at the moment but assert a warning/error when something is unexpectedly breaking. question is wether it is worth the investment or not, that is for the CDT to judge.
How about releasing the current RC via the Steam Beta feature? I'm sure you'll get plenty of volunteers to test-drive it... pretty sure there will be servers, too.
Pros:
- More eyeballs and test time
- More different hardware/driver configurations
- Pacify the rampaging hor... err, the community
Cons:
- A bit of effort to set up the beta channel.. not sure how much?
While certainly true, you dont want to release with a bug that crashes all marine players at the end of the round, or one that crashes the server if anybody is still in the commander mode when changing maps. Both appeared yesterday when the game went into release candidate mode. So yes, bugs will always be there, but certain types of bugs can not be released (but dont worry, both bugs have been fixed, now its just a matter to test the fixes and not releasing on fridays)
The problem with unit tests is that they test small-scale features. They are definitely useful, but they can only test precisely what they've been set up for.
Example: Create some Hydra entities; create some hostile entities (Marines, ARCs, buildings) in different positions (range, line-of-sight etc.); check which Hydra chooses which target to attack. Compare with the expected result.
To test something as complex as a multi-player game, you would have a much better chance to catch bugs with higher-level integration tests.
Example: Set up a server; connect two simulated clients (using as much real code as possible); simulate one client going Fade, the other going Shotgun Marine; move both to defined starting positions; send timed movement and aim commands; send a "pull trigger" command when the Marine should have the Fade in its crosshairs; let simulated time pass. Compare the damage feedback from the server to both clients with the expected result.
Building on this, you can create more complex tests: For example, let the Fade kill the Marine a moment before the shot is registered in game-time, and check that the Fade does not take damage (or vice versa).
As you can imagine, this gets complicated *very* fast. And still, you'd have to do this for a huge number of scenarios. Evolve in various places and check where the new lifeform ends up... Have many Marines use a Phase Gate at the same time... Check how Spores affect Sentries... Check how MACs and Drifters react to movement commands... etc etc.
So in some cases, creating tests can be useful, but for all the complex interactions in the game, you're usually better off testing manually.
99 little bugs
Take one down, patch it around
117 little bugs in the code
anyway, this is getting quite off topic so i'll shut up now
I was just trying to point out that it is not easy to either predict or fix all of the bugs that may come up, and that you must weigh the benefit of fixing bugs against the cost, and only work on the ones that tip the scale.
I've been developer for 15 years, so I understand all too well that releases must be delayed when a showstopper comes along. My only problem is that people (especially in projects without hard deadlines) often place such a high bar on the quality of the release that it never happens. (I'm not claiming the CDT is doing this). For example, a popular mod for another game I enjoy spent two years in beta. By the time they finally released, the next version of the game came out (which had many of the features that the mod was adding), and as a result nobody really ever could play the mod.
We have not done this because the last 2 bugs are so game breaking, that we would not be able to get any feedback.
On the plus side, we think we've found the issue and will be testing a fix later today. Once that's done, stress testing over the next few days, through the weekend will inform us on what day next week we will release.
Also, please tune in for this:
And thanks for all your support guys. We really do appreciate it and all your thoughts!
The suspense is killing me.
That's called a WasabiFail® ;-) August 3rd will be the ClogCast.
Then you have from now till next week sometime for random people to test and possibly find bugs before you plan on an actual release.
That is perhaps the worst analogy I have ever seen, but thanks for the effort Para lol
MrFangs literally asked the same question - you cant actually play the game because it's so broken in our beta branch. As in, a round cant be completed. At all due to this bug.
We will be testing the fix for it soon, once we find you can play NS2 again, the patch will be ready to go as everything else has been checked already.
Not that I don't want you guys to have a weekend, but this might help spread the workload a bit.
The weekend is when the playercount is highest - there will be a lot of frustration for people who only play in weekends and who can't play because of a botched patch. It's better to wait till monday or what ever.
We can't publish to Steam ourselves, we have to go through UWE, we're not going to force them to work a weekend for us, are we? :P
Well, no name and shame here in the forum.