Pondering Rapid Iteration - Natural Selection 2

SystemSystem Join Date: 2013-01-29 Member: 182599Members, Super Administrators, Reinforced - Diamond Posts: 466 vanilla
edited December 2015 in NS2 General Discussion

imagePondering Rapid Iteration - Natural Selection 2

One of the things I’ve been rambling on about a lot lately is the ‘How’ of NS2 development. That right now the ‘How’ is more important than the ‘What.’ Specifically, I spend...

Read the full story here


«134

Comments

  • HughHugh Cameraman San Francisco, CA Join Date: 2010-04-18 Member: 71444Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Reinforced - Silver, Reinforced - Onos, WC 2013 - Shadow, Subnautica Developer, Pistachionauts Posts: 1,779 admin
    Bump to achieve comment thread liftoff, initiation, startup, etc
    The fastest way to contact me is via @hugh_jeremy on Twitter! Add me on Steam: Strayan. I do not check forum PMs.
    Keep up to date on what is going on with: Unknown Worlds | Natural Selection 2 | Subnautica
    Chat with the UWE team on Twitter | The best way to be notified about Subnautica updates is to sign up to our development newsletters
  • WyzcrakWyzcrak Pot Pie Aficionado Join Date: 2002-12-04 Member: 10447Forum Moderators, Constellation, NS2 Playtester, Squad Five Blue Posts: 920 mod
    My first thought was A/B testing, but I'm not sure how the details would work out.

    https://en.wikipedia.org/wiki/A/B_testing
    TGNS Portal | NS2 side-by-side game viewer
    IronHorse wrote: »
    TG's win or lose mod that replaces the concede function is the best of both worlds and should be official.
  • FrozenFrozen New York, NY Join Date: 2010-07-02 Member: 72228Members, Constellation Posts: 1,480 Advanced user
    I can't read through the purple
    mattji104
  • IeptBarakatIeptBarakat The most difficult name to speak ingame. Join Date: 2009-07-10 Member: 68107Members, Constellation, NS2 Playtester, Squad Five Blue, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow Posts: 2,679 Advanced user
    How are servers going to handle this rapid fire iteration? Usually you have to email servers owners ahead of time to let them know to download an upcoming update but with this it sounds like server owners would have to frequently check for updates and drop filled servers to stay afloat in the sea of updates. This also means players would have to frequently close the game to keep updating if the devs iterate on a whim.

    This doesn't sound practical in a multiplayer environment.
    ZEROibis
  • ObraxisObraxis Subnautica Animator & QA Lead, NS2 Person Join Date: 2004-07-24 Member: 30071Super Administrators, Forum Admins, NS1 Playtester, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Silver, WC 2013 - Supporter, Subnautica Developer, Pistachionauts Posts: 3,003 admin
    This doesn't sound practical in a multiplayer environment.

    It actually already happens. With NS2+.

    Mendasp can update it as much as he wants, as often as he wants. As an example, if you put the latest build of NS2 into a mod on Steam, it will auto-update everyone & servers at the end of a round, or on map-change. And with more than 1 person developing this 'mod' (which is NS2) iteration can be faster, and bugs fixed quicker. A new Hive backend can be constructed so that if Steam fails, the 'mod' can always be got.

    I'm not saying this is the best way to do it, but to me, it's the most practical. Emulate NS2+ and how Mendasp has updated and worked on his mod, and apply it to NS2's own development. He proved that it can work very well, it might be time to emulate his methods for NS2 itself.


    If NS2 fails, follow my Nuke From Orbit Guide.

    NS2 Discord: https://discord.gg/0TKQwZWu7JEPeAgW

    Subnautica Discord: https://discord.gg/subnautica

    Saffron_bakerUncleCrunchCmdrKeen
  • Scout255Scout255 Canada Join Date: 2015-01-24 Member: 201015Members, NS2 Map Tester Posts: 18 Advanced user
    Unless the system has a method of updating servers automatically (as workshop mods do) combined with a method to guarantee that no mods are broken with your changes, your going to kill the two main reasons we can maintain the player numbers we do. These are server communities (DMD, Ibis, wooza, etc) and 3rd party mods that make the game livable (shine, ns2+, various server mods, etc.)

    Right now I just see this rapid prototyping being a means to ending the game rather than saving it.....

    I also believe it will push developers away from risky changes with big payoffs that require extensive QA (I.e. all the hit reg fixes, all the hitches and file system changes, network improvements, engine bugs, etc.). These likely would have never happened had it not been for a build-test-break-fix iterative process, as the first builds have far too many bugs. In essence your taking the game back to a beta or alpha stage and making us all testers that paid for a final product.

    When I get off work, the last thing I want to find is a game breaking bug preventing me from enjoying my favorite game. This method increases the chances of that happening as a regular occorance. I'd rather have what happened this week that you so vocally decried as a massive failure of the monolithic building process than what you are proposing.

    Regardless, I'm addicted I'll probably stick around because I love the game and playing with many of the senior playerbase. However, when the scales tip from the game being fun to being an aggravation, I can see lots of loyal players leaving (the same ones who have kept the game alive since UWE left). Quite the gamble when I don't see a particularly big potential gain. Considering the up side hasn't been communicated to any of us...... Though I suppose communication has never been a strong point under UWE, of past performance is anything to go by.
    NordicZEROibis
  • IeptBarakatIeptBarakat The most difficult name to speak ingame. Join Date: 2009-07-10 Member: 68107Members, Constellation, NS2 Playtester, Squad Five Blue, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow Posts: 2,679 Advanced user
    Obraxis wrote: »
    This doesn't sound practical in a multiplayer environment.

    It actually already happens. With NS2+.

    Mendasp can update it as much as he wants, as often as he wants. As an example, if you put the latest build of NS2 into a mod on Steam, it will auto-update everyone & servers at the end of a round, or on map-change. And with more than 1 person developing this 'mod' (which is NS2) iteration can be faster, and bugs fixed quicker. A new Hive backend can be constructed so that if Steam fails, the 'mod' can always be got.

    I'm not saying this is the best way to do it, but to me, it's the most practical. Emulate NS2+ and how Mendasp has updated and worked on his mod, and apply it to NS2's own development. He proved that it can work very well, it might be time to emulate his methods for NS2 itself.

    That's for mods in the steam workshop, unless you plan on delivering updates to the game through the workshop then it's an entirely different matter.
  • ObraxisObraxis Subnautica Animator & QA Lead, NS2 Person Join Date: 2004-07-24 Member: 30071Super Administrators, Forum Admins, NS1 Playtester, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Silver, WC 2013 - Supporter, Subnautica Developer, Pistachionauts Posts: 3,003 admin
    That's for mods in the steam workshop, unless you plan on delivering updates to the game through the workshop then it's an entirely different matter.

    But...That's what I said? It's certainly a way to do it. NS2+ already proved it can be done.

    If NS2 fails, follow my Nuke From Orbit Guide.

    NS2 Discord: https://discord.gg/0TKQwZWu7JEPeAgW

    Subnautica Discord: https://discord.gg/subnautica

    UncleCrunch
  • Kouji_SanKouji_San Sr. Hινε Uρкεερεг - EUPT Deputy The Netherlands Join Date: 2003-05-13 Member: 16271Members, NS2 Playtester, Squad Five Blue Posts: 15,780 Advanced user
    edited December 2015
    @Obraxis that seems all well and good, but NS2+ is an addon mod, which if there are huge issue doesn't kill the game and can be disabled if it is severe enough. NS2 IS the game, if something breaks there the game is broken. And all the logistics behind it... You know... I think most of the major concerns have already been brought up in the last few posts.

    I guess we'd also love to hear an answer on this risk vs reward factor @Hugh is so keen about... We've been waiting for... What 2 or 3 weeks now? Would you please stop with the vagueness already :dizzy:

    Guardian of the "magic cookiejar" 

    Retired forum Admin, I mostly used a flamethrower tank for disputes... Mostly

    Retired EUPT Deputy | Moral Support | Squad 5 Blue | 102 1HP Skulk escapes and counting

  • IeptBarakatIeptBarakat The most difficult name to speak ingame. Join Date: 2009-07-10 Member: 68107Members, Constellation, NS2 Playtester, Squad Five Blue, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow Posts: 2,679 Advanced user
    Obraxis wrote: »
    That's for mods in the steam workshop, unless you plan on delivering updates to the game through the workshop then it's an entirely different matter.

    But...That's what I said? It's certainly a way to do it. NS2+ already proved it can be done.

    Can that be done with engine changes? Surely you'd have to restart regardless for some updates.
    Saffron_baker
  • SantaClawsSantaClaws Denmark Join Date: 2012-07-31 Member: 154491Members, Reinforced - Shadow Posts: 1,082 Advanced user
    @Kouji_San - Yes, if they release an update that breaks the game - then they break the game.... Not any different from the situation we have now.

    My concern is more the fact that I'm not interested in downloading new updates constantly. To me, the rapid iteration model, even if it works perfectly with no issues, is not anything I desire.

    I can see the value in being able to hotfix rapidly. But other than that, I really prefer the slower in nature updates.

    Oh well, we will have to see I guess - maybe this is a brilliant idea in practise, who knows.
    Saffron_baker
  • ceribikceribik Join Date: 2009-11-24 Member: 69492Members, Reinforced - Supporter, Reinforced - Silver Posts: 102 Fully active user
    edited December 2015
    .
    Post edited by ceribik on
  • CalegoCalego Join Date: 2013-01-24 Member: 181848Members, NS2 Map Tester Posts: 763 Advanced user
    This is an excellent post describing a fair chunk of questions we were actually hoping you already knew the answers to, had weighed the consequences of, and had decided was worth it in the end (with good reasons that we're all curious to hear). I mean, why else would you just go ahead and initiate this plan unless you had thought through the multitude of issues that could arise?

    Surely UWE didn't sit up one day and say "We can do this, therefore lets do it!." And then come up with a long list of questions, talk about them, then ask the community about some more questions they might have.

    We've posited questions on these forums, we've been asking these things since the initial vauge references were made, we want to know the answers to these questions. We don't want to know the answers after s*** hits the fan, if s*** hits the fan.
    Obraxis wrote: »
    This doesn't sound practical in a multiplayer environment.

    It actually already happens. With NS2+.

    Mendasp can update it as much as he wants, as often as he wants. As an example, if you put the latest build of NS2 into a mod on Steam, it will auto-update everyone & servers at the end of a round, or on map-change. And with more than 1 person developing this 'mod' (which is NS2) iteration can be faster, and bugs fixed quicker. A new Hive backend can be constructed so that if Steam fails, the 'mod' can always be got.

    I'm not saying this is the best way to do it, but to me, it's the most practical. Emulate NS2+ and how Mendasp has updated and worked on his mod, and apply it to NS2's own development. He proved that it can work very well, it might be time to emulate his methods for NS2 itself.


    This is an interesting point, Perhaps something for you at UWE to consider is to make an Official "Update" Mod (Which I seem to remember being suggested like gazillions of times before), and update that as frequently as your hearts desire. Server ops can opt into having this mod active on their servers, nothing horrible breaks, and new content/ideas are delivered in realtime to the players. The mod can be disabled at any time if something terribad happens.

    Limitations of this method are that major engine changes won't work (right?). And since those are the biggest baddest most possible to break everything changes, that's completely fine. Those changes go out to Playtesters and Beta build curious people in the processes that have been refined in the years since this game has been released untill they've been deemed stable.

    At that point, you compile all the engine changes beta things, and all the stuff from the Mod you want to, and push out a new 'monolithic' build. Rinse and repeat.

    Just like that you've made everything possible to happen quickly, efficently (as far as steam can move efficiently), and you've put the ultimate decision power in the hands of us, the people who play the game and run the servers. Until everything is stable and accepted, then you push things out permanently.

    You haven't yet responded about the point behind this, but usually the point behind an agile development cycle is to test the product in the real world and see if the consumers like the changes/new features quickly. So ultimately your goal is to make the game better for us, the players/community right? So you're not going to ignore us when we cry out in dismay at a change and disable the mod on the servers until something is changed back, right? Just checking because it seems a lot of the time that we're talking at a one way mirror.
    Saffron_bakerJektCmdrKeen
  • CalegoCalego Join Date: 2013-01-24 Member: 181848Members, NS2 Map Tester Posts: 763 Advanced user
    Some more things to consider (from a server-op point of view):

    Rented servers are a thing, and they're not the easiest thing to update or control. Often times you have to get on the horn with someone at support to get things updated. It's been nice having advanced warning of such updates so you know when to yell at who, with rapid-fire (even if it's not every half-hour) updates will be even more of a pain for admins who don't have the ability to write/use scripts on their boxes to check for/update automatically.

    Furthermore, every update to the game requires a complete shutdown/restart of the server, unlike a mod update, which is just a mapchange. This is basically a "Kiss your seeded server population goodbye" because once they see that redplug, they'll move on (assuming they're not a community based server, and even then). So there's that too.

    Basically, it's rough enough as is to keep lights on in a rented server, increasing the game update rate will only make it tougher, and seeing as how there's no more UWE Official servers, some strong consideration to the server admins should be there.
    JektScout255
  • Saffron_bakerSaffron_baker Sweden Join Date: 2015-06-09 Member: 205352Members Posts: 492 Advanced user
    Give it a try 3 weeks mabey? Then people can vote.
  • KartoshkaKartoshka Join Date: 2012-01-13 Member: 140302Members, Reinforced - Shadow Posts: 32 Fully active user
    I must admit, i am not 100% opposed to the concept, but I like the idea that has been posed of an official opt-in update mod that is incorporated into the main branch once it has been verified. At the end of the day, a lot of the success of this style of development will come down to how responsible the developers will be with it. It will take time to establish that trust.
    Saffron_baker
  • PelargirPelargir Join Date: 2013-07-02 Member: 185857Members, Forum Moderators, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Supporter, Reinforced - Silver, WC 2013 - Silver, Forum staff Posts: 1,199 mod
    @Calego: we're currently working on a new article that should, hopefully, get published later this week. We asked a couple of questions to Hugh and we'll put down all the answers. They might not satisfy all the questions you may have in your mind but it will address some. We'll keep you updated, but you can also stay tuned by visiting http://ns2news.org/
    "All we have to decide is what to do with the time that is given to us"

    "However beautiful the strategy, you should occasionally look at the results."


  • elodeaelodea Editlodea Join Date: 2009-06-20 Member: 67877Members, Reinforced - Shadow Posts: 1,786 Fully active user
    Will you be doing changelogs for each update?
    Gorge bilebomb forever.
    philoglKouji_SanDecoy
  • .trixX..trixX. Budapest Join Date: 2007-10-11 Member: 62605Members Posts: 856 Advanced user
    This sounds like a batshit crazy idea. This scheme could work for beta and early access-games, because there's not many things to upset. But cmon, NS2 has been stable for a long time, what if you mess up balance? How will the few regular players react? I'd quit and leave that shit for a few days. I don't really think NS2 needs this if you want to gain more players...
    NO Cyril, when they're dead they're just hookers!
    Jekt[AwE]SentinelOscarTheCouchDecoy
  • GhoulofGSG9GhoulofGSG9 Join Date: 2013-03-31 Member: 184566Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Reinforced - Supporter, WC 2013 - Supporter, Pistachionauts Posts: 2,832 admin
    edited December 2015
    To be honest i think everyone focuses too much on "faster iterations" while this really is about agile software development.

    The Agile Manifesto

    Everyone of us is very much aware that daily builds are not an option at the moment due to:
    1. We have no way to automatically validate builds
    2. NS2 relies on mods that we can't afford to break
    3. Server admin take about 1 day to update
    4. We have no way to gather user feedback fast enough to react on critical changes.

    But if we find a way to solve some of those issues a faster release cycle (not directly daily builds) is possible to do.

    What's the idea behind using agile development for NS2 is basically that the past has shown that often features which were developed over month had absolute no effect on the player base and went unnoticed/unused. From now on we do not want to invest any time on a feature that you neither want nor use in the end.

    Secondly NS2 builds often got rather complex with many features depending onto each other. Then when one of those feature turned out to be broken we often had no other choice but to revert a complete set of features. That should not happen anymore. Of course we don't want to use the player base as testers but there will be always edge cases all our internal tests won't cover. Those issues have to be uncovered and reacted on much faster than they have been reacted on in the past 2 years.

    @elodea Every update will come with a change log as we have to make sure you are actually aware what to give feedback on. We even already went a step further and made the ns2 repositories commit history public (still WIP): http://goo.gl/hDlYAU
    Post edited by GhoulofGSG9 on
    Developer, Modder and Server Admin of Survival of the Fattest - Ingame Nick: Ghoul
    Saffron_bakerelodeavartija
  • .trixX..trixX. Budapest Join Date: 2007-10-11 Member: 62605Members Posts: 856 Advanced user
    edited December 2015
    To be honest i think everyone focuses too much on "faster iterations" while this really is about agile software development.

    It doesn't really matter what you call it... xP The idea outlined in the blog post hints at a "trialanderror" development mode, which to me means that devs dream up and code a change then push it to the public to see what will happen.

    On the other hand, if the idea is still to thoroughly test any changes made to the game, why announce this in the first place? As far as I know, that's the current practice.
    Don't get me wrong, I'd love to see the game getting even more awesome, but the fanbase is a bit fickle. You really don't want to mess up the playtime the few us have, otherwise the game will die if even 1/10 times there is some problem with an update...
    NO Cyril, when they're dead they're just hookers!
  • simple_simple_ Join Date: 2013-09-10 Member: 188102Members Posts: 41 Fully active user
    I in favor of moving to a faster development circle.

    One approach to the "updates break" problematic would be something like this:
    - At all times there are always two version of the game (a and b) on all clients and servers.
    - If a clients joins, the server tells him which version he is currently running and that one gets used.
    - If an update is release (while version b is considered stable), version a gets removed everywhere and the new version is added.
    - Server admins can switch between both versions at will.

    With that, if a release is broken server admins can just "rollback" to the previous version till everything is fixed.
    Server admins can test a new version while they watch for mod problems and can give reports.
    Modders have more time fixing broken mods, knowing that their mod still can used with the older version.

    Of course that would involve some work on uwe side and some patches may not work with that system and have to be release normally.

    Contact me, if I should elaborate.
Sign In or Register to comment.