Don't allow compiled lua plugins

devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
LUA has an optional compiler that can be used to prevent the code of a plugin or addon from being visible. There is a decompiler, but it's better to not have to reply on that. The community will ultimately benefit if the source of all the addons is visible for people to look at and make changes to.
«1

Comments

  • WhiteZeroWhiteZero That Guy Join Date: 2004-06-24 Member: 29511Members, Constellation
    Chances are, the NS2 Engine couldn't even interpret "compiled" LUA code.
    But yeah, I agree. NS2 should be much like Garry's MOD, and have all LUA sources available.
  • InsaneInsane Anomaly Join Date: 2002-05-13 Member: 605Members, Super Administrators, Forum Admins, NS1 Playtester, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, NS2 Map Tester, Subnautica Developer, Pistachionauts, Future Perfect Developer
    Given their desire to make NS2 extremely moddable, I think it's probably unlikely that they'd be using "compiled" LUA code.
  • devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
    Compiled code is something that lua supports by default, and needs to be explicitly disabled.
  • RobBRobB TUBES OF THE INTERWEB Join Date: 2003-08-11 Member: 19423Members, Constellation, Reinforced - Shadow
    As they employ their own interpreter as far as I know, I doubt it will understand compiled code.
  • devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
    <!--quoteo(post=1712470:date=Jun 16 2009, 03:32 AM:name=RobB)--><div class='quotetop'>QUOTE (RobB @ Jun 16 2009, 03:32 AM) <a href="index.php?act=findpost&pid=1712470"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->As they employ their own interpreter as far as I know, I doubt it will understand compiled code.<!--QuoteEnd--></div><!--QuoteEEnd-->

    Er, I've heard nothing similar to that. It doesn't make much sense to write your own interpreter, and still pick the rest of a syntax like lua's.
  • FocusedWolfFocusedWolf Join Date: 2005-01-09 Member: 34258Members
    Ummm no... you guys are not smart on this topic.

    When you play on a server... you want scripts to fast download and execute. I don't want my computer spending 15 minutes to compile ######. It would drastically hurt performance, use up tons of ram, and ruin the experience of the game. DO YOU WANT SERVERS TO HAVE TO DOWNLOAD THOUSANDS OF LUA FILES LIKE GARRYSMOD.... DO YOU WANT TO HAVE TO USE SOME SVN PROGRAM TO UPDATE EACH AN EVERY CUSTOM PACKAGE SOMEONE MAKES LIKE GARRYSMOD!!!

    It's very much faster to crc check against a single solitary binary representing 9000 lines of lua then to crc check against 9000 files.

    Imagine the following, with your thinking of not using binary scripts:

    you just joined a server. you play on this server every day... what's your computer doing... ohh it has to do a lot of crc checking on thousands of 4kb files... your harddrive spins and spins... It takes me about 3-4 minutes to join a spacebuild3 server in garrysmod... that's what a couple of custom packs of lua will give you.

    Now with binary scripts:
    The server is running 10 addins... no problem, you already downloaded them... in less then a millisecond your computer checked all 10 assemblies against the server and your game loads into memory efficiently... now your playing.

    =====================================

    Now for a reality check.... someone already mentioned a lua decompiler... and alot of the scripters will be releasing the code... what more do you want??? Do you want to be able to fiddle, and corrupt with your little experience in scripting, something that someone else made and in the process have it so load times between maps takes ages??? possibly killing peoples harddrives with all the unnecessary disk accessing??

    .....

    At this point i know we dont have to worry here... the devs already wrote a ton of lua code and they already know to compile it to binary for performance... therefore binary scripts will be supported.
  • devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
    edited June 2009
    Edit: Upon further review, I'm not falling for your trolling.
  • solcottsolcott Join Date: 2003-11-04 Member: 22307Members
    <!--quoteo(post=1712612:date=Jun 16 2009, 09:32 PM:name=devicenull)--><div class='quotetop'>QUOTE (devicenull @ Jun 16 2009, 09:32 PM) <a href="index.php?act=findpost&pid=1712612"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Edit: Upon further review, I'm not falling for your trolling.<!--QuoteEnd--></div><!--QuoteEEnd-->
    Winner!

    Also, agreed with the whole interpreting vs loading precompiled LUA subject. Interpreting takes like half a second, you lose far more than you gain by compiling.
  • flying_mooseflying_moose Join Date: 2009-06-03 Member: 67676Members, Constellation
    edited June 2009
    Oh oh oh! I'll take the bait!

    1) When you play on a server - some things are handled client side and some things are handled server side. Unless something specifically adds an image or model, (client data) its unlikely you'll need to download it. Most scripts will be executed server-side.
    2) Lua scripts have very fast compile time, I expect the entire time it takes to compile the NS2 LUA scripts to take less time then it takes you to load a webpage.
    3) Believe it or not, compiled LUA files are actually larger then un-compiled - even tho the comment lines are removed on compilation, so you'd have to spend more time downloading. (Don't ask me why, its a known fact). In addition a given LUA script will take up the same memory size compiled or not.
    4) SVN has absolutely nothing to do with this - they release their packages via SVN by their choice.
    5) While I'm not specifically aware of how CRC hashes: Due to its nature of comparing memory blocks, in theory, any number of files of total size X will take the same amount of time to CRC as a single file of size X assuming all are initially cached before checking. (as that will lessen overall seek times.) Add that to option 3 and guess what. Un-compiled LUA scripts take LESS time to CRC.
    6) The reason garrys mod has to re-download everything is because of the source engine - not because of LUA.

    Finally) devicenull IS the main community scripter right now. Hes not modifying someone else's admin menu. Hes writing the one most people will end up using.

    Summary - Ummm... No. You not so smart on this topic.
  • FocusedWolfFocusedWolf Join Date: 2005-01-09 Member: 34258Members
    <!--quoteo(post=1712626:date=Jun 17 2009, 12:16 AM:name=flying_moose)--><div class='quotetop'>QUOTE (flying_moose @ Jun 17 2009, 12:16 AM) <a href="index.php?act=findpost&pid=1712626"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Oh oh oh! I'll take the bait!

    1) When you play on a server - some things are handled client side and some things are handled server side. Unless something specifically adds an image or model, (client data) its unlikely you'll need to download it. Most scripts will be executed server-side.
    2) Lua scripts have very fast compile time, I expect the entire time it takes to compile the NS2 LUA scripts to take less time then it takes you to load a webpage.
    3) Believe it or not, compiled LUA files are actually larger then un-compiled - even tho the comment lines are removed on compilation, so you'd have to spend more time downloading. (Don't ask me why, its a known fact). In addition a given LUA script will take up the same memory size compiled or not.
    4) SVN has absolutely nothing to do with this - they release their packages via SVN by their choice.
    5) While I'm not specifically aware of how CRC hashes: Due to its nature of comparing memory blocks, in theory, any number of files of total size X will take the same amount of time to CRC as a single file of size X assuming all are initially cached before checking. (as that will lessen overall seek times.) Add that to option 3 and guess what. Un-compiled LUA scripts take LESS time to CRC.
    6) The reason garrys mod has to re-download everything is because of the source engine - not because of LUA.

    Finally) devicenull IS the main community scripter right now. Hes not modifying someone else's admin menu. Hes writing the one most people will end up using.

    Summary - Ummm... No. You not so smart on this topic.<!--QuoteEnd--></div><!--QuoteEEnd-->

    1. I read some dev video that said they wanted a great deal of both server and clientside scripting and a great deal of syncronization between the two.

    2. a lua script will compile in the order of milliseconds but regardless.. if it's anything like garrysmod then it will take ages to join a server

    3. hmm i'll believe it that the lua assemblies are bigger then the individual scripts but it is still faster to download 1 big file then 1000 small files... i can just imagine alot more disk activity both on the users computer, the other users connected to the server, and the server itself. Ofcourse that can be eliminated if the engine compresses all the scripts in a zip and sends that as a single binary... although theirs no reason that cant be done with a combiled lua assembly inside.

    4. SVN... well that depends i mean if the game has heavy clientside mods (like how CO will be a lua mod) that get heavily accepted by the community... then their will be alot of mismatching of versions between differant servers and the official svn... it could get messy.

    5. you might be right about the time it takes to crc but their has to be some overhead in terms of loading all these files and unloading... everytime you join a server..zzz Their will be millions of script files... every other server will be custom all the way... imagine how many zombie mod servers their will be lol. I'm just describing what is garrysmod, soon to be ns2. I.e. every server is a minigame depending on the module it's running.

    6. I'm not sure what you mean on 6. When i play... it's only if your missing the files that it will download... But i have all of the files because i svn update many packages everytime, before i play... nonetheless when i join a fast server it still takes about 4 minutes to get in game... about 2 just to start garrysmod... this is strictly because of all the custom content... i could just as easily delete all the files and find garrysmod load as fast as any other hl2 game. On that note... it takes alot longer to delete folders with a bazillion small script files then it does to delete the reference to 1 big file... that's the performance aspect that i dont see this game getting around... big binaries, even when gigantic with respect to their source will always be faster then the source. This is because of what compiling a script means... depends on lua but just changing the labels of differant variables to be byte symbols is enough to improve execution performance. Atleast that's how my TI89 works xD
  • flying_mooseflying_moose Join Date: 2009-06-03 Member: 67676Members, Constellation
    <!--quoteo(post=1712629:date=Jun 16 2009, 10:59 PM:name=FocusedWolf)--><div class='quotetop'>QUOTE (FocusedWolf @ Jun 16 2009, 10:59 PM) <a href="index.php?act=findpost&pid=1712629"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->1. I read some dev video that said they wanted a great deal of both server and clientside scripting and a great deal of syncronization between the two.

    2. a lua script will compile in the order of milliseconds but regardless.. if it's anything like garrysmod then it will take ages to join a server

    3. hmm i'll believe it that the lua assemblies are bigger then the individual scripts but it is still faster to download 1 big file then 1000 small files... i can just imagine alot more disk activity both on the users computer, the other users connected to the server, and the server itself. Ofcourse that can be eliminated if the engine compresses all the scripts in a zip and sends that as a single binary... although theirs no reason that cant be done with a combiled lua assembly inside.

    4. SVN... well that depends i mean if the game has heavy clientside mods (like how CO will be a lua mod) that get heavily accepted by the community... then their will be alot of mismatching of versions between differant servers and the official svn... it could get messy.

    5. you might be right about the time it takes to crc but their has to be some overhead in terms of loading all these files and unloading... everytime you join a server..zzz Their will be millions of script files... every other server will be custom all the way... imagine how many zombie mod servers their will be lol. I'm just describing what is garrysmod, soon to be ns2. I.e. every server is a minigame depending on the module it's running.

    6. I'm not sure what you mean on 6. When i play... it's only if your missing the files that it will download... But i have all of the files because i svn update many packages everytime, before i play... nonetheless when i join a fast server it still takes about 4 minutes to get in game... about 2 just to start garrysmod... this is strictly because of all the custom content... i could just as easily delete all the files and find garrysmod load as fast as any other hl2 game. On that note... it takes alot longer to delete folders with a bazillion small script files then it does to delete the reference to 1 big file... that's the performance aspect that i dont see this game getting around... big binaries, even when gigantic with respect to their source will always be faster then the source. This is because of what compiling a script means... depends on lua but just changing the labels of differant variables to be byte symbols is enough to improve execution performance. Atleast that's how my TI89 works xD<!--QuoteEnd--></div><!--QuoteEEnd-->

    1) Yea, I guess in the end we'll have to just wait and see how its implemented. While both client and server will have scripted elements, we don't know if those elements will NEED to be shared.

    3) I didn't even consider zipping. because of LUA's language structure I think zipping would compress them down to a pretty damn small file size. On the other hand, compressing binary files dont usually yield very good compression rates.

    4) SVN would only be necessary for mods that update very frequently - and once again - thats the choice of the mod author.

    5) Well, I think it will be like l4d. Most servers will either be pretty standard with mostly extra admin features, and maybe a few minor modifications. I don't think we'll be seeing TOO many different overall game types - although I'm sure there will be more then a few servers that have their own custom mods.

    6) Now I'm confused so I'll just drop it.

    What it comes down to is with uncompiled scripts - yes, there will be a slightly longer load time when the game starts. and yes, it will take a little bit more processing power in order to process the scripts. I believe both will be negligible. However the net gains by having an open-source community are far greater then the benefits of compiling the code and will yield a much better collection of mods in the long run. - I think.
  • locallyunscenelocallyunscene Feeder of Trolls Join Date: 2002-12-25 Member: 11528Members, Constellation
    <!--quoteo(post=1712636:date=Jun 17 2009, 02:14 AM:name=flying_moose)--><div class='quotetop'>QUOTE (flying_moose @ Jun 17 2009, 02:14 AM) <a href="index.php?act=findpost&pid=1712636"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->What it comes down to is with uncompiled scripts - yes, there will be a slightly longer load time when the game starts. and yes, it will take a little bit more processing power in order to process the scripts. I believe both will be negligible. However the net gains by having an open-source community are far greater then the benefits of compiling the code and will yield a much better collection of mods in the long run. - I think.<!--QuoteEnd--></div><!--QuoteEEnd-->
    I agree. There might be compiled versions with a checksum so that there can be "vanilla" mods(votemap, votekick, etc), but that code will have to be at least available to the devs if not everyone. it makes sense to have open source in this context.
    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I read some dev video<!--QuoteEnd--></div><!--QuoteEEnd-->
    No offence FocusedWolf, but that quote is gold.
  • ThaldarinThaldarin Alonzi&#33; Join Date: 2003-07-15 Member: 18173Members, Constellation
    It's a different engine to Source, comparing it to Garrysmod doesn't make sense to me.

    Yes, that quote is gold.
  • PrivatePrivate Join Date: 2007-06-10 Member: 61204Members, Constellation
    Right, I believe we are (supposed to be) discussing pre-compiled Lua plugins.

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->LUA has an optional compiler that can be used to prevent the code of a plugin or addon from being visible. There is a decompiler, but it's better to not have to reply on that. The community will ultimately benefit if the source of all the addons is visible for people to look at and make changes to.<!--QuoteEnd--></div><!--QuoteEEnd-->

    In my opinion, this is a matter of trust. As a server administrator, do you trust the authors of plugins enough to give them time on your CPU (and in turn, CPU time on every client connecting to your server). Personally, I am much more inclined to trust someone distributing source rather than binaries, but I don't believe robbing people of the choice will have any positive effects. I don't believe there will be any security benefits, and no one can force authors to comply to standards even if all the code is available. I know for a fact that neither of us will be rewriting that naked Barbie fade mod, even though it is plain as day that it is really crappy work that crashes every 7 minutes. The only real option is to rely on server administrators to chose their plugins carefully. A task that I think they are quite capable of shouldering.

    Public sources have always been a double edged sword. As a developer, there will always be someone who holds you responsible for your code, even though it has been heavily modified by a third party. Having a community of busybodies all making different changes to your code is bound to land you a great deal of angry emails from server operators wondering why their combat server crashes every other map change. I understand full well why someone might chose to distribute binaries to attempt to keep a lid on shoddy modifications.

    On a further note, this is <b>completely</b> unrelated to performance issues. Lua code is not interpreted in it's textual format. It is compiled (all of it) into bytecode which is then interpreted. At least if you are based on the source from Lua.org.

    As to the most of FocusedWolf's comments - I would consider him a troll if he didn't use such an old account and post so persistently across the boards. I guess I should consider him an old and persistent troll.
  • Dalin SeivewrightDalin Seivewright 0x0000221E Join Date: 2007-10-20 Member: 62685Members, Constellation
    edited June 2009
    I think the option of having compiled Lua plugins should be there. Yes uncompiled plugins can be of use to the community, but the community also has the source code for the game that they can draw from, as well as anyone of the plugin developers who wish to release their plugins in an uncompiled format. There are likely certain parts of my code that I would prefer to have remain hidden (perhaps validation methods, calculations) and other parts of my code that I probably wouldn't bother with.

    The fact is, I would probably a learn a lot more from talking to a fellow developer in the community than simply looking at his source code. While you can learn a decent amount of something from source code, it is usually only limited to <b>how</b> you do something and not necessarily <b>why</b> you should do something that way. I don't think we want a million different versions of a plugin flying around either, as Private mentioned. That can not only kill a plugin, but also kill a developer. If someone is always stealing my code, I don't think I would be too happy and I might feel discouraged and start to lose faith in the community I'm a part of.

    While most of my source code will probably be released in an uncompiled format, I want the option of compiling reserved for when I deem it needed. This comes in handy when writing security for your plugin or even a plugin for securing a server entirely.

    Also, decompilers do exist however from what I've seen of them they're all still very experimental and in many cases don't work correctly.
  • PollyPolly Join Date: 2009-06-04 Member: 67686Members
    I'm not sure about this, compiled plugins could be useful but there's dangers too. With compiled plugins you can't look at the source (duh), so you can't know if someone has added anything harmful in the code, like people trying to give themselves admin access in Garry's Mod by adding their SteamID in.
    On the other side, they can be useful for private scripts or for keeping the contents of clientside scripts a secret.

    <!--quoteo(post=1712629:date=Jun 17 2009, 06:59 AM:name=FocusedWolf)--><div class='quotetop'>QUOTE (FocusedWolf @ Jun 17 2009, 06:59 AM) <a href="index.php?act=findpost&pid=1712629"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->2. a lua script will compile in the order of milliseconds but regardless.. if it's anything like garrysmod then it will take ages to join a server<!--QuoteEnd--></div><!--QuoteEEnd-->

    That wouldn't be because of the lua scripts, it's because of the way Source engine loads the addons by wildcarding and going through every folder. For faster loading put the contents of addons directly to the garrysmod folder.
  • devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
    <!--quoteo(post=1713226:date=Jun 21 2009, 03:38 AM:name=Dalin Seivewright)--><div class='quotetop'>QUOTE (Dalin Seivewright @ Jun 21 2009, 03:38 AM) <a href="index.php?act=findpost&pid=1713226"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I think the option of having compiled Lua plugins should be there. Yes uncompiled plugins can be of use to the community, but the community also has the source code for the game that they can draw from, as well as anyone of the plugin developers who wish to release their plugins in an uncompiled format. There are likely certain parts of my code that I would prefer to have remain hidden (perhaps validation methods, calculations) and other parts of my code that I probably wouldn't bother with.<!--QuoteEnd--></div><!--QuoteEEnd-->

    I don't really see a lot that can be done in a plugin that should remain secret to the people running it.

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->The fact is, I would probably a learn a lot more from talking to a fellow developer in the community than simply looking at his source code. While you can learn a decent amount of something from source code, it is usually only limited to <b>how</b> you do something and not necessarily <b>why</b> you should do something that way.<!--QuoteEnd--></div><!--QuoteEEnd-->
    True, but there are a lot of people who "write" plugins by copy and pasting from other plugins. This is a kind of important thing that we don't really want to limit.

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I don't think we want a million different versions of a plugin flying around either, as Private mentioned. That can not only kill a plugin, but also kill a developer. If someone is always stealing my code, I don't think I would be too happy and I might feel discouraged and start to lose faith in the community I'm a part of.<!--QuoteEnd--></div><!--QuoteEEnd-->
    This largely isn't an issue. It doesn't happen with amxmodx, nor sourcemod plugins. Both require that the source be released anyway.

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->While most of my source code will probably be released in an uncompiled format, I want the option of compiling reserved for when I deem it needed. This comes in handy when writing security for your plugin or even a plugin for securing a server entirely.<!--QuoteEnd--></div><!--QuoteEEnd-->
    Security through obscurity is *not* security. If people are able to bypass the security of a plugin just by looking at it's source, then it's not secure, and compiling it will only delay the inevitable.

    @locallyunscene: If you are hashing plugins to ensure they are vanilla, it doesn't matter if they are compiled or not. The hash will change if you change a text file or a binary file. The type of file doesn't matter to a hash, just the contents.
  • PrivatePrivate Join Date: 2007-06-10 Member: 61204Members, Constellation
    I still don't quite follow you dev - I'm not perfectly clear on what benefits the modding community would have from forced open sources.

    (I'll be speaking to you, but this being public and all, I've attempted to make some points that I consider beneath you. I know you to be fairly intelligent.)

    I've attempted to make up a few benefits myself, just in order to examine and discard them. Here goes.

    <b><u>Copy paste</u></b>
    You mentioned this yourself. There's plenty of code being written not from the bottom up, but by copying and modifying a piece of existing code. This makes it somewhat easy to figure out how to hook up a new gun, for instance, as you can simply find a popular plugin and copy the way they do it.

    First of all, I have a personal rule of thumb that states that every time I copy-paste a piece of code I'm doing something wrong. There is, of course, exceptions, but I find that it is usually an early sign of poor program structure. It is not something I would ever encourage in anyone, but that is mostly a matter of personal bias.
    Now, I have no doubt that most mods and plugins to NS2 will be distributed as sources rather than binaries, so there will be plenty of plugins to copy from. In addition, the authors of the available code will intend others to read it (or at least acknowledge that someone might read it). However, should some unfortunate soul be forced to make a number of ugly hacks to achieve weird functionality or meet a deadline, he does have the option of distributing binaries. A pretty effective way to say 'please don't read/use/copy my code'. By removing this option, you do give aspiring Lua programmers a broader base to copy and learn from, but you do so by including code that is almost certainly of poor quality.

    <b><u>Standard compliance</u></b>
    Forcing all mods to be open source makes it very easy to see whether the authors follows certain standards. For instance, it makes it very easy to see whether the mod hooks up new weapons the 'right' way, whether the mod modifies some core NS2 files, or some other discouraged behavior.

    Nekkid Barbie Fade plugin - Fades turn nekkid barbies with huge breasts that bounces as they blink around. Great for Friday night drunken shooting. However, as you are reading through the code to figure out why the server crashes every other map change, you realize pretty quickly that the original author ignored pretty much every rule, guideline and practice established in the modding community. You have three options.
    1) Remove the plugin from the server, forget it ever existed. - My personal favorite.
    2) Fix everything yourself. - Who doesn't feel like rewriting a mod about naked breasts?
    3) Email the original author and pester him about his crappy work.
    Now, the plugin doesn't conform to to the community's expectations, either because the author didn't know how to meet them, or because he didn't care. If he didn't care, pestering him through email won't get you anywhere. If breast-mod is his first mod and he didn't know how to comply to standards, pestering him with emails about poor work will only discourage him from ever trying to do another mod. Unless, of course, you take it upon yourself to help this poor guy out in fixing his mod. This will be strangely similar to rewriting the whole thing yourself, only with a stranger between you and the keyboard.
    In the end, having the source available doesn't help you. The difference between a crappy mod and a crappy mod with sources would be that you would waste time scrolling through the sources before binning the whole thing.

    <b><u>The inner workings of a plugin is immediately (...) obvious to server admins</u></b>
    The point about trust I made before relates to this, but I am not sure the opening of my previous post was sufficiently clear. Having all the code available to server admins makes it relatively easy to run through it, develop some idea of what it does, and more importantly, what it should not do. As a server admin, you can (somewhat) easily spot plugins that modify core NS2 files, for instance, and decide that you are not okay with that.

    I have warped this slightly. Server admins will never read their plugins, even though the sources are provided. But if the sources are available, there's a fair chance that someone from the modding community might take a look around in them and raise the alarm if something is not right. Especially if we make it a habit to help others with their bugs (It's not kill-stealing, it's teamwork!). If no sources are available, bad things would have an easier time getting all the way onto servers and clients.
    I find it convenient to consider unintentionally harmful code grossly mis-followed practices. Then we only have to consider intentionally malicious code. And this, in the end, comes down to trust. Even if the engine didn't accept binary plugins, there is no NS2 modding police intently reading every line of Lua out there that might be shoved up the tailpipe of an NS2 server. Clients have to trust the server admins to pick safe plugins, and admins have to trust providers to provide safe plugins.


    To me, it seems that binaries makes very little difference. As long as there is a strong NS2 modding community to hunt bugs, support newcomers and keep some good practices alive, I really don't think there will be any problems with distributing binary code once in a while. In the end, having a strong community maintaining a practice of never distributing binaries might be enough to make sure no binaries are ever distributed.

    Sorry about the word count.
  • devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
    Well, while I agree copy and paste isn't the best way to go if you are somewhat skilled in writing plugins, I've had a number of people mention that they started writing plugins by copy and pasting code from other plugins to form what they wanted. Making it easy for people to get started is important, I think. This may be part of the reason behind the popularity of something like PHP, which has snippets (however terrible they may be) for many many things.

    Granted, binary plugins do let you hide things from people if you feel the code isn't good enough quality. I'm not sure if that's enough reason to allow it. Many of the people writing terrible code aren't going to realize they are writing terrible code.

    About standard compliance, both amxmodx and sourcemod (I keep going back to these as they are the two I know the best) have plugin guidelines. For a plugin to be "approved", it has to follow some very basic rules (<a href="https://forums.alliedmods.net/showthread.php?t=57142" target="_blank">Sourcemod's</a>, and <a href="https://forums.alliedmods.net/showthread.php?t=21956" target="_blank">amxx's</a>). These don't guarentee the plugin works, in fact it never gets compiled/tested before it is approved. Rather, it ensures they are all somewhat consistent, and that there are no major issues. There are a few "plugin approvers" for each one, who go through all the plugins and ensure they follow the rules.

    I know as well as you do that if a mod isn't popular, no one is going to waste their time fixing it. Instead, take this example: A NS2:Combat mod is released, as a binary only plugin (the developer said "I spent X weeks writing this, I don't want anyone else to modify it"). It becomes very popular, until a fatal crash bug is discovered in the plugin, and the developer is unwilling to fix it. As it's binary only, there's three choices: 1) Give up on it, 2) Rewrite it completely from scratch, 3) Attempt to decompile it. None of these are really good choices when compared to having the source available for this.

    I'm well aware of the fact that server admins don't read plugins, we can barely get them to read documentation. Let's say you have a backdoor in a plugin that gives you rcon access to the server when you use some command (a real enough example, it's happened before when someone distributed binary only plugins). While the server admin may not catch it, at some point the code is going to be looked at, and someone is going to figure out that it's there, assuming the source is available. This may not happen right away, it may not happen until it's exploited, but it will definitely happen at some point.

    Even if there is an exploit that doesn't get caught right away, eventually someone is going to go looking for it. This is far easier to do if the source to the plugins is available.

    It's possible that I'm just being influenced by my thinking that everything is better open source :P However, look at it this way: We can easily go from not allowing compiled plugins to allowing compiled plugins at some point if we determine it is useful. It is very difficult to go the other way (You break plugins that may no longer have active developers)
  • puzlpuzl The Old Firm Join Date: 2003-02-26 Member: 14029Retired Developer, NS1 Playtester, Forum Moderators, Constellation
    Although I certainly agree with devicenull from a broad perspective, I can see one reason to allow compilation: Anyone planning to do a total conversion of NS2 might find it difficult to accept that their mod can be forked at any point in the future.
  • PrivatePrivate Join Date: 2007-06-10 Member: 61204Members, Constellation
    edited June 2009
    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I know as well as you do that if a mod isn't popular, no one is going to waste their time fixing it. Instead, take this example: A NS2:Combat mod is released, as a binary only plugin (the developer said "I spent X weeks writing this, I don't want anyone else to modify it"). It becomes very popular, until a fatal crash bug is discovered in the plugin, and the developer is unwilling to fix it. As it's binary only, there's three choices: 1) Give up on it, 2) Rewrite it completely from scratch, 3) Attempt to decompile it. None of these are really good choices when compared to having the source available for this.<!--QuoteEnd--></div><!--QuoteEEnd-->

    First of all, I'll assume the developer is unable to fix the bug. Say, his computer is in an advanced state of death. (ns_nancy anyone?). A developer being unwilling to fix (or allow others to fix) a fatal bug seems a little out there.
    I think that it is perfectly reasonable that he get's to chose how his code is distributed. He did the work, after all. Not wanting a slew of NS2:Combat_xmenu floating around with his name on it seems like plenty reason to distribute binaries. I'm not saying distributing binaries is a good idea, I don't think anyone should, I know I won't. But people should have the right to chose. Should sources be lost through unfortunate events, I expect the original developers to spearhead a recovery. Should the original developers be lost along with the sources I think a rewrite is in order.

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I'm well aware of the fact that server admins don't read plugins, we can barely get them to read documentation. Let's say you have a backdoor in a plugin that gives you rcon access to the server when you use some command (a real enough example, it's happened before when someone distributed binary only plugins). While the server admin may not catch it, at some point the code is going to be looked at, and someone is going to figure out that it's there, assuming the source is available. This may not happen right away, it may not happen until it's exploited, but it will definitely happen at some point.

    Even if there is an exploit that doesn't get caught right away, eventually someone is going to go looking for it. This is far easier to do if the source to the plugins is available.<!--QuoteEnd--></div><!--QuoteEEnd-->

    Forcing the source to be available will certainly increase the likelihood of someone stumbling on any backdoors.

    I don't think either of us will endorse a modification that we are not reasonably sure doesn't contain any security risks, nor do I think any major NS2 modding community will. To me, this means that I will not endorse binary distributions. I expect that a server admin would have to venture well off the trodden path of modifications to find a deliberate backdoor, even if binaries were still accepted by the engine. I don't think disabling binaries will benefit anyone determined to run odd stuff on their servers. They are bound to suffer a healthy dose of bad luck either way.

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->We can easily go from not allowing compiled plugins to allowing compiled plugins at some point if we determine it is useful. It is very difficult to go the other way (You break plugins that may no longer have active developers)<!--QuoteEnd--></div><!--QuoteEEnd-->

    This is flawed. After having forced the author of NS2:Combat to make his code available for the first 2 major updates, suffer numerous unwanted modifications and code floating freely on the internet we can't just change our minds and tell him "Sorry, we were wrong! Hide you code if you want." That's simply not good enough.

    Edit: Re-phrased some things for clarity. I was tired when I wrote this.
  • JirikiJiriki retired ns1 player Join Date: 2003-01-04 Member: 11780Members, NS1 Playtester, Squad Five Silver
    edited June 2009
    I strongly <b>disagree</b> with this mandatory open-source. We have ENSL plugin which is closed source for a reason. This is to prevent abuse, especially when it interacts with our database to fetch player information. I know its reverse-engineerable, but its a piece of cake with open source. There's a challenge-verification algorithm so its not possible to use proxying.
  • JazzXJazzX cl_labelmaps ∞ Join Date: 2002-11-19 Member: 9285Members, Retired Developer, NS1 Playtester
    <!--QuoteBegin-puzl+--><div class='quotetop'>QUOTE (puzl)</div><div class='quotemain'><!--QuoteEBegin-->Although I certainly agree with devicenull from a broad perspective, I can see one reason to allow compilation: Anyone planning to do a total conversion of NS2 might find it difficult to accept that their mod can be forked at any point in the future.<!--QuoteEnd--></div><!--QuoteEEnd--> <!--QuoteBegin-Jiriki+--><div class='quotetop'>QUOTE (Jiriki)</div><div class='quotemain'><!--QuoteEBegin-->I strongly <b>disagree</b> with this mandatory open-source. We have ENSL plugin which is closed source for a reason. This is to prevent abuse, especially when it interacts with our database to fetch player information. I know its reverse-engineerable, but its a piece of cake with open source. I have a challenge-verification algorithm so its not possible to use proxying.<!--QuoteEnd--></div><!--QuoteEEnd-->
    I don't know enough about Lua or its compiler to really know what I'm talking about here, but from what I do know, and from the comments in this thread I'm guessing that compiled means the byte code is pre-generated instead of being created at run time from source. If that's true then unless Lua is doing something very different from other interpreted languages I've worked with, there is minimal obfuscation taking place through compilation. And to me it almost seems like the situation is worse in assuming this is more secure than having source files lying around.

    It will stop some people from making their own versions or from learning how the plugin works in order to "beat it". But if anyone is seriously interested in doing these thing it doesn't seem like compiling to byte code ahead of time is gonna get you that much additional protection. And if this becomes a selling point, "you can also use compiled Lua code so your Intellectual Property is safe", the claim is almost disingenuous, even if its well intended.
  • JirikiJiriki retired ns1 player Join Date: 2003-01-04 Member: 11780Members, NS1 Playtester, Squad Five Silver
    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->It will stop some people from making their own versions or from learning how the plugin works in order to "beat it". But if anyone is seriously interested in doing these thing it doesn't seem like compiling to byte code ahead of time is gonna get you that much additional protection. And if this becomes a selling point, "you can also use compiled Lua code so your Intellectual Property is safe", the claim is almost disingenuous, even if its well intended.<!--QuoteEnd--></div><!--QuoteEEnd-->
    The ability to run compiled plugins will have little drawback. I don't see why this "community control" is necessary to promote open source plugins. Licensing is a better option.

    You have to understand that the gained security of something is a function of two variables, <b>intent</b> and security. Our website is very weak and vulnerable but it hasn't been hacked in many years because nobody cares. I don't have bank-class security at my home for a reason.

    CAL plugin wasn't open source, and it was made by you. Thanks for the code base again though.
  • devicenulldevicenull Join Date: 2003-04-30 Member: 15967Members, NS2 Playtester, Squad Five Blue
    Jiriki, having that be closed source really doesn't get you much. What's the worst that happens? Someone is able to query your database? Like JazzX said, compiled LUA code is not strong protection at all. It's easily decompiled, though you lose things like comments and possibly nice formatting. Still it's enough to figure out how your plugin works.

    JazzX, It's not very strong protection at all. The best it really gets you is the ability to say "don't decompile my ######". Still, new developers are not going to know there is a way to decompile stuff.
  • JazzXJazzX cl_labelmaps ∞ Join Date: 2002-11-19 Member: 9285Members, Retired Developer, NS1 Playtester
    edited June 2009
    <!--QuoteBegin-Jikri+--><div class='quotetop'>QUOTE (Jikri)</div><div class='quotemain'><!--QuoteEBegin-->You have to understand that the gained security of something is a function of two variables, <b>intent</b> and security. Our website is very weak and vulnerable but it hasn't been hacked in many years because nobody cares. I don't have bank-class security at my home for a reason.

    CAL plugin wasn't open source, and it was made by you. Thanks for the code base again though.<!--QuoteEnd--></div><!--QuoteEEnd--> I think that you and puzl both gave perfect examples where having the option of Closed-Source/compiled plugins would be beneficial. But in general I agree with devicenull too (are you the devnull that I remember from the #ns IRC channel, from way back?). I think most of the time having plugin source code available is how people learn, and how things get improved.

    My biggest problem with the idea of Compiled plugins is that you are not much more secure than you were before, and its even less safe to pretend like that isn't the case. Not sure if this saying is a universal saying, but I liken it to "locks keep honest people honest". Nearly anyone who would be stopped from using your code as base because its compiled would also be stopped if you ask them not to do it (in a license or readme or whatever). There will be some people without the technical sophistication to get past the compilation, but those people would likely be the ones who'd get the most from seeing how "real" code works anyway.

    I'd argue that a false sense of security is a major drawback in allowing the compiled plugins.
  • JirikiJiriki retired ns1 player Join Date: 2003-01-04 Member: 11780Members, NS1 Playtester, Squad Five Silver
    edited June 2009
    <!--quoteo(post=1713689:date=Jun 24 2009, 07:23 AM:name=devicenull)--><div class='quotetop'>QUOTE (devicenull @ Jun 24 2009, 07:23 AM) <a href="index.php?act=findpost&pid=1713689"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Jiriki, having that be closed source really doesn't get you much. What's the worst that happens? Someone is able to query your database? Like JazzX said, compiled LUA code is not strong protection at all. It's easily decompiled, though you lose things like comments and possibly nice formatting. Still it's enough to figure out how your plugin works.

    JazzX, It's not very strong protection at all. The best it really gets you is the ability to say "don't decompile my ######". Still, new developers are not going to know there is a way to decompile stuff.<!--QuoteEnd--></div><!--QuoteEEnd-->
    Well if the compiled LUA is easily decompilable then its bad news because we have to provide (and fund) the servers ourself if we want to enforce ENSL plugin. Now teams' own servers have been capable of using the plugin. The plugin has brought has lots of <a href="http://ensl.zanith.nl/index.php?view=ensl_plugin" target="_blank">features</a>, especially player checking (this is a rather tidious manual work for the refs, which we have little anyway and this is something for which I want avoid being frauded) and forfeit clock (to avoid 30 minute wait times). This is all in vain if the plugin is easily editable. Many of these issues can of course addressed by manual means, but hard-to-decompile plugins has helped us, especially in terms of preventing drama (when the plugin can prevent problems).

    <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I'd argue that a false sense of security is a major drawback in allowing the compiled plugins.<!--QuoteEnd--></div><!--QuoteEEnd-->
    Its only false sense of security if one thinks its unbreakable. Nothing is absolute, the point of these things is to limit damage, to make it harder to break. The fact its not absolute, does not say it does not have a gain.

    Its not impossible to make cheats either, but if the game developers use some the bytecode encoders, there will be probably a lot less of them. Just like DRM. Its not impossible to break, but its still annoying, which its objective, not that I like drm or anything.

    But yeah you are right, if the LUA is very easily decompilable, then its totally useless for security purposes.
  • puzlpuzl The Old Firm Join Date: 2003-02-26 Member: 14029Retired Developer, NS1 Playtester, Forum Moderators, Constellation
    Jiriki, you do know that you *have* to release the source to the ENSL plugin, right? Amxmodx requires plugin source to be licensed under the GPL, unless you get an exception.

    But putting that aside, if you depend on it being difficult to decompile a plugin for its operation then you have some major issues in your assumptions.

    Even compiled binaries are easy to decompile, maybe not into easily readable C code, but the very existence of metamod demonstrates that even fully compiled programs
    are not free from inspection by users. Your example of DRM does nothing to support your premise, in fact it undermines it deeply.
  • RobRob Unknown Enemy Join Date: 2002-01-24 Member: 25Members, NS1 Playtester
    <!--quoteo(post=1713759:date=Jun 24 2009, 12:26 PM:name=puzl)--><div class='quotetop'>QUOTE (puzl @ Jun 24 2009, 12:26 PM) <a href="index.php?act=findpost&pid=1713759"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Jiriki, you do know that you *have* to release the source to the ENSL plugin, right? Amxmodx requires plugin source to be licensed under the GPL, unless you get an exception.

    But putting that aside, if you depend on it being difficult to decompile a plugin for its operation then you have some major issues in your assumptions.

    Even compiled binaries are easy to decompile, maybe not into easily readable C code, but the very existence of metamod demonstrates that even fully compiled programs
    are not free from inspection by users. Your example of DRM does nothing to support your premise, in fact it undermines it deeply.<!--QuoteEnd--></div><!--QuoteEEnd-->

    Meh, if you're talking about someone who's willing to damage their eyesight looking through assembler, all bets are off anyway.
  • KillerBroetchenKillerBroetchen Join Date: 2004-08-10 Member: 30508Members
    edited June 2009
    <!--quoteo(post=1713424:date=Jun 22 2009, 04:54 PM:name=devicenull)--><div class='quotetop'>QUOTE (devicenull @ Jun 22 2009, 04:54 PM) <a href="index.php?act=findpost&pid=1713424"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->...but Security through obscurity is *not* security.<!--QuoteEnd--></div><!--QuoteEEnd-->

    Wrong. In general, obscurity is part of a security chain. For example: Making your web server not exposing his name and version will not make it completely secure against any exploits for this version, yes. But it adds an obstacle for eve, which has to fire up more effort to look if your server is vulnerable. Like a door on a simple house, you can easily crack it, but this is already too much effort for most people.

    Well, I totally understand your concern from the point of security. (if you could compromise server administration and such, for example) But your posts sounds somehow like "Weehh, I want to look, copy & paste from other mods". I'm going to script for NS2 too, as soon as it arrives, but since my scripts will be server-side only (to be able to offer some unique game server :)), I don't really care. Depending on what kind of damage/fraud could be done from within a script, of course.
Sign In or Register to comment.