Created a new topic for this mod in anticipation of it's completion, which shouldn't be more than a week away. This also gives me time to write up a proper documentation.
The way I've written our DB library we use associative arrays for row column lookup. We do for instance row["player_name"], etc. It feels like you're taking an OOP approach to a DB structure. Message me if you want to share ideas.
Ah yes, I suppose it's my C programming-style that served as the bases for this approach. The reason I shoved everything in functions is because it allows for proper error-handling, where I presume all Lua does in case of an incorrect key is return nil, which I find a bit insufficient. To further clarify here, you should realize that the mod, by design, does not keep any kind of buffer. Thus each call is guaranteed to return the most up-to-date information. This brings with it that it will query along the entiry hierarchy each time, which in turn means it can fault at any point along that hierachy (server->database->table->field->entry) for whichever reason (invalid database, invalid table, invalid field et cetera). This is why I considered it wise to provide very extensive error-handling at each call.
What you are demonstrating can be reasonably approximated by simply chaining the calls (ea. GetDB(dbname):GetTable(tablename):GetField(fieldname):GetEntry(stringvalue):Getr ow). But I'm glad you mentioned it as it certainly doesn't hurt to add it as an option for the brave reckless scripters among us, and it's but a small effort so I'll have it in before I put up the initial version.
Well how I've done it is instead of chaining you initialize them and hold them together. I do web programming and a lot of the stuff we do is highly transactional and heavy load, so we try and save as few method calls as possible. I can understand the hierarchy but under heavy load I don't know if it will stand up. Thus for ours I wrote the database to be a singleton. Maybe you have something similar underneath. Also are you hand rolling the query?
I see where you're coming from. A state-less instance that has to be requery'd every time is indeed more costly than keeping track of what has been performed already. I suppose it's a design-decision, security over performance in this case. Whether it proves to be too expensive I'm not sure, it depends on what is being done with it on a NS2-server. Queries to retrieve admin-lists or reserve-slot lists each map-change aren't very costly, and can easely be done this way. Keeping track of real-time and very detailed statistics (down to the number of bullets in your pistol) might be a different matter. I suppose we will also have to see how much of a performance burden the Lua-VM proves to be, in terms of number of calls made.
If performance proves to be an issue, I can always add functions to the upper-class ('NS2DtBase_MySQL') that forego the use of instances and execute a direct command. And yes, everything is hand-rolled.
devicenullJoin Date: 2003-04-30Member: 15967Members, NS2 Playtester, Squad Five Blue
Are the queries threaded (I'd guess not)?
Blocking queries have been a rather large source of lag in things like amxmodx/sourcemod. Adding some type of threaded or nonblocking queries would make the larger mods somewhat less laggy.
McGlaspiewww.team156.comJoin Date: 2010-07-26Member: 73044Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Onos, WC 2013 - Gold, Subnautica Playtester
Feel free to correct me if I'm wrong, but I don't believe we have any kind of threading model in the LUA VM. I suspect the closest thing you could get to it is (somehow) create a new VM just for the database. Problems may arise with this approach as (I could be wrong) the Server would potentially run at a different tick than the DB. You'd end up with stale queries or data collisions.
It would probably be a good idea to always add some kind of transaction anti-locking. Granted you could end up with dirty data, but the database won't have to lock a table/row/etc when running a query. This is only useful when running a SELECT statements. Transaction locking could be optional, but (non-locking) read-only transactions are faster.
More info here: <a href="http://dev.mysql.com/doc/refman/5.0/en/set-transaction.html" target="_blank">http://dev.mysql.com/doc/refman/5.0/en/set-transaction.html</a>
@OP: Great work on this mod! Seeing projects like this make me wonder if a group of server's could be linked via central database, and have this central data source setup/tweak/etc the global game rules (queries could retrieve data, and rules parse that data into static globals). Either way, good stuff and nice job.
Blocking queries have been a rather large source of lag in things like amxmodx/sourcemod. Adding some type of threaded or nonblocking queries would make the larger mods somewhat less laggy.<!--QuoteEnd--></div><!--QuoteEEnd--> I have investigated multi-threading for Lua before (for one of my other works), and it does <i>somewhat</i> support it, but it is certainly lacking. From what I gathered <a href="http://www.lua.org/pil/9.4.html" target="_blank">here</a>, it does support non-blocking script-execution, but has no support for thread-communication or any kind of semaphore-support, which means even if you shove script into a thread it will still end up blocking other operations along the way (even for read-only operations as I understand it).
<!--quoteo(post=1814294:date=Dec 8 2010, 07:51 AM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Dec 8 2010, 07:51 AM) <a href="index.php?act=findpost&pid=1814294"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Feel free to correct me if I'm wrong, but I don't believe we have any kind of threading model in the LUA VM. I suspect the closest thing you could get to it is (somehow) create a new VM just for the database. Problems may arise with this approach as (I could be wrong) the Server would potentially run at a different tick than the DB. You'd end up with stale queries or data collisions.<!--QuoteEnd--></div><!--QuoteEEnd--> I believe the Lua-VM's are essentially what the Server- and Client-DLL's were for HL1. Although in NS2's case they might share script, they are in no way connected to each other, and cannot communicate with each other (other than the obvious network-connection between the two), ea. variable\class instances aren't shared.
There is no reason, however, to have the multi-threading be performed on the Lua-level. As each call is pretty much sent directly to the DLL, the DLL can provide for a threading-model with a little Lua-API to go with it for feedback to the script. You could consider this on the to-do list.
<!--quoteo(post=1814294:date=Dec 8 2010, 07:51 AM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Dec 8 2010, 07:51 AM) <a href="index.php?act=findpost&pid=1814294"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->It would probably be a good idea to always add some kind of transaction anti-locking. Granted you could end up with dirty data, but the database won't have to lock a table/row/etc when running a query. This is only useful when running a SELECT statements. Transaction locking could be optional, but (non-locking) read-only transactions are faster.<!--QuoteEnd--></div><!--QuoteEEnd--> <!--quoteo(post=1814354:date=Dec 8 2010, 04:05 PM:name=Martin)--><div class='quotetop'>QUOTE (Martin @ Dec 8 2010, 04:05 PM) <a href="index.php?act=findpost&pid=1814354"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Remember Oracle locks the row based on the index. MySQL locks the whole table. ;) ... just some words of wisdom<!--QuoteEnd--></div><!--QuoteEEnd--> Chances are write-operations will introduce a number of new headaches, hence I went for read-only for the moment, just until I have a good implementation nailed.
Comments
What you are demonstrating can be reasonably approximated by simply chaining the calls (ea. GetDB(dbname):GetTable(tablename):GetField(fieldname):GetEntry(stringvalue):Getr
ow). But I'm glad you mentioned it as it certainly doesn't hurt to add it as an option for the brave reckless scripters among us, and it's but a small effort so I'll have it in before I put up the initial version.
If performance proves to be an issue, I can always add functions to the upper-class ('NS2DtBase_MySQL') that forego the use of instances and execute a direct command. And yes, everything is hand-rolled.
Blocking queries have been a rather large source of lag in things like amxmodx/sourcemod. Adding some type of threaded or nonblocking queries would make the larger mods somewhat less laggy.
It would probably be a good idea to always add some kind of transaction anti-locking. Granted you could end up with dirty data, but the database won't have to lock a table/row/etc when running a query. This is only useful when running a SELECT statements. Transaction locking could be optional, but (non-locking) read-only transactions are faster.
More info here: <a href="http://dev.mysql.com/doc/refman/5.0/en/set-transaction.html" target="_blank">http://dev.mysql.com/doc/refman/5.0/en/set-transaction.html</a>
@OP: Great work on this mod! Seeing projects like this make me wonder if a group of server's could be linked via central database, and have this central data source setup/tweak/etc the global game rules (queries could retrieve data, and rules parse that data into static globals). Either way, good stuff and nice job.
Blocking queries have been a rather large source of lag in things like amxmodx/sourcemod. Adding some type of threaded or nonblocking queries would make the larger mods somewhat less laggy.<!--QuoteEnd--></div><!--QuoteEEnd-->
I have investigated multi-threading for Lua before (for one of my other works), and it does <i>somewhat</i> support it, but it is certainly lacking. From what I gathered <a href="http://www.lua.org/pil/9.4.html" target="_blank">here</a>, it does support non-blocking script-execution, but has no support for thread-communication or any kind of semaphore-support, which means even if you shove script into a thread it will still end up blocking other operations along the way (even for read-only operations as I understand it).
<!--quoteo(post=1814294:date=Dec 8 2010, 07:51 AM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Dec 8 2010, 07:51 AM) <a href="index.php?act=findpost&pid=1814294"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Feel free to correct me if I'm wrong, but I don't believe we have any kind of threading model in the LUA VM. I suspect the closest thing you could get to it is (somehow) create a new VM just for the database. Problems may arise with this approach as (I could be wrong) the Server would potentially run at a different tick than the DB. You'd end up with stale queries or data collisions.<!--QuoteEnd--></div><!--QuoteEEnd-->
I believe the Lua-VM's are essentially what the Server- and Client-DLL's were for HL1. Although in NS2's case they might share script, they are in no way connected to each other, and cannot communicate with each other (other than the obvious network-connection between the two), ea. variable\class instances aren't shared.
There is no reason, however, to have the multi-threading be performed on the Lua-level. As each call is pretty much sent directly to the DLL, the DLL can provide for a threading-model with a little Lua-API to go with it for feedback to the script. You could consider this on the to-do list.
<!--quoteo(post=1814294:date=Dec 8 2010, 07:51 AM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Dec 8 2010, 07:51 AM) <a href="index.php?act=findpost&pid=1814294"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->It would probably be a good idea to always add some kind of transaction anti-locking. Granted you could end up with dirty data, but the database won't have to lock a table/row/etc when running a query. This is only useful when running a SELECT statements. Transaction locking could be optional, but (non-locking) read-only transactions are faster.<!--QuoteEnd--></div><!--QuoteEEnd-->
<!--quoteo(post=1814354:date=Dec 8 2010, 04:05 PM:name=Martin)--><div class='quotetop'>QUOTE (Martin @ Dec 8 2010, 04:05 PM) <a href="index.php?act=findpost&pid=1814354"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Remember Oracle locks the row based on the index. MySQL locks the whole table. ;) ... just some words of wisdom<!--QuoteEnd--></div><!--QuoteEEnd-->
Chances are write-operations will introduce a number of new headaches, hence I went for read-only for the moment, just until I have a good implementation nailed.