Clustering

SpiffyJrSpiffyJr Join Date: 2002-12-06 Member: 10518Members
<div class="IPBDescription">is it worth it?</div> Basically I got couple cpus left over and I was wondering if hlds_l would benefit from clustering. It is a single process so would it be able to benefit from both CPUs or not? Any information would be great.
«1

Comments

  • verboseverbose Join Date: 2002-11-25 Member: 9968Members, Constellation
    SMP and clustering are two different things. HLDS is not threaded, so additional CPUs in a SMP setup will not be utilized.
  • BytorBytor Join Date: 2002-11-19 Member: 9323Members
    Verbose, any idea if there is a way with SMP linux kernels to assign a given process to one CPU or another? Example, could I run 2 NS server daemons and have one on each CPU in an SMP system?
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--Bytor+Feb 9 2003, 11:46 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (Bytor @ Feb 9 2003, 11:46 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Verbose, any idea if there is a way with SMP linux kernels to assign a given process to one CPU or another? Example, could I run 2 NS server daemons and have one on each CPU in an SMP system? <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    <a href='http://www.tech9.net/rml/linux/' target='_blank'>http://www.tech9.net/rml/linux/</a>

    u have to patch the kernel
  • BytorBytor Join Date: 2002-11-19 Member: 9323Members
    Looks like he's just providing procfs and syscall interfaces to assign CPU affinity. Isn't there a way with standard SMP kernels to assign a process to a specific processor?
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--Bytor+Feb 10 2003, 01:25 AM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (Bytor @ Feb 10 2003, 01:25 AM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Looks like he's just providing procfs and syscall interfaces to assign CPU affinity. Isn't there a way with standard SMP kernels to assign a process to a specific processor? <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    that is the answer got when i asked the same question. I don't have a dualy so i dunno.

    www.tldp.org <!--emo&;)--><img src='http://www.unknownworlds.com/forums/html/emoticons/wink.gif' border='0' style='vertical-align:middle' alt='wink.gif'><!--endemo-->
  • verboseverbose Join Date: 2002-11-25 Member: 9968Members, Constellation
    <!--QuoteBegin--Bytor+Feb 10 2003, 12:25 AM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (Bytor @ Feb 10 2003, 12:25 AM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Looks like he's just providing procfs and syscall interfaces to assign CPU affinity. Isn't there a way with standard SMP kernels to assign a process to a specific processor? <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    Not that I'm aware of, but my experience in Linux SMP is limited. AFAIK, FreeBSD 5.0's SMPng does not have this ability either, although measures have been taken to reduce the overhead introduced by juggling a process between CPUs.
  • PetitMortePetitMorte Join Date: 2002-11-06 Member: 7232Members
    Here's my $0.02

    I'm running a dual processor machine with 2 celeron 533's ($20 each!) and the game works better when I use 2 processors as opposed to one. Bear in mind that they are Celeron 533's tho.

    Anyhoo, I'm using GameHost to run the server, and it lets me choose what processors I want to run it on.

    What I've been doing is making the game run on processor #1. This frees up cycles from processor #0 where my OS and other stuff runs. #0 runs at about 20%, and #1 runs at about 60%-80% depending on the map, how many turrets the marines place, etc.

    In my case, it runs fine, and my only lag bottleneck is my WAN connection.

    <!--emo&::skulk::--><img src='http://www.unknownworlds.com/forums/html/emoticons/skulk.gif' border='0' style='vertical-align:middle' alt='skulk.gif'><!--endemo--> grrrr! growl! heheheheh
  • mathyoumathyou Join Date: 2002-11-28 Member: 10131Members
    Well of course the game will run better if it is the only thing using cycles on a CPU. If you can specifiy which processes run on which CPU, you can eliminate most of the SMP overhead. The issue is that linux doesn't seem to have an easy way to do that.
  • masterswordmanmasterswordman Join Date: 2002-12-21 Member: 11303Members
    DOES Microsoft Denial Of Service do that easily?
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--masterswordman+Feb 10 2003, 05:33 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (masterswordman @ Feb 10 2003, 05:33 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> DOES Microsoft Denial Of Service do that easily? <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    your post means nothing and is stupid!!!

    plz go back the the lame "off-topic" forums.

    DoSing has nothing to do with SMP




    btw, Linux can is u use the kernel patch and install a new kernel. "omg, i dont' wana screw up my system"

    hahaha, only newbies do that. if i get a few request i will get you a newbie-proof way of getting a new kernel into your system. As being once a newbie, i learned from my mistakes.
  • masterswordmanmasterswordman Join Date: 2002-12-21 Member: 11303Members
    Who the hell are you, annother person that thinks just because he's admin could "minorly" <img src='http://www.snowtroopers.ca/forumicons/flamer.gif' border='0' alt='user posted image'> people. First of people with even a little sense of humor would <img src='http://www.ezboard.com/images/emoticons/laugh.gif' border='0' alt='user posted image'> at the MS-DOS joke. Secondly it also was a ligidimate ?. I think MS can handle putting system tasks on one processor, and the HLDS on the other, I was just asking.
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--masterswordman+Feb 10 2003, 08:50 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (masterswordman @ Feb 10 2003, 08:50 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Who the hell are you, annother person that thinks just because he's admin could "minorly" <img src='http://www.snowtroopers.ca/forumicons/flamer.gif' border='0' alt='user posted image'> people. <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    your right, i should get a warning for flaming u. But i'm not the only one tired of your irrate and off topic posts.

    <!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->First of people with even a little sense of humor would <img src='http://www.ezboard.com/images/emoticons/laugh.gif' border='0' alt='user posted image'> at the MS-DOS joke. .<!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->
    I can't see the joke. This isn't a humorus topic....it was a serious discussion before u got here.

    <!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->Secondly it also was a ligidimate ?<!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->

    Are u saying your question was ligit? Because it wasn't. It is completely off topic. DoS is network flooding tactic, mainly a security issue and not related to SMP in anyway.

    <!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->I think MS can handle putting system tasks on one processor, and the HLDS on the other, I was just asking.<!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->

    That was already discussed previously you, (like 1000 other people) should read the entire thread before posting. Your not the only one who does this....but thats only my pet peave.
  • SpiffyJrSpiffyJr Join Date: 2002-12-06 Member: 10518Members
    and my simple question has turned into a flame war... why even bother?
  • masterswordmanmasterswordman Join Date: 2002-12-21 Member: 11303Members
    Microsoft Denial Of Service MEANS = MS-DOS, refering to the line of Microsoft Operating Systems.
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--SpiffyJr+Feb 10 2003, 09:15 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (SpiffyJr @ Feb 10 2003, 09:15 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> and my simple question has turned into a flame war... why even bother? <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    well, i dind't want it to.


    anyway. the only way to use SMP affinity in linux is to patch the kernel, if u want help doing that pm me...otherwise this thread should be locked.
  • disqdisq Join Date: 2002-11-30 Member: 10312Members, NS1 Playtester
    BACK ON TOPIC guys :)

    patch the kernel, then download and install schedutils.

    to set affinity, use the "taskset" program that comes with schedutils.

    examples:
    /usr/bin/taskset -p 01 9381
    /usr/bin/taskset -p 02 132
    /usr/bin/taskset -p 03 3049

    first line tells kernel to run process #9381 (this is the pid you get in <i>ps</i>) on the first cpu. second line tells kernel to run process #132 on the second cpu.
    third line, tells the kernel to run process #3049 on both cpus (3=1+2, bitwise)

    ie. i have a quad box and i want to run process 85 on the third cpu. <i>/usr/bin/taskset -p 04 85</i> would do it. or, to run it on cpus 1 and 4, i'd use <i>-p 09</i>.
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    edited February 2003
    so proc assignment is in bits

    cpu0 = 1
    cpu1 = 2
    cpu3 = 4
    cpu4 = 8

    ? is that what u ment?

    <b>edit:

    [21:33] <CrackerJackMack> when u mean bitwise you mean cpu0=1, cpu1 = 2, cpu2=4, cpu3=8?
    [21:33] <disq> yep

    </b>
  • scottlscottl Join Date: 2002-12-20 Member: 11232Members
    Just thaught i would throw this out.

    <a href='http://openmosix.sourceforge.net/' target='_blank'>http://openmosix.sourceforge.net/</a>

    One day i will get around to trying out a clustering software but if I do that one got my attention. I ran across that a while back and just found it in my favorites. I thaught it was pretty good, and OpenSource!
  • gamingfort_comgamingfort_com Join Date: 2002-11-19 Member: 9288Members
    HLDS will benifiet from SMP and Clustering, but only in some instances.

    In a SMP, you have more room for "overhead" like system processes and running multiple instances of HLDS. In linux, we usually don't patch our kernels or set CPU affinity. The linux kernel does a very good job of balancing the loads, and when you start getting 6 or so instances of HLDS running its good to let it balance out itself. Some servers will be full, and others will be idle, so setting affinity will probably leave you with one hammered CPU and one with unused resources.

    Now clustering gets very very trickey. We currently have a prototype game server cluster project, but its proving to be an expensive endevor. For cost/yields single and dual machines are showing to be the most effective. Now when you start looking at dozens to hundreds of machines, the administration of a cluster starts to make sence, but for your average user to your small and medium size hosting company its not a cost effective solution. I won't get into the specific problems with clustering here, but unless your using very expencive server grade components you will have a nest of issues.
  • SpiffyJrSpiffyJr Join Date: 2002-12-06 Member: 10518Members
    What's does SMP stand for exactly? I'm assuing dual processors or what not on the same mobo. I'm also assuming I can't use my other pc for it. If I'm assuming wrong, let me know cause I want to make use of this other server.
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--SpiffyJr+Feb 12 2003, 11:41 AM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (SpiffyJr @ Feb 12 2003, 11:41 AM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> What's does SMP stand for exactly? I'm assuing dual processors or what not on the same mobo. I'm also assuming I can't use my other pc for it. If I'm assuming wrong, let me know cause I want to make use of this other server. <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    SMP
    Last modified: Monday, May 18, 1998

    (1) Short for Symmetric Multiprocessing, a computer architecture that provides fast performance by making multiple CPUs available to complete individual processes simultaneously (multiprocessing). Unlike asymmetrical processing, any idle processor can be assigned any task, and additional CPUs can be added to improve performance and handle increased loads. A variety of specialized operating systems and hardware arrangements are available to support SMP. Specific applications can benefit from SMP if the code allows multithreading.

    SMP uses a single operating system and shares common memory and disk input/output resources. Both UNIX and Windows NT support SMP.

    (2) Short for Simple Management Protocol, another name for SNMP2. SNMP2 is an enhanced version of the Simple Network Management Protocol (SNMP) with features required to support larger networks operating at high data transmission rates. SNMP2 also supports multiple network management workstations organized in a hierarchical fashion.




    <a href='http://www.webopedia.com/TERM/S/SMP.html' target='_blank'>http://www.webopedia.com/TERM/S/SMP.html</a>
  • verboseverbose Join Date: 2002-11-25 Member: 9968Members, Constellation
    <!--QuoteBegin--gamingfort.com+Feb 12 2003, 03:03 AM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (gamingfort.com @ Feb 12 2003, 03:03 AM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Now clustering gets very very trickey. We currently have a prototype game server cluster project, but its proving to be an expensive endevor. For cost/yields single and dual machines are showing to be the most effective. Now when you start looking at dozens to hundreds of machines, the administration of a cluster starts to make sence, but for your average user to your small and medium size hosting company its not a cost effective solution. I won't get into the specific problems with clustering here, but unless your using very expencive server grade components you will have a nest of issues. <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    Please explain in further detail. Specifiically, I would like to know how you're distributing the computations one instance of HLDS requires over multiple machines. Clustering is defined, after all, as parallel processing on multiple machines, most often in a single-master-multiple-slaves arrangement. Such a setup requires custom programming, the difiiculty of which is decreasing due to the increasing availability of mature MPI libraries, and the release of complete development environments tailored to clustering. Or perhaps you're defining clustering differently?
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    I've done alot of reading on MS clusters (EEWWW) here is how it typically works

    Each node has a bootup disk (raid1), external scsi card, attached to a KVM, dual nics (or more)

    The external scsi is for all nodes to attacth to a "shared-scsi" quorum disk. How that works i still haven't gotten a straight answer.

    The nics are 1 for the interenal communication (seperate, non-public, node-only access) and the other nics are for public access +1 IP for the cluster itself.


    When u boot the frist node, it takes an election...then makes itself the master....then the other nodes become slaves.....if the master goes down...the slaves see this and start a new election (not sure how the election works..) but u get a new master and the cluster is back up.




    I'm not sure if this is how Linux, disk-less image stations work as well. but thats the best explanation i got right now.
  • masterswordmanmasterswordman Join Date: 2002-12-21 Member: 11303Members
    Clustering is mostly worthless. I mean what really can you do with it, except for storage, and mass processing power?
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--masterswordman+Feb 14 2003, 10:45 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (masterswordman @ Feb 14 2003, 10:45 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> mass processing power? <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    u answered your own question.
  • BytorBytor Join Date: 2002-11-19 Member: 9323Members
    edited February 2003
    <!--QuoteBegin--masterswordman+Feb 14 2003, 10:45 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (masterswordman @ Feb 14 2003, 10:45 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Clustering is mostly worthless. I mean what really can you do with it, except for storage, and mass processing power? <!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->
    LOL ...

    Cars are useless. I mean, what are they really good for, besides transportation?

    Best use of clustering I've seen is computer graphics rendering farms. Every feature film you see these days that contains any computer-generated material was rendered on a cluster of some sort.
  • Phant0mxPhant0mx Join Date: 2002-12-05 Member: 10496Members
    I have a few ?'s:

    -Which Microsoft OS's use SMP Architecture?

    -Will Dual Proc's provide enough performance gain to be worth the cost? (I.E. Multiple NS servers in a box, rather than just 1)

    Thanks,
    jonathan
  • cracker_jackmaccracker_jackmac Join Date: 2002-11-04 Member: 6891Members, Constellation, Reinforced - Shadow
    <!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->-Which Microsoft OS's use SMP Architecture?<!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->

    Anything based on the NT kernel (NT4, 2000[pro,server,adv], XP, .NET)

    <!--QuoteBegin--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> </td></tr><tr><td id='QUOTE'><!--QuoteEBegin-->-Will Dual Proc's provide enough performance gain to be worth the cost? (I.E. Multiple NS servers in a box, rather than just 1)
    <!--QuoteEnd--></td></tr></table><span class='postcolor'><!--QuoteEEnd-->

    Just bind each hlds to a single proc.
  • verboseverbose Join Date: 2002-11-25 Member: 9968Members, Constellation
    <!--QuoteBegin--Phant0m51+Feb 18 2003, 04:41 PM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (Phant0m51 @ Feb 18 2003, 04:41 PM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> -Will Dual Proc's provide enough performance gain to be worth the cost? (I.E. Multiple NS servers in a box, rather than just 1) <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    Basically, a CPU is good for one NS server (at this point in time). So, 2 CPUs gets you two servers. You save money by sharing memory, mobo, disk, etc. However, you spend extra for that MP mobo and chips. Common reasons people run multi-CPU servers are:

    - rack rental space is expensive, so cram as much as you can into a small spot
    - reducing the total number of machines you have to maintain
    - "cool" factor
  • JebJeb Join Date: 2002-12-30 Member: 11616Members
    I think linux should share the load between the 2 processors fine. (with the kernel patched.)

    It is true that halflife will not use both processors at the same time, but linux should be ok with the balancing of the processes between the two. With a good OS, you shouldn't need to tell the OS what to put on what CPU... Although it is a nice feature in the NT based systems, I have always felt that if MS did its job properly and balanced the load between both processors, then people shouldn't ever need to assign anything to one cpu.
Sign In or Register to comment.