Creating Groups of Nodes in TORQUE

Despite being a simple first in/first out (FIFO) scheduler, pbs_sched can use node properties to emulate host groups. This can be useful if you have different types of nodes that provide different types of resources.

The nodes available in TORQUE are controlled by the file /var/spool/torque/server_priv/nodes. The most basic configuration simply lists the nodes and the number of processor cores per node:

node01.cluster np=12
node02.cluster np=12
node03.cluster np=12
node04.cluster np=12

Let’s say that there’s a job that needs access to fast scratch space and an SSD has been added to node03 and node04. We want the user to be able to say, “I only want my job to run on the nodes with an SSD.” To do this we’ll add a property to the nodes named ssd.

node01.cluster np=12
node02.cluster np=12
node03.cluster np=12 ssd
node04.cluster np=12 ssd

To request these nodes, we will add “ssd” to the properties list when running qsub:

qsub -l nodes=2:ppn=12:ssd submission_script.sh

The above example requests two nodes with 12 processors per node (ppn) and the property we added, ssd. This job will only run when the two nodes tagged with that property are available.