Submitting job arrays to the queue

You can organize many related jobs into a single script using the job array feature with:

#PBS -t <range>

The range specification can be either a single integer N, which will create parameters 0 ... N-1 or a rangeN-M, which will create parameters N, N+1 ... M. For example, 10 and 0-9 are equivalent. You can also explicitly specify the parameters using a comma-separated list, e.g. 0,2,4.

Note: the number of jobs in a job array on Oscar is currently limited to 1024.

When you submit a batch script with an array in it with qsub, it will return a single job id that you can use to alter or delete the entire array of jobs. When you list the jobs, you will see an entry for each parameter with the form {job_id}-{array_id}. For example, a job with id 123467 and parameters 0-3 will display as:

1234567-0
1234567-1
1234567-2
1234567-3

and will create separate output files with these names, but you could delete the entire array with qdel 1234567. You can pass the parameter to your executable using the $PBS_ARRAYID environment variable.

Here is an example of a script that uses job arrays for a parameter sweep over 64 different parameters in Matlab (notice that the Matlab function must accept the parameter ID as input) by launching 8 instances on each of 8 nodes:

#!/bin/bash
#PBS -N MATLAB
#PBS -t 0-7
#PBS -l nodes=1:ppn=8
#PBS -l walltime=1:00:00

pbsdsh bash -l -c '''
  export ID=$((PBS_NUM_PPN*PBS_ARRAYID+PBS_VNODENUM));
  echo "Starting job $ID on $HOSTNAME (CPU $PBS_VNODENUM)";
  matlab -nodisplay -nojvm -r "MyMatlabFunction($ID); quit;"
'''