Examples of cluster job scripts

::MITgcm cluster facility

Example MITgcm scripts using PBS and MPI

A simple example MPI job that uses the PBS queues to compile and run on the myrinet-3 cluster can be found in the cluster sub-directory /s00/examples/basic_mpi.
An example script that queues, compiles and runs a coupled form of MITgcm can be found in the cluster sub-directory /s00/examples/jmcl.

Case 1. - Example in /s00/examples/basic_mpi

The job script basic_mpi.pbs for this example is shown below. To execute the example copy the file /s00/examples/basic_mpi/basic_mpi.pbs to your home directory. Then submit the job using the command /usr/pbs/bin/qsub basic_mpi.pbs. The script will execute and create output files called stdout and stderr. The machinefile and make file that are used will be left in a directory called basic_mpi_example.

#!/bin/csh
#
# Example PBS script to run a job on the myrinet-3 cluster.
# The lines beginning #PBS set various queuing parameters.
#
# o -N Job Name
#PBS -N basic_mpi
#
# o -M Where to send mail
# o -m Conditions for which mail is sent a=abort, e=terminates, b=begins
#
# o -l resource lists that control where job goes
# p4 sends to myrinet-3 cluster.
#PBS -l nodes=3:p4
#
# o Where to write output
#PBS -e stderr
#PBS -o stdout
#
# o Export all my environment variables to the job
#PBS -V
#
cd ~
mkdir basic_mpi_example
cd basic_mpi_example
cp /s00/examples/basic_mpi/* .
make
cat $PBS_NODEFILE > mf
set ncpus = ( `wc -l mf | awk '{print $1}'` )
/usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/intel/bin/mpirun.ch_gm -machinefile mf -
-gm-kill 5 -v -np $ncpus ./a.out

Case 2. - Example in /s00/examples/jmcl

This example shows how to build and execute an MITgcm configuration on the myrinet-3 cluster. When submitted the job script (shown below) installs the source and starting inputs for a coupled MITgcm simulation ( a users guide to this coupled configuration can be found here) in a sub-directory called jmcl_example. The script then compiles the three executables that comprise the coupled run and starts an MPI job on the nodes provided by the PBS queue system. The MPI job uses the -pg option of mpirun to specify the executables to invoke and the -wd option to control where those executables are located.

#!/bin/csh
#
# Example PBS script to run a job on the myrinet-3 cluster.
# The lines beginning #PBS set various queuing parameters.
#
# o -N Job Name
#PBS -N jmcl_example
#
# o -M Where to send mail
# o -m Conditions for which mail is sent a=abort, e=terminates, b=begins
#
# o -l resource lists that control where job goes
#      p4 sends to myrinet-3 cluster.
#PBS -l nodes=5:p4
#
# o Where to write output
#PBS -e stderr
#PBS -o stdout
#
# o Export all my environment variables to the job
#PBS -V
#
set verbose
cd ~
mkdir jmcl_example
cd jmcl_example
pushd /s00/examples/jmcl
tar -cf - . | tcsh -c "cd ~/jmcl_example; tar -xf -"
popd
./build.sh
set mlist = ( `cat  $PBS_NODEFILE` )
set plist = ( cpl/run/mitgcmuv ocn/run/mitgcmuv ocn/run/mitgcmuv atm/run/mitgcmu
v atm/run/mitgcmuv )
set plist = ( ./mitgcmuv ./mitgcmuv ./mitgcmuv ./mitgcmuv ./mitgcmuv )
echo > pr5_group
@ i = 1
set j = 0
foreach m ( $mlist )
echo $m $j $plist[$i] >> pr5_group
@ i = $i + 1
set j = 1
end
./run_myrinet.sh