6.18.4 exch2, SIZE.h, and Multiprocessing

Once the topology configuration files are created, the Fortran PARAMETERs in SIZE.h must be configured to match. Section 4.3.1 Specifying a decomposition provides a general description of domain decomposition within MITgcm and its relation to SIZE.h. The current section specifies constraints that the exch2 package imposes and describes how to enable parallel execution with MPI.

As in the general case, the parameters sNx and sNy define the size of the individual tiles, and so must be assigned the same respective values as tnx and tny in driver.m.

The halo width parameters OLx and OLy have no special bearing on exch2 and may be assigned as in the general case. The same holds for Nr, the number of vertical levels in the model.

The parameters nSx, nSy, nPx, and nPy relate to the number of tiles and how they are distributed on processors. When using exch2, the tiles are stored in the dimension, and so nSy=1 in all cases. Since the tiles as configured by exch2 cannot be split up accross processors without regenerating the topology, nPy=1 as well.

The number of tiles MITgcm allocates and how they are distributed between processors depends on nPx and nSx. nSx is the number of tiles per processor and nPx is the number of processors. The total number of tiles in the topology minus those listed in blanklist.txt must equal nSx*nPx. Note that in order to obtain maximum usage from a given number of processors in some cases, this restriction might entail sharing a processor with a tile that would otherwise be excluded because it is topographically outside of the domain and therefore in blanklist.txt. For example, suppose you have five processors and a domain decomposition of thirty-six tiles that allows you to exclude seven tiles. To evenly distribute the remaining twenty-nine tiles among five processors, you would have to run one ``dummy'' tile to make an even six tiles per processor. Such dummy tiles are not listed in blanklist.txt.

The following is an example of SIZE.h for the twelve-tile configuration illustrated in figure 6.5 running on one processor:

      PARAMETER (
     &           sNx =  16,
     &           sNy =  32,
     &           OLx =   2,
     &           OLy =   2,
     &           nSx =  12,
     &           nSy =   1,
     &           nPx =   1,
     &           nPy =   1,
     &           Nx  = sNx*nSx*nPx,
     &           Ny  = sNy*nSy*nPy,
     &           Nr  =   5)

The following is an example for the twenty-four-tile topology in figure 6.4 running on six processors:

      PARAMETER (
     &           sNx =  16,
     &           sNy =  16,
     &           OLx =   2,
     &           OLy =   2,
     &           nSx =   4,
     &           nSy =   1,
     &           nPx =   6,
     &           nPy =   1,
     &           Nx  = sNx*nSx*nPx,
     &           Ny  = sNy*nSy*nPy,
     &           Nr  =   5)

Copyright © 2002 Massachusetts Institute of Technology