• 7.3.1.1 Introduction
• 7.3.1.2 Using MDSIO
• 7.3.1.3 Important Considerations

7.3.1 MDSIO

7.3.1.1 Introduction

The mdsio package is currently the primary method for MITgcm I/O, but it is not being actively extended or enhanced. Instead, the mnc netCDF package (see Section 7.2) is expected to gain all of the current mdsio functionality and, eventually, replace it. For the short term, every effort has been made to allow mnc and mdsio to peacefully co-exist. In may cases, the model can read one format and write to the other. This side-by-side functionality can be used to, for instance, help convert pickup files or other data sets between the two formats.

7.3.1.2 Using MDSIO

C     Example of a "2D" array
      _RL anArray(1-OLx:sNx+OLx,1-OLy:sNy+OLy,nSx,nSy)

C     Example of a "3D" array
      _RL anArray(1-OLx:sNx+OLx,1-OLy:sNy+OLy,1:Nr,nSx,nSy)

In order to write output, the mdsio package is called with a function such as:

      CALL MDSWRITEFIELD(fn,prec,lgf,typ,Nr,arr,irec,myIter,myThid)

fn

is a CHARACTER string containing a file ``base'' name which will then be used to create file names that contain tile and/or model iteration indicies

prec

is an integer that contains one of two globally defined values (precFloat64 or precFloat32)

lgf

is a LOGICAL that typically contains the globally defined globalFile option which specifies the creation of globally (spatially) concatenated files

typ

is a CHARACTER string that specifies the type of the variable being written ('RL' or 'RS')

Nr

is an integer that specifies the number of vertical levels within the variable being written

arr

is the variable (array) to be written

irec

is the starting record within the output file that will contain the array

myIter,myThid

are integers containing, respectively, the current model iteration count and the unique thread ID for the current context of execution

1. open either a per-tile file such as:
   uVel.0000302400.003.001.data
   or a ``global'' file such as
   uVel.0000302400.data
2. byte-swap (as necessary) the input array and write its contents (minus any halo information) to the binary file - or to the correct location within the binary file if the globalfile option is used, and
3. create an ASCII-text metadata file (same name as the binary but with a .meta extension) describing the binary file contents (often, for later use with the MatLAB rdmds() utility).

Reading output with mdsio is very similar to writing it. A typical function call is

      CALL MDSREADFIELD(fn,prec,typ,Nr,arr,irec,myThid)

7.3.1.3 Important Considerations

Byte-swapping

is, for the most part, gracefully handled. All files intended for reading/writing by mdsio should contain big-endian (sometimes called ``network byte order'') data. By handling byte-swapping within the model, MITgcm output is more easily ported between different machines, architectures, compilers, etc. Byteswapping can be turned on/off at compile time within mdsio using the _BYTESWAPIO CPP macro which is usually set within a genmake2 options file or ``optfile'' which are located in

      MITgcm/tools/build_options

Additionally, some compilers may have byte-swap options that are speedier or more convenient to use.

Types

are currently limited to single- or double-precision floating point values. These values can be converted, on-the-fly, from one to the other so that any combination of either single- or double-precision variables can be read from or written to files containing either single- or double-precision data.

Array sizes

are limited. The mdsio package is very much geared towards the reading/writing of per-tile (that is, domain-decomposed and halo-ed) arrays. Data that cannot be made to ``fit'' within these assumed sizes can be challenging to read or write with mdsio.

Tiling

or domain decomposition is automatically handled by mdsio for logically rectangular grid topologies (eg. lat-lon grids) and ``standard'' cubesphere topologies. More complicated topologies will probably not be supported. The mdsio package can, without any coding changes, read and write to/from files that were run on the same global grid but with different tiling (grid decomposition) schemes. For example, mdsio can use and/or create identical input/output files for a ``C32'' cube when the model is run with either 6, 12, or 24 tiles (corresponding to 1, 2 or 4 tiles per cubesphere face). Currently, this is one of the primary advantages that the mdsio package has over mnc.

Single-CPU I/O

can be specified with the flag

       useSingleCpuIO = .TRUE.,

in the PARM01 namelist within the main data file. Single-CPU I/O mode is appropriate for computers (eg. some SGI systems) where it can either speed overall I/O or solve problems where the operating system or file systems cannot correctly handle multiple threads or MPI processes simultaneously writing to the same file.

Meta-data

is written by MITgcm on a per-file basis using a second file with a .meta extension as described above. MITgcm itself does not read the *.meta files, they are there primarly for convenience during post-processing. One should be careful not to delete the meta-data files when using MatLAB post-processing scripts such as rdmds() since it relies upon them.

Numerous files

can be written by mdsio due to its typically per-time-step and per-variable orientation. The creation of both a binary (*.data) and ASCII text meta-data (*.meta) file for each output type step tends to exacerbate the problem. Some (mostly, older) operating systems do not gracefully handle large numbers (eg. many thousands) of files within one directory. So care should be taken to split output into smaller groups using subdirectories.

Overwriting

is the default behavior for mdsio. If a model tries to write to a file name that already exists, the older file will be deleted. For this reason, MITgcm users should be careful to move output that that wish to keep into, for instance, subdirectories before performing subsequent runs that may over-lap in time or otherwise produce files with identical names (eg. Monte-Carlo simulations).

No ``halo'' information

is written or read by mdsio. Along the horizontal dimensions, all variables are written in an sNx-by-sNy fashion. So, although variables (arrays) may be defined at different locations on Arakawa grids [U (right/left horizontal edges), V (top/bottom horizontal edges), M (mass or cell center), or Z (vorticity or cell corner) points], they are all written using only interior (1:sNx and 1:sNy) values. For quantities defined at U, V, and M points, writing 1:sNx and 1:sNy for every tile is sufficient to ensure that all values are written globally for some grids (eg. cubesphere, re-entrant channels, and doubly-periodic rectangular regions). For Z points, failing to write values at the sNx+1 and sNy+1 locations means that, for some tile topologies, not all values are written. For instance, with a cubesphere topology at least two corner values are ``lost'' (fail to be written for any tile) if the sNx+1 and sNy+1 values are ignored. To fix this problem, the mnc package writes the sNx+1 and sNy+1 grid values for the U, V, and Z locations. Also, the mnc package is capable of reading and/or writing entire halo regions and more complicated array shapes which can be helpful when debugging-features that do not exist within mdsio.

Copyright © 2006 Massachusetts Institute of Technology

Last update 2018-01-23