Downloading the code

setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/u0/gcmpack
setenv CVS_RSH ssh
setenv CVSEDITOR vi

cvs login ( CVS password: cvsanon )
cvs co -d directory_name -P -r ecco-branch MITgcm

Configuring your platform

 case IRIX64+mpi:
    echo "Configuring for SGI Mips with MPI"
    set DEFINES    = ( ${DEFINES} '-DTARGET_SGI -DWORDLENGTH=4' )
    set FC         = ( 'mpif77' )
    set LINK       = ( 'mpif77' )
    set FFLAGS     = ( '-extend_source -bytereclen -r10000 -mips4' )
    set INCLUDES   = ( '-I/usr/local/mpi/include' )
    set LIBS       = ( '-L/usr/local/mpi/lib/ -lfmpich -lmpich' )
    breaksw

Configuring your setup

• code/ contains all relevant configuration headers. The most important are:
  • .genmakerc: specifies which packages are enabled/disabled
  • SIZE.h: sets the domain size and the composition into tiles/processors
  • CPP_OPTIONS.h: main switches to enable/disable various aspects of MITgcm's dynamical kernel
  • CPP_EEOPTIONS.h: WRAPPER/platform specific settings.
  • ECCO_CPPOPTIONS.h: main switches for the ECCO environment. Switches for packages autodiff, cost, ctrl, exf, grdchk are set here.
  • tamc.h: settings (i.e. interval sizes) for TAF's n-level checkpointing
• input/ contains all runtime parameter files

Building the code (without the adjoint)

cd bin
ln -s ../verification/global2x2_tot/code/.genmakerc .
ln -s ../verification/global2x2_tot/code/*.h .
../tools/genmake -mpi -platform=my_platform -makefile 
make depend
make

Building the code including the generation of the adjoint code

cd bin/
ln -s ../verification/global2x2_tot/code/.genmakerc .
ln -s ../verification/global2x2_tot/code/*.h .
../tools/genmake -mpi -platform=my_platform -makefile 
make depend
cd ../adjoint/
make adtaf
make adchange
cd ../bin/
make

Building the code if TAF is on a different than your compile platform

• on your compile platform:

  # from base directory
  cd bin/
  ln -s ../verification/global2x2_tot/code/.genmakerc .
  ln -s ../verification/global2x2_tot/code/*.h .
  ../tools/genmake -mpi -platform=my_platform -makefile 
  make depend
  make small_f
  cd ../adjoint/
  make adrestore
  make allcode
  

  Copy the file tamc_code_ecco.f over to the platform where taf is installed.
  
  
• on the TAF platform:
  When setting up the system for the first time you need to "mirror" a stripped-down version of the directory tree structure of your compile platform on your TAF platform. To do this:

  # create base directory
  mkdir directory_name
  cd directory_name
  scp -r user@compile_platform:YOURPATH/directory_name/tools .
  scp -r user@compile_platform:YOURPATH/directory_name/adjoint .
  

  With this setup you can now proceed as follows to generate the adjoint code:

  # from base directory
  cd adjoint/
  make admodeltaf
  

  Copy the file tamc_code_ecco_ad.f back to your compile platform
  
  
• on your compile platform:

  # from base directory
  cd adjoint/
  make adchange
  cd ../bin/
  make
  

Running the code

# from base directory
cd exe/
ln -s ../verification/global2x2_tot/input/* .
ln -s ${fieldStorageDir}/* .
mpirun -np 6 ./mitgcmuv

Using the line search routine lsopt

J.C. Gilbert & C. Lemarechal
Some numerical experiments with variable-storage quasi-Newton algorithms
Mathematical Programming 45 (1989), pp. 407-435

• lsopt/

  contains the line search routines and a BLAS1 library (a subset of the LAPACK library) which is used extensively in lsopt.
  
  In Makefile, you need to set your compiler type $FC and compiler flags $FFLAGS. Then,

  make ecco
  

  will produce a library of object files, called

  liblsopt_ecco.a
  

  This library will be linked with the optim/ routines (see next item).



• optim/

  contains the driver routine for lsopt and the interface to read/write the control and gradient vector in the format defined by MITgcm.
  
  In Makefile, you need to set your compiler type $FC and compiler flags $FFLAGS. Then,

  make
  

  will
  • compile the code in optim/
  • link the library liblsopt_ecco.a (from lsopt/)
  • link the library libblas1.a (from lsopt/)
  • generate an executable optim.x



• jobs/

  contains a script,

  run_optim.csh
  

  to run the line search. Some parameters need to be set in the header of the script, a typical setup of which is (see documentations for details):

  # set parameters for namelist ECCO_OPTIM
  set optimcycle = 0
  set numiter    = 1
  set nfunc      = 3
  set fmin       = 1.D+5
  set iprint     = 10
  set nupdate    = 4
  
  # set parameters for namelist ECCO_PARMS
  set expId = MIT_CE_000
  
  # set parameters for namelist CTRL_NML and CTRL_PACKNAMES
  set ctrlname = ecco_ctrl
  set costname = ecco_cost
  
  # set system parameters
  set mitgcmdir = [your_mitgcm_exe_dir]
  set optimdir = [your_optim_exe_dir]
  
  # IDEALLY, NO MORE EDITING BEYOND THIS LINE.
  # ---------------------------------------------------------------------
  

  The script will then do the following:
  
  
  • change dir to [your_optim_exe_dir]
  • from the parameters provided in the header, three runtime parameter files data.optim, data.ecco, data.ctrl are generated
  • the control and gradient vector files are linked from [your_mitgcm_exe_dir]
  • optim.x is executed, producing
    1. a new control vector file (optimcycle increased by one)
    2. OPWARMI, an index file containing index infos for the for the variable storage file OPWARMD
    3. OPWARMD, a storage file, which contains the last nupdate gradients to approximate the Hessian
  
  The new control vector file serves as input for a new forward/adjoint integration in a subsequent iteration.