5. Automatic Differentiation

Author: Patrick Heimbach

Automatic differentiation (AD), also referred to as algorithmic (or, more loosely, computational) differentiation, involves automatically deriving code to calculate partial derivatives from an existing fully non-linear prognostic code. (see Griewank [2000]). A software tool is used that parses and transforms source files according to a set of linguistic and mathematical rules. AD tools are like source-to-source translators in that they parse a program code as input and produce a new program code as output (we restrict our discussion to source-to-source tools, ignoring operator-overloading tools). However, unlike a pure source-to-source translation, the output program represents a new algorithm, such as the evaluation of the Jacobian, the Hessian, or higher derivative operators. In principle, a variety of derived algorithms can be generated automatically in this way.

MITgcm has been adapted for use with the Tangent linear and Adjoint Model Compiler (TAMC) and its successor TAF (Transformation of Algorithms in Fortran), developed by Ralf Giering (Giering and Kaminski [1998], Giering [1999,2000]). The first application of the adjoint of MITgcm for sensitivity studies has been published by Marotzke et al. [1999]. Stammer et al. [002a,1997] use MITgcm and its adjoint for ocean state estimation studies. In the following we shall refer to TAMC and TAF synonymously, except were explicitly stated otherwise.

As of mid-2007 we are also able to generate fairly efficient adjoint code of the MITgcm using a new, open-source AD tool, called OpenAD (see Utke et al. [2008]; Naumann et al. [2006]. This enables us for the first time to compare adjoint models generated from different AD tools, providing an additional accuracy check, complementary to finite-difference gradient checks. OpenAD and its application to MITgcm is described in detail in section 5.5.

The AD tool exploits the chain rule for computing the first derivative of a function with respect to a set of input variables. Treating a given forward code as a composition of operations - each line representing a compositional element, the chain rule is rigorously applied to the code, line by line. The resulting tangent linear or adjoint code, then, may be thought of as the composition in forward or reverse order, respectively, of the Jacobian matrices of the forward code's compositional elements.

• 5.1 Some basic algebra
  • 5.1.1 Forward or direct sensitivity
  • 5.1.2 Reverse or adjoint sensitivity
  • 5.1.3 Storing vs. recomputation in reverse mode
  
  
• 5.2 TLM and ADM generation in general
  • 5.2.1 General setup
  • 5.2.2 Building the AD code using TAF
  • 5.2.3 The AD build process in detail
  • 5.2.4 The cost function (dependent variable)
  • 5.2.5 The control variables (independent variables)
  
  
• 5.3 The gradient check package
  • 5.3.1 Code description
  • 5.3.2 Code configuration
  
  
• 5.4 Adjoint dump & restart - divided adjoint (DIVA)
  • 5.4.1 Introduction
  • 5.4.2 Recipe 1: single processor
  • 5.4.3 Recipe 2: multi processor (MPI)
  
  
• 5.5 Adjoint code generation using OpenAD
  • 5.5.1 Introduction
  • 5.5.2 Downloading and installing OpenAD
  • 5.5.3 Building MITgcm adjoint with OpenAD

Copyright © 2006 Massachusetts Institute of Technology

Last update 2011-01-09