5. Automatic Differentiation

Author: Patrick Heimbach

Automatic differentiation (AD), also referred to as algorithmic (or, more loosely, computational) differentiation, involves automatically deriving code to calculate partial derivatives from an existing fully non-linear prognostic code. (see Griewank [2000]). A software tool is used that parses and transforms source files according to a set of linguistic and mathematical rules. AD tools are like source-to-source translators in that they parse a program code as input and produce a new program code as output. However, unlike a pure source-to-source translation, the output program represents a new algorithm, such as the evaluation of the Jacobian, the Hessian, or higher derivative operators. In principle, a variety of derived algorithms can be generated automatically in this way.

MITgcm has been adapted for use with the Tangent linear and Adjoint Model Compiler (TAMC) and its successor TAF (Transformation of Algorithms in Fortran), developed by Ralf Giering (Giering and Kaminski [1998], Giering [1999,2000]). The first application of the adjoint of MITgcm for sensitivity studies has been published by Marotzke et al. [1999]. []; Stammer et al. [1997] use MITgcm and its adjoint for ocean state estimation studies. In the following we shall refer to TAMC and TAF synonymously, except were explicitly stated otherwise.

TAMC exploits the chain rule for computing the first derivative of a function with respect to a set of input variables. Treating a given forward code as a composition of operations - each line representing a compositional element, the chain rule is rigorously applied to the code, line by line. The resulting tangent linear or adjoint code, then, may be thought of as the composition in forward or reverse order, respectively, of the Jacobian matrices of the forward code's compositional elements.

• 5.1 Some basic algebra
  • 5.1.1 Forward or direct sensitivity
  • 5.1.2 Reverse or adjoint sensitivity
  • 5.1.3 Storing vs. recomputation in reverse mode
  
  
• 5.2 TLM and ADM generation in general
  • 5.2.1 General setup
  • 5.2.2 Building the AD code
  • 5.2.3 The AD build process in detail
  • 5.2.4 The cost function (dependent variable)
  • 5.2.5 The control variables (independent variables)
  
  
• 5.3 The gradient check package
  • 5.3.1 Code description
  • 5.3.2 Code configuration
  
  
• 5.4 Adjoint dump & restart - divided adjoint (DIVA)
  • 5.4.1 Introduction
  • 5.4.2 Recipe 1: single processor
  • 5.4.3 Recipe 2: multi processor (MPI)

Copyright © 2006 Massachusetts Institute of Technology