Syntax Supported by the IML Procedure and the iml Action

The following example demonstrates how to call the FDDSOLVE subroutine. In the unconstrained Rosenbrock problem, the objective function is

The gradient and the Hessian are

At the point , these expressions evaluate to

The following statements define the Rosenbrock function and use the FDDSOLVE subroutine to compute the gradient and the Hessian. The results are shown in Figure 117.

start F_ROSEN(x);
   y1 = 10 * (x[2] - x[1] * x[1]);
   y2 =  1 - x[1];
   f  = 0.5 * (y1 * y1 + y2 * y2);
   return(f);
finish F_ROSEN;
x = {2 7};
call fddsolve(f, grad, Hess, "F_ROSEN", x);
print f, grad, Hess;

Figure 117 shows forward-difference approximations to the derivatives. You can compute central-difference derivatives by using the OPT keyword: OPT={1 1}.

If you define a function that evaluates the gradient of the function, you can use the GRD keyword to use the analytic gradient. For example, the following call to the FDDSOLVE subroutine uses the G_ROSEN function to evaluate the analytical gradient. The Hessian is computed by using first-order differences of the gradient function.

start G_ROSEN(x);
   g = j(1,2,0.);
   g[1] = -200*x[1]*(x[2]-x[1]*x[1]) - (1-x[1]);
   g[2] =  100*(x[2]-x[1]*x[1]);
   return(g);
finish;

call fddsolve(f, grad, Hess, "F_ROSEN", x) grd="G_ROSEN";

If the Rosenbrock problem is considered from a least squares perspective, the two component functions are

The Jacobian and the crossproduct of the Jacobian are

At the point , these matrices evaluate to

The following statements define the Rosenbrock problem in a least squares framework and use the FDDSOLVE subroutine to compute the Jacobian and the crossproduct matrix. The OPT keyword is used to specify the vector opt. Because the last element of opt is 2, the FDDSOLVE subroutine allocates memory for a least squares problem with two functions, and .

start F_ROSEN(x);
   y = j(2, 1, 0);
   y[1] = 10 * (x[2] - x[1] * x[1]);
   y[2] =  1 - x[1];
   return(y);
finish F_ROSEN;
x     = {2 7};
parms = {. . 2};
call fddsolve(f, jac, crpj, "F_ROSEN", x) opt=parms;
print f, jac, crpj;

For this example, the domain and the range of the objective function are both two-dimensional. However, in general, you can analyze the derivatives of a function for any values of n and m. The dimension of the domain is determined by the number of elements in the x0 vector. The dimension of the range is determined by the last element of the vector opt.

In the following formulas for gradients, Jacobians, and Hessians, the vector is the ith basis row vector , where the 1 is in the ith coordinate position. The symbol denotes machine precision, which is approximately 2.22E–16.

The FDDSOLVE subroutine approximates derivatives by using finite-difference formulas. The formulas depend on whether you request a forward-difference or central-difference derivative.

If you do not provide an analytic gradient function, then the following formulas are used:

The finite-difference formulas for a Jacobian are similar to the formulas for the gradient. The th position of the Jacobian matrix is the partial derivative of the ith component function with respect to the jth variable: .

If you do not provide an analytical gradient, then the Hessian matrix is approximated by using a second-order finite-difference formula.

If you provide an analytic gradient function, the Hessian is the result of first-order calls to the gradient function, as follows: