The NLMIXED Procedure

Covariance Matrix

The estimated covariance matrix of the parameter estimates is computed as the inverse Hessian matrix, and for unconstrained problems it should be positive definite. If the final parameter estimates are subjected to n Subscript a c t Baseline greater-than 0 active linear inequality constraints, the formulas of the covariance matrices are modified similar to Gallant (1987) and Cramer (1986, p. 38) and additionally generalized for applications with singular matrices.

There are several steps available that enable you to tune the rank calculations of the covariance matrix.

  1. You can use the ASINGULAR=, MSINGULAR=, and VSINGULAR= options to set three singularity criteria for the inversion of the Hessian matrix bold upper H. The singularity criterion used for the inversion is

    StartAbsoluteValue d Subscript j comma j Baseline EndAbsoluteValue less-than-or-equal-to max left-parenthesis ASING comma VSING asterisk StartAbsoluteValue upper H Subscript j comma j Baseline EndAbsoluteValue comma MSING asterisk max left-parenthesis StartAbsoluteValue upper H Subscript 1 comma 1 Baseline EndAbsoluteValue comma ellipsis comma StartAbsoluteValue upper H Subscript n comma n Baseline EndAbsoluteValue right-parenthesis right-parenthesis

    where d Subscript j comma j is the diagonal pivot of the matrix bold upper H, and ASING, VSING, and MSING are the specified values of the ASINGULAR=, VSINGULAR=, and MSINGULAR= options, respectively. The default values are as follows:

    • ASING: the square root of the smallest positive double-precision value

    • MSING: 1E–12 if you do not specify the SINGHESS= option and max left-parenthesis 10 epsilon comma 1 normal upper E minus 4 times SINGHESS right-parenthesis otherwise, where epsilon is the machine precision

    • VSING: 1E–8 if you do not specify the SINGHESS= option and the value of SINGHESS otherwise

    Note that, in many cases, a normalized matrix bold upper D Superscript negative 1 Baseline bold upper A bold upper D Superscript negative 1 is decomposed, and the singularity criteria are modified correspondingly.

  2. If the matrix bold upper H is found to be singular in the first step, a generalized inverse is computed. Depending on the G4= option, either a generalized inverse satisfying all four Moore-Penrose conditions is computed (a g 4-inverse) or a generalized inverse satisfying only two Moore-Penrose conditions is computed (a g 2-inverse, Pringle and Rayner 1971). If the number of parameters n of the application is less than or equal to G4=i, a g 4-inverse is computed; otherwise, only a g 2-inverse is computed. The g 4-inverse is computed by the (computationally very expensive but numerically stable) eigenvalue decomposition, and the g 2-inverse is computed by Gauss transformation. The g 4-inverse is computed using the eigenvalue decomposition bold upper A equals bold upper Z bold upper Lamda bold upper Z prime, where bold upper Z is the orthogonal matrix of eigenvectors and bold upper Lamda is the diagonal matrix of eigenvalues, bold upper Lamda equals normal d normal i normal a normal g left-parenthesis lamda 1 comma ellipsis comma lamda Subscript n Baseline right-parenthesis. The g 4-inverse of bold upper H is set to

    bold upper A Superscript minus Baseline equals bold upper Z bold upper Lamda Superscript minus Baseline bold upper Z prime

    where the diagonal matrix bold upper Lamda Superscript minus Baseline equals normal d normal i normal a normal g left-parenthesis lamda 1 Superscript minus Baseline comma ellipsis comma lamda Subscript n Superscript minus Baseline right-parenthesis is defined using the COVSING= option:

    lamda Subscript i Superscript minus Baseline equals StartLayout Enlarged left-brace 1st Row 1st Column 1 slash lamda Subscript i Baseline 2nd Column normal i normal f StartAbsoluteValue lamda Subscript i Baseline EndAbsoluteValue greater-than normal upper C normal upper O normal upper V normal upper S normal upper I normal upper N normal upper G 2nd Row 1st Column 0 2nd Column normal i normal f StartAbsoluteValue lamda Subscript i Baseline EndAbsoluteValue less-than-or-equal-to normal upper C normal upper O normal upper V normal upper S normal upper I normal upper N normal upper G EndLayout

    If you do not specify the COVSING= option, the nr smallest eigenvalues are set to zero, where nr is the number of rank deficiencies found in the first step.

For optimization techniques that do not use second-order derivatives, the covariance matrix is computed using finite-difference approximations of the derivatives.

Last updated: December 09, 2022