The GLIMMIX Procedure

Maximum Likelihood

The GLIMMIX procedure forms the log likelihoods of generalized linear models as

upper L left-parenthesis bold-italic mu comma phi semicolon bold y right-parenthesis equals sigma-summation Underscript i equals 1 Overscript n Endscripts f Subscript i Baseline l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis

where l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis is the log likelihood contribution of the ith observation with weight w Subscript i and f Subscript i is the value of the frequency variable. For the determination of w Subscript i and f Subscript i, see the WEIGHT and FREQ statements. The individual log likelihood contributions for the various distributions are as follows.

Beta
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals log left-brace StartFraction normal upper Gamma left-parenthesis phi slash w Subscript i Baseline right-parenthesis Over normal upper Gamma left-parenthesis mu Subscript i Baseline phi slash w Subscript i Baseline right-parenthesis normal upper Gamma left-parenthesis left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis phi slash w Subscript i Baseline right-parenthesis EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus left-parenthesis mu Subscript i Baseline phi slash w Subscript i Baseline minus 1 right-parenthesis log left-brace y Subscript i Baseline right-brace 3rd Row 1st Column Blank 2nd Column plus left-parenthesis left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis phi slash w Subscript i Baseline minus 1 right-parenthesis log left-brace 1 minus y Subscript i Baseline right-brace EndLayout

normal upper V normal a normal r left-bracket upper Y right-bracket equals mu left-parenthesis 1 minus mu right-parenthesis slash left-parenthesis 1 plus phi right-parenthesis comma phi greater-than 0. See Ferrari and Cribari-Neto (2004).

Binary
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline left-parenthesis y Subscript i Baseline log left-brace mu Subscript i Baseline right-brace plus left-parenthesis 1 minus y Subscript i Baseline right-parenthesis log left-brace 1 minus mu Subscript i Baseline right-brace right-parenthesis

normal upper V normal a normal r left-bracket upper Y right-bracket equals mu left-parenthesis 1 minus mu right-parenthesis comma phi identical-to 1.

Binomial
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline left-parenthesis y Subscript i Baseline log left-brace mu Subscript i Baseline right-brace plus left-parenthesis n Subscript i Baseline minus y Subscript i Baseline right-parenthesis log left-brace 1 minus mu Subscript i Baseline right-brace right-parenthesis 2nd Row 1st Column Blank 2nd Column plus w Subscript i Baseline left-parenthesis log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline minus y Subscript i Baseline plus 1 right-parenthesis right-brace right-parenthesis EndLayout

where y Subscript i and n Subscript i are the events and trials in the events/trials syntax, and 0 less-than mu less-than 1. normal upper V normal a normal r left-bracket upper Y slash n right-bracket equals mu left-parenthesis 1 minus mu right-parenthesis slash n comma phi identical-to 1.

Exponential
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals StartLayout Enlarged left-brace 1st Row 1st Column minus log left-brace mu Subscript i Baseline right-brace minus y Subscript i Baseline slash mu Subscript i Baseline 2nd Column w Subscript i Baseline equals 1 2nd Row 1st Column w Subscript i Baseline log left-brace StartFraction w Subscript i Baseline y Subscript i Baseline Over mu Subscript i Baseline EndFraction right-brace minus StartFraction w Subscript i Baseline y Subscript i Baseline Over mu Subscript i Baseline EndFraction minus log left-brace y Subscript i Baseline normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis right-brace 2nd Column w Subscript i Baseline not-equals 1 EndLayout

normal upper V normal a normal r left-bracket upper Y right-bracket equals mu squared comma phi identical-to 1.

Gamma
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline phi log left-brace StartFraction w Subscript i Baseline y Subscript i Baseline phi Over mu Subscript i Baseline EndFraction right-brace minus StartFraction w Subscript i Baseline y Subscript i Baseline phi Over mu Subscript i Baseline EndFraction minus log left-brace y Subscript i Baseline right-brace minus log left-brace normal upper Gamma left-parenthesis w Subscript i Baseline phi right-parenthesis right-brace

normal upper V normal a normal r left-bracket upper Y right-bracket equals phi mu squared comma phi greater-than 0.

Geometric
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals y Subscript i Baseline log left-brace StartFraction mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace minus left-parenthesis y Subscript i Baseline plus w Subscript i Baseline right-parenthesis log left-brace 1 plus StartFraction mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus log left-brace StartFraction normal upper Gamma left-parenthesis y Subscript i Baseline plus w Subscript i Baseline right-parenthesis Over normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis EndFraction right-brace EndLayout

normal upper V normal a normal r left-bracket upper Y right-bracket equals mu plus mu squared comma phi identical-to 1.

Inverse Gaussian
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals minus one-half left-bracket StartFraction w Subscript i Baseline left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over y Subscript i Baseline phi mu Subscript i Superscript 2 Baseline EndFraction plus log left-brace StartFraction phi y Subscript i Superscript 3 Baseline Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace right-bracket

normal upper V normal a normal r left-bracket upper Y right-bracket equals phi mu cubed comma phi greater-than 0.

"Lognormal"
l left-parenthesis mu Subscript i Baseline comma phi semicolon log left-brace y Subscript i Baseline right-brace comma w Subscript i Baseline right-parenthesis equals minus one-half left-bracket StartFraction w Subscript i Baseline left-parenthesis log left-brace y Subscript i Baseline right-brace minus mu Subscript i Baseline right-parenthesis squared Over phi EndFraction plus log left-brace StartFraction phi Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace right-bracket

normal upper V normal a normal r left-bracket log left-brace upper Y right-brace right-bracket equals phi comma phi greater-than 0.If you specify DIST=LOGNORMAL with response variable Y, the GLIMMIX procedure assumes that log left-brace upper Y right-brace tilde upper N left-parenthesis mu comma sigma squared right-parenthesis. Note that the preceding density is not the density of Y.

Multinomial
l left-parenthesis bold-italic mu Subscript i Baseline comma phi semicolon bold y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline sigma-summation Underscript j equals 1 Overscript upper J Endscripts y Subscript i j Baseline log left-brace mu Subscript i j Baseline right-brace

phi identical-to 1.

Negative Binomial
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals y Subscript i Baseline log left-brace StartFraction k mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace minus left-parenthesis y Subscript i Baseline plus w Subscript i Baseline slash k right-parenthesis log left-brace 1 plus StartFraction k mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus log left-brace StartFraction normal upper Gamma left-parenthesis y Subscript i Baseline plus w Subscript i Baseline slash k right-parenthesis Over normal upper Gamma left-parenthesis w Subscript i Baseline slash k right-parenthesis normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis EndFraction right-brace EndLayout

normal upper V normal a normal r left-bracket upper Y right-bracket equals mu plus k mu squared comma k greater-than 0 comma phi identical-to 1. For a given k, the negative binomial distribution is a member of the exponential family. The parameter k is related to the scale of the data, because it is part of the variance function. However, it cannot be factored from the variance, as is the case with the phi parameter in many other distributions. The parameter k is designated as "Scale" in the "Parameter Estimates" table of the GLIMMIX procedure.

Normal (Gaussian)
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals minus one-half left-bracket StartFraction w Subscript i Baseline left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over phi EndFraction plus log left-brace StartFraction phi Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace right-bracket

normal upper V normal a normal r left-bracket upper Y right-bracket equals phi comma phi greater-than 0.

Poisson
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline left-parenthesis y Subscript i Baseline log left-brace mu Subscript i Baseline right-brace minus mu Subscript i Baseline minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace right-parenthesis

normal upper V normal a normal r left-bracket upper Y right-bracket equals mu comma phi identical-to 1.

Shifted T
StartLayout 1st Row 1st Column z Subscript i 2nd Column equals minus 0.5 log left-brace phi slash w Subscript i Baseline right-brace plus log left-brace normal upper Gamma left-parenthesis 0.5 left-parenthesis nu plus 1 right-parenthesis right-brace 2nd Row 1st Column Blank 2nd Column minus log left-brace normal upper Gamma left-parenthesis 0.5 nu right-parenthesis right-brace minus 0.5 times log left-brace pi nu right-brace 3rd Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals minus left-parenthesis nu slash 2 plus 0.5 right-parenthesis log left-brace 1 plus StartFraction w Subscript i Baseline Over nu EndFraction StartFraction left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over phi EndFraction right-brace plus z Subscript i EndLayout

phi greater-than 0 comma nu greater-than 0 comma normal upper V normal a normal r left-bracket upper Y right-bracket equals phi nu slash left-parenthesis nu minus 2 right-parenthesis.

Define the parameter vector for the generalized linear model as bold-italic theta equals bold-italic beta, if phi identical-to 1, and as bold-italic theta equals left-bracket bold-italic beta prime comma phi right-bracket prime otherwise. bold-italic beta denotes the fixed-effects parameters in the linear predictor. For the negative binomial distribution, the relevant parameter vector is bold-italic theta equals left-bracket bold-italic beta prime comma k right-bracket prime. The gradient and Hessian of the negative log likelihood are then

bold g equals minus StartFraction partial-differential upper L left-parenthesis bold-italic theta semicolon bold y right-parenthesis Over partial-differential bold-italic theta EndFraction bold upper H equals minus StartFraction partial-differential squared upper L left-parenthesis bold-italic theta semicolon bold y right-parenthesis Over partial-differential bold-italic theta partial-differential bold-italic theta prime EndFraction

The GLIMMIX procedure computes the gradient vector and Hessian matrix analytically, unless your programming statements involve functions whose derivatives are determined by finite differences. If the procedure is in scoring mode, bold upper H is replaced by its expected value. PROC GLIMMIX is in scoring mode when the number n of SCORING=n iterations has not been exceeded and the optimization technique uses second derivatives, or when the Hessian is computed at convergence and the EXPHESSIAN option is in effect. Note that the objective function is the negative log likelihood when the GLIMMIX procedure fits a GLM model. The procedure performs a minimization problem in this case.

In models for independent data with known distribution, parameter estimates are obtained by the method of maximum likelihood. No parameters are profiled from the optimization. The default optimization technique for GLMs is the Newton-Raphson algorithm, except for Gaussian models with identity link, which do not require iterative model fitting. In the case of a Gaussian model, the scale parameter is estimated by restricted maximum likelihood, because this estimate is unbiased. The results from the GLIMMIX procedure agree with those from the GLM and REG procedures for such models. You can obtain the maximum likelihood estimate of the scale parameter with the NOREML option in the PROC GLIMMIX statement. To change the optimization algorithm, use the TECHNIQUE= option in the NLOPTIONS statement.

Standard errors of the parameter estimates are obtained from the inverse of the (observed or expected) second derivative matrix bold upper H.

Last updated: December 09, 2022