The FMM Procedure

Log-Likelihood Functions for Response Distributions

The FMM procedure calculates the log likelihood that corresponds to a particular response distribution according to the following formulas. The response distribution is the distribution specified (or chosen by default) through the DIST= option in the MODEL statement. The parameterizations used for log-likelihood functions of these distributions were chosen to facilitate expressions in terms of mean parameters that are modeled through an (inverse) link functions and in terms of scale parameters. These are not necessarily the parameterizations in which parameters of prior distributions are specified in a Bayesian analysis of homogeneous mixtures. See the section Prior Distributions for details about the parameterizations of prior distributions.

The FMM procedure includes all constant terms in the computation of densities or mass functions. In the expressions that follow, l denotes the log-likelihood function, phi denotes a general scale parameter, mu Subscript i is the "mean", and w Subscript i is a weight from the use of a WEIGHT statement.

For some distributions (for example, the Weibull distribution) mu Subscript i is not the mean of the distribution. The parameter mu Subscript i is the quantity that is modeled as g Superscript negative 1 Baseline left-parenthesis bold x single-turned-comma-quotation-mark bold-italic beta right-parenthesis, where g Superscript negative 1 Baseline left-parenthesis dot right-parenthesis is the inverse link function and the bold x vector is constructed based on the effects in the MODEL statement. Situations in which the parameter mu does not represent the mean of the distribution are explicitly mentioned in the list that follows.

The parameter phi is frequently labeled as a "Scale" parameter in output from the FMM procedure. It is not necessarily the scale parameter of the particular distribution.

Beta left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals log left-brace StartFraction normal upper Gamma left-parenthesis phi slash w Subscript i Baseline right-parenthesis Over normal upper Gamma left-parenthesis mu Subscript i Baseline phi slash w Subscript i Baseline right-parenthesis normal upper Gamma left-parenthesis left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis phi slash w Subscript i Baseline right-parenthesis EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus left-parenthesis mu Subscript i Baseline phi slash w Subscript i Baseline minus 1 right-parenthesis log left-brace y Subscript i Baseline right-brace 3rd Row 1st Column Blank 2nd Column plus left-parenthesis left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis phi slash w Subscript i Baseline minus 1 right-parenthesis log left-brace 1 minus y Subscript i Baseline right-brace EndLayout

This parameterization of the beta distribution is due to Ferrari and Cribari-Neto (2004) and has properties normal upper E left-bracket upper Y right-bracket equals mu, normal upper V normal a normal r left-bracket upper Y right-bracket equals mu left-parenthesis 1 minus mu right-parenthesis slash left-parenthesis 1 plus phi right-parenthesis comma phi greater-than 0.

Beta-binomial left-parenthesis n semicolon mu comma phi right-parenthesis
StartLayout 1st Row 1st Column phi 2nd Column equals left-parenthesis 1 minus rho squared right-parenthesis slash rho squared 2nd Row 1st Column l left-parenthesis mu Subscript i Baseline comma rho semicolon y Subscript i Baseline right-parenthesis 2nd Column equals log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace 3rd Row 1st Column Blank 2nd Column minus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline minus y Subscript i Baseline plus 1 right-parenthesis right-brace 4th Row 1st Column Blank 2nd Column plus log left-brace normal upper Gamma left-parenthesis phi right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus phi right-parenthesis right-brace plus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus phi mu Subscript i Baseline right-parenthesis right-brace 5th Row 1st Column Blank 2nd Column plus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline minus y Subscript i Baseline plus phi left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis phi mu Subscript i Baseline right-parenthesis right-brace 6th Row 1st Column Blank 2nd Column minus log left-brace normal upper Gamma left-parenthesis phi left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis right-parenthesis right-brace 7th Row 1st Column l left-parenthesis mu Subscript i Baseline comma rho semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline l left-parenthesis mu Subscript i Baseline comma rho semicolon y Subscript i Baseline right-parenthesis EndLayout

where y Subscript i and n Subscript i are the events and trials in the events/trials syntax and 0 less-than mu less-than 1. This parameterization of the beta-binomial model presents the distribution as a special case of the Dirichlet-Multinomial distribution—see, for example, Neerchal and Morel (1998). In this parameterization, normal upper E left-bracket upper Y right-bracket equals n mu and normal upper V normal a normal r left-bracket upper Y right-bracket equals n mu left-parenthesis 1 minus mu right-parenthesis left-parenthesis 1 plus left-parenthesis n minus 1 right-parenthesis slash left-parenthesis phi plus 1 right-parenthesis right-parenthesis comma 0 less-than-or-equal-to rho less-than-or-equal-to 1. The FMM procedure models the parameter phi and labels it "Scale" on the procedure output. For other parameterizations of the beta-binomial model, see Griffiths (1973) or Williams (1975).

Binomial left-parenthesis n semicolon mu right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline right-parenthesis 2nd Column equals y Subscript i Baseline log left-brace mu Subscript i Baseline right-brace plus left-parenthesis n Subscript i Baseline minus y Subscript i Baseline right-parenthesis log left-brace 1 minus mu Subscript i Baseline right-brace 2nd Row 1st Column Blank 2nd Column plus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace 3rd Row 1st Column Blank 2nd Column minus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline minus y Subscript i Baseline plus 1 right-parenthesis right-brace 4th Row 1st Column l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline right-parenthesis EndLayout

where y Subscript i and n Subscript i are the events and trials in the events/trials syntax and 0 less-than mu less-than 1. In this parameterization normal upper E left-bracket upper Y right-bracket equals n mu, normal upper V normal a normal r left-bracket upper Y right-bracket equals n mu left-parenthesis 1 minus mu right-parenthesis.

Binomial Cluster left-parenthesis n semicolon mu comma pi right-parenthesis
StartLayout 1st Row 1st Column z 2nd Column equals log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline minus y Subscript i Baseline plus 1 right-parenthesis right-brace 2nd Row 1st Column mu Subscript i Superscript asterisk 2nd Column equals left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis pi Subscript i EndLayout
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma pi Subscript i Baseline semicolon y Subscript i Baseline right-parenthesis equals z plus log left-brace 2nd Column pi Subscript i Baseline left-parenthesis mu Subscript i Superscript asterisk Baseline plus mu Subscript i Baseline right-parenthesis Superscript y Super Subscript i Baseline left-parenthesis 1 minus mu Subscript i Superscript asterisk Baseline minus mu Subscript i Baseline right-parenthesis Superscript n Super Subscript i Superscript minus y Super Subscript i 2nd Row 1st Column Blank 2nd Column plus left-parenthesis 1 minus pi Subscript i Baseline right-parenthesis left-parenthesis mu Subscript i Superscript asterisk Baseline right-parenthesis Superscript y Super Subscript i Superscript Baseline left-parenthesis 1 minus mu Subscript i Superscript asterisk Baseline right-parenthesis Superscript n Super Subscript i Superscript minus y Super Subscript i Superscript Baseline right-brace EndLayout
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma pi Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline l left-parenthesis mu Subscript i Baseline comma pi Subscript i Baseline semicolon y Subscript i Baseline right-parenthesis EndLayout

In this parameterization, normal upper E left-bracket upper Y right-bracket equals n pi and normal upper V normal a normal r left-bracket upper Y right-bracket equals n pi left-parenthesis 1 minus pi right-parenthesis StartSet 1 plus mu squared left-parenthesis n minus 1 right-parenthesis EndSet. The binomial cluster model is a two-component mixture of a binomialleft-parenthesis n comma mu Superscript asterisk Baseline plus mu right-parenthesis and a binomialleft-parenthesis n comma mu Superscript asterisk Baseline right-parenthesis random variable. This mixture is unusual in that it fixes the number of components and because the mixing probability pi appears in the moments of the mixture components. For further details, see Morel and Nagaraj (1993); Morel and Neerchal (1997); Neerchal and Morel (1998) and Example 46.1 in this chapter. The expressions for the mean and variance in the binomial cluster model are identical to those of the beta-binomial model shown previously, with pi Subscript b c Baseline equals mu Subscript b b, mu Subscript b c Baseline equals rho Subscript b b.

The FMM procedure models the parameter mu through the MODEL statement and the parameter pi through the PROBMODEL statement.

Constant (c)
l left-parenthesis y Subscript i Baseline right-parenthesis equals StartLayout Enlarged left-brace 1st Row 1st Column 0 2nd Column StartAbsoluteValue y Subscript i Baseline minus c EndAbsoluteValue less-than epsilon 2nd Row 1st Column minus 1 normal upper E Baseline 20 2nd Column StartAbsoluteValue y Subscript i Baseline minus c EndAbsoluteValue greater-than-or-equal-to epsilon EndLayout

The extreme value when StartAbsoluteValue y Subscript i Baseline minus c EndAbsoluteValue greater-than-or-equal-to epsilon is an approximation of normal l normal o normal g left-parenthesis 0 right-parenthesis equals negative normal infinity, chosen so that exp left-brace l left-parenthesis y Subscript i Baseline right-parenthesis right-brace yields a likelihood of 0. You can change this value by using the INVALIDLOGL= option in the PROC FMM statement. The constant distribution is useful for modeling overdispersion due to zero-inflation (or inflation of the process at support c).

The DIST=CONSTANT distribution is useful for modeling an inflated probability of observing a particular value (0, by default) in data from other discrete distributions, as demonstrated in the section Modeling Zero-Inflation: Is It Better to Fish Poorly or Not to Have Fished at All?. Although it is syntactically valid to mix a constant distribution with a continuous distribution, such as DIST=LOGNORMAL, such a mixture is not mathematically appropriate, because the constant log likelihood is the logarithm of a probability, whereas a continuous log likelihood is the logarithm of a probability density function. If you want to mix a constant distribution with a continuous distribution, you could model the constant as a very narrow continuous distribution, such as DIST=UNIFORM(c minus delta, c plus delta) for a small value delta. However, fitting this type of mixture model can be sensitive to roundoff error and other numerical inaccuracies. Instead, the following approach is mathematically equivalent and more numerically stable:

  1. Estimate the mixing probability normal upper P left-parenthesis upper Y equals c right-parenthesis as the proportion of observations in the data set such that StartAbsoluteValue y Subscript i Baseline minus c EndAbsoluteValue less-than epsilon.

  2. Estimate the parameters of the continuous distribution from the observations for which StartAbsoluteValue y Subscript i Baseline minus c EndAbsoluteValue greater-than-or-equal-to epsilon.

Dirichlet-multinomial left-parenthesis n comma bold-italic mu comma phi right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis bold-italic mu Subscript i Baseline comma phi semicolon bold y Subscript i Baseline right-parenthesis 2nd Column equals log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace plus log left-brace normal upper Gamma left-parenthesis phi right-parenthesis right-brace plus sigma-summation Underscript j equals 1 Overscript upper M Endscripts log left-brace normal upper Gamma left-parenthesis y Subscript i j Baseline plus phi mu Subscript i j Baseline right-parenthesis right-brace 2nd Row 1st Column Blank 2nd Column minus sigma-summation Underscript j equals 1 Overscript upper M Endscripts log left-brace normal upper Gamma left-parenthesis y Subscript i j Baseline plus 1 right-parenthesis right-brace minus sigma-summation Underscript j equals 1 Overscript upper M Endscripts log left-brace normal upper Gamma left-parenthesis phi mu Subscript i j Baseline right-parenthesis right-brace minus log left-brace normal upper Gamma left-parenthesis sigma-summation Underscript j equals 1 Overscript upper M Endscripts left-bracket y Subscript i j Baseline plus phi mu Subscript i j Baseline right-bracket right-parenthesis right-brace 3rd Row 1st Column l left-parenthesis bold-italic mu Subscript i Baseline comma phi semicolon bold y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline l left-parenthesis bold-italic mu Subscript i Baseline comma phi semicolon bold y Subscript i Baseline right-parenthesis EndLayout

where M is the number of levels of the response variable, n Subscript i Baseline equals sigma-summation Underscript j equals 1 Overscript upper M Endscripts y Subscript i j, 0 less-than mu Subscript i j Baseline less-than 1 for each j, and mu Subscript i upper M Baseline equals 1 minus sigma-summation Underscript j equals 1 Overscript upper M minus 1 Endscripts mu Subscript i j.

The mean and variance are normal upper E left-bracket bold upper Y right-bracket equals n bold-italic mu and normal upper V normal a normal r left-bracket bold upper Y right-bracket equals n left-parenthesis Diag left-parenthesis bold-italic mu right-parenthesis minus bold-italic mu bold-italic mu Superscript upper T Baseline right-parenthesis left-parenthesis StartFraction n plus phi Over 1 plus phi EndFraction right-parenthesis.

Exponential left-parenthesis mu right-parenthesis
l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals StartLayout Enlarged left-brace 1st Row 1st Column minus log left-brace mu Subscript i Baseline right-brace minus y Subscript i Baseline slash mu Subscript i Baseline 2nd Column w Subscript i Baseline equals 1 2nd Row 1st Column w Subscript i Baseline log left-brace StartFraction w Subscript i Baseline y Subscript i Baseline Over mu Subscript i Baseline EndFraction right-brace minus StartFraction w Subscript i Baseline y Subscript i Baseline Over mu Subscript i Baseline EndFraction minus log left-brace y Subscript i Baseline normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis right-brace 2nd Column w Subscript i Baseline not-equals 1 EndLayout

In this parameterization, normal upper E left-bracket upper Y right-bracket equals mu and normal upper V normal a normal r left-bracket upper Y right-bracket equals mu squared.

Folded Normal left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals 2nd Column minus one-half log left-brace 2 pi right-brace minus one-half log left-brace phi slash w Subscript i Baseline right-brace 2nd Row 1st Column plus 2nd Column log left-brace exp left-brace StartFraction minus w Subscript i Baseline left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over 2 phi EndFraction right-brace plus exp left-brace StartFraction minus w Subscript i Baseline left-parenthesis y Subscript i Baseline plus mu Subscript i Baseline right-parenthesis squared Over 2 phi EndFraction right-brace right-brace EndLayout

If X has a normal distribution with mean mu and variance phi, then upper Y equals StartAbsoluteValue upper X EndAbsoluteValue has a folded normal distribution and log-likelihood function l left-parenthesis mu comma phi semicolon y comma w right-parenthesis for y greater-than-or-equal-to 0. The folded normal distribution arises, for example, when normally distributed measurements are observed, but their signs are not observed. The mean and variance of the folded normal in terms of the underlying normal upper N left-parenthesis mu comma phi right-parenthesis distribution are

StartLayout 1st Row 1st Column normal upper E left-bracket upper Y right-bracket equals 2nd Column StartFraction 1 Over StartRoot 2 pi phi EndRoot EndFraction exp left-brace minus StartFraction mu squared Over 2 slash phi EndFraction right-brace plus mu left-parenthesis 1 minus 2 normal upper Phi left-parenthesis negative mu slash StartRoot phi EndRoot right-parenthesis right-parenthesis 2nd Row 1st Column normal upper V normal a normal r left-bracket upper Y right-bracket equals 2nd Column phi plus mu squared minus normal upper E left-bracket upper Y right-bracket squared EndLayout

The FMM procedure models the folded normal distribution through the mean mu and variance phi of the underlying normal distribution. When the FMM procedure computes output statistics for the response variable (for example when you use the OUTPUT statement), the mean and variance of the response Y are reported. Similarly, the fit statistics apply to the distribution of upper Y equals StartAbsoluteValue upper X EndAbsoluteValue, not the distribution of X. When you model a folded normal variable, the response input variable should be positive; the FMM procedure treats negative values of Y as a support violation.

Gamma left-parenthesis mu comma phi right-parenthesis
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline phi log left-brace StartFraction w Subscript i Baseline y Subscript i Baseline phi Over mu Subscript i Baseline EndFraction right-brace minus StartFraction w Subscript i Baseline y Subscript i Baseline phi Over mu Subscript i Baseline EndFraction minus log left-brace y Subscript i Baseline right-brace minus log left-brace normal upper Gamma left-parenthesis w Subscript i Baseline phi right-parenthesis right-brace

In this parameterization, normal upper E left-bracket upper Y right-bracket equals mu and normal upper V normal a normal r left-bracket upper Y right-bracket equals mu squared slash phi comma phi greater-than 0. This parameterization of the gamma distribution differs from that in the GLIMMIX procedure, which expresses the log-likelihood function in terms of 1 slash phi in order to achieve a variance function suitable for mixed model analysis.

Geometric left-parenthesis mu right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals y Subscript i Baseline log left-brace StartFraction mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace minus left-parenthesis y Subscript i Baseline plus w Subscript i Baseline right-parenthesis log left-brace 1 plus StartFraction mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus log left-brace StartFraction normal upper Gamma left-parenthesis y Subscript i Baseline plus w Subscript i Baseline right-parenthesis Over normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis EndFraction right-brace EndLayout

In this parameterization, normal upper E left-bracket upper Y right-bracket equals mu and normal upper V normal a normal r left-bracket upper Y right-bracket equals mu plus mu squared. The geometric distribution is a special case of the negative binomial distribution with phi equals 1.

Generalized Poisson left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column xi Subscript i 2nd Column equals 1 minus exp left-brace negative phi right-brace slash w Subscript i Baseline 2nd Row 1st Column mu Subscript i Superscript asterisk 2nd Column equals mu Subscript i Baseline minus xi left-parenthesis mu Subscript i Baseline minus y Subscript i Baseline right-parenthesis 3rd Row 1st Column l left-parenthesis mu Subscript i Superscript asterisk Baseline comma xi Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals log left-brace mu Subscript i Superscript asterisk Baseline minus xi Subscript i Baseline y Subscript i Baseline right-brace plus left-parenthesis y Subscript i Baseline minus 1 right-parenthesis log left-brace mu Subscript i Superscript asterisk Baseline right-brace 4th Row 1st Column Blank 2nd Column minus mu Subscript i Superscript asterisk minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace EndLayout

In this parameterization, normal upper E left-bracket upper Y right-bracket equals mu, normal upper V normal a normal r left-bracket upper Y right-bracket equals mu slash left-parenthesis 1 minus xi right-parenthesis squared comma and phi greater-than-or-equal-to 0. The FMM procedure models the mean mu through the effects in the MODEL statement and applies a log link by default. The generalized Poisson distribution provides an overdispersed alternative to the Poisson distribution; phi equals xi Subscript i Baseline equals 0 produces the mass function of a regular Poisson random variable. For details about the generalized Poisson distribution and a comparison with the negative binomial distribution, see Joe and Zhu (2005).

Inverse Gaussian left-parenthesis mu comma phi right-parenthesis
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals minus one-half left-bracket StartFraction w Subscript i Baseline left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over y Subscript i Baseline phi mu Subscript i Superscript 2 Baseline EndFraction plus log left-brace StartFraction phi y Subscript i Superscript 3 Baseline Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace right-bracket

The variance is normal upper V normal a normal r left-bracket upper Y right-bracket equals phi mu cubed comma phi greater-than 0.

Lognormal left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column z Subscript i 2nd Column equals log left-brace y Subscript i Baseline right-brace minus mu Subscript i Baseline 2nd Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals minus one-half left-parenthesis 2 log left-brace y Subscript i Baseline right-brace plus log left-brace StartFraction phi Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace plus StartFraction w Subscript i Baseline z Subscript i Superscript 2 Baseline Over phi EndFraction right-parenthesis EndLayout

If upper X equals log left-brace upper Y right-brace has a normal distribution with mean mu and variance phi, then Y has the log-likelihood function l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis. The FMM procedure models the lognormal distribution and not the "shortcut" version you can obtain by taking the logarithm of a random variable and modeling that as normally distributed. The two approaches are not equivalent, and the approach taken by PROC FMM is the actual lognormal distribution. Although the lognormal model is a member of the exponential family of distributions, it is not in the "natural" exponential family because it cannot be written in canonical form.

In terms of the parameters mu and phi of the underlying normal process for X, the mean and variance of Y are normal upper E left-bracket upper Y right-bracket equals exp left-brace mu right-brace StartRoot omega EndRoot and normal upper V normal a normal r left-bracket upper Y right-bracket equals exp left-brace 2 mu right-brace omega left-parenthesis omega minus 1 right-parenthesis, respectively, where omega equals exp left-brace phi right-brace. When you request predicted values with the OUTPUT statement, the FMM procedure computes normal upper E left-bracket upper Y right-bracket and not mu.

Multinomial left-parenthesis n comma bold-italic mu right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis bold-italic mu Subscript i Baseline semicolon bold y Subscript i Baseline right-parenthesis 2nd Column equals sigma-summation Underscript j equals 1 Overscript upper M Endscripts y Subscript i j Baseline log left-parenthesis mu Subscript i j Baseline right-parenthesis plus log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace minus sigma-summation Underscript j equals 1 Overscript upper M Endscripts log left-brace normal upper Gamma left-parenthesis y Subscript i j Baseline plus 1 right-parenthesis right-brace 2nd Row 1st Column l left-parenthesis bold-italic mu Subscript i Baseline semicolon bold y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline l left-parenthesis bold-italic mu Subscript i Baseline comma bold y Subscript i Baseline right-parenthesis EndLayout

where M is the number of levels of the response variable, n Subscript i Baseline equals sigma-summation Underscript j equals 1 Overscript upper M Endscripts y Subscript i j, 0 less-than mu Subscript i j Baseline less-than 1 for each j, and sigma-summation Underscript j equals 1 Overscript upper M Endscripts mu Subscript i j Baseline equals 1.

The mean and variance are normal upper E left-bracket bold upper Y right-bracket equals n bold-italic mu and normal upper V normal a normal r left-bracket bold upper Y right-bracket equals n left-parenthesis Diag left-parenthesis bold-italic mu right-parenthesis minus bold-italic mu bold-italic mu Superscript upper T Baseline right-parenthesis, respectively, where bold-italic mu equals left-parenthesis mu 1 comma ellipsis comma mu Subscript upper M minus 1 Baseline right-parenthesis.

Multinomial Cluster left-parenthesis n comma mu comma bold-italic pi right-parenthesis
StartLayout 1st Row 1st Column upper K left-parenthesis mu Subscript i Baseline comma bold-italic pi Subscript i Baseline comma bold y Subscript i Baseline right-parenthesis 2nd Column equals sigma-summation Underscript h equals 1 Overscript upper M Endscripts StartSet pi Subscript i h Baseline product Underscript g equals 1 Overscript upper M Endscripts left-bracket pi Subscript i g Baseline left-parenthesis 1 minus mu Subscript i Baseline right-parenthesis plus mu Subscript i Baseline 1 left-parenthesis g equals h right-parenthesis right-bracket Superscript y Super Subscript i g Superscript Baseline EndSet 2nd Row 1st Column l left-parenthesis mu Subscript i Baseline comma bold-italic pi Subscript i Baseline comma bold y Subscript i Baseline right-parenthesis 2nd Column equals log left-brace normal upper Gamma left-parenthesis n Subscript i Baseline plus 1 right-parenthesis right-brace minus sigma-summation Underscript g equals 1 Overscript upper M Endscripts log left-brace normal upper Gamma left-parenthesis y Subscript i j Baseline plus 1 right-parenthesis right-brace plus log left-brace upper K left-parenthesis mu Subscript i Baseline comma bold-italic pi Subscript i Baseline comma bold y Subscript i Baseline right-parenthesis right-brace 3rd Row 1st Column l left-parenthesis mu Subscript i Baseline comma bold-italic pi Subscript i Baseline comma bold y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline l left-parenthesis mu Subscript i Baseline comma bold-italic pi Subscript i Baseline comma bold y Subscript i Baseline right-parenthesis EndLayout

The multinomial cluster model for a response that has M levels is an M-component mixture of multinomials. The parameter vector for each component multinomial is based on the mixing probability vector bold-italic pi and an additional factor mu. As is true for the binomial cluster model, the number of components is fixed and the mixing probabilities in bold-italic pi appear in the moments for the mixture components. For more information, see Morel and Nagaraj (1993) and Example 46.4 in this chapter.

The mean and variance are normal upper E left-bracket bold upper Y right-bracket equals n bold-italic pi and normal upper V normal a normal r left-bracket bold upper Y right-bracket equals n left-parenthesis Diag left-parenthesis bold-italic pi right-parenthesis minus bold-italic pi bold-italic pi Superscript upper T Baseline right-parenthesis StartSet 1 plus mu squared left-parenthesis n minus 1 right-parenthesis EndSet, respectively.

Negative Binomial left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals y Subscript i Baseline log left-brace StartFraction phi mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace minus left-parenthesis y Subscript i Baseline plus w Subscript i Baseline slash phi right-parenthesis log left-brace 1 plus StartFraction phi mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus log left-brace StartFraction normal upper Gamma left-parenthesis y Subscript i Baseline plus w Subscript i Baseline slash phi right-parenthesis Over normal upper Gamma left-parenthesis w Subscript i Baseline slash phi right-parenthesis normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis EndFraction right-brace EndLayout

The variance is normal upper V normal a normal r left-bracket upper Y right-bracket equals mu plus phi mu squared comma phi greater-than 0.

For a given phi, the negative binomial distribution is a member of the exponential family. The parameter phi is related to the scale of the data because it is part of the variance function. However, it cannot be factored from the variance, as is the case with the phi parameter in many other distributions.

Normal left-parenthesis mu comma phi right-parenthesis
l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals minus one-half left-bracket StartFraction w Subscript i Baseline left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over phi EndFraction plus log left-brace StartFraction phi Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace right-bracket

The mean and variance are normal upper E left-bracket upper Y right-bracket equals mu and normal upper V normal a normal r left-bracket upper Y right-bracket equals phi, respectively, phi greater-than 0

Poisson left-parenthesis mu right-parenthesis
l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline left-parenthesis y Subscript i Baseline log left-brace mu Subscript i Baseline right-brace minus mu Subscript i Baseline minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace right-parenthesis

The mean and variance are normal upper E left-bracket upper Y right-bracket equals mu and normal upper V normal a normal r left-bracket upper Y right-bracket equals mu.

(Shifted) T left-parenthesis nu semicolon mu comma phi right-parenthesis
StartLayout 1st Row 1st Column z Subscript i 2nd Column equals minus 0.5 log left-brace phi slash w Subscript i Baseline right-brace plus log left-brace normal upper Gamma left-parenthesis 0.5 left-parenthesis nu plus 1 right-parenthesis right-brace 2nd Row 1st Column Blank 2nd Column minus log left-brace normal upper Gamma left-parenthesis 0.5 nu right-parenthesis right-brace minus 0.5 times log left-brace pi nu right-brace 3rd Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals minus left-parenthesis StartFraction nu plus 1 Over 2 EndFraction right-parenthesis log left-brace 1 plus StartFraction w Subscript i Baseline Over nu EndFraction StartFraction left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over phi EndFraction right-brace plus z Subscript i EndLayout

nu greater-than 0 comma phi greater-than 0. In this parameterization, normal upper E left-bracket upper Y right-bracket equals mu when nu greater-than 1 and normal upper V normal a normal r left-bracket upper Y right-bracket equals phi nu slash left-parenthesis nu minus 2 right-parenthesis when nu greater-than 2. Note that this form of the t distribution is not a noncentral distribution, but that of a shifted central t random variable.

Truncated Exponential left-parenthesis mu semicolon a comma b right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline semicolon a comma b comma y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals w Subscript i Baseline log left-brace StartFraction w Subscript i Baseline y Subscript i Baseline Over mu Subscript i Baseline EndFraction right-brace minus StartFraction w Subscript i Baseline y Subscript i Baseline Over mu Subscript i Baseline EndFraction minus log left-brace y Subscript i Baseline normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis right-brace 2nd Row 1st Column Blank 2nd Column minus log left-bracket StartStartFraction gamma left-parenthesis w Subscript i Baseline comma StartFraction w Subscript i Baseline b Over mu Subscript i Baseline EndFraction right-parenthesis OverOver normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis EndEndFraction minus StartStartFraction gamma left-parenthesis w Subscript i Baseline comma StartFraction w Subscript i Baseline a Over mu Subscript i Baseline EndFraction right-parenthesis OverOver normal upper Gamma left-parenthesis w Subscript i Baseline right-parenthesis EndEndFraction right-bracket EndLayout

where

gamma left-parenthesis c 1 comma c 2 right-parenthesis equals integral Subscript 0 Superscript c 2 Baseline t Superscript c 1 minus 1 Baseline exp left-parenthesis negative t right-parenthesis normal d t

is the lower incomplete gamma function. The mean and variance are

StartLayout 1st Row 1st Column normal upper E left-bracket upper Y right-bracket 2nd Column equals StartFraction left-parenthesis a plus mu Subscript i Baseline right-parenthesis exp left-parenthesis negative a slash mu Subscript i Baseline right-parenthesis minus left-parenthesis b plus mu Subscript i Baseline right-parenthesis exp left-parenthesis negative b slash mu Subscript i Baseline right-parenthesis Over exp left-parenthesis negative a slash mu Subscript i Baseline right-parenthesis minus exp left-parenthesis negative b slash mu Subscript i Baseline right-parenthesis EndFraction 2nd Row 1st Column normal upper V normal a normal r left-bracket upper Y right-bracket 2nd Column equals StartFraction left-parenthesis a squared plus 2 a mu Subscript i Baseline plus 2 mu Subscript i Superscript 2 Baseline right-parenthesis exp left-parenthesis negative a slash mu Subscript i Baseline right-parenthesis minus left-parenthesis b squared plus 2 b mu Subscript i Baseline plus 2 mu Subscript i Superscript 2 Baseline right-parenthesis exp left-parenthesis negative b slash mu Subscript i Baseline right-parenthesis Over exp left-parenthesis negative a slash mu Subscript i Baseline right-parenthesis minus exp left-parenthesis negative b slash mu Subscript i Baseline right-parenthesis EndFraction 3rd Row 1st Column Blank 2nd Column minus left-parenthesis normal upper E left-bracket upper Y right-bracket right-parenthesis squared EndLayout
Truncated Lognormal left-parenthesis mu comma phi semicolon a comma b right-parenthesis
StartLayout 1st Row 1st Column z Subscript i 2nd Column equals log left-brace y Subscript i Baseline right-brace minus mu Subscript i Baseline 2nd Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon a comma b comma y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals minus one-half left-parenthesis 2 log left-brace y Subscript i Baseline right-brace plus log left-brace StartFraction phi Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace plus StartFraction w Subscript i Baseline z Subscript i Superscript 2 Baseline Over phi EndFraction right-parenthesis 3rd Row 1st Column Blank 2nd Column minus log left-brace normal upper Phi left-bracket StartRoot w Subscript i Baseline slash phi EndRoot left-parenthesis log b minus mu Subscript i Baseline right-parenthesis right-bracket minus normal upper Phi left-bracket StartRoot w Subscript i Baseline slash phi EndRoot left-parenthesis log a minus mu Subscript i Baseline right-parenthesis right-bracket right-brace EndLayout

where normal upper Phi left-parenthesis dot right-parenthesis is the cumulative distribution function of the standard normal distribution. The mean and variance are

StartLayout 1st Row 1st Column normal upper E left-bracket upper Y right-bracket 2nd Column equals exp left-parenthesis mu Subscript i Baseline plus 0.5 phi right-parenthesis StartStartFraction normal upper Phi left-parenthesis StartRoot phi EndRoot minus StartFraction log a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis StartRoot phi EndRoot minus StartFraction log b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis OverOver normal upper Phi left-parenthesis StartFraction log b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis StartFraction log a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis EndEndFraction 2nd Row 1st Column normal upper V normal a normal r left-bracket upper Y right-bracket 2nd Column equals exp left-parenthesis 2 mu Subscript i Baseline plus 2 phi right-parenthesis StartStartFraction normal upper Phi left-parenthesis 2 StartRoot phi EndRoot minus StartFraction log a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis 2 StartRoot phi EndRoot minus StartFraction log b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis OverOver normal upper Phi left-parenthesis StartFraction log b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis StartFraction log a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis EndEndFraction minus left-parenthesis normal upper E left-bracket upper Y right-bracket right-parenthesis squared EndLayout
Truncated Negative Binomial left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals y Subscript i Baseline log left-brace StartFraction phi mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace minus left-parenthesis y Subscript i Baseline plus w Subscript i Baseline slash phi right-parenthesis log left-brace 1 plus StartFraction phi mu Subscript i Baseline Over w Subscript i Baseline EndFraction right-brace 2nd Row 1st Column Blank 2nd Column plus log left-brace StartFraction normal upper Gamma left-parenthesis y Subscript i Baseline plus w Subscript i Baseline slash phi right-parenthesis Over normal upper Gamma left-parenthesis w Subscript i Baseline slash phi right-parenthesis normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis EndFraction right-brace 3rd Row 1st Column Blank 2nd Column minus log left-brace 1 minus left-parenthesis StartFraction phi mu Subscript i Baseline Over w Subscript i Baseline EndFraction plus 1 right-parenthesis Superscript minus w Super Subscript i Superscript slash phi Baseline right-brace EndLayout

The mean and variance are

StartLayout 1st Row 1st Column normal upper E left-bracket upper Y right-bracket 2nd Column equals mu Subscript i Baseline StartSet 1 minus left-parenthesis phi mu Subscript i Baseline plus 1 right-parenthesis Superscript negative 1 slash phi Baseline EndSet Superscript negative 1 Baseline 2nd Row 1st Column normal upper V normal a normal r left-bracket upper Y right-bracket 2nd Column equals left-parenthesis 1 plus phi mu Subscript i Baseline plus mu Subscript i Baseline right-parenthesis normal upper E left-bracket upper Y right-bracket minus left-parenthesis normal upper E left-bracket upper Y right-bracket right-parenthesis squared EndLayout
Truncated Normal left-parenthesis mu comma phi semicolon a comma b right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon a comma b comma y Subscript i Baseline comma w Subscript i Baseline right-parenthesis 2nd Column equals minus one-half left-bracket StartFraction w Subscript i Baseline left-parenthesis y Subscript i Baseline minus mu Subscript i Baseline right-parenthesis squared Over phi EndFraction plus log left-brace StartFraction phi Over w Subscript i Baseline EndFraction right-brace plus log left-brace 2 pi right-brace right-bracket 2nd Row 1st Column Blank 2nd Column minus log left-brace normal upper Phi left-bracket StartRoot w Subscript i Baseline slash phi EndRoot left-parenthesis b minus mu Subscript i Baseline right-parenthesis right-bracket minus normal upper Phi left-bracket StartRoot w Subscript i Baseline slash phi EndRoot left-parenthesis a minus mu Subscript i Baseline right-parenthesis right-bracket right-brace EndLayout

where normal upper Phi left-parenthesis dot right-parenthesis is the cumulative distribution function of the standard normal distribution. The mean and variance are

StartLayout 1st Row 1st Column normal upper E left-bracket upper Y right-bracket 2nd Column equals mu Subscript i Baseline plus StartRoot phi EndRoot StartStartFraction normal p normal h normal i left-parenthesis StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal p normal h normal i left-parenthesis StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis OverOver normal upper Phi left-parenthesis StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis EndEndFraction 2nd Row 1st Column normal upper V normal a normal r left-bracket upper Y right-bracket 2nd Column equals phi left-bracket 1 plus StartStartFraction StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction normal p normal h normal i left-parenthesis StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction normal p normal h normal i left-parenthesis StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis OverOver normal upper Phi left-parenthesis StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis EndEndFraction 3rd Row 1st Column Blank 2nd Column minus StartSet StartStartFraction normal p normal h normal i left-parenthesis StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal p normal h normal i left-parenthesis StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis OverOver normal upper Phi left-parenthesis StartFraction b minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis minus normal upper Phi left-parenthesis StartFraction a minus mu Subscript i Baseline Over StartRoot phi EndRoot EndFraction right-parenthesis EndEndFraction EndSet squared right-bracket EndLayout

where normal p normal h normal i left-parenthesis dot right-parenthesis is the probability density function of the standard normal distribution.

Truncated Poisson left-parenthesis mu right-parenthesis
l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals w Subscript i Baseline left-parenthesis y Subscript i Baseline log left-brace mu Subscript i Baseline right-brace minus log left-brace exp left-parenthesis mu Subscript i Baseline right-parenthesis minus 1 right-brace minus log left-brace normal upper Gamma left-parenthesis y Subscript i Baseline plus 1 right-parenthesis right-brace right-parenthesis

The mean and variance are

StartLayout 1st Row 1st Column normal upper E left-bracket upper Y right-bracket 2nd Column equals StartFraction mu Subscript i Baseline Over 1 minus exp left-parenthesis minus mu Subscript i Baseline right-parenthesis EndFraction 2nd Row 1st Column normal upper V normal a normal r left-bracket upper Y right-bracket 2nd Column equals StartFraction mu Subscript i Baseline left-bracket 1 minus exp left-parenthesis minus mu Subscript i Baseline right-parenthesis minus mu Subscript i Baseline exp left-parenthesis minus mu Subscript i Baseline right-parenthesis right-bracket Over left-bracket 1 minus exp left-parenthesis minus mu Subscript i Baseline right-parenthesis right-bracket squared EndFraction EndLayout
Uniform left-parenthesis a comma b right-parenthesis
l left-parenthesis mu Subscript i Baseline semicolon y Subscript i Baseline comma w Subscript i Baseline right-parenthesis equals minus log left-brace b minus a right-brace

The mean and variance are normal upper E left-bracket upper Y right-bracket equals 0.5 left-parenthesis a plus b right-parenthesis and normal upper V normal a normal r left-bracket upper Y right-bracket equals left-parenthesis b minus a right-parenthesis squared slash 12.

Weibull left-parenthesis mu comma phi right-parenthesis
StartLayout 1st Row 1st Column l left-parenthesis mu Subscript i Baseline comma phi semicolon y Subscript i Baseline right-parenthesis 2nd Column equals minus StartFraction phi minus 1 Over phi EndFraction log left-brace StartFraction y Subscript i Baseline Over mu Subscript i Baseline EndFraction right-brace minus log left-brace mu Subscript i Baseline phi right-brace 2nd Row 1st Column Blank 2nd Column minus exp left-brace log left-brace StartFraction y Subscript i Baseline Over mu Subscript i Baseline EndFraction right-brace slash phi right-brace EndLayout

In this particular parameterization of the two-parameter Weibull distribution, the mean and variance of the random variable Y are normal upper E left-bracket upper Y right-bracket equals mu normal upper Gamma left-parenthesis 1 plus phi right-parenthesis and normal upper V normal a normal r left-bracket upper Y right-bracket equals mu squared StartSet normal upper Gamma left-parenthesis 1 plus 2 phi right-parenthesis minus normal upper Gamma squared left-parenthesis 1 plus phi right-parenthesis EndSet.

Last updated: December 09, 2022