The SEVSELECT Procedure

This example extends Example 23.6: Scale Regression with Rich Regression Effects to illustrate the model selection features of the SEVSELECT procedure. There are two phases of model selection when you fit a scale regression model. First, you want to select an optimal set of regression effects for each distribution’s scale regression model. The SELECTION statement provides several methods to do that. Second, you want to identify the best severity distribution by comparing the final selected scale regression models for each distribution. PROC SEVSELECT displays tables to help you do that.

The following SEVSELECT step uses the same data table as in Example 23.6 to find the best set of scale regression effects for three severity distributions—lognormal (Logn), Burr, and Weibull:

proc sevselect data=mycas.losses outest=mycas.est print=all;
   loss lossAmount / lt=deductible rc=limit;
   class carType gender education;
   scalemodel carType gender carSafety income education*carType
              income*gender carSafety*income;
   selection;
   dist logn burr weibull;
   output out=mycas.score copyvars=(carType gender education carSafety income)
          functions=(mean) quantiles=(points=0.5 0.975 names=(median var));
run;

The SELECTION statement without any options uses the stepwise selection method by default. PROC SEVSELECT reports the settings of the selection process in the "Selection Information" table, as shown in Output 23.7.1.

For each severity distribution, PROC SEVSELECT reports the summary of the selection process, as shown in Output 23.7.2 for the lognormal distribution. The "Selection Summary" table shows the order in which regression effects enter or leave the model. In this case, no effect leaves the model after entering the model. In the first step, PROC SEVSELECT fits seven models, each with an intercept and one of the seven eligible effects in the SCALEMODEL statement. The model that contains the intercept and carType*education effect has the lowest value of the Schwarz Bayesian criterion (SBC) statistic in the first step, and hence it enters the model. In the second step, PROC SEVSELECT compares the model that does not contain the existing effect, carType*education, with models that contain that effect and one of the remaining six effects. The model that contains the intercept, carType*education, and carSafety effects is the best in the second step according to the SBC statistic. The process of removing an effect from the model or adding an effect that is not in the model continues until the stop criterion does not improve or the stop horizon condition is met. As the results in Output 23.7.2 show, the optimal value of the SBC criterion is achieved in the fourth step. The final scale regression model for the lognormal distribution contains the following effects: intercept, gender, carSafety, carType*education, and income*gender.

The selection summary information for the Weibull distribution is shown in Output 23.7.3. It shows that the best model in the sixth step is obtained by removing the carSafety*income effect, which was added in the second step. The model in the sixth step also has the lowest value of the SBC criterion; hence, it is the best model and contains the following effects: intercept, gender, carSafety, income, and carType*education.

PROC SEVSELECT reports the optimization summary, fit statistics, and parameter estimates for the final selected model of each severity distribution. The results for the lognormal distribution are shown in Output 23.7.4. The "Parameter Estimates" table shows that all regression parameters in the final model are statistically significant. Comparison of these parameter estimates with those in Output 23.6.4, which were obtained without using the SELECTION statement, shows that the statistically insignificant effect, carSafety*income, is not present in the final selected model. The comparison also shows that the stepwise selection process prefers income*gender effect over the income effect and, because of the exclusion of the income effect, the income*gender effect has become statistically significant.

After the effect selection process chooses the best scale regression model for each severity distribution, the next phase of model selection is to compare the final models across all severity distributions and identify the best severity distribution. You can do that by using the "All Fit Statistics" table, which helps you compare the final selected models according to all the fit statistics. For this example, Output 23.7.5 compares the final models of the three severity distributions—lognormal, Burr, and Weibull.

The lognormal distribution’s final selected model is the best according to all the likelihood-based statistics, the Burr distribution’s final selected model is the best according to the Anderson-Darling (AD) statistic, and the Weibull distribution’s final selected model is the best according to the Kolmogorov-Smirnov (KS) and Cramér–von Mises (CvM) statistics. You can choose the fit statistic that is most suitable for your application and use it to identify the best severity distribution. Alternatively, if your goal is to estimate some loss distribution statistic, such as the value-at-risk (VaR), for a particular scenario, then instead of choosing one severity distribution, you might want to compute and compare the desired statistic for the final scale regression model of each severity distribution that wins according to at least one fit statistic. The OUTPUT statement is useful for such evaluation, because it helps you score each observation of the input data table that you specify in the DATA= option. In the context of PROC SEVSELECT, scoring an observation is equivalent to computing particular statistics for the severity distribution of the losses. The previous PROC SEVSELECT step specifies the following OUTPUT statement to write the scores for each observation to the mycas.Score output table:

output out=mycas.score copyvars=(carType gender education carSafety income)
       functions=(mean) quantiles=(cdf=0.5 0.975 names=(median var));

When you specify the scale regression model, the scale parameter of each severity distribution depends on the values of the regression effects in the final selected model of that distribution. For each observation in the input table, PROC SEVSELECT computes the estimate of the scale parameter and uses it to evaluate the scoring and quantile functions that you specify in the FUNCTIONS= and QUANTILES= options, respectively. The OUTPUT statement of this example specifies that three statistics be computed for the final model of each severity distribution: the mean scoring function and two quantile functions, and , where denotes the value of the quantile function for a distribution dist that is evaluated at the CDF value of p for the estimated distribution parameters . The variable that contains the estimate of the mean function is named mean. The variables that contain the quantile function estimate for p=0.5 and p=0.975 are named median and var, respectively; var denotes the value-at-risk (VaR). In order to compute a function F in the FUNCTIONS= option, for each severity distribution dist, the dist_F function must be available in the function library search path that you specify in the CMPLIB= system option. Also, in order to compute the quantile functions faster, it is recommended that the dist_QUANTILE function be available in the CMPLIB= search path. If PROC SEVSELECT does not find a quantile function’s definition, it computes the quantile function by numerically inverting the cumulative distribution function (CDF). For each predefined distribution, such as the lognormal, Burr, and Weibull distributions of this example, the dist_MEAN and dist_QUANTILE functions are already defined and available to you in the function library, Sashelp.Svrtdist, that PROC SEVSELECT includes by default.

Output 23.7.6, Output 23.7.7, and Output 23.7.8 compare the estimates of the mean, median, and value-at-risk (var), respectively, of the three severity distributions for the first 10 observations that are fetched from the mycas.Score table. Note: The estimates are all available in the same table but are shown here separately for better comparison among different severity distributions. These results illustrate the following:

PROC SEVSELECT always includes the intercept in all scale regression models, so the selection process never attempts to fit a model without an intercept. In addition to the intercept, you can force other regression effects to be included in each selected model by specifying the INCLUDE= option in the SCALEMODEL statement. The following SEVSELECT step illustrates this by forcing the carSafety effect into every selected model:

proc sevselect data=mycas.losses outest=mycas.est print=all;
   loss lossAmount / lt=deductible rc=limit;
   class carType gender education;
   scalemodel carType gender carType*gender carSafety income education
              income*gender carSafety*income / include=(carSafety);
   selection method=forward(stop=aicc) hierarchy=single stophorizon=2 details=all;
   dist logn weibull;
run;

The various options in the SELECTION statement do the following:

The "Selection Information" table in Output 23.7.9 summarizes the selection settings.

The selection summary for the lognormal distribution, shown in Output 23.7.10, illustrates the following:

The selection summary for the Weibull distribution is shown in Output 23.7.11. It indicates that the selection stopped at the local minimum of the AICC statistic at step 5 because of the shorter stop horizon.

As the results in Output 23.7.10 and Output 23.7.11 show, although the SELECT= criterion is reported in the "Selection Summary" table, it does not decide the final selected model. Instead, the SELECT= criterion is used to choose the best effect to add or remove in each step. To illustrate that, Output 23.7.12 shows the details of step 1 for the Weibull distribution. PROC SEVSELECT produces the step details because of the DETAILS=ALL option in the SELECTION statement. Step 1 is the step after the initial step (step 0), which fits the model with the intercept and carSafety effect that the INCLUDE= option forces in. The "Entry Candidates" table of step 1 shows the candidates that PROC SEVSELECT evaluates for entering the model. For each candidate, it shows the value of the SELECT= criterion, which is the SBC statistic, of the model that includes that candidate. The interaction effects carType*gender and carSafety*income are not present in this list of candidates, because of the constraint that the HIERARCHY=SINGLE option imposes. The model that includes the carType effect has the smallest SBC statistic, so PROC SEVSELECT selects it in step 1. The "Optimization Summary" table shows that PROC SEVSELECT uses the trust region optimization method, which is the default optimization method for fitting a model in each step. You can use the SELECTNLOTECH= option in the PROC SEVSELECT statement to specify a different optimization method to use during each step of the selection. The "Parameter Estimates" table shows that step 1’s model includes the best entry candidate, carType, and the carSafety effect, which entered the model in step 0. The estimate of the intercept effect is reflected in the estimate of the scale parameter Theta, as described in the section Reporting Estimates of Regression Parameters.

The fit summary of the final model for the Weibull distribution is shown in Output 23.7.13. It shows that PROC SEVSELECT uses the trust region optimization method to estimate the parameters of the final model. The "Parameter Estimates" table shows that some regression parameters are not significant at the usual 95% confidence level. None of the selection criteria that PROC SEVSELECT supports depend directly on the p-values of regression effects. So a statistically insignificant effect might be included in the final model, as this example illustrates. If you do not want such effects in the final model, you can modify the selection method settings to search for a different model that contains all statistically significant effects. Alternatively, you can use your domain knowledge to decide which effects you want to retain from the selected model and then refit the model by appropriately modifying the SCALEMODEL statement.