LOGSELECT Procedure

(View the complete code for this example.)

This example uses data from a cancer study that attempts to determine whether a patient has type 1 leukemia (acute lymphoblastic leukemia) or type 2 leukemia (acute myeloid leukemia) on the basis of the patient’s gene expression measurements (Golub et al. 1999; Zou and Hastie 2005). The data sets, which are specified in the following DATA steps, contain 38 observations for training and 34 observations for testing prediction accuracy. Each observation contains 7,129 continuous variables that are gene expression measurements. The response variable is a binary indicator that is coded as 1 for type 1 leukemia and –1 for type 2 leukemia. An additional role variable is added to the training data set so that a subset of training observations are used for computing the validation error.

data LeuTrain;
   set Sashelp.LeuTrain;
   call streamInit(123);
   if rand('UNIFORM')<0.6 then role='train'; else role='valid';
run;
data mylib.LeuTrain;
   set LeuTrain;
run;
data mylib.LeuTest;
   set Sashelp.LeuTest;
run;

This study tries to find genes that can help identify the leukemia type most effectively. Because the number of variables is much greater than the number of observations, and because many measurements are highly correlated, model selection methods that are based on maximum likelihood estimation are not suitable choices. In this case, penalized selection methods provide solutions that can achieve a better trade-off between bias and variance. In particular, elastic net selection inherits advantages from both LASSO selection, which can return sparse models, and ridge regression, which can handle wide data sets that contain highly correlated variables. The following program builds a predictive model by using the SELECTION statement with the elastic net method, then stores the model by using the STORE statement. The SELECTION statement uses the average square error that is computed from the validation data as the criterion to determine the best candidate. The PARTITION statement uses the role variable to divide the training data so that approximately 40% of the observations are used for validation.

proc logselect data=mylib.LeuTrain partfit;
   model y=x1-x7129;
   selection method=elasticnet(choose=validate);
   partition role=role(train='train' validate='valid');
   store mylib.ElasticNetModel / note="Microarray data with Elastic Net.";
run;

The "Selection Information" table in Output 18.8.1 displays information about the selection process. The maximum lambda value is the upper bound of the regularization parameter for the LASSO penalty such that there can be at least one nonzero regression parameter estimate before lambda reaches that value and all estimates reduce to zero when lambda increases further. There are a total of 20 regularization parameters that elastic net selection uses to fit candidate models. By default, elastic net uses a balanced LASSO and ridge penalty.

The "Selected Effects" table in Output 18.8.2 shows the genes that are selected from the total of 7,129 genes by the elastic net selection method.

Because the data for assessing test error are in a separate data set, you can compute the assessment statistics by using the stored model in several ways. One way is to use the RESTORE option in the PROC LOGSELECT statement to load the selected model and compute values from the new data. The following statements restore the selected model mylib.ElasticNetModel and compute the test error from the test data set mylib.LeuTest. Output 18.8.3 displays the fit statistics, including the misclassification rate. Compared to other selection methods—including LASSO, which returns models that have the number of selected variables only up to the training sample size—elastic net selection can select more variables and thus provide better prediction accuracy.

proc logselect restore=mylib.ElasticNetModel data=mylib.LeuTest partfit;
run;