LOGSELECT Procedure
Example 18.7 Restoring a Stored Model
(View the complete code for this example.)
The Sashelp.JunkMail data set from Example 18.3 is used here to show how to store a model and later how to restore that same model for further processing. The following program fits a model by using a SELECTION statement, then stores the model by using a STORE statement. The NOTE= option in the STORE statement stores a few notes along with the model; you might also want to store the submitted program. The DISPLAY statement suppresses all output.
data mylib.JunkMail;
set Sashelp.JunkMail;
run;
proc logselect data=mylib.JunkMail;
model Class(event='1')=Make Address All _3d Our Over Remove Internet Order
Mail Receive Will People Report Addresses Free Business Email You
Credit Your Font _000 Money HP HPL George _650 Lab Labs Telnet _857
Data _415 _85 Technology _1999 Parts PM Direct CS Meeting Original
Project RE Edu Table Conference Semicolon Paren Bracket Exclamation
Dollar Pound CapAvg CapLong CapTotal;
display / excludeall;
selection method=forward(maxstep=5);
store mylib.mymodel /
note="Generated with stepwise selection.";
run;
The following program replays the standard tables from the fitted model. You can also specify the STB, CLB, CORRB, COVB, and TYPE3 options and the CODE and ODDSRATIO statements, because they do not require access to data.
proc logselect restore=mylib.mymodel stb clb corrb covb type3;
code;
oddsratio;
run;
Information about the item store is displayed in Output 18.7.1. The Replay section is displayed in Output 18.7.2 through Output 18.7.5.
Output 18.7.1: Store Information
| Store Information | |
|---|---|
| Name | MYMODEL |
| Data Source | JUNKMAIL |
| Data Libname | <<your library>> |
| Procedure | LOGSELECT |
| Action | regression.logistic |
| Action Version | 1.0.6 |
| Date | 19May2023:14:02:33 |
| Generated with stepwise selection. |
The standard tables from PROC LOGSELECT are displayed next, beginning with the "Model Information" table in Output 18.7.2.
Output 18.7.2: Replay Section
| Model Information | |
|---|---|
| Response Variable | Class |
| Distribution | Binary |
| Link Function | Logit |
| Optimization Technique | Newton-Raphson with Ridging |
| Number of Observations Read | 4601 |
|---|---|
| Number of Observations Used | 4601 |
| Response Profile | ||
|---|---|---|
| Ordered Value | Class | Total Frequency |
| 1 | 0 | 2788 |
| 2 | 1 | 1813 |
| Probability modeled is Class = 1. |
Output 18.7.3: Replay Section (continued)
| Convergence criterion (ABSGCONV=1E-7) satisfied. |
| Dimensions | |
|---|---|
| Columns in Design | 6 |
| Number of Effects | 6 |
| Max Effect Columns | 1 |
| Rank of Design | 6 |
| Parameters in Optimization | 6 |
| Testing Global Null Hypothesis: BETA=0 | |||
|---|---|---|---|
| Test | DF | Chi-Square | Pr > ChiSq |
| Likelihood Ratio | 5 | 2677.9370 | <.0001 |
The fit statistics shown in Output 18.7.4 are those that are computed when the original model is fit. If you specify the PARTFIT option or a PARTITION statement when the original model is fit, then this table always includes the additional statistics. However, specifying a PARTFIT option when you restore the model does not include the additional statistics in this table in the Replay section. Instead, the same table that appears in the Restore section is expanded; see Output 18.7.8.
Output 18.7.4: Replay Section (continued)
| Fit Statistics | |
|---|---|
| -2 Log Likelihood | 3492.21579 |
| AIC (smaller is better) | 3504.21579 |
| AICC (smaller is better) | 3504.23408 |
| SBC (smaller is better) | 3542.81997 |
| Parameter Estimates | ||||||||
|---|---|---|---|---|---|---|---|---|
| Parameter | DF | Estimate | Standard Error | Chi-Square | Pr > ChiSq | 95% Confidence Limits | Standardized Estimate | |
| Intercept | 1 | -2.007045 | 0.057444 | 1220.7512 | <.0001 | -2.11963 | -1.89446 | |
| Remove | 1 | 4.674731 | 0.371458 | 158.3777 | <.0001 | 3.94669 | 5.40278 | 1.008867 |
| Free | 1 | 1.057249 | 0.093489 | 127.8902 | <.0001 | 0.87401 | 1.24048 | 0.481347 |
| Your | 1 | 0.465818 | 0.035907 | 168.2990 | <.0001 | 0.39544 | 0.53619 | 0.308391 |
| _000 | 1 | 4.582674 | 0.461809 | 98.4719 | <.0001 | 3.67754 | 5.48780 | 0.885020 |
| Dollar | 1 | 8.208586 | 0.631423 | 169.0034 | <.0001 | 6.97102 | 9.44615 | 1.112771 |
Output 18.7.5: Replay Section (continued)
| Parameter Estimates Covariance Matrix | ||||||
|---|---|---|---|---|---|---|
| Parameter | Intercept | Remove | Free | Your | _000 | Dollar |
| Intercept | 0.003300 | -0.00318 | -0.00128 | -0.00094 | -0.00300 | -0.00847 |
| Remove | -0.00318 | 0.1380 | -0.00136 | -0.00089 | -0.00279 | -0.00730 |
| Free | -0.00128 | -0.00136 | 0.008740 | -0.00024 | 0.000223 | -0.00156 |
| Your | -0.00094 | -0.00089 | -0.00024 | 0.001289 | -0.00025 | -0.00225 |
| _000 | -0.00300 | -0.00279 | 0.000223 | -0.00025 | 0.2133 | -0.05709 |
| Dollar | -0.00847 | -0.00730 | -0.00156 | -0.00225 | -0.05709 | 0.3987 |
| Parameter Estimates Correlation Matrix | ||||||
|---|---|---|---|---|---|---|
| Parameter | Intercept | Remove | Free | Your | _000 | Dollar |
| Intercept | 1.0000 | -0.1488 | -0.2393 | -0.4550 | -0.1131 | -0.2335 |
| Remove | -0.1488 | 1.0000 | -0.0393 | -0.0667 | -0.0163 | -0.0311 |
| Free | -0.2393 | -0.0393 | 1.0000 | -0.0706 | 0.0052 | -0.0264 |
| Your | -0.4550 | -0.0667 | -0.0706 | 1.0000 | -0.0149 | -0.0994 |
| _000 | -0.1131 | -0.0163 | 0.0052 | -0.0149 | 1.0000 | -0.1958 |
| Dollar | -0.2335 | -0.0311 | -0.0264 | -0.0994 | -0.1958 | 1.0000 |
Output 18.7.6: Replay Section (continued)
| Model Analysis of Variance (Type III) | |||
|---|---|---|---|
| Effect | DF | Chi-Square | Pr > ChiSq |
| Remove | 1 | 158.3777 | <.0001 |
| Free | 1 | 127.8902 | <.0001 |
| Your | 1 | 168.2990 | <.0001 |
| _000 | 1 | 98.4719 | <.0001 |
| Dollar | 1 | 169.0034 | <.0001 |
| Odds Ratios | |||||
|---|---|---|---|---|---|
| Description | Estimate | 95% Wald Confidence Limits | p-Value | Percent Change | |
| _000 | 97.776 | 39.549 | 241.726 | <.0001 | 9677.55 |
| Dollar | >999.999 | >999.999 | >999.999 | <.0001 | 367134.6 |
| Free | 2.878 | 2.397 | 3.457 | <.0001 | 187.84 |
| Remove | 107.204 | 51.764 | 222.022 | <.0001 | 10620.38 |
| Your | 1.593 | 1.485 | 1.709 | <.0001 | 59.33 |
Output 18.7.7: Replay Timing
| Task Timing | ||
|---|---|---|
| Task | Seconds | Percent |
| Model Restoration | 0.00 | 0.30% |
| Setup and Parsing | 0.10 | 37.06% |
| Producing Score Code | 0.00 | 1.27% |
| Type III Sums of Squares | 0.00 | 0.99% |
| Display | 0.00 | 1.70% |
| Cleanup | 0.00 | 0.00% |
| Total | 0.27 | 100.00% |
If you specify the CTABLE, LACKFIT, ASSOCIATION, or PARTFIT option or the OUTPUT statement, then you must also specify an input data table. The following program requests these tables; the Replay section tables are produced but not displayed here.
proc logselect restore=mylib.mymodel data=mylib.JunkMail
association ctable(cutpt=0.1 to 0.9 by 0.1) lackfit partfit;
output out=mylib.out p cbar;
proc print data=mylib.out(obs=1);
run;
The first table that is displayed in the Restore section (Output 18.7.8) is the "Postfit Information" table, which contains the name of the input data table and information usually displayed in an "NObs" table. The next two tables are the "Response Profile" and "Fit Statistics" tables, which are computed from the JunkMail data.
Output 18.7.8: Restore Section
| Postfit Information | |
|---|---|
| Postfit Data Source | JUNKMAIL |
| Number of Observations Read | 4601 |
| Number of Observations Used | 4601 |
| Response Profile | ||
|---|---|---|
| Ordered Value | Class | Total Frequency |
| 1 | 0 | 2788 |
| 2 | 1 | 1813 |
| Probability modeled is Class = 1. |
| Fit Statistics | |
|---|---|
| -2 Log Likelihood | 3492.21579 |
| AIC (smaller is better) | 3504.21579 |
| AICC (smaller is better) | 3504.23408 |
| SBC (smaller is better) | 3542.81997 |
| Average Square Error | 0.10819 |
| -2 Log L (Intercept-only) | 6170.15284 |
| R-Square | 0.44124 |
| Max-rescaled R-Square | 0.59754 |
| McFadden's R-Square | 0.43401 |
| Misclassification Rate | 0.14606 |
| Difference of Means | 0.52695 |
The "Association" and "Classification" tables in Output 18.7.9 also provide information that is computed from the input data table.
Output 18.7.9: Restore Section (continued)
| Association of Predicted Probabilities and Observed Responses | |
|---|---|
| Concordance Index (AUC) | 0.9125 |
| Somers' D | 0.8250 |
| Gamma | 0.8589 |
| Tau-a | 0.3940 |
| Pairs | 5054644 |
| Percent Concordant | 89.2764 |
| Percent Discordant | 6.7789 |
| Percent Tied | 3.9447 |
| Classification Table | ||||
|---|---|---|---|---|
| Probability Level | Correct | Incorrect | ||
| Event | Nonevent | Event | Nonevent | |
| 0.1000 | 1813 | 0 | 2788 | 0 |
| 0.2000 | 1613 | 2230 | 558 | 200 |
| 0.3000 | 1483 | 2491 | 297 | 330 |
| 0.4000 | 1363 | 2598 | 190 | 450 |
| 0.5000 | 1289 | 2640 | 148 | 524 |
| 0.6000 | 1213 | 2675 | 113 | 600 |
| 0.7000 | 1132 | 2696 | 92 | 681 |
| 0.8000 | 1029 | 2722 | 66 | 784 |
| 0.9000 | 858 | 2750 | 38 | 955 |
The two Hosmer-Lemeshow tables whose content is computed from the input data are shown in Output 18.7.10.
Output 18.7.10: Restore Section (continued)
| Partition for the Hosmer and Lemeshow Test | |||||
|---|---|---|---|---|---|
| Group | Total | Observed 1 | Expected 1 | Observed 0 | Expected 0 |
| 1 | 1763 | 121 | 208.85 | 1642 | 1554.15 |
| 2 | 461 | 45 | 68.63 | 416 | 392.37 |
| 3 | 460 | 110 | 95.92 | 350 | 364.08 |
| 4 | 460 | 235 | 158.10 | 225 | 301.90 |
| 5 | 461 | 355 | 315.79 | 106 | 145.21 |
| 6 | 460 | 429 | 431.12 | 31 | 28.88 |
| 7 | 536 | 518 | 534.59 | 18 | 1.41 |
| Hosmer and Lemeshow Goodness-of-Fit Test | ||
|---|---|---|
| Chi-Square | DF | Pr > ChiSq |
| 322.2997 | 5 | <.0001 |
The first line of the output data table is displayed in Output 18.7.11. Note that the CBAR statistic is not produced because the computation requires the original input data table.
Output 18.7.11: First Observation from Output Data Table
| Obs | _PRED_ |
|---|---|
| 1 | 0.22767 |
You can specify the FITDATA option to indicate that the DATA= table is the original input table so that computation of the CBAR statistic is valid. The first observation in the output data table is shown in Output 18.7.12 and displays the CBAR statistic.
proc logselect restore=mylib.mymodel data=mylib.JunkMail fitdata;
output out=mylib.out p cbar;
proc print data=mylib.out(obs=1);
run;
Output 18.7.12: First Observation from Output Data Table with FITDATA Option
| Obs | _PRED_ | _CBAR_ |
|---|---|---|
| 1 | 0.22767 | .001559475 |