The MULTTEST Procedure

Example 80.2 Freeman-Tukey and t Tests with Bootstrap Resampling

(View the complete code for this example.)

The data for this example are the same as for Example 80.1, except that a continuous variable T, which indicates the time of death of the animal, has been added.

data a;
   input S1 S2 T Dose @@;
   datalines;
0 1 104 1   1 0  80 1   0 1 104 1
0 1 104 1   0 1 100 1   1 0 104 1
1 0  85 2   1 0  60 2   0 1  89 2
1 0  96 2   0 1  96 2   1 0  99 2
1 0  60 3   1 0  50 3   1 0  80 3
0 1  98 3   0 1  99 3   1 0  50 3
;
proc multtest data=a bootstrap nsample=10000 seed=37081 outsamp=res;
   test ft(S1 S2 / lowertailed) mean(T / lowertailed);
   class Dose;
   contrast 'Linear Trend' 0 1 2;
run;
proc print data=res(obs=36);
run;

The BOOTSTRAP option in the PROC MULTTEST statement requests bootstrap resampling, and NSAMPLE=10000 requests 10,000 bootstrap samples. The SEED=37081 option provides a starting value for the random number generator. The OUTSAMP=res option creates an output SAS data set res containing the 10,000 bootstrap samples.

The TEST statement specifies the Freeman-Tukey test for S1 and S2 and specifies the t test for T. Both tests are lower-tailed. The grouping variable in the CLASS statement is Dose, and the coefficients across the levels of Dose are 0, 1, and 2, as specified in the CONTRAST statement. The PROC PRINT statement displays the first 36 observations of the res data set containing the bootstrap samples.

The results from this analysis are listed in Output 80.2.1 through Output 80.2.5.

The "Model Information" table in Output 80.2.1 corresponds to the specifications in the invocation of PROC MULTTEST.

Output 80.2.1: FT and t tests with Bootstrap Resampling

The Multtest Procedure

Model Information
Test for discrete variablesFreeman-Tukey
Test for continuous variablesMean t-test
Degrees of Freedom MethodPooled
Tails for discrete testsLower-tailed
Tails for continuous testsLower-tailed
Strata weightsNone
P-value adjustmentBootstrap
Center continuous variablesYes
Number of resamples10000
Seed37081


The "Contrast Coefficients" table in Output 80.2.2 shows the coefficients from the CONTRAST statement after centering, and they model a linear trend.

Output 80.2.2: Contrast Coefficients

Contrast Coefficients
Contrast Dose
123
Linear TrendCentered-101


The summary statistics are displayed in Output 80.2.3. The values for the discrete variables S1 and S2 are the same as those from Example 80.1. The mean, standard deviation, and sample size for the continuous variable T at each level of Dose are displayed in the "Continuous Variable Tabulations" table.

Output 80.2.3: Summary Statistics

Discrete Variable Tabulations
VariableDoseCountNumObsPercent
S112633.33
S124666.67
S134666.67
S214666.67
S222633.33
S232633.33

Continuous Variable Tabulations
VariableDoseNumObsMeanStandard Deviation
T1699.33339.6056
T2687.500014.4326
T3672.833322.7017


The p-values, displayed in Output 80.2.4, are from the Freeman-Tukey test for S1 and S2, and are from the t test for T.

Output 80.2.4: p-Values

p-Values
VariableContrastRawBootstrap
S1Linear Trend0.85471.0000
S2Linear Trend0.14530.4605
TLinear Trend0.00700.0281


The Raw column in Output 80.2.4 contains the results from the tests on the original data, while the Bootstrap column contains the bootstrap resampled adjustment to raw_p. Note that the adjusted p-values are larger than the raw p-values for all three variables. The adjusted p-values more accurately reflect the correlation of the raw p-values, the small size of the data, and the multiple testing.

Output 80.2.5 displays the first 36 observations of the SAS data set resulting from the OUTSAMP=RES option in the PROC MULTTEST statement. The entire data set has 180,000 observations, which is 10,000 times the number of observations in the data set.

Output 80.2.5: Resampling Data Set

Obs_sample__class__obs_S1S2T
111170126.1667
211810-27.5000
3115010.6667
4119011.5000
511710-2.5000
6113014.6667
712121011.5000
812121011.5000
9121410-22.8333
1012170126.1667
11121014.6667
121215107.1667
13134014.6667
1413170126.1667
15131410-22.8333
161315107.1667
171315107.1667
18136104.6667
19216104.6667
2021170126.1667
2121810-27.5000
22211310-12.8333
23219011.5000
2421810-27.5000
25229011.5000
26221810-22.8333
272215107.1667
28221410-22.8333
29229011.5000
3022170126.1667
3123160125.1667
322311018.5000
33231410-22.8333
34231810-22.8333
35231810-22.8333
362310108.5000


The _sample_ variable is the sample indicator and _class_ indicates the resampling group—that is, the level of the CLASS variable Dose assigned to the new observation. The number of the observation in the original data set is represented by _obs_. Also listed are the values of the original test variables, S1 and S2, and the mean-centered values of T.