Kernel Principal Component Analysis Action Set

Letter Classification

This section contains PROC CAS code.

Note: Input data must be accessible in your CAS session, either as a CAS table or as a transient-scope table. A CAS table has a two-level name: the first level is your CAS engine libref, and the second level is the table name. You refer to this table in the CAS procedure by specifying only the second level. For more information about two-level names, see Chapter 2, Shared Concepts (SAS Visual Data Mining and Machine Learning: Procedures). A transient-scope table is called directly from the action and exists in memory for the duration of the action. For more information about accessing data, see SAS Viya: System Programming Guide. For more information about PROC CAS and programming in CASL, see SAS Cloud Analytic Services: CASL Programmer’s Guide and SAS Cloud Analytic Services: CASL Reference.

In this example, the kPca action is used as a preprocessor to the linear discriminant model of letter recognition.

The letter recognition data set that this example uses comes from the UCI Machine Learning Repository (Dua and Graff 2019). The data set includes a large number of black-and-white pixel images of rectangular shape, each of them corresponding to one of the 26 capital letters in the English alphabet. These letter pictures use 20 different fonts; each was randomly altered to generate a total of 20,000 unique instances. Each instance was then transformed into 16 statistical attributes (edge counts and statistical moments). All these attributes were further scaled to a range of integer values from 0 to 15 (Frey and Slate 1991). The objective is to classify each pixel image as one of the 26 capital letters. In this example, a training set of size 16,000 is randomly selected, and the remaining 4,000 instances are used as a test set. Because this is a big data set, fast KPCA is a more appropriate method to use for training while still achieving performance comparable to that of the exact method. To apply fast KPCA, 200 centroids are selected. After the model is trained, the projection values of training and scoring data onto the kernel principal components undergo a multilabel linear discriminant analysis (LDA) for classification.

In the following code, the kPca action applies both exact and fast KPCA to the data table mycas.letter_train. The statements assume that your CAS engine libref is named mycas, but you can substitute any appropriately defined CAS engine libref. Next, the PROC DISCRIM statement applies linear discriminant analysis to the projections onto the principal component directions. Here letter_train and letter_test are the names of the training data and test data, respectively.

   data letter;
infile datalines delimiter=',';
input capital $ var1-var16;
datalines;
T,2,8,3,5,1,8,13,0,6,6,10,8,0,8,0,8

   ... more lines ...


data letter;
   set letter;
   obsid=_n_;
run;

/* split data set to train (16000) and test(4000) */
proc surveyselect data=letter
   method=srs n=16000
   seed=100 out=letterTrain;
run;

proc sql;
   create table letterTest as
   select * from letter
   where obsid not in (select obsid from letterTrain);
quit;

/* load data sets to CAS server */
data mycas.letterTrain; set letterTrain; run;
data mycas.letterTest; set letterTest; run;

/* apply exact KPCA to training data set, extract 200 principal components*/
proc cas;
   loadactionset "KernelPCA";
   run;
   action kPca result = r/
       table  = {name='letterTrain'},
       output ={casout={name="score", replace=TRUE}, copyvars={"capital"}, npc=200},
       saveState={name="STATE", replace=TRUE},
       input=${var1-var16},
                method = "EXACT",
                kerType="RBF",
                kerParam=7.071;
run;
print r;
quit;

/* score test data */
proc cas  ;
 action aStore.score /
        table={name='letterTest'},
        out={name='outscore', replace=true},
        rstore={name='state'},
        copyVars={'capital'};
run;
quit;


/* apply linear discriminant analysis and get multilabel classification error */
proc discrim data=mycas.score method=normal pool=yes short testdata=mycas.outscore;
 class capital;
 testclass capital;
run;

/* apply fast KPCA to training data, extract 200 principal components */
proc cas;
    loadactionset "KernelPCA";
    run;
    action kPca result = r/
        table  = {name='letterTrain'},
  output ={casout={name="score_fast", replace=TRUE}, copyvars={"capital"}, npc=200},
  saveState={name="state_fast", replace=TRUE},
        input=${var1-var16},
  method = "APPROXIMATE",
        clusMethod="KMPP",
        clusRandseed=1234,
        maxclus=200,
        kerType="RBF",
        kerParam=7.071;
run;
print r;
quit;


/* fast score test data */
proc cas  ;
 action aStore.score /
        table={name='letterTest'},
  out={name='outscore_fast', replace=true},
  rstore={name='state_fast'},
   copyVars={'capital'};
  run;
  quit;

/* apply linear discriminant analysis and get multilabel classification error */
proc discrim data=mycas.score_fast method=normal pool=yes short
 testdata=mycas.outscore_fast;
 class capital;
 testclass capital;
run;

In this example, the multilabel classification errors that are obtained by fast KPCA and exact KPCA are close: 0.1662 for fast KPCA and 0.1593 for exact KPCA. However, the training time for fast KPCA is only 8.88 seconds, compared to 256.01 seconds for exact KPCA, as shown in the "Task Timing" tables in Output 21.2.1 and Output 21.2.2, respectively. The example demonstrates the efficiency of fast KPCA in significantly reducing the run time while not compromising much on the quality of the principal components that it generates. Moreover, in exact KPCA all the training data, along with the eigenvector matrix, must be stored in the state file for scoring the test data, whereas in fast KPCA only the k-means centroids and eigenvector matrix must be stored in the state file. This greatly reduces the space that is required for scoring new data.

Output 21.2.1: Timing Table—Fast KPCA

Task Timing
Task	Seconds	Percent
Kernel Matrix Construction	0.08	1.50%
k-means Clustering	5.39	96.66%
Eigendecomposition	0.02	0.27%
Score Training Data	0.02	0.32%
Other	0.07	1.25%
Total	5.57	100.00%

Output 21.2.2: Timing Table—Exact KPCA

Task Timing
Task	Seconds	Percent
Kernel Matrix Construction	10.41	5.52%
Eigendecomposition	178.00	94.43%
Score Training Data	0.02	0.01%
Other	0.08	0.04%
Total	188.50	100.00%

You can use the following code to also apply PCA (equivalent to KPCA with linear kernel) to the training data and extract all 16 components to use in linear discriminant analysis:


/* apply PCA to training data, extract all 16 principal components */
proc cas;
loadactionset "KernelPCA";
run;
action kPca result = r/
    table  = {name='letterTrain'},
  output ={casout={name="score_pca", replace=TRUE}, copyvars={"capital"}, npc=16},
 saveState={name="state_pca", replace=TRUE},
 input=${var1-var16},
 method = "EXACT",
 kerType="LINEAR",
 kerParam=7.071;
run;
print r;
quit;


/* score test data set with PCA */
proc cas;
 action aStore.score /
        table={name='letterTest'},
  out={name='outscore_pca', replace=true},
  rstore={name='state_pca'},
        copyVars={'capital'};
run;
quit;

/* apply linear discriminant analysis and get multilabel classification error */
proc discrim data=mycas.score_pca method=normal pool=yes short
 testdata=mycas.outscore_pca;
 class capital;
 testclass capital;
run;

Letter Classification

This section contains Lua code for the analysis in the CASL version of this example, which contains details about the results.

Note: In order to run this code, the data that are described in the CASL version need to be accessible to the CAS server. One way to do this is to convert the letterTrain data to the comma-separated-value (CSV) file letterTrain.csv, convert the letterTest data to the CSV file letterTest.csv, and then use the following code to load the CSV files into CAS:

s:loadtable{casLib="casuser", path="letterTrain.csv"}
s:loadtable{casLib="casuser", path="letterTest.csv"}

For more information about coding in Lua, see Getting Started with SAS Viya for Lua and SAS Viya: System Programming Guide.

-- apply exact KPCA to training data, extract 200 principal components --
res = s:kernelPca_kPca{table={name='letterTrain'},
                            method="EXACT",
                            kerParam= 7.071,
                            kerType="RBF",
                            inputs={{name="variable1"},{name="variable2"},
                                    {name="variable3"},{name="variable4"},
                                    {name="variable5"},{name="variable6"},
                                    {name="variable7"},{name="variable8"},
                                    {name="variable9"},{name="variable10"},
                                    {name="variable11"},{name="variable12"},
                                    {name="variable13"},{name="variable14"},
                                    {name="variable15"},{name="variable16"}},
                                    output={casout={name="score", replace=true},
                                            copyVars={"capital"}, npc=200},
                                    savestate={name = "state"}
}
print (res)
r=s:fetch{table={name="score"},to=10}
print(r)

-- score test data --
res = s:aStore_score{table={name='letterTest'},
                            out={name='outsocre',
                            compress=false,
                            replace=true,
                            replication=1,
                            promote=false},
                            rstore={name='state'}
}
print (res)
r=s:fetch{table={name="outscore"},to=10}
print (r)

-- apply fast KPCA to training data, extract 200 principal components --
res = s:kernelPca_kPca{table={name='letterTrain'},
                                    method="APPROXIMATE",
                                    kerParam= 7.071,
                                    kerType="RBF",
                                    inputs={{name="variable1"},{name="variable2"},
                                            {name="variable3"},{name="variable4"},
                                            {name="variable5"},{name="variable6"},
                                            {name="variable7"},{name="variable8"},
                                            {name="variable9"},{name="variable10"},
                                            {name="variable11"},{name="variable12"},
                                            {name="variable13"},{name="variable14"},
                                            {name="variable15"},{name="variable16"}},
                                    clusMethod="KMPP",
                                    clusRandseed=1234,
                                    maxclus=200,
                                    output={casout={name="score_fast", replace=true},
                                            copyVars={"capital"}, npc=200},
                                    savestate={name = 'state_fast'}
}
print (res)
r=s:fetch{table={name="score_fast"},to=10}
print(r)

-- fast score test data --
res = s:aStore_score{table={name='letterTest'},
                                    out={name='outscore_fast',
                                    compress=false,
                                    replace=true,
                                    replication=1,
                                    promote=false},
                                    rstore={name='state_fast'}
}

print (res)
r=s:fetch{table={name="outscore_fast"},to=10}
print (r)

-- [[ apply KPCA with linear kernel (PCA) to training data,
extract all 16 principal components ]]
res = s:kernelPca_kPca{table={name="letterTrain"},
                                    method="EXACT",
                                    kerParam= 7.071,
                                    kerType="LINEAR",
                                    inputs={{name="variable1"},{name="variable2"},
                                            {name="variable3"},{name="variable4"},
                                            {name="variable5"},{name="variable6"},
                                            {name="variable7"},{name="variable8"},
                                            {name="variable9"},{name="variable10"},
                                            {name="variable11"},{name="variable12"},
                                            {name="variable13"},{name="variable14"},
                                            {name="variable15"},{name="variable16"}},
                                    output={casout={name="score_pca", replace=true},
                                            copyVars={"capital"}, npc=16},
                                    savestate={name = "state_pca"}
}

print (res)
r=s:fetch{table={name="train_score"},to=10}
print(r)

-- score test data with PCA results--
re = s:aStore_score{table={name='letterTest'},
                                out={name='outscore_pca',
                                compress=false,
                                replace=true,
                                replication=1,
                                promote=false},
                                rstore={name='state_pca'}
}

print (result)
r=s:fetch{table={name="outscore_pca"},to=10}
print (r)

Letter Classification

This section contains Python code for the analysis in the CASL version of this example, which contains details about the results.

s.upload_file('letterTrain.csv')
s.upload_file('letterTest.csv')

For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.

s.loadactionset('kernelPca')

# generate variable names
index=range(1,17)
var_names=['variable'+str(i) for i in index]


# apply exact KPCA to training data, extract 200 principal components
res = s.kpca.kPca(inputs= var_names
         ,output={'casout':{'name':'score', 'replace':True},
                  'copyVars':["capital"], 'npc':200}
         ,table=table('letterTrain')
         ,method="EXACT"
         ,savestate = {"name": "state"}
         ,kerType="RBF"
         ,kerParam=7.071)

print(res)


score = s.fetch(table = {"name":"train_score"}, to =10)
print (score)


# call aStore action for scoring
r = s.loadactionset(actionset='astore')
res = s.aStore.score(out = {"name":"outscore","replace":True}
            ,rstore = {"name":"state"}
            ,table=table('letterTest')
            ,copyVars= {"name":"capital"})
print (res)

outscore = s.fetch(table = {"name":"outscore"}, to =10)
print (outscore)

# apply fast KPCA to training data, extract 200 principal components
res = s.kpca.kPca(inputs= var_names
         ,output={'casout':{'name':'score_fast', 'replace':True},
                  'copyVars':["capital"], 'npc':200}
         ,table=table('letterTrain')
         ,method="APPROXIMATE"
         ,savestate = {"name": "state_fast"}
         ,clusMethod="KMPP"
         ,maxclus=200
         ,kerType="RBF"
         ,kerParam=7.071)

print(res)


score_fast = s.fetch(table = {"name":"score_fast"}, to =10)
print (score_fast)


# call aStore action for fast scoring
r = s.loadactionset(actionset='astore')
res = s.aStore.score(out = {"name":"outscore_fast","replace":True}
            ,rstore = {"name":"state_fast"}
            ,table=table('letterTest')
            ,copyVars= {"name":"capital"})
print (res)

outscore_fast = s.fetch(table = {"name":"outscore_fast"}, to =10)
print (outscore_fast)

# apply PCA to training data, extract all 16 principal components
res = s.kpca.kPca(inputs= var_names
        ,output={'casout':{'name':'score_pca', 'replace':True},
                 'copyVars':["capital"], 'npc':16}
        ,table=table('letterTrain')
        ,method="EXACT"
        ,savestate = {"name": "state_pca"}
        ,kerType="LINEAR"
        ,kerParam=7.071)

print(res)


score_pca = s.fetch(table = {"name":"score_pca"}, to =10)
print (score_pca)


# call aStore action for PCA scoring
r = s.loadactionset(actionset='astore')
res = s.aStore.score(out = {"name":"outscore_pca","replace":True}
           ,rstore = {"name":"state_pca"}
           ,table=table('letterTest')
           ,copyVars= {"name":"capital"})
print (res)

outscore_pca = s.fetch(table = {"name":"outscore_pca"}, to =10)
print (outscore_pca)

Letter Classification

This section contains R code for the analysis in the CASL version of this example, which contains details about the results.

m <- cas.read.csv(s, "letterTrain.csv", casOut=list(name="letterTrain"))
m <- cas.read.csv(s, "letterTest.csv", casOut=list(name="letterTest"))

For more information about coding in R, see Getting Started with SAS Viya for R and SAS Viya: System Programming Guide.

The SAS Scripting Wrapper for Analytics Transfer (SWAT) is an R package that enables you to interface with SAS Cloud Analytics Services (CAS). You can use the SWAT package to write R code to connect to a CAS server and analyze data. For more information about SWAT package; see Getting Started with SAS Viya for R.

The following code assumes that the training data set and the scoring data set are uploaded to CAS tables using appropriate functions available in the SWAT package.

library("swat")
loadActionSet(s, "kernelPca")

#generate variable names
var_names<-paste0("variable", 1:16)

#apply exact KPCA to training data, extract 200 principal components
results <- cas.kernelPca.kPca(s,kerParam = 7.071,
                              method="EXACT",
                              kerType="RBF",
                              inputs = var_names,
                              output=list(casOut=list(name="score",replace=TRUE),
                                       copyVars=list("capital"),npc=200),
                              savestate = list(name = "state",replace= TRUE),
                              table=list(name="letterTrain"))
print(results)
score <- cas.table.fetch(s, table = list(name = "score"), to = 10)
score_df = score$Fetch

#score test data set
loadActionSet(s, "astore")
results <- cas.astore.score (s, out = list(name = "outscore", replace = TRUE),
                             rstore = "state", table = "letterTest")

outscore <- cas.table.fetch(s, table = list(name = "outscore"), to = 10)
outscore_df = outscore$Fetch

#apply fast KPCA to training dataset, extract 200 principal components
results <- cas.kernelPca.kPca(s,kerParam = 7.071,
                              method="APPROXIMATE",
                              kerType="RBF",
                              inputs = var_names,
                              output=list(casOut=list(name="score_fast",replace=TRUE),
                                       copyVars=list("capital"),npc=200),
                              clusMethod="KMPP",
                              clusRandseed=1234,
                              maxclus=200,
                              savestate = list(name = "state_fast",replace= TRUE),
                              table=list(name="letterTrain"))
print(results)

score_fast <- cas.table.fetch(s, table = list(name = "score_fast"), to = 10)
score_fast_df = score_fast$Fetch

#fast score test data set
loadActionSet(s, "astore")

results <- cas.astore.score (s, out = list(name = "outscore_fast", replace = TRUE),
                             rstore = "state_fast", table = "letterTest")
test_score <- cas.table.fetch(s, table = list(name = "outscore_fast"), to = 10)
outscore_fast_df = outscore_fast$Fetch

#apply KPCA with linear kernel to training data, extract all 16 principal components
results <- cas.kernelPca.kPca(s,
                              method="EXACT",
                              kerType="LINEAR",
                              inputs = var_names,
                              output=list(casOut=list(name="score_pca",replace=TRUE),
                                       copyVars=list("capital"),npc=16),
                              savestate = list(name = "state_pca",replace= TRUE),
                              table=list(name="letterTrain"))

print(results)

score_pca <- cas.table.fetch(s, table = list(name = "score_pca"), to = 10)
score_pca_df = score_pca$Fetch

#score test data set with PCA results
loadActionSet(s, "astore")
results <- cas.astore.score (s, out = list(name = "outscore_pca", replace = TRUE),
                             rstore = "state_pca", table = "letterTest")

outscore_pca <- cas.table.fetch(s, table = list(name = "outscore_pca"), to = 10)
outscore_pca_df = outscore_pca_df$Fetch

Last updated: September 10, 2021