Bulk Loading and Bulk Unloading for Google BigQuery

Options to Use with Bulk Loading and Unloading

Bulk loading is the fastest way to insert large numbers of rows into a Google BigQuery table. You must enable either the BULKLOAD= LIBNAME option or the BULKLOAD= data set option in order to use the bulk-load interface. The Google BigQuery bulk-load interface moves data from SAS to the Google BigQuery database.

Similarly, the fastest way to retrieve a large number of rows from Google BigQuery is to use bulk unloading. You must enable either the BULKUNLOAD= LIBNAME option or the BULKUNLOAD= data set option to use the Google BigQuery Extractor API.

Here are the Google BigQuery bulk-load and bulk-unload options.

• BL_BUCKET= LIBNAME option and data set option
• BL_DEFAULT_DIR= LIBNAME option and data set option
• BL_DELETE_DATAFILE= LIBNAME option and data set option
• BL_DELIMITER= LIBNAME option and data set option
• BL_NUM_READ_THREADS= LIBNAME option and data set option
• BULKLOAD= LIBNAME option and data set option
• BULKUNLOAD= BULKUNLOAD= LIBNAME Statement Option LIBNAME option and BULKUNLOAD= Data Set Option data set option

Examples

This example shows how you can use a SAS data set, Sasflt.Flt98, to create and load a large Google BigQuery table, net_air.flights98.

libname sasflt 'SAS-library';
libname net_air bigquery project='Project1' schema='USdata'
            cred_path='/u/fedadmin/BigQuery/xxx-yyy-9e99c10a8888.json';

proc sql;
create table net_air.flights98(bulkload=YES)
      as select * from sasflt.flt98;
quit;

Bulk Loading with PROC FEDSQL and PROC DS2

Any bulk-load options that you specify in a SAS/ACCESS LIBNAME statement are automatically included in the connection string that is passed from PROC FEDSQL or PROC DS2. You can override these values by specifying table options within these procedures.