DEEPPRICE Procedure

This example demonstrates the effect of functional form misspecification on the estimated demand curve and shows how to compare demand curves across customer characteristics and purchasing time.

The data set has 10,000 simulated observations, which include customer type, customer purchase time, price, and demand. The treatment variable, price, is the selling price of a product. The outcome variable, demand, is the sales quantity of the product. Table 4 describes the variables that are used in the model. The type can be T (treatment), Y (outcome), (a covariate in the treatment model), and/or (a covariate in the outcome model).

The following statements create the data set and load the data into your session:

 data deepPriceData;
    input i customertype time price demand  ;
    datalines;
    1    10 3.2517  0.82502 0.59985
    2     8 9.6777  1.47760 0.35881
    3     1 6.4239  1.82441 2.24215
    4     4 10.0217 1.56411 1.18040
    5     3 11.2567 2.13610 1.39896

   ... more lines ...   

    9995  8 3.4223  1.04572 0.38379
    9996  6 11.1666 1.65613 1.29208
    9997  3 2.9016  1.11613 1.97467
    9998  6 3.8167  0.97021 1.05249
    9999  7 3.7612  1.17288 0.35869
    10000 4 1.2474  1.13899 1.51599
    ;

data mylib.deeppricedata;
   set deeppricedata;
run;

It is first assumed that the demand curve is linear. The following statements use PROC DEEPPRICE to estimate the intercept and slope functions and then to save the estimation details:

proc deepprice data=mylib.deeppricedata;
   id i;
   tmodel price = customerType time /
         dnn=(nodes=(32 32 32 32)
         train=(optimizer=(miniBatchSize=500 regL1=0.0001 maxEpochs=32000
         algorithm=adam) nthreads=20 seed=12345 recordseed=67890));
   model demand = customerType time /
         dnn=(nodes=(32 32 32 32)
         train=(optimizer=(miniBatchSize=500 regL1=0.001 maxEpochs=32000
         algorithm=adam) nthreads=20 seed=12345 recordseed=67890));
   infer out=mylib.oest outdetails=mylib.outtabdetails;
   score outt=mylib.outtdata outa=mylib.outadata outb=mylib.outbdata;
run;

For this example, the same covariates are used in both the treatment model and the outcome model. The model estimation details are saved in the output data table outtabdetails.

The estimation results are shown in Output 16.2.1.

This predicted demand might not be accurate, because the covariates might affect both price and demand. The following statements show how you can construct the grid of price and calculate the predicted demand by using the influence functions _if_alpha_ and _if_beta_ that you saved in the outtabdetails data set in the initial call to PROC DEEPPRICE and then how you can plot the demand curve:

data myGrid_if;
   array gridPrice[100];
   array gridY[100];
   set  mylib.outtabdetails;
   do j = 1 to 100;
      gridPrice[j] = 0.7 + 2.8*j / 101;
      gridY[j] = _if_alpha_ + _if_beta_*gridPrice[j] ;
   end;
run;

The following statements compute the average demand, its standard error, and its 95% confidence interval at each of the 100 price values and then plot the results:

proc univariate data=mygrid_if;
   var gridPrice1-gridPrice100 gridy1-gridy100;
   output out= estCurve_if
   mean=gridPricem1-gridPricem100 gridym1-gridym100
   std=gridPrices1-gridPrices100 gridys1-gridys100;
run;

data estCurve_if_ci;
   set estCurve_if;
   array prices[100] gridPricem1-gridPricem100;
   array demands[100] gridym1-gridym100;
   array stds[100] gridys1-gridys100;
   do i = 1 to 100;
      price=prices[i];
      estDemand=demands[i];
      estDemandLb = estDemand - 1.96*stds[i]/sqrt(10000);
      estDemandUb = estDemand + 1.96*stds[i]/sqrt(10000);
      output;
   end;
   keep price estDemand estDemandLb estDemandUb;
run;

proc sgplot data= estCurve_if_ci;
   title 'Demand Curve';
   band  x=price lower=estDemandLb upper=estDemandUb /
   legendlabel="95% Confidence Band " name="band";
   scatter x=price y=estDemand / markerattrs=(color=blue size=3 symbol=circle)
   legendlabel="Estimated Demand" name="estDemand";
   refline 0;
   xaxis label='Price';
   yaxis label='Demand' ;
run;

The generated demand curve and 95% pointwise confidence band are shown in Output 16.2.2.

The curve looks unrealistic, especially given that a large portion of the curve is below zero. This indicates that the linear demand curve assumption might be incorrect. There is another way to model price and demand: by using log transformation.

The following DATA step computes the logarithm of price and demand data:

data mylib.deeppricedata_log;
   set deepPriceData;
   logy=log(demand);
   logp=log(price);
run;

The following statements use PROC DEEPPRICE to estimate the price effect and save the estimation details to the output table outtabdetails_log:

proc deepprice data=mylib.deeppricedata_log;
    id i;
    tmodel logp = customerType time /
         dnn=(nodes=(32 32 32 32)
         train=(optimizer=(miniBatchSize=500 regL1=0.0001 maxEpochs=32000
         algorithm=adam) nthreads=20 seed=12345 recordseed=67890));
    model logy = customerType time /
         dnn=(nodes=(32 32 32 32)
         train=(optimizer=(miniBatchSize=500 regL1=0.001 maxEpochs=32000
         algorithm=adam) nthreads=20 seed=12345 recordseed=67890));
    infer out=mylib.oest_log outdetails=mylib.outtabdetails_log;
    score outt=mylib.outtdata_log outa=mylib.outadata_log outb=mylib.outbdata_log;
run;

The estimation results are shown in Output 16.2.3. The average slope is negative, indicating that the impact of price on demand is negative. This is consistent with expectations.

The following DATA step constructs a grid of price values and calculates the corresponding expected demand, whose influence function is given by

data myGrid_log_if;
   array gridPrice[100];
   array gridY[100];
   set mylib.outtabdetails_log;
   do j = 1 to 100;
      gridPrice[j] = 0.7 + 2.8*j / 101 ;
      gridY[j] = exp(_alpha_ + _beta_ * log(gridPrice[j]))*(1+(_if_alpha_-_alpha_)
               +log(gridPrice[j])*(_if_beta_-_beta_));
      end;
run;

The following PROC UNIVARIATE and DATA steps compute the average demand, its standard error, and its 95% confidence interval at each of the 100 price values:

proc univariate data=myGrid_log_if noprint;
   var gridPrice1-gridPrice100 gridy1-gridy100;
   output out=estCurve_log_if mean=gridPricem1-gridPricem100 gridym1-gridym100
   std=gridPrice1-gridPrice100 gridys1-gridys100;
run;

data estCurve_log_if_ic;
   set estCurve_log_if;
   array prices[100] gridPricem1-gridPricem100;
   array demands[100] gridym1-gridym100;
   array stds[100] gridys1-gridys100;
   do i = 1 to 100;
      price=prices[i];
      estDemand=demands[i];
      estDemandLb = estDemand - 1.96*stds[i]/sqrt(10000);
      estDemandUb = estDemand + 1.96*stds[i]/sqrt(10000);
      output;
   end;
   keep price estDemand estDemandLb estDemandUb;
run;

Then the demand curve and 95% pointwise confidence intervals are displayed as shown in Output 16.2.4, using the following statements:

proc sgplot data=estCurve_log_if_ic;
   title 'Demand Curve';
   band  x=price lower=estDemandLb upper=estDemandUb /
   legendlabel="95% Confidence Band " name="band";
   scatter x=price y=estDemand / markerattrs=(color=blue size=3 symbol=circle)
   legendlabel="Estimated Demand" name="estDemand";
   xaxis label='Price';
   yaxis label='Demand';
run;

The curve looks considerably better. As expected, the demand curve slopes downward. In fact, the data generating process (DGP) is available, and the objective of this section of the example is to assess how well the estimated demand curves approximate the truth.

The true DGP is defined as follows:

The treatment model is given as

where describes the seasonality.

The outcome model is given as

The following DATA steps generate the data by using the true DGP:

data deepPriceDGP;
   array gridPrice[100] gridPrice1-gridPrice100;
   array gridY[100] gridY1-gridY100;
   call streaminit('pcg', 12440);
   do i = 1 to 10000;
      customertype = floor(rand('Uniform')*10+1);
      time = rand('Uniform')*12;
      eta = 0.1*rand('Normal',0,1);
      * z: seasonality;
      z = 2*(50/3*((time-6)/12)**4
          +exp(-400*((time-6)/12)**2)+time/12-41/20);
      price = exp(1+0.3*z + eta);
      a = exp(2 + 0.1*customertype*z);
      b = 0.1*customertype*z - 1.5;
      demand = a * exp(log(price) * b);
      do j = 1 to 100;
         gridPrice[j] = 0.7 + 2.8*j / 101;
         gridY[j] = a * exp( log(gridPrice[j]) * b);
      end;
      output;
   end;
run;

data mylib.deepPriceDataFull;
  set deepPriceDGP;
run;

Then, using the code segments that follow, you can plot the true and estimated demand curves.

First, the following code calculates the mean of the true price and demand:

proc univariate data=mylib.deepPriceDataFull noprint;
   var  gridPrice1-gridPrice100 gridy1-gridy100;
   output out=trueCurve mean=gridPricem1-gridPricem100 gridym1-gridym100;
run;

data trueCurve2;
   set trueCurve;
   array prices[100] gridPricem1-gridPricem100;
   array demands[100] gridym1-gridym100;
   do i = 1 to 100;
      price=prices[i];
      demand=demands[i];
      output;
   end;
   keep price demand;
run;

The following code calculates the mean of estimated price and demand, and then the true and estimated demand curves and 95% pointwise confidence intervals are displayed in Output 16.2.5 and Output 16.2.6:

%macro createcurve(infile,outfile,bothfile);
   proc univariate data=&infile noprint;
      var gridPrice1-gridPrice100 gridy1-gridy100;
      output out=&outfile mean=gridPricem1-gridPricem100
      gridym1-gridym100 std=gridPrices1-gridPrices100 gridys1-gridys100;
   run;

   data &outfile%str(_mean);
      set &outfile;
      array prices[100] gridPricem1-gridPricem100;
      array demands[100] gridym1-gridym100;
      array stds[100] gridys1-gridys100;
      do i = 1 to 100;
         price=prices[i];
         estDemand=demands[i];
         estDemandLb = estDemand - 1.96*stds[i]/100;
         estDemandUb = estDemand + 1.96*stds[i]/100;
         output;
      end;
      keep price estDemand estDemandLb estDemandUb;
   run;

   data &bothfile;
      set trueCurve2;
      set &outfile%str(_mean);
   run;

   proc sgplot data=&bothfile;
      title 'True and Estimated Demand Curves';
      scatter x=price y=demand / markerattrs=(color=red size=5 symbol=trianglefilled)
      legendlabel="True Demand" name="demand";
      band  x=price lower=estDemandLb upper=estDemandUb /
      legendlabel="95% Confidence Band " name="band";
      scatter x=price y=estDemand / markerattrs=(color=blue size=3 symbol=circle)
      legendlabel="Estimated Demand" name="estDemand";
         refline 0;
      xaxis label='Price';
      yaxis label='Demand';
   run;
%mend;

The results of using a linear demand curve assumption as follows are shown in Output 16.2.5:

%let infile=mygrid_if;
%let outfile=estCurve_if;
%let bothfile=bothCurves_if;
%createcurve(&infile.,&outfile.,&bothfile.);

The results of using a log-log model as follows are shown in Output 16.2.6:

%let infile=mygrid_log_if;
%let outfile=estCurve_log_if;
%let bothfile=bothCurves_log_if;
%createcurve(&infile.,&outfile.,&bothfile.);

The coverage rate of the 95% confidence interval for the linear curve is 0%, whereas that for the log-log curve is 93%. The linear demand curve assumption is clearly wrong for this example. However, in the real world, when there is no true demand curve or it is too expensive to get such a curve through randomized controlled trials (RCTs), how can you know what is the correct model specification? You can use the nonparametric framework, but that is beyond the scope of this example.

Considering that different types of customers and different purchase times can have an impact on the estimated results, the following code segments show how to produce the conditional demand curve. There are two situations: the first is the demand curve for different types of customers, and the second is the demand curve in different time periods. The following statements generate subsets of DGP based on various conditions:

%macro createtrue(n,where);
   %do j=1 %to &n;
      %let l=%eval(&j-1);
      proc univariate data=mylib.deepPriceDataFull(where=(&where)) noprint;
         var  gridPrice1-gridPrice100 gridy1-gridy100;
         output out=trueCurve_&j mean=gridPricem1-gridPricem100
         gridym1-gridym100;
      run;

      data trueCurve2_&j;
         set trueCurve_&j;
         array prices[100] gridPricem1-gridPricem100;
         array demands[100] gridym1-gridym100;
         do i = 1 to 100;
            price=prices[i];
            demand=demands[i];
            output;
         end;
         keep price demand;
      run;
   %end;
%mend;

The following statements generate subsets of estimated price and demand data sets under various conditions:

%macro createdata(infile,outfile,bothfile,k);
   proc univariate data=&infile noprint;
      var gridPrice1-gridPrice100 gridy1-gridy100;
      output out=&outfile mean=gridPricem1-gridPricem100
      gridym1-gridym100
      std=gridPrices1-gridPrices100 gridys1-gridys100;
   run;

   data "&outfile%scan(_2,-1,'')"n;
      set &outfile;
      array prices[100] gridPricem1-gridPricem100;
      array demands[100] gridym1-gridym100;
      array stds[100] gridys1-gridys100;
      do i = 1 to 100;
         price=prices[i];
         estDemand=demands[i];
         estDemandLb = estDemand - 1.96*stds[i]/sqrt(10000);
         estDemandUb = estDemand + 1.96*stds[i]/sqrt(10000);
         output;
      end;
      keep price estDemand estDemandLb estDemandUb;
   run;

   data &bothfile;
      set trueCurve2_&k;
      set "&outfile%scan(_2,-1,'')"n;
   run;
%mend;

The following statements merge the data sets and plot the demand curves under various conditions:

 %macro createcurve(n,where,title,cols,rows,type);
   %do k=1 %to &n;
   %let l=%eval(&k-1);
   data mygrid_log_if_&k;
      set mygrid_log_if(where=(&where));
   run;
   %let infile=mygrid_log_if_&k;
   %let outfile=estCurve_log_if_&k;
   %let bothfile=bothCurves_log_if_&k;
   %createdata(&infile.,&outfile.,&bothfile.,&k);

   data bothCurves_log_if_&k;
      set bothCurves_log_if_&k;
      &type=&k;
   run;
   %if &k=1 %then %do;
      data bothCurves_log_if_all;
         set bothCurves_log_if_&k;
      run;
   %end;
   %if &k>1 %then %do;
      data bothCurves_log_if_all;
         merge bothCurves_log_if_all bothCurves_log_if_&k;
         by &type;
      run;
   %end;
   %end;

   %let varname=;

   %if &type=time %then %do;
      data bothCurves_log_if_all;
         set bothCurves_log_if_all;
         month=put(time,NLSTRMON3.1);
      run;
      %let type=time month;
      %let varname=novarname;
   %end;

   proc sgpanel data=bothCurves_log_if_all;
      title "&title";
      panelby &type /layout=panel columns=&cols rows=&rows uniscale=column &varname;
      scatter x=price y=demand / markerattrs=(color=red size=5 symbol=trianglefilled)
      legendlabel="True Demand" name="demand";
      band  x=price lower=estDemandLb upper=estDemandUb /
      legendlabel="95% Confidence Band " name="band";
      scatter x=price y=estDemand / markerattrs=(color=blue size=3 symbol=circle)
      legendlabel="Estimated Demand" name="estDemand";
      refline 1.5 /AXIS=X lineattrs=( color=green thickness=1);
      refline 1 /AXIS=Y   lineattrs=( color=green thickness=1);
      colaxis label='Price';
      rowaxis label='Demand' values=(0 to 10 by 1) ;
   run;
%mend;

The following statements pass parameters to the macro, select criteria for classification, add a title, and format the display:

%let n=10;
%let where= customertype %quote(=)%str(&)j;
%createtrue(&n,&where);
%let where= customertype %quote(=)%str(&)k;
%let title=Demand Curves for Different Types of Customers;
%let cols=2;
%let rows=5;
%let type=customertype;
%createcurve(&n,&where,&title,&cols,&rows,&type);

The results are shown in Output 16.2.7

Different types of customers have varying degrees of sensitivity to prices. Customers of customertype1 and customertype2 are the most price-sensitive, especially when price < 1.5. Customers of customertype4 and customertype5 are somewhat price-sensitive, whereas customers of customertype7, customertype8, customertype9, and customertype10 are relatively price-insensitive.

The following statements pass parameters to the macro, select criteria for purchasing time, add a title, and format the display:

%let n=12;
%let where=time%quote(>=)%str(&)l and time%quote(<=)%str(&)j;
%createtrue(&n,&where)
%let where=time%quote(>=)%str(&)l and time%quote(<=)%str(&)k;
%let title=Demand Curves for Each Month;
%let type=time;
%let cols=3;
%let rows=4;
%createcurve(&n,&where,&title,&cols,&rows,&type);

The results are shown in Output 16.2.8:

The price sensitivity varies in different months. In February, March, April, and May, customers are not very sensitive to price changes, but in November and December, customers are very sensitive to price changes. The closer to the end of the year, the higher the price sensitivity. At the beginning of the year, the price sensitivity is relatively low.

In summary, only by specifying the correct functional form in PROC DEEPPRICE can you correctly estimate the demand curve. The shape of the estimated demand curves varies across customer types and over time.