Time Series Model Package

RNNSPEC Methods

RNNSPEC.AddTF Method

rc = obj.AddTF (XName);

This option adds a transfer function to the RNN model for the specified XName variable. Unlike the ARIMASPEC object, all other parameters except the variable name in AddTF are disabled. The RNNSPEC AddTF method provides a basic transfer function, which is described in the TNF package.

Input Arguments

You must specify the following input argument:

XName: is a character string that specifies the name of the X variable.

RNNSPEC.Close Method

rc=obj.Close ();

Finalizes the RNNSPEC object to prepare the RNN model to be used in a TSM object or to be imported to a TSMSPEC object for printing or for storage in a model repository.

Arguments

There are no arguments associated with this method.

RNNSPEC.GetLabel Method

rc=obj.GetLabel ();

Retrieves the label of the specified RNN model and stores it in the return variable, rc.

Arguments

There are no arguments associated with this method.

RNNSPEC.Open Method

rc=obj.Open ();

Initializes an empty RNNSPEC object for configuration.

Arguments

There are no arguments associated with this method.

RNNSPEC.SetOption Method

rc=obj.SetOption ('Name', Value <,'Name',Value,…>);

Specifies RNN model options. Options are ('Name',Value) pairs, where 'Name' is a case-insensitive character string and Value depends on the 'Name'.

Input Arguments

You must specify at least one of the following 'Names' and its associated Value:

'BATCHSELECTION'

takes a string Value that specifies how to make a minibatch data set for training networks. You can specify one of the following values:

RANDOM | RAN: selects the minibatch data set randomly.
SEQUENCE | SEQ: selects the minibatch data set sequentially according to the order of observations in the training data.
SHUFFLE | SHUF: selects the minibatch data set sequentially after shuffling the training data.

The default is SHUFFLE.

'FCSTLOWERBOUND'

takes a numeric Value that specifies a lower bound for the forecast. Forecast values that are below the specified value are truncated to the Value. A missing value indicates that no lower-bound truncation should occur. The default is a missing value.

'FCSTUPPERBOUND'

takes a numeric Value that specifies an upper bound for the forecast. Forecast values that are above the specified value are truncated to the Value. A missing value indicates that no upper-bound truncation should occur. The default is a missing value.

'NINPUT'

takes a positive integer Value that specifies the length of input series for the network. It is the length of the sampled subsequence for the input layer. The default is 6.

'NLAYER'

takes a positive integer Value that specifies the number of neural network layers. The first layer is the input layer, and the last layer is the output layer. The default is 3, which means that the neural network has one hidden layer.

'NNEURONH'

takes a positive integer Value that specifies the number of neurons for each hidden node. The default is 10.

'NORMALIZE'

takes a string Value that specifies a normalization method for the input sequence and the target sequence. You can specify one of the following values:

MIDRANGE: uses the midrange normalization, which scales variables so that the midrange is 0 and the half-range is 1. The result is that variables have a minimum value of –1 and a maximum value of 1.
NONE: means that variables are not modified.
STD: uses the normal standardization, which scales variables so that the mean is 0 and the standard deviation is 1.

The default is STD.

'NVALIDATION'

takes a nonnegative integer Value that specifies the number of validation samples. The validation sample is used to validate models. The default is 0. When the value is set to 0, the training error is used for model validation.

'OUTPUTINITBIAS'

takes a numeric Value that specifies the initial bias for the output layer. The default is 0.

'POSTTRAIN'

takes a string Value that specifies whether the optimization will run on the validation samples after the training process ends. You can specify one of the following values:

NO: means that the post-training process is not performed.
YES: means that the post-training process is performed.

The default is YES if the size of the validation sample is greater than 0. Otherwise, the default is NO.

'RECURRENTINITBIAS'

takes a numeric Value that specifies the initial bias for the recurrent layers. The default is 0.

'RNNTYPE'

takes a string Value that specifies a recurrent network type for the hidden layers. You can specify one of the following values:

GRU: uses the gated recurrent unit.
LSTM: uses the long short-term memory unit.
RNN: uses the original recurrent neural network.

The default is LSTM.

'SEED'

takes a nonnegative integer Value that specifies the seed to generate random numbers for the optimization process and data shuffling of minibatch selection. When the value is 0, the seed is generated by using the time of day from the computer’s clock. The default is 12345.

'USEERRORACF'

takes a string Value that specifies whether to use the autocorrelation (ACF) to calculate forecast standard errors for forecast confidence intervals.

NO: means that the standard error of the h-step forecast is calculated by the residual standard deviation multiplied by the square root of h.
YES: means that the standard error of the h-step forecast is calculated by the residual standard deviation multiplied by the square root of the cumulative ACF up to the h lag.

The default is YES.

'WEIGHTINIT'

takes a string Value that specifies the weight initialization function for the neurons in each hidden layer. You can specify one of the following values:

CAUCHY: initializes weights so that the median is 0 and the scale is 1.
NORMAL: initializes weights so that the mean is 0 and the standard deviation is 1.
UNIFORM: initializes weights so that the mean is 0 and the half-range is 1.
XAVIER: initializes weights according to the method of Glorot and Bengio (2010) for a normal distribution.

The default is XAVIER.

RNNSPEC.SetOptimizer Method

rc=obj.SetOptimizer ('Name', Value <,'Name',Value,…>);

Specifies optimizer options for the RNNSPEC object.

Input Arguments

You must specify at least one of the following 'Names' and its associated Value:

'ALGORITHM'

takes a string Value that specifies one of the following stochastic gradient descent (SGD) optimization algorithms:

ADAM: uses adaptive moment estimation, which computes adaptive learning rates for each parameter.
MOMENTUM: uses an exponential smoothing model to accelerate SGD in the relevant direction and to dampen oscillation.
VANILLA: uses the plain stochastic gradient descent method.

The default is ADAM.

'BETA1'

takes a positive numeric Value that specifies the exponential decay rate for the first moment in the Adam learning algorithm. The value should be between 0 (inclusive) and 1 (exclusive). The default is 0.9.

'BETA2'

takes a positive numeric Value that specifies the exponential decay rate for the second moment in the Adam learning algorithm. The value should be between 0 (inclusive) and 1 (exclusive). The default is 0.999.

'CLIPGRADMAX'

takes a positive numeric Value that specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value. The default is 1,000. When you specify the missing value, it means that there is no restriction on the gradient values.

'CLIPGRADMIN'

takes a negative numeric Value that specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value. The default is –1,000. When you specify the missing value, it means that there is no restriction on the gradient values.

'DROPOUT'

takes a numeric Value that specifies the probability that the output of a neuron in a fully connected layer will be set to 0 during training. The specified probability is recalculated each time an observation is processed. The value should be between 0 (inclusive) and 1 (exclusive). The default is 0.

'DROPOUTINPUT'

takes a numeric Value that specifies the probability that an input variable will be set to 0 during training. The specified probability is recalculated each time an observation is processed. The value should be between 0 (inclusive) and 1 (exclusive). The default is 0.

'DROPOUTTYPE'

takes a string Value that specifies one of the following dropout types:

INVERTED: uses the inverted dropout.
STANDARD: uses the standard dropout.

The default is STANDARD.

'EPSILON'

takes a positive numeric Value that specifies a very small number to prevent any division by zero in the Adam learning algorithm. The default is 1E–08.

'GAMMA'

takes a positive numeric Value that specifies the gamma parameter for the learning rate policy. The value should be between 0 and 1, inclusive. The default is 0.1.

'LEARNINGPOLICY'

takes a string Value that specifies one of the following learning rate policies:

FIXED: specifies a fixed learning rate.
INV: sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as
POLY: sets the learning rate parameter value after each epoch, according to the initial learning rate, the maximum number of epochs, and the value of the power parameter. The rate is calculated as
STEP: sets the learning rate parameter by multiplying the current learning rate value by the gamma parameter value. The number of steps is specified in the stepSize parameter. The learning rate is recalculated for each group of epochs according to the step size. The rate is calculated as

The default is FIXED.

'LEARNINGRATE'

takes a positive numeric Value that specifies the learning rate for stochastic gradient descent. The default is 0.001.

'MAXEPOCHS'

takes a positive integer Value that specifies the maximum number of epochs. The default is 100.

'MINIBATCHSIZE'

takes a positive integer Value that specifies the number of observations per thread in a minibatch. The default is 1.

'MOMENTUM'

takes a numeric Value that specifies the momentum parameter value for stochastic gradient descent. The value should be between 0 and 1, inclusive. The default is 0.9.

'NMINIBATCH'

takes a positive integer Value that specifies the maximum number of minibatch sets. The default is the maximum number of 64-bit signed integers.

'POWER'

takes a nonnegative numeric Value that specifies the power for the learning rate policy. The default is 0.75.

'REGL1'

takes a nonnegative numeric Value that specifies the weight for the L1 regularization term. The default is 0, which means that the L1 regularization is not performed. It is suggested that the weight parameter begin with a small value, such as 1E–6. L1 regularization can be combined with L2 regularization.

'REGL2'

takes a nonnegative numeric Value that specifies the weight for the L2 regularization term. The default is 0, which means that the L2 regularization is not performed. It is suggested that the weight parameter begin with a small value, such as 1E–3. L2 regularization can be combined with L1 regularization.

'STAGNATION'

takes a nonnegative integer Value that specifies the number of successive training errors when a validation set is not given or the number of successive validation errors without improvement that will cause early termination. The default is 20. A value of 0 means that no stagnation early stopping is applied.

'STEPSIZE'

takes a positive integer Value that specifies the step size when the learning rate policy is set to STEP. The default is 10.

'THRESHOLD'

takes a nonnegative numeric value that specifies the threshold that is used to determine whether the validation score is improving or stagnating. When

abs left-parenthesis currentScore hyphen hyphen previousScore right-parenthesis less-than-or-equal-to abs left-parenthesis currentScore right-parenthesis asterisk THRESHOLD

the current iteration does not improve the optimization and the stagnation counter is incremented. Otherwise, the stagnation counter is set to 0. When a validation (holdout) data set is not available, the training score is used. The default value is 1E–08, and the minimum value is 0.

'WARMUPEPOCHS'

takes a nonnegative integer Value that specifies the number of epochs for the learning rate warmup. The number should not exceed the maximum number of epochs. The default is 0, which means that there is no warmup.

Last updated: January 27, 2023