How to use a Keras RNN model to forecast for future dates or events?
Solution 1:
Well, you need a stateful=True model, so you can feed it one prediction after another to get the next and keep the model thinking that each input is not a new sequence, but a sequel to the previous.
Fixing the code and training
I see in the code that there is an attempt to make your y be a shifte x (a good option for predicting the next steps). But there is also a big problem in the preprocessing here:
training_set = df_train.values
training_set = min_max_scaler.fit_transform(training_set)
x_train = training_set[0:len(training_set)-1]
y_train = training_set[1:len(training_set)]
x_train = np.reshape(x_train, (len(x_train), 1, 1))
Data for LSTM layers must be shaped as (number_of_sequences, number_of_steps,features).
So, you're clearly creating sequences of 1 step only, meaning that your LSTM is not learning sequences at all. (There is no sequence with only one step).
Assuming that your data is a single unique sequence with 1 feature, it should definitely be shaped as (1, len(x_train), 1).
Naturally, y_train should also have the same shape.
This, in its turn, will require that your LSTM layers be return_sequences=True - The only way to make y have a length in steps. Also, for having a good prediction, you may need a more complex model (because now it will be trully learning).
This done, you train your model until you get a satisfactory result.
Predicting the future
For predicting the future, you will need stateful=True LSTM layers.
Before anything, you reset the model's states: model.reset_states() - Necessary every time you're inputting a new sequence into a stateful model.
Then, first you predict the entire X_train (this is needed for the model to understand at which point of the sequence it is, in technical words: to create a state).
predictions = model.predict(`X_train`) #this creates states
And finally you create a loop where you start with the last step of the previous prediction:
future = []
currentStep = predictions[:,-1:,:] #last step from the previous prediction
for i in range(future_pred_count):
currentStep = model.predict(currentStep) #get the next step
future.append(currentStep) #store the future steps
#after processing a sequence, reset the states for safety
model.reset_states()
Example
This code does this with a 2-feature sequence, a shifted future step prediction, and a method that is a little different from this answer, but based on the same principle.
I created two models (one stateful=False, for training without needing to reset states every time - never forget to reset states when you're starting a new sequence - and the other stateful=True, copying the weights from the trained model, for predicting the future)
https://github.com/danmoller/TestRepo/blob/master/TestBookLSTM.ipynb
Solution 2:
What you need to do in order to predict future values with RNNs is to provide data as sequences. Something like this:
[0 1 2] --> [3]
[1 2 3] --> [4]
[2 3 4] --> [5]
[3 4 5] --> [6]
[4 5 6] --> [7]
RNNs learn the structure of sequences, and therefore need a unique input shape:
(n_samples, time_steps, n_features)
For instance, the time steps could be 7 if you use every day of the last week.
How can I create a dataset for RNNs?
tf.keras.preprocessing.timeseries_dataset_from_array
What you'll need to do is provide this function with a) present values, and b) future values. Here, seq_length is the number of time steps to use.
import tensorflow as tf
seq_length = 3
x = tf.range(25)[:-seq_length]
y = tf.range(25)[seq_length:]
ds = tf.keras.preprocessing.timeseries_dataset_from_array(x, y,
sequence_length=seq_length,
batch_size=1)
for present_values, next_value in ds.take(5):
print(tf.squeeze(present_values).numpy(), '-->', next_value.numpy())
[0 1 2] --> [3]
[1 2 3] --> [4]
[2 3 4] --> [5]
[3 4 5] --> [6]
[4 5 6] --> [7]
You can also do the same for multiple variables:
import tensorflow as tf
seq_length = 3
x = tf.concat([
tf.reshape(tf.range(25, dtype=tf.float32)[:-seq_length], (-1, 1)),
tf.reshape(tf.linspace(0., .24, 25) [:-seq_length], (-1, 1))], axis=-1)
y = tf.concat([
tf.reshape(tf.range(25, dtype=tf.float32)[seq_length:], (-1, 1)),
tf.reshape(tf.linspace(0., .24, 25) [seq_length:], (-1, 1))], axis=-1)
ds = tf.keras.preprocessing.timeseries_dataset_from_array(x, y,
sequence_length=seq_length,
batch_size=1)
for present_values, next_value in ds.take(5):
print(tf.squeeze(present_values).numpy(), '-->', tf.squeeze(next_value).numpy())
model = tf.keras.Sequential([
tf.keras.layers.LSTM(8),
tf.keras.layers.Dense(8, activation='relu'),
tf.keras.layers.Dense(2)
])
model.compile(loss='mae', optimizer='adam')
history = model.fit(ds)
[[0. 0. ]
[1. 0.01]
[2. 0.02]] --> [3. 0.03]
[[1. 0.01]
[2. 0.02]
[3. 0.03]] --> [4. 0.04]
[[2. 0.02]
[3. 0.03]
[4. 0.04]] --> [5. 0.05]
[[3. 0.03]
[4. 0.04]
[5. 0.05]] --> [6. 0.06]
[[4. 0.04]
[5. 0.05]
[6. 0.06]] --> [7. 0.07]
- This function
import tensorflow as tf
import numpy as np
x = np.arange(25)
def univariate_data(dataset, start_index, end_index, history_size, target_size):
data, labels = [], []
start_index = start_index + history_size
if end_index is None:
end_index = len(dataset) - target_size
for i in range(start_index, end_index):
indices = np.arange(i-history_size, i)
data.append(np.reshape(dataset[indices], (history_size, 1)))
labels.append(dataset[i:i+target_size])
return np.array(data), np.array(labels)
present_values, future_values = univariate_data(x, 0, 9, 3, 3)
for present, next_val in zip(present_values, future_values):
print(tf.squeeze(present).numpy(), '-->', tf.squeeze(next_val).numpy())
[0 1 2] --> [3 4]
[1 2 3] --> [4 5]
[2 3 4] --> [5 6]
[3 4 5] --> [6 7]
[4 5 6] --> [7 8]
[5 6 7] --> [8 9]
And now for multiple variables:
import tensorflow as tf
import numpy as np
history_size = 3
x = np.concatenate([np.expand_dims(np.arange(25), 1)[:-history_size],
np.expand_dims(np.linspace(0., .24, 25), 1)[:-history_size]], axis=1)
y = np.concatenate([np.expand_dims(np.arange(25), 1)[history_size:],
np.expand_dims(np.linspace(0., .24, 25), 1)[history_size:]], axis=1)
def multivariate_data(dataset, target, start_index, end_index, history_size,
target_size, step, single_step=False):
data = []
labels = []
start_index = start_index + history_size
if end_index is None:
end_index = len(dataset) - target_size
for i in range(start_index, end_index):
indices = range(i-history_size, i, step)
data.append(dataset[indices])
if single_step:
labels.append(target[i+target_size])
else:
labels.append(target[i:i+target_size])
return np.array(data), np.array(labels)
present_values, future_values = multivariate_data(x, y, 0, 8, history_size, 1, 1)
for present, next_val in zip(present_values, future_values):
print(tf.squeeze(present).numpy(), '-->', tf.squeeze(next_val).numpy())
[[0. 0. ]
[1. 0.01]
[2. 0.02]] --> [6. 0.06]
[[1. 0.01]
[2. 0.02]
[3. 0.03]] --> [7. 0.07]
[[2. 0.02]
[3. 0.03]
[4. 0.04]] --> [8. 0.08]
[[3. 0.03]
[4. 0.04]
[5. 0.05]] --> [9. 0.09]
[[4. 0.04]
[5. 0.05]
[6. 0.06]] --> [10. 0.1]
tf.data.Dataset.window
import tensorflow as tf
import numpy as np
history_size = 3
lookahead = 2
x = tf.range(8)
ds = tf.data.Dataset.from_tensor_slices(x)
ds = ds.window(history_size + lookahead, shift=1, drop_remainder=True)
ds = ds.flat_map(lambda window: window.batch(history_size + lookahead))
ds = ds.map(lambda window: (window[:-lookahead], window[-lookahead:]))
for present_values, next_value in ds:
print(present_values.numpy(), '-->', next_value.numpy())
[0 1 2] --> [3 4]
[1 2 3] --> [4 5]
[2 3 4] --> [5 6]
[3 4 5] --> [6 7]
With multiple variables:
import tensorflow as tf
import numpy as np
history_size = 3
lookahead = 2
x = tf.concat([
tf.reshape(tf.range(20, dtype=tf.float32), (-1, 1)),
tf.reshape(tf.linspace(0., .19, 20), (-1, 1))], axis=-1)
ds = tf.data.Dataset.from_tensor_slices(x)
ds = ds.window(history_size + lookahead, shift=1, drop_remainder=True)
ds = ds.flat_map(lambda window: window.batch(history_size + lookahead))
ds = ds.map(lambda window: (window[:-lookahead], window[-lookahead:]))
for present_values, next_value in ds.take(8):
print(tf.squeeze(np.round(present_values, 2)).numpy(), '-->',
tf.squeeze(np.round(next_value, 2)).numpy())
print()
[[0. 0. ]
[1. 0.01]
[2. 0.02]] --> [[3. 0.03]
[4. 0.04]]
[[1. 0.01]
[2. 0.02]
[3. 0.03]] --> [[4. 0.04]
[5. 0.05]]
[[2. 0.02]
[3. 0.03]
[4. 0.04]] --> [[5. 0.05]
[6. 0.06]]
[[3. 0.03]
[4. 0.04]
[5. 0.05]] --> [[6. 0.06]
[7. 0.07]]
[[4. 0.04]
[5. 0.05]
[6. 0.06]] --> [[7. 0.07]
[8. 0.08]]
[[5. 0.05]
[6. 0.06]
[7. 0.07]] --> [[8. 0.08]
[9. 0.09]]