Skip to content

Instantly share code, notes, and snippets.

@springcoil
Created August 28, 2016 17:19
Show Gist options
  • Save springcoil/bed76b65f17c0fa046adb1ce238caf8d to your computer and use it in GitHub Desktop.
Save springcoil/bed76b65f17c0fa046adb1ce238caf8d to your computer and use it in GitHub Desktop.
Here's an example of Cross-validation for PyFlux - needs some refactoring
def time_series_cross_validation_2(model, data, params, number_folds=10):
"""
Performs a k-fold cross-validation on a Time Series as described by
Rob Hyndman.
See Also:
http://robjhyndman.com/hyndsight/crossvalidation/
Arguments:
model: a model somewhere...
database (numpy.ndarray): uses 'data' matrix to perform
cross-validation.
params (dict): dictionary of parameters to train/test.
number_folds (int): number of folds to be created from training and
testing matrices.
Returns:
tuple: tuple of AIC between training and testing for out of sample
"""
if number_folds < 2:
print("Error: Must have at least 2-folds.")
return
number_patterns = data.shape[0]
fold_size = round(number_patterns / number_folds)
folds = []
for k in range(number_folds):
folds.append(database[k * fold_size:(k + 1) * fold_size])
training_errors = []
testing_errors = []
training_matrix = folds[0]
testing_matrix = []
for k in range(number_folds - 1):
if k > 0:
training_matrix = \
np.concatenate((training_matrix, testing_matrix), axis=0)
testing_matrix = folds[k + 1]
print("Testing matrix:", testing_matrix)
tr_error = model(training_matrix, **params)
model_tr = tr_error.fit('BBVI',iterations=1000).aic
te_error = tr_error.predict(testing_matrix.shape[0]).values
print("Rmse error", rmse(te_error[0], testing_matrix))
return print("Training AIC: ", training_errors)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment