RMSE in catboost 0.2.5

results.txt

Borders for float features generated
0:	learn 70.50475406	test 40.44999897	bestTest 40.44999897		total: 47.1ms	remaining: 424ms
1:	learn 65.63694777	test 53.45852055	bestTest 40.44999897		total: 47.6ms	remaining: 191ms
2:	learn 57.7842572	test 100.5151111	bestTest 40.44999897		total: 48.1ms	remaining: 112ms
3:	learn 55.30496775	test 128.7490654	bestTest 40.44999897		total: 48.6ms	remaining: 72.8ms
4:	learn 52.89929452	test 136.0044652	bestTest 40.44999897		total: 49ms	remaining: 49ms
5:	learn 51.9437037	test 150.0426779	bestTest 40.44999897		total: 49.6ms	remaining: 33ms
6:	learn 51.70084336	test 158.4656055	bestTest 40.44999897		total: 50.1ms	remaining: 21.5ms
7:	learn 50.61418096	test 162.2118236	bestTest 40.44999897		total: 50.6ms	remaining: 12.6ms
8:	learn 50.49985141	test 165.7670929	bestTest 40.44999897		total: 51.1ms	remaining: 5.67ms
9:	learn 49.9656332	test 167.9084788	bestTest 40.44999897		total: 51.6ms	remaining: 0us

bestTest = 40.44999897
bestIteration = 0

score 28193.257265, root score 167.908479
Predictions:  [-162.90847884]  get_test_eval:  [[-162.90847883612867]]
Test: mse 28193.257265, rmse 167.908479
Train: mse 14355.246379, rmse 119.813381, wrmse 49.965634

test_rmse.py

import numpy as np
from catboost import CatBoostRegressor


def predict_and_score(dataset, labels):
    pred = fit_model.predict(dataset)
    mse = np.mean((pred - labels) ** 2)
    rmse = np.sqrt(mse)
    return pred, mse, rmse

trainset = np.array([[1, 4, 5, 6], [4, 5, 6, 7], [30, 40, 50, 60], [20, 15, 85, 60]])
train_labels = [1.2, -333.4, 119.5, -24.5]
weights = [1, 1, 1, 20]
model = CatBoostRegressor(iterations=10, learning_rate=1, depth=1, loss_function='RMSE', random_seed=1)
testset = [[8, 9, 10, 11]]
test_labels = [5]
fit_model = model.fit(trainset, train_labels, sample_weight=weights, verbose=True, eval_set=(testset, test_labels))
score = fit_model.score(testset, test_labels)
print 'score %f, root score %f' % (score, np.sqrt(score))
pred, mse, rmse = predict_and_score(testset, test_labels)
print 'Predictions: ', pred, ' get_test_eval: ', fit_model.get_test_eval()
print 'Test: mse %f, rmse %f' % (mse, rmse)
pred, mse, rmse = predict_and_score(trainset, train_labels)
wrmse = np.sqrt(np.sum(weights * (pred - train_labels) ** 2)/np.sum(weights))
print 'Train: mse %f, rmse %f, wrmse %f' % (mse, rmse, wrmse)