Skip to content

Instantly share code, notes, and snippets.

@njwilson

njwilson/README.rst

Created August 9, 2012 20:13
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save njwilson/3307706 to your computer and use it in GitHub Desktop.
Save njwilson/3307706 to your computer and use it in GitHub Desktop.
Download ZIP
Machine Learning Throwdown
Raw
README.rst

Services and Algorithms

This section describes each of the services compared in the throwdown and the algorithms/models used.

BigML

Decision trees.

Google Prediction API

Unknown black box model(s).

Prior Knowledge

Veritable API - Nonparametric Bayesian model.

Weka

10 popular algorithms (5 classification and 5 regression) were chosen to evaluate Weka. These were chosen to evaluate a wide variety of algorithms rather than to optimize performance.

Classification

Trees (J48)

  • Classifier: weka.classifiers.trees.J48
  • Parameters: -C 0.25 -M 2

Boosted Trees (Adaboost Classifier with J48 as weak learner)

  • Classifier: weka.classifiers.meta.AdaBoostM1
  • Parameters: -P 100 -S 1 -I 10 -W weka.classifiers.trees.J48 -- -C 0.25 -M 2

Naive Bayes

  • Classifier: weka.classifiers.bayes.NaiveBayes
  • No parameters

SVM (with RBF kernel function)

  • Classifier: weka.classifiers.functions.LibSVM
  • Parameters: -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 40.0 -C 1.0 -E 0.0010 -P 0.1

k-Nearest Neighbor (k = 3)

  • Classifier: weka.classifiers.lazy.IBk
  • Parameters: -K 3 -W 0 -A "weka.core.neighboursearch.LinearNNSearch -A \"weka.core.EuclideanDistance -R first-last\""

Regression

Trees (M5P)

  • Classifier: weka.classifiers.trees.M5P
  • Parameters: -M 4.0

Additive Regression (with M5P as weak classifier)

  • Classifier: weka.classifiers.meta.AdditiveRegression
  • Parameters: -S 1.0 -I 10 -W weka.classifiers.trees.M5P -- -M 4.0

Linear Regression

  • Classifier: weka.classifiers.functions.LinearRegression
  • Parameters: -S 0 -R 1.0E-8

SMOreg (support vector regression)

  • Classifier: weka.classifiers.functions.SMOreg
  • Parameters: -C 1.0 -N 0 -I "weka.classifiers.functions.supportVector.RegSMOImproved -L 0.0010 -W 1 -P 1.0E-12 -T 0.0010 -V" -K "weka.classifiers.functions.supportVector.PolyKernel -C 250007 -E 1.0"

k-Nearest Neighbor (k = 3)

  • Classifier: weka.classifiers.lazy.IBk
  • Parameters: -K 3 -W 0 -A "weka.core.neighboursearch.LinearNNSearch -A \"weka.core.EuclideanDistance -R first-last\""

Datasets

The datasets come from the UCI Machine Learning Repository and are relatively clean by machine learning standards. They are split into two categories, classification and regression, based on the type of the field we are trying to predict.

The links in the tables below point to the description of the original datasets on the UCI repository. Some of the datasets were downsampled or modified slightly to get them in a common CSV format for our benchmarking software. The actual data used for the throwdown can be found here.

Classification

In a classification problem, the field we are trying to predict has one of a finite number of possible values. Examples include predicting the type of an iris plant or predicting whether a tumor is malignant or benign.

Dataset # Instances # Attributes Missing Values CSV Size
Breast Cancer Wisconsin (Diagnostic) 699 11 Yes 24K
Glass Identification 214 11 No 17K
Iris 150 5 No 4.5K
Pen-Based Recognition of Handwritten Digits 7494 17 No 381K
Pima Indians Diabetes 768 9 No 28K
Wine 178 14 No 11K

Regression

In a regression problem, the field we are trying to predict has a numeric value. Examples include predicting the fuel efficiency of a car or predicting the number of violent crimes in a community.

Dataset # Instances # Attributes Missing Values CSV Size
Abalone 4177 9 No 187K
Auto MPG 398 8 Yes 14K
Communities and Crime 5822 127 Yes 1.0M
Concrete Compressive Strength 1030 9 No 48K
Insurance Company Benchmark (COIL 2000) 5822 86 No 983K

Results

Dataset: Breast Cancer Wisconsin (Diagnostic) (Classification)

# Algorithm Accuracy
1 Prior Knowledge - Veritable - Classification 0.97
2 Weka k-Nearest Neighbor (k = 3) classifier 0.97
3 Weka Adaboost Classifier with J48 as weak learner 0.97
4 Weka Naive Bayes 0.96
5 Google Predict Classifier 0.96
6 BigML Classification Tree 0.95
7 Weka J48 Tree 0.94
8 Weka SVM (with RBF kernel function) 0.66
# Algorithm Macro Average F1 Score
1 Prior Knowledge - Veritable - Classification 0.96
2 Weka Adaboost Classifier with J48 as weak learner 0.95
3 Weka k-Nearest Neighbor (k = 3) classifier 0.95
4 Weka Naive Bayes 0.94
5 Google Predict Classifier 0.94
6 BigML Classification Tree 0.93
7 Weka J48 Tree 0.91
8 Weka SVM (with RBF kernel function) 0.05
# Algorithm Macro Average Phi Coefficient
1 Prior Knowledge - Veritable - Classification 0.94
2 Weka Adaboost Classifier with J48 as weak learner 0.92
3 Weka k-Nearest Neighbor (k = 3) classifier 0.92
4 Weka Naive Bayes 0.92
5 Google Predict Classifier 0.90
6 BigML Classification Tree 0.89
7 Weka J48 Tree 0.87
8 Weka SVM (with RBF kernel function) 0.10

Dataset: Pima Indians Diabetes (Classification)

# Algorithm Accuracy
1 Google Predict Classifier 0.76
2 Prior Knowledge - Veritable - Classification 0.76
3 Weka Naive Bayes 0.75
4 Weka k-Nearest Neighbor (k = 3) classifier 0.74
5 Weka J48 Tree 0.74
6 Weka Adaboost Classifier with J48 as weak learner 0.73
7 BigML Classification Tree 0.70
8 Weka SVM (with RBF kernel function) 0.65
# Algorithm Macro Average F1 Score
1 Weka Naive Bayes 0.62
2 Google Predict Classifier 0.61
3 Weka Adaboost Classifier with J48 as weak learner 0.59
4 Weka J48 Tree 0.59
5 Weka k-Nearest Neighbor (k = 3) classifier 0.59
6 Prior Knowledge - Veritable - Classification 0.58
7 BigML Classification Tree 0.56
8 Weka SVM (with RBF kernel function) 0.00
# Algorithm Macro Average Phi Coefficient
1 Google Predict Classifier 0.45
2 Weka Naive Bayes 0.44
3 Prior Knowledge - Veritable - Classification 0.44
4 Weka J48 Tree 0.41
5 Weka k-Nearest Neighbor (k = 3) classifier 0.40
6 Weka Adaboost Classifier with J48 as weak learner 0.40
7 BigML Classification Tree 0.34
8 Weka SVM (with RBF kernel function) 0.00

Dataset: Glass Identification (Classification)

# Algorithm Accuracy
1 Weka SVM (with RBF kernel function) 0.98
2 Weka J48 Tree 0.98
3 Weka Adaboost Classifier with J48 as weak learner 0.98
4 BigML Classification Tree 0.97
5 Google Predict Classifier 0.96
6 Prior Knowledge - Veritable - Classification 0.93
7 Weka k-Nearest Neighbor (k = 3) classifier 0.90
8 Weka Naive Bayes 0.83
# Algorithm Macro Average F1 Score
1 Weka SVM (with RBF kernel function) 0.95
2 Weka J48 Tree 0.94
3 Weka Adaboost Classifier with J48 as weak learner 0.94
4 BigML Classification Tree 0.93
5 Google Predict Classifier 0.89
6 Prior Knowledge - Veritable - Classification 0.83
7 Weka Naive Bayes 0.80
8 Weka k-Nearest Neighbor (k = 3) classifier 0.79
# Algorithm Macro Average Phi Coefficient
1 Weka SVM (with RBF kernel function) 0.95
2 Weka J48 Tree 0.94
3 Weka Adaboost Classifier with J48 as weak learner 0.94
4 BigML Classification Tree 0.93
5 Google Predict Classifier 0.89
6 Prior Knowledge - Veritable - Classification 0.83
7 Weka k-Nearest Neighbor (k = 3) classifier 0.78
8 Weka Naive Bayes 0.77

Dataset: Iris (Classification)

# Algorithm Accuracy
1 Google Predict Classifier 0.97
2 Weka SVM (with RBF kernel function) 0.97
3 Prior Knowledge - Veritable - Classification 0.95
4 Weka Naive Bayes 0.95
5 BigML Classification Tree 0.95
6 Weka k-Nearest Neighbor (k = 3) classifier 0.95
7 Weka J48 Tree 0.95
8 Weka Adaboost Classifier with J48 as weak learner 0.93
# Algorithm Macro Average F1 Score
1 Google Predict Classifier 0.97
2 Weka SVM (with RBF kernel function) 0.96
3 BigML Classification Tree 0.95
4 Weka J48 Tree 0.94
5 Weka k-Nearest Neighbor (k = 3) classifier 0.92
6 Prior Knowledge - Veritable - Classification 0.92
7 Weka Naive Bayes 0.92
8 Weka Adaboost Classifier with J48 as weak learner 0.89
# Algorithm Macro Average Phi Coefficient
1 Google Predict Classifier 0.96
2 Weka SVM (with RBF kernel function) 0.95
3 BigML Classification Tree 0.93
4 Weka J48 Tree 0.92
5 Prior Knowledge - Veritable - Classification 0.90
6 Weka Naive Bayes 0.90
7 Weka k-Nearest Neighbor (k = 3) classifier 0.90
8 Weka Adaboost Classifier with J48 as weak learner 0.86

Dataset: Pen-Based Recognition of Handwritten Digits (Classification)

# Algorithm Accuracy
1 Weka k-Nearest Neighbor (k = 3) classifier 0.99
2 Weka Adaboost Classifier with J48 as weak learner 0.99
3 Google Predict Classifier 0.98
4 BigML Classification Tree 0.97
5 Weka J48 Tree 0.96
6 Weka Naive Bayes 0.88
7 Weka SVM (with RBF kernel function) 0.10
# Algorithm Macro Average F1 Score
1 Weka k-Nearest Neighbor (k = 3) classifier 0.99
2 Weka Adaboost Classifier with J48 as weak learner 0.99
3 Google Predict Classifier 0.98
4 BigML Classification Tree 0.96
5 Weka J48 Tree 0.96
6 Weka Naive Bayes 0.88
7 Weka SVM (with RBF kernel function) 0.03
# Algorithm Macro Average Phi Coefficient
1 Weka k-Nearest Neighbor (k = 3) classifier 0.99
2 Weka Adaboost Classifier with J48 as weak learner 0.99
3 Google Predict Classifier 0.98
4 BigML Classification Tree 0.96
5 Weka J48 Tree 0.96
6 Weka Naive Bayes 0.87
7 Weka SVM (with RBF kernel function) 0.04

Dataset: Wine (Classification)

# Algorithm Accuracy
1 Prior Knowledge - Veritable - Classification 0.97
2 Weka Naive Bayes 0.97
3 Google Predict Classifier 0.97
4 Weka Adaboost Classifier with J48 as weak learner 0.97
5 Weka k-Nearest Neighbor (k = 3) classifier 0.96
6 Weka J48 Tree 0.95
7 BigML Classification Tree 0.92
8 Weka SVM (with RBF kernel function) 0.44
# Algorithm Macro Average F1 Score
1 Prior Knowledge - Veritable - Classification 0.98
2 Weka Naive Bayes 0.97
3 Google Predict Classifier 0.97
4 Weka k-Nearest Neighbor (k = 3) classifier 0.96
5 Weka Adaboost Classifier with J48 as weak learner 0.96
6 Weka J48 Tree 0.94
7 BigML Classification Tree 0.90
8 Weka SVM (with RBF kernel function) 0.27
# Algorithm Macro Average Phi Coefficient
1 Prior Knowledge - Veritable - Classification 0.96
2 Weka Naive Bayes 0.95
3 Google Predict Classifier 0.95
4 Weka Adaboost Classifier with J48 as weak learner 0.94
5 Weka k-Nearest Neighbor (k = 3) classifier 0.94
6 Weka J48 Tree 0.92
7 BigML Classification Tree 0.86
8 Weka SVM (with RBF kernel function) 0.13

Dataset: Abalone (Regression)

# Algorithm Mean Squared Error
1 Weka Additive Regression (with M5P as weak classifier) 4.55
2 Weka M5P Tree 4.55
3 Weka Linear Regression 4.91
4 Google Predict Regressor 4.92
5 Weka SMOreg (support vector regression) 5.08
6 Prior Knowledge - Veritable - Regression 5.08
7 Weka k-Nearest Neighbor (k = 3) regressor 5.61
8 BigML Regression Tree 6.97
# Algorithm R-Squared Score
1 Weka Additive Regression (with M5P as weak classifier) 0.56
2 Weka M5P Tree 0.56
3 Weka Linear Regression 0.52
4 Google Predict Regressor 0.52
5 Prior Knowledge - Veritable - Regression 0.51
6 Weka SMOreg (support vector regression) 0.51
7 Weka k-Nearest Neighbor (k = 3) regressor 0.46
8 BigML Regression Tree 0.32

Dataset: Auto MPG (Regression)

# Algorithm Mean Squared Error
1 Weka M5P Tree 7.89
2 Weka Additive Regression (with M5P as weak classifier) 7.90
3 Weka k-Nearest Neighbor (k = 3) regressor 8.88
4 Weka Linear Regression 11.53
5 Weka SMOreg (support vector regression) 12.12
6 BigML Regression Tree 13.45
7 Prior Knowledge - Veritable - Regression 14.92
8 Google Predict Regressor 85.17
# Algorithm R-Squared Score
1 Weka M5P Tree 0.86
2 Weka Additive Regression (with M5P as weak classifier) 0.86
3 Weka k-Nearest Neighbor (k = 3) regressor 0.85
4 Weka Linear Regression 0.80
5 Weka SMOreg (support vector regression) 0.79
6 BigML Regression Tree 0.76
7 Prior Knowledge - Veritable - Regression 0.73
8 Google Predict Regressor -0.52

Dataset: Insurance Company Benchmark (COIL 2000) (Regression)

# Algorithm Mean Squared Error
1 Google Predict Regressor 0.05
2 Weka M5P Tree 0.05
3 Weka Linear Regression 0.05
4 Weka Additive Regression (with M5P as weak classifier) 0.05
5 Weka SMOreg (support vector regression) 0.06
6 Weka k-Nearest Neighbor (k = 3) regressor 0.07
7 BigML Regression Tree 0.10
# Algorithm R-Squared Score
1 Google Predict Regressor 0.04
2 Weka M5P Tree 0.04
3 Weka Linear Regression 0.04
4 Weka Additive Regression (with M5P as weak classifier) 0.02
5 Weka SMOreg (support vector regression) -0.06
6 Weka k-Nearest Neighbor (k = 3) regressor -0.22
7 BigML Regression Tree -0.78

Dataset: Communities and Crime (Regression)

# Algorithm Mean Squared Error
1 Weka M5P Tree 0.02
2 Weka SMOreg (support vector regression) 0.02
3 Weka Linear Regression 0.02
4 Weka Additive Regression (with M5P as weak classifier) 0.02
5 BigML Regression Tree 0.03
6 Weka k-Nearest Neighbor (k = 3) regressor 0.09
7 Google Predict Regressor 0.17
# Algorithm R-Squared Score
1 Weka SMOreg (support vector regression) 0.64
2 Weka M5P Tree 0.64
3 Weka Linear Regression 0.64
4 Weka Additive Regression (with M5P as weak classifier) 0.64
5 BigML Regression Tree 0.36
6 Weka k-Nearest Neighbor (k = 3) regressor -0.65
7 Google Predict Regressor -2.14

Dataset: Concrete Compressive Strength (Regression)

# Algorithm Mean Squared Error
1 Weka Additive Regression (with M5P as weak classifier) 37.46
2 Weka M5P Tree 38.73
3 BigML Regression Tree 48.41
4 Weka k-Nearest Neighbor (k = 3) regressor 81.20
5 Google Predict Regressor 109.04
6 Weka Linear Regression 109.27
7 Prior Knowledge - Veritable - Regression 116.80
8 Weka SMOreg (support vector regression) 119.75
# Algorithm R-Squared Score
1 Weka Additive Regression (with M5P as weak classifier) 0.86
2 Weka M5P Tree 0.86
3 BigML Regression Tree 0.82
4 Weka k-Nearest Neighbor (k = 3) regressor 0.71
5 Google Predict Regressor 0.61
6 Weka Linear Regression 0.60
7 Prior Knowledge - Veritable - Regression 0.58
8 Weka SMOreg (support vector regression) 0.56
Raw
results.csv
cross_validator cross_validator_seed dataset estimator score_mean score_metric score_std type
10-fold 1 breast_cancer bigml_classification_tree 0.951226309921962 accuracy 0.016386917052496 classification
10-fold 1 breast_cancer bigml_classification_tree 0.928196003807927 macro_avg_f1 0.028101360669502 classification
10-fold 1 breast_cancer bigml_classification_tree 0.891608738468192 macro_avg_phi_coef 0.039332492035009 classification
10-fold 1 breast_cancer weka_boosted_trees 0.965384615384615 accuracy 0.022793309418442 classification
10-fold 1 breast_cancer weka_boosted_trees 0.950151005198885 macro_avg_f1 0.034046100841199 classification
10-fold 1 breast_cancer weka_boosted_trees 0.924307743944268 macro_avg_phi_coef 0.050785360819803 classification
10-fold 1 breast_cancer weka_j48_tree 0.940914158305462 accuracy 0.0233316048344 classification
10-fold 1 breast_cancer weka_j48_tree 0.913608253477239 macro_avg_f1 0.036645992774037 classification
10-fold 1 breast_cancer weka_j48_tree 0.871394669196381 macro_avg_phi_coef 0.052594800049204 classification
10-fold 1 breast_cancer weka_knn_3_clf 0.965551839464883 accuracy 0.0118029615281 classification
10-fold 1 breast_cancer weka_knn_3_clf 0.949103825339267 macro_avg_f1 0.020051339461581 classification
10-fold 1 breast_cancer weka_knn_3_clf 0.923664082753319 macro_avg_phi_coef 0.028459383222066 classification
10-fold 1 breast_cancer weka_naive_bayes 0.960869565217391 accuracy 0.018389242812246 classification
10-fold 1 breast_cancer weka_naive_bayes 0.943736439119476 macro_avg_f1 0.028897005829922 classification
10-fold 1 breast_cancer weka_naive_bayes 0.915430987712097 macro_avg_phi_coef 0.041641500115954 classification
10-fold 1 breast_cancer weka_svm_rbf 0.664102564102564 accuracy 0.034545435278461 classification
10-fold 1 breast_cancer weka_svm_rbf 0.049404942883204 macro_avg_f1 0.0407654741452 classification
10-fold 1 breast_cancer weka_svm_rbf 0.101693553095898 macro_avg_phi_coef 0.083587522106414 classification
10-fold 1 diabetes bigml_classification_tree 0.704573934837093 accuracy 0.068278635967424 classification
10-fold 1 diabetes bigml_classification_tree 0.559577181464978 macro_avg_f1 0.10007245150889 classification
10-fold 1 diabetes bigml_classification_tree 0.341016458258455 macro_avg_phi_coef 0.147369740712292 classification
10-fold 1 diabetes weka_boosted_trees 0.734210526315789 accuracy 0.025514104512718 classification
10-fold 1 diabetes weka_boosted_trees 0.59437996031381 macro_avg_f1 0.069297670369576 classification
10-fold 1 diabetes weka_boosted_trees 0.400992375706477 macro_avg_phi_coef 0.078136961714044 classification
10-fold 1 diabetes weka_j48_tree 0.735275689223058 accuracy 0.039855977882827 classification
10-fold 1 diabetes weka_j48_tree 0.594165600696767 macro_avg_f1 0.080564270124875 classification
10-fold 1 diabetes weka_j48_tree 0.405325558780458 macro_avg_phi_coef 0.096995343829099 classification
10-fold 1 diabetes weka_knn_3_clf 0.738784461152882 accuracy 0.034612846809588 classification
10-fold 1 diabetes weka_knn_3_clf 0.588586128735434 macro_avg_f1 0.08722174201729 classification
10-fold 1 diabetes weka_knn_3_clf 0.403278772386714 macro_avg_phi_coef 0.09220359589005 classification
10-fold 1 diabetes weka_naive_bayes 0.753571428571428 accuracy 0.048636287267255 classification
10-fold 1 diabetes weka_naive_bayes 0.623834750948419 macro_avg_f1 0.081635695460394 classification
10-fold 1 diabetes weka_naive_bayes 0.444925808695859 macro_avg_phi_coef 0.112899862337749 classification
10-fold 1 diabetes weka_svm_rbf 0.651503759398496 accuracy 0.037567659955953 classification
10-fold 1 diabetes weka_svm_rbf 0 macro_avg_f1 0 classification
10-fold 1 diabetes weka_svm_rbf 0 macro_avg_phi_coef 0 classification
10-fold 1 glass bigml_classification_tree 0.967428571428572 accuracy 0.03021855386567 classification
10-fold 1 glass bigml_classification_tree 0.934943454116964 macro_avg_f1 0.106892252401758 classification
10-fold 1 glass bigml_classification_tree 0.931575850170558 macro_avg_phi_coef 0.108091047226758 classification
10-fold 1 glass weka_boosted_trees 0.976952380952381 accuracy 0.023148147543462 classification
10-fold 1 glass weka_boosted_trees 0.936527260179434 macro_avg_f1 0.083523592777672 classification
10-fold 1 glass weka_boosted_trees 0.935405712742695 macro_avg_phi_coef 0.083430615126526 classification
10-fold 1 glass weka_j48_tree 0.976952380952381 accuracy 0.023148147543462 classification
10-fold 1 glass weka_j48_tree 0.936527260179434 macro_avg_f1 0.083523592777672 classification
10-fold 1 glass weka_j48_tree 0.935405712742695 macro_avg_phi_coef 0.083430615126526 classification
10-fold 1 glass weka_knn_3_clf 0.90152380952381 accuracy 0.075376846141507 classification
10-fold 1 glass weka_knn_3_clf 0.78677456083648 macro_avg_f1 0.166784058864142 classification
10-fold 1 glass weka_knn_3_clf 0.778539149581789 macro_avg_phi_coef 0.172134608192175 classification
10-fold 1 glass weka_naive_bayes 0.830857142857143 accuracy 0.078511318998501 classification
10-fold 1 glass weka_naive_bayes 0.796917707160102 macro_avg_f1 0.107159628016662 classification
10-fold 1 glass weka_naive_bayes 0.766899316690241 macro_avg_phi_coef 0.117273292868926 classification
10-fold 1 glass weka_svm_rbf 0.977714285714286 accuracy 0.028698763869182 classification
10-fold 1 glass weka_svm_rbf 0.948903232790701 macro_avg_f1 0.079168104014354 classification
10-fold 1 glass weka_svm_rbf 0.945836818243419 macro_avg_phi_coef 0.08189026708681 classification
10-fold 1 iris bigml_classification_tree 0.953333333333333 accuracy 0.042687494916219 classification
10-fold 1 iris bigml_classification_tree 0.949955632602692 macro_avg_f1 0.047659951792652 classification
10-fold 1 iris bigml_classification_tree 0.929905393298776 macro_avg_phi_coef 0.065947279050746 classification
10-fold 1 iris weka_boosted_trees 0.926666666666667 accuracy 0.046666666666667 classification
10-fold 1 iris weka_boosted_trees 0.892455632602692 macro_avg_f1 0.097614092271965 classification
10-fold 1 iris weka_boosted_trees 0.859587932981316 macro_avg_phi_coef 0.108120729266006 classification
10-fold 1 iris weka_j48_tree 0.946666666666667 accuracy 0.049888765156986 classification
10-fold 1 iris weka_j48_tree 0.943288965936025 macro_avg_f1 0.053870960056862 classification
10-fold 1 iris weka_j48_tree 0.919905393298777 macro_avg_phi_coef 0.075641454807929 classification
10-fold 1 iris weka_knn_3_clf 0.946666666666667 accuracy 0.049888765156986 classification
10-fold 1 iris weka_knn_3_clf 0.921175676175676 macro_avg_f1 0.1031606716506 classification
10-fold 1 iris weka_knn_3_clf 0.898418606543324 macro_avg_phi_coef 0.114343263027383 classification
10-fold 1 iris weka_naive_bayes 0.953333333333333 accuracy 0.042687494916219 classification
10-fold 1 iris weka_naive_bayes 0.919814444814445 macro_avg_f1 0.103448583106163 classification
10-fold 1 iris weka_naive_bayes 0.901060510032628 macro_avg_phi_coef 0.113109152168249 classification
10-fold 1 iris weka_svm_rbf 0.966666666666667 accuracy 0.044721359549996 classification
10-fold 1 iris weka_svm_rbf 0.962696007696008 macro_avg_f1 0.050723817236708 classification
10-fold 1 iris weka_svm_rbf 0.948492918081535 macro_avg_phi_coef 0.069874459271534 classification
10-fold 1 pendigits bigml_classification_tree 0.965168431746977 accuracy 0.00368184138007 classification
10-fold 1 pendigits bigml_classification_tree 0.964857053721874 macro_avg_f1 0.003889018461871 classification
10-fold 1 pendigits bigml_classification_tree 0.96112946093136 macro_avg_phi_coef 0.004260609361135 classification
10-fold 1 pendigits weka_boosted_trees 0.989725299957269 accuracy 0.002391162807042 classification
10-fold 1 pendigits weka_boosted_trees 0.989678544944292 macro_avg_f1 0.002708145678401 classification
10-fold 1 pendigits weka_boosted_trees 0.9885946106445 macro_avg_phi_coef 0.002942036102772 classification
10-fold 1 pendigits weka_j48_tree 0.96010422041252 accuracy 0.00705779364467 classification
10-fold 1 pendigits weka_j48_tree 0.960209218586964 macro_avg_f1 0.007099435386359 classification
10-fold 1 pendigits weka_j48_tree 0.955942975008047 macro_avg_phi_coef 0.007889031922313 classification
10-fold 1 pendigits weka_knn_3_clf 0.99439677870627 accuracy 0.002213478571591 classification
10-fold 1 pendigits weka_knn_3_clf 0.994310722301037 macro_avg_f1 0.002253871244657 classification
10-fold 1 pendigits weka_knn_3_clf 0.993713103609172 macro_avg_phi_coef 0.002490454943369 classification
10-fold 1 pendigits weka_naive_bayes 0.880571173250922 accuracy 0.009518035393995 classification
10-fold 1 pendigits weka_naive_bayes 0.877762238744393 macro_avg_f1 0.008662422210273 classification
10-fold 1 pendigits weka_naive_bayes 0.867212588318019 macro_avg_phi_coef 0.009333127378116 classification
10-fold 1 pendigits weka_svm_rbf 0.098745206091522 accuracy 0.008869676911991 classification
10-fold 1 pendigits weka_svm_rbf 0.030874916853079 macro_avg_f1 0.012590420370936 classification
10-fold 1 pendigits weka_svm_rbf 0.035143388900732 macro_avg_phi_coef 0.021177266591916 classification
10-fold 1 wine bigml_classification_tree 0.915294117647059 accuracy 0.063963657501497 classification
10-fold 1 wine bigml_classification_tree 0.898911388425118 macro_avg_f1 0.070986035269931 classification
10-fold 1 wine bigml_classification_tree 0.864080579687316 macro_avg_phi_coef 0.098251804250762 classification
10-fold 1 wine weka_boosted_trees 0.966588235294118 accuracy 0.027816502035071 classification
10-fold 1 wine weka_boosted_trees 0.955805239594713 macro_avg_f1 0.0432054621438 classification
10-fold 1 wine weka_boosted_trees 0.942457117293388 macro_avg_phi_coef 0.05087614091305 classification
10-fold 1 wine weka_j48_tree 0.948941176470588 accuracy 0.041294620409207 classification
10-fold 1 wine weka_j48_tree 0.937564498853973 macro_avg_f1 0.048608671890465 classification
10-fold 1 wine weka_j48_tree 0.916864597922742 macro_avg_phi_coef 0.062585529460819 classification
10-fold 1 wine weka_knn_3_clf 0.958588235294118 accuracy 0.04633800894152 classification
10-fold 1 wine weka_knn_3_clf 0.957058099852218 macro_avg_f1 0.048217487710731 classification
10-fold 1 wine weka_knn_3_clf 0.939477585662414 macro_avg_phi_coef 0.06667042612509 classification
10-fold 1 wine weka_naive_bayes 0.968470588235294 accuracy 0.041497237995386 classification
10-fold 1 wine weka_naive_bayes 0.968590456384574 macro_avg_f1 0.040022051233726 classification
10-fold 1 wine weka_naive_bayes 0.954068112954002 macro_avg_phi_coef 0.058353386558566 classification
10-fold 1 wine weka_svm_rbf 0.440235294117647 accuracy 0.120614114870036 classification
10-fold 1 wine weka_svm_rbf 0.270209880212169 macro_avg_f1 0.092280636765713 classification
10-fold 1 wine weka_svm_rbf 0.133644503654394 macro_avg_phi_coef 0.121144000422805 classification
10-fold 1 breast_cancer prior_veritable_clf 0.971014492753623 accuracy 0.018332044406773 classification
10-fold 1 breast_cancer prior_veritable_clf 0.957713898504596 macro_avg_f1 0.029131025455308 classification
10-fold 1 breast_cancer prior_veritable_clf 0.936453832370214 macro_avg_phi_coef 0.042031386113793 classification
10-fold 1 diabetes prior_veritable_clf 0.758270676691729 accuracy 0.042923046544939 classification
10-fold 1 diabetes prior_veritable_clf 0.581134247690686 macro_avg_f1 0.073936655458044 classification
10-fold 1 diabetes prior_veritable_clf 0.440124541197298 macro_avg_phi_coef 0.091454780652115 classification
10-fold 1 glass prior_veritable_clf 0.932380952380952 accuracy 0.072961332286837 classification
10-fold 1 glass prior_veritable_clf 0.828878277463552 macro_avg_f1 0.198970939100008 classification
10-fold 1 glass prior_veritable_clf 0.825152346272562 macro_avg_phi_coef 0.202883687133711 classification
10-fold 1 iris prior_veritable_clf 0.953333333333333 accuracy 0.042687494916219 classification
10-fold 1 iris prior_veritable_clf 0.920406445406446 macro_avg_f1 0.103560626299866 classification
10-fold 1 iris prior_veritable_clf 0.901620527055097 macro_avg_phi_coef 0.113127432503581 classification
10-fold 1 wine prior_veritable_clf 0.974352941176471 accuracy 0.031941988939943 classification
10-fold 1 wine prior_veritable_clf 0.975441011230485 macro_avg_f1 0.031458321543618 classification
10-fold 1 wine prior_veritable_clf 0.963103874705891 macro_avg_phi_coef 0.046093331168768 classification
10-fold 1 diabetes google_predict_clf 0.760401002506266 accuracy 0.048088472399905 classification
10-fold 1 diabetes google_predict_clf 0.6149472288215 macro_avg_f1 0.069268995528017 classification
10-fold 1 diabetes google_predict_clf 0.45482121493494 macro_avg_phi_coef 0.109245349668621 classification
10-fold 1 glass google_predict_clf 0.961904761904762 accuracy 0.041513323271816 classification
10-fold 1 glass google_predict_clf 0.89 macro_avg_f1 0.147359474805032 classification
10-fold 1 glass google_predict_clf 0.888869652339296 macro_avg_phi_coef 0.147177689218105 classification
10-fold 1 iris google_predict_clf 0.973333333333333 accuracy 0.044221663871405 classification
10-fold 1 iris google_predict_clf 0.969088134088134 macro_avg_f1 0.050917237553235 classification
10-fold 1 iris google_predict_clf 0.957794361139438 macro_avg_phi_coef 0.069914132640715 classification
10-fold 1 wine google_predict_clf 0.966588235294118 accuracy 0.038285758552737 classification
10-fold 1 wine google_predict_clf 0.96756205644313 macro_avg_f1 0.035681382678228 classification
10-fold 1 wine google_predict_clf 0.952334058224315 macro_avg_phi_coef 0.052106419253482 classification
10-fold 1 pendigits google_predict_clf 0.983052569428561 accuracy 0.004226069705338 classification
10-fold 1 pendigits google_predict_clf 0.982762376081588 macro_avg_f1 0.004642463924349 classification
10-fold 1 pendigits google_predict_clf 0.98095032468926 macro_avg_phi_coef 0.005086024305822 classification
10-fold 1 breast_cancer google_predict_clf 0.956522 accuracy 0.012551 classification
10-fold 1 breast_cancer google_predict_clf 0.936725 macro_avg_f1 0.023127 classification
10-fold 1 breast_cancer google_predict_clf 0.904882 macro_avg_phi_coef 0.031323 classification
10-fold 1 abalone weka_additive_m5p 4.54992467826173 mean_squared_error 0.69959856521352 regression
10-fold 1 abalone weka_additive_m5p 0.561441208377698 r2_score 0.043725376290232 regression
10-fold 1 abalone weka_knn_3_reg 5.60703580539286 mean_squared_error 0.703635281332986 regression
10-fold 1 abalone weka_knn_3_reg 0.45766452388648 r2_score 0.05292216267921 regression
10-fold 1 abalone weka_linear_regression 4.91433700570902 mean_squared_error 0.627884170118274 regression
10-fold 1 abalone weka_linear_regression 0.524356912405225 r2_score 0.049632216308787 regression
10-fold 1 abalone weka_m5p_tree 4.55110670311694 mean_squared_error 0.699744317807047 regression
10-fold 1 abalone weka_m5p_tree 0.561339510933719 r2_score 0.043634828015723 regression
10-fold 1 abalone weka_smo_reg 5.07651287058334 mean_squared_error 0.627170523728217 regression
10-fold 1 abalone weka_smo_reg 0.50927437907874 r2_score 0.04184211388264 regression
10-fold 1 coil_2000 weka_additive_m5p 0.05488533321859 mean_squared_error 0.005790933239239 regression
10-fold 1 coil_2000 weka_additive_m5p 0.022713030026238 r2_score 0.034373206426247 regression
10-fold 1 coil_2000 weka_knn_3_reg 0.068336849984112 mean_squared_error 0.006212479887392 regression
10-fold 1 coil_2000 weka_knn_3_reg -0.219439432668953 r2_score 0.062711362336138 regression
10-fold 1 coil_2000 weka_linear_regression 0.054065457252389 mean_squared_error 0.005004972623676 regression
10-fold 1 coil_2000 weka_linear_regression 0.035925346751917 r2_score 0.037764920820893 regression
10-fold 1 coil_2000 weka_m5p_tree 0.053966083939415 mean_squared_error 0.005419128007854 regression
10-fold 1 coil_2000 weka_m5p_tree 0.038481984868146 r2_score 0.03670255821291 regression
10-fold 1 coil_2000 weka_smo_reg 0.059667425227722 mean_squared_error 0.006258612638789 regression
10-fold 1 coil_2000 weka_smo_reg -0.06175230019412 r2_score 0.007244003730246 regression
10-fold 1 abalone bigml_regression_tree 6.97421088643514 mean_squared_error 0.579670182091557 regression
10-fold 1 abalone bigml_regression_tree 0.322677436860679 r2_score 0.06526995008768 regression
10-fold 1 coil_2000 bigml_regression_tree 0.099562076226359 mean_squared_error 0.008063907499774 regression
10-fold 1 coil_2000 bigml_regression_tree -0.784941265905169 r2_score 0.187188130886462 regression
10-fold 1 concrete bigml_regression_tree 48.4123000459789 mean_squared_error 19.516299059764 regression
10-fold 1 concrete bigml_regression_tree 0.823146416957591 r2_score 0.071772768042928 regression
10-fold 1 concrete weka_additive_m5p 37.4607368087379 mean_squared_error 6.70626396022337 regression
10-fold 1 concrete weka_additive_m5p 0.862849721550737 r2_score 0.027888157379538 regression
10-fold 1 concrete weka_knn_3_reg 81.1957176951456 mean_squared_error 17.8455929410339 regression
10-fold 1 concrete weka_knn_3_reg 0.70602437564346 r2_score 0.058452678003436 regression
10-fold 1 concrete weka_linear_regression 109.274176883495 mean_squared_error 15.2013648348219 regression
10-fold 1 concrete weka_linear_regression 0.60452015593516 r2_score 0.039183883223734 regression
10-fold 1 concrete weka_m5p_tree 38.7255184359223 mean_squared_error 5.13677494328826 regression
10-fold 1 concrete weka_m5p_tree 0.858964009431649 r2_score 0.018994556251257 regression
10-fold 1 concrete weka_smo_reg 119.754272932039 mean_squared_error 22.439945387393 regression
10-fold 1 concrete weka_smo_reg 0.56419613372891 r2_score 0.081814819129096 regression
10-fold 1 communities weka_additive_m5p 0.019464509409114 mean_squared_error 0.002293895461941 regression
10-fold 1 communities weka_additive_m5p 0.635899303641782 r2_score 0.050071880383268 regression
10-fold 1 communities weka_knn_3_reg 0.087348456835904 mean_squared_error 0.010139704385443 regression
10-fold 1 communities weka_knn_3_reg -0.651912667472884 r2_score 0.332459741843092 regression
10-fold 1 communities weka_linear_regression 0.019405039394014 mean_squared_error 0.002481980609832 regression
10-fold 1 communities weka_linear_regression 0.636433465091892 r2_score 0.057352807562634 regression
10-fold 1 communities weka_m5p_tree 0.019202958667723 mean_squared_error 0.001792032582392 regression
10-fold 1 communities weka_m5p_tree 0.640025896469511 r2_score 0.049425870692006 regression
10-fold 1 communities weka_smo_reg 0.01922243913657 mean_squared_error 0.003092471920479 regression
10-fold 1 communities weka_smo_reg 0.641103992684875 r2_score 0.056554694297154 regression
10-fold 1 communities bigml_regression_tree 0.033733325583782 mean_squared_error 0.004224716239447 regression
10-fold 1 communities bigml_regression_tree 0.36247891024832 r2_score 0.139668357939383 regression
10-fold 1 auto_mpg weka_additive_m5p 7.90250490654664 mean_squared_error 2.72153795691769 regression
10-fold 1 auto_mpg weka_additive_m5p 0.86428278948242 r2_score 0.034031319668874 regression
10-fold 1 auto_mpg weka_knn_3_reg 8.88153111451173 mean_squared_error 4.12348660223311 regression
10-fold 1 auto_mpg weka_knn_3_reg 0.848152834944264 r2_score 0.059070982707936 regression
10-fold 1 auto_mpg weka_linear_regression 11.532176500982 mean_squared_error 3.56939821173046 regression
10-fold 1 auto_mpg weka_linear_regression 0.802126652606469 r2_score 0.03247599931947 regression
10-fold 1 auto_mpg weka_m5p_tree 7.89340639536279 mean_squared_error 2.72334051708283 regression
10-fold 1 auto_mpg weka_m5p_tree 0.864490667055228 r2_score 0.033868028722113 regression
10-fold 1 auto_mpg weka_smo_reg 12.1218403062193 mean_squared_error 4.72454728862622 regression
10-fold 1 auto_mpg weka_smo_reg 0.794097443903468 r2_score 0.048443160094442 regression
10-fold 1 auto_mpg bigml_regression_tree 13.4463127683323 mean_squared_error 4.69563179693518 regression
10-fold 1 auto_mpg bigml_regression_tree 0.76487398046169 r2_score 0.076827702909823 regression
10-fold 1 auto_mpg prior_veritable_reg 14.9248392432455 mean_squared_error 4.47928145806367 regression
10-fold 1 auto_mpg prior_veritable_reg 0.734196141178405 r2_score 0.087080143963041 regression
10-fold 1 concrete prior_veritable_reg 116.801297102311 mean_squared_error 14.4501273100566 regression
10-fold 1 concrete prior_veritable_reg 0.576098821804962 r2_score 0.040996457696531 regression
10-fold 1 abalone prior_veritable_reg 5.08258053934211 mean_squared_error 0.701936050215711 regression
10-fold 1 abalone prior_veritable_reg 0.509517745364222 r2_score 0.043877113232832 regression
10-fold 1 concrete google_predict_reg 109.037305585005 mean_squared_error 15.2893425822412 regression
10-fold 1 concrete google_predict_reg 0.605384256805329 r2_score 0.039555602560217 regression
10-fold 1 abalone google_predict_reg 4.9150511508671 mean_squared_error 0.62649331426591 regression
10-fold 1 abalone google_predict_reg 0.524291566090047 r2_score 0.049415635171014 regression
10-fold 1 coil_2000 google_predict_reg 0.053883740736006 mean_squared_error 0.005111250515672 regression
10-fold 1 coil_2000 google_predict_reg 0.039387872600548 r2_score 0.036871967301275 regression
10-fold 1 auto_mpg google_predict_reg 85.1716775639449 mean_squared_error 107.159988934036 regression
10-fold 1 auto_mpg google_predict_reg -0.518749288218349 r2_score 1.70531183149744 regression
10-fold 1 communities google_predict_reg 0.166552670127419 mean_squared_error 0.036545666627393 regression
10-fold 1 communities google_predict_reg -2.14488294659385 r2_score 0.872124135741322 regression
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment