Skip to content

Instantly share code, notes, and snippets.

@michelkana

michelkana/boosting_simplified.py

Created August 31, 2021 11:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save michelkana/7ea3a9d26a5e512ed6131f5e3ae94330 to your computer and use it in GitHub Desktop.
Save michelkana/7ea3a9d26a5e512ed6131f5e3ae94330 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
boosting_simplified.py
# Single decision tree1 trained on original dataset
tree1 = DecisionTreeClassifier(max_depth=3).fit(X_train, y_train)
y_train_predicted_tree1 = tree1.predict(X_train)
y_test_predicted_tree1 = tree1.predict(X_test)
# Modified dataset, weighted by residuals
y_train_predicted_tree1_bool = y_train_predicted_tree1 == y_train
sample_weights = np.ones(len(X_train))
sample_weights[np.logical_not(y_train_predicted_tree1_bool)] = 2
# Single decision tree2 trained on modified dataset
tree2 = DecisionTreeClassifier(max_depth=3).fit(X_train, y_train, sample_weight=sample_weights)
y_train_predicted_tree2 = tree2.predict(X_train)
y_test_predicted_tree2 = tree2.predict(X_test)
# Combined predictions as average of both tree1 and tree2
y_train_predicted_boosted = ((tree1.predict_proba(X_train)[:,1] + tree2.predict_proba(X_train)[:,1])/2 > 0.5).astype('int')
y_test_predicted_boosted = ((tree1.predict_proba(X_test)[:,1] + tree2.predict_proba(X_test)[:,1])/2 > 0.5).astype('int')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment