Skip to content

Instantly share code, notes, and snippets.

Last active April 30, 2020 13:40
  • Star 23 You must be signed in to star a gist
  • Fork 18 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Analysis for customer segmentation blog post
import pandas as pd
df_offers = pd.read_excel("./WineKMC.xlsx", sheetname=0)
df_offers.columns = ["offer_id", "campaign", "varietal", "min_qty", "discount", "origin", "past_peak"]
df_transactions = pd.read_excel("./WineKMC.xlsx", sheetname=1)
df_transactions.columns = ["customer_name", "offer_id"]
df_transactions['n'] = 1
# join the offers and transactions table
df = pd.merge(df_offers, df_transactions)
# create a "pivot table" which will give us the number of times each
# customer responded to a given variable
matrix = df.pivot_table(index=['customer_name'], columns=['offer_id'], values='n')
# a little tidying up. fill NA values with 0 and make the index into a column
matrix = matrix.fillna(0).reset_index()
x_cols = matrix.columns[1:]
from sklearn.cluster import KMeans
cluster = KMeans(n_clusters=5)
# slice matrix so we only include the 0/1 indicator columns in the clustering
matrix['cluster'] = cluster.fit_predict(matrix[x_cols])
from ggplot import *
ggplot(matrix, aes(x='factor(cluster)')) + geom_bar() + xlab("Cluster") + ylab("Customers\n(# in cluster)")
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
matrix['x'] = pca.fit_transform(matrix[x_cols])[:,0]
matrix['y'] = pca.fit_transform(matrix[x_cols])[:,1]
matrix = matrix.reset_index()
customer_clusters = matrix[['customer_name', 'cluster', 'x', 'y']]
df = pd.merge(df_transactions, customer_clusters)
df = pd.merge(df_offers, df)
from ggplot import *
ggplot(df, aes(x='x', y='y', color='cluster')) + \
geom_point(size=75) + \
ggtitle("Customers Grouped by Cluster")
cluster_centers = pca.transform(cluster.cluster_centers_)
cluster_centers = pd.DataFrame(cluster_centers, columns=['x', 'y'])
cluster_centers['cluster'] = range(0, len(cluster_centers))
ggplot(df, aes(x='x', y='y', color='cluster')) + \
geom_point(size=75) + \
geom_point(cluster_centers, size=500) +\
ggtitle("Customers Grouped by Cluster")
df['is_4'] = df.cluster==4
df.groupby("is_4")[['min_qty', 'discount']].mean()
Copy link

Hi, Thanks for this! I have a few questions though as I get different clusters and different centers. My results differ from those on the blog.

Copy link

@glamp Line 25 differs from the blog post on Yhat.

Copy link

bjhaveri commented Nov 8, 2016

When I try to run visualize a similar dataset using ggplot, it just hangs. I have over a million records. Any ideas on how I can visualize the clusters.

Copy link

Awilonk commented Nov 16, 2016

I find it hard to display the center.
my ggplot seems can only display one size...
I tried older ggplot which can even no plot two data.

plz let me know the version of your ggplot

Copy link

Awilonk commented Nov 16, 2016

this is your code running on my pc..
do you know how to fix it?

Copy link

alitrack commented May 3, 2017

when execute

cluster_centers = pca.transform(cluster.cluster_centers_)

I got the following error

ValueError                                Traceback (most recent call last)
<ipython-input-16-989caa1671e5> in <module>()
----> 1 cluster_centers = pca.transform(cluster.cluster_centers_)

/home/ubuntu/tensorflow/lib/python3.4/site-packages/sklearn/decomposition/ in transform(self, X, y)
    130         X = check_array(X)
    131         if self.mean_ is not None:
--> 132             X = X - self.mean_
    133         X_transformed = fast_dot(X, self.components_.T)
    134         if self.whiten:

ValueError: operands could not be broadcast together with shapes (5,31) (32,) 

I use python3

Copy link

avinash-mishra commented Jun 5, 2018

Where is the blog post?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment