Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Excellent work!!!

I have put your code in a reusable function and added an stop parameter used when valid centroids values are reached:

import tensorflow as tf

def kMeansCluster(vector_values, num_clusters, max_num_steps, stop_coeficient = 0.0):
  vectors = tf.constant(vector_values)
  centroids = tf.Variable(tf.slice(tf.random_shuffle(vectors),
                                   [0,0],[num_clusters,-1]))
  old_centroids = tf.Variable(tf.zeros([num_clusters,2]))
  centroid_distance = tf.Variable(tf.zeros([num_clusters,2]))

  expanded_vectors = tf.expand_dims(vectors, 0)
  expanded_centroids = tf.expand_dims(centroids, 1)

  print expanded_vectors.get_shape()
  print expanded_centroids.get_shape()

  distances = tf.reduce_sum(
    tf.square(tf.sub(expanded_vectors, expanded_centroids)), 2)
  assignments = tf.argmin(distances, 0)

  means = tf.concat(0, [
    tf.reduce_mean(
        tf.gather(vectors,
                  tf.reshape(
                    tf.where(
                      tf.equal(assignments, c)
                    ),[1,-1])
                 ),reduction_indices=[1])
    for c in xrange(num_clusters)])

  save_old_centroids = tf.assign(old_centroids, centroids)

  update_centroids = tf.assign(centroids, means)
  init_op = tf.initialize_all_variables()

  performance = tf.assign(centroid_distance, tf.sub(centroids, old_centroids))
  check_stop = tf.reduce_sum(tf.abs(performance))

  with tf.Session() as sess:
    sess.run(init_op)
    for step in xrange(max_num_steps):
      print "Running step " + str(step)
      sess.run(save_old_centroids)
      _, centroid_values, assignment_values = sess.run([update_centroids,
                                                        centroids,
                                                        assignments])
      sess.run(check_stop)
      current_stop_coeficient = check_stop.eval()
      print "coeficient:", current_stop_coeficient
      if current_stop_coeficient <= stop_coeficient:
        break

    return centroid_values, assignment_values
Owner

narphorium commented Jan 29, 2016

Thanks @germanramos! That looks great.

vlad17 commented Apr 27, 2016

This looks like it serializes the centroids and assignments, copies them from the backend to the python process, and then sends them back to the engine in the next step. Is there any way to avoid this copying without making max_num_steps ops?

Owner

narphorium commented Jun 6, 2016

That's a good point @vlad17. You can do iteration in TF with tf.tf.while_loop but it is a bit more advanced.

Good tutorial,

We can simplify the code of calculating the means by using tf.boolean_mask instead of tf.reshape(tf.where(..)):

means = tf.pack([
    tf.reduce_mean(
        tf.boolean_mask(
            vectors, tf.equal(assignments, c)
        ), 0) 
    for c in xrange(num_clusters)])

I think it's more intuitive

h4p commented Sep 3, 2016

Hello,

when I input values of shape (1000,1), I'm getting a lot of NaNs in the centroid list.

array([[-0.0615779 ],
       [ 0.        ],
       [-0.01855482],
       [        nan],
       [        nan],
       [        nan],
       [        nan],
       [-0.03768255],
       [ 0.01288017],
       [ 0.01535422],
       [ 0.04958867],
       [        nan],
       [-0.01960552],
       [ 0.09472825],
       [-0.09461572],
       [        nan]]

Basically I want to do the same as this MATLAB code does:

  >> load fisheriris
  >> X = meas(:,3); 
  >> [idx,C] = kmeans(X,3);
  >> size(X) => [150,1]
  >> size(idx) => [150,1]
  >> size(C) => [3,1]

I think there's problem with the calculation of means, because this is where the assignment for centroids is coming from, but I'm not sure where the nan is coming from. Can somebody please give me a hint to fix? :)

tf.sub need changes to tf.subtract
and

means = tf.concat(0, [
    tf.reduce_mean(
        tf.gather(vectors,
                  tf.reshape(
                    tf.where(
                      tf.equal(assignments, c)
                    ),[1,-1])
                 ),reduction_indices=[1])
    for c in xrange(num_clusters)])

to

means = tf.concat([
    tf.reduce_mean(
        tf.gather(vectors,
                  tf.reshape(
                    tf.where(
                      tf.equal(assignments, c)
                    ),[1,-1])
                 ),reduction_indices=[1])
    for c in xrange(num_clusters)], 0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment