Skip to content

Instantly share code, notes, and snippets.

@acharles7
Created July 18, 2019 21:14
Show Gist options
  • Save acharles7/4fd284e068c2f109c2e5f40563dbe05d to your computer and use it in GitHub Desktop.
Save acharles7/4fd284e068c2f109c2e5f40563dbe05d to your computer and use it in GitHub Desktop.
calculate entropy in decision tree algorithm
# x = np.random.rand(10,2) * 10
# y = np.random.rand(10,1) * 10
def calculate_entropy(pi):
total = 0
for p in pi:
p = p / sum(pi)
if p != 0:
total += p * np.log2(p)
else:
total += 0
total *= -1
return total
# entropy of data in ascending order
def entropy(x, y):
entropies = []
for i in x:
entropies.append(calculate_entropy(i))
entropies.sort()
return entropies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment