Given the Cross Entroy Cost Formula:
where:
J
is the averaged cross entropy costm
is the number of samples- super script
[L]
corresponds to output layer - super script
(i)
corresponds to thei
th sample - A is the activation matrix
- Y is the true output label
log()
is the natural logarithm
We can implement this in Numpy in either the np.sum
or np.dot
styles, like this:
The cross entropy lost is defined as (using the np.sum
style):
cost = -(1.0/m) * np.sum(Y*np.log(A) + (1-Y)*np.log(1-A))
Note: A
is the Activation Matrix in the output layer L
, and Y
is the true label matrix at that same layer. Both have dimensions (n_y, m)
, where n_y
is number of nodes at output layer, and m
is number of samples.
This is equivalent as the followings "dot product" style:
cost = -(1.0/m) * (np.dot(np.log(A), Y.T) + np.dot(np.log(1-A), (1-Y).T))
The np.sum
method is probably easier to read. Computational efficiency wise I wonder which one is preferred? (requires validation)
- deeplearning.ai course
Dear @kumarsubodh, as the dot products are already scalars (single numbers) the np.sum (which sums the array of products in the other style - https://docs.scipy.org/doc/numpy/reference/generated/numpy.sum.html ) is not needed in "np dot style".