Skip to content

Instantly share code, notes, and snippets.

@hietalajulius
Created July 28, 2023 12:40
Show Gist options
  • Save hietalajulius/9e101f3d1715d54e952da5003a6fb82f to your computer and use it in GitHub Desktop.
Save hietalajulius/9e101f3d1715d54e952da5003a6fb82f to your computer and use it in GitHub Desktop.
/// Perform forward pass through the ReLU activation function.
fn forward(&mut self, x: &Array2<f64>) -> Array2<f64> {
// Calculate dy/dx for backpropagation
let dy_dx = x.mapv(|val| if val > 0.0 { 1.0 } else { 0.0 });
// Store dy/dx for later use in backward pass
self.dy_dx = Some(dy_dx);
// Get the output of the ReLU activation function
self.get_output(x)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment