Instantly share code, notes, and snippets.

# apaszke/jacobian_hessian.py

Last active August 3, 2023 08:26
Star You must be signed in to star a gist
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters

### Ronnypetson commented Jun 26, 2020

Hello Adam! How could I give credit to you if I use this code? Can it be a doc-string in documentation, paper citation or something?

### guanshaoheng commented Apr 17, 2021 • edited

Now the function `torch.autograd.functional.jacobian` can do the same thing, I think.

``````
def jacobian(y, x, create_graph=False):
# xx, yy = x.detach().numpy(), y.detach().numpy()
jac = []
flat_y = y.reshape(-1)
for i in range(len(flat_y)):

def hessian(y, x):
return jacobian(jacobian(y, x, create_graph=True), x)

def f(xx):
# y = x * x * torch.arange(4, dtype=torch.float)
matrix = torch.tensor([[0.2618, 0.2033, 0.7280, 0.8618],
[0.1299, 0.6498, 0.6675, 0.0527],
[0.3006, 0.9691, 0.0824, 0.8513],
y = torch.einsum('ji, i -> j', (matrix, xx))
return y

if __name__ == "__main__":
# matrix = torch.rand(4, 4, requires_grad=True)
# print(matrix)
print(jacobian(f(x), x))
# print(hessian(f(x, matrix), x))
``````

output

``````        [0.1299, 0.6498, 0.6675, 0.0527],
[0.3006, 0.9691, 0.0824, 0.8513],
[[0.2618 0.2033 0.728  0.8618]
[0.1299 0.6498 0.6675 0.0527]
[0.3006 0.9691 0.0824 0.8513]
[0.7914 0.2796 0.3717 0.9483]]```
``````

### AjinkyaBankar commented May 12, 2021

Hi,
I want to find a Hessian matrix for the loss function of the pre-trained neural network with respect to the parameters of the network. How can I use this method? Can someone please share an example? Thanks.

### maryamaliakbari commented Oct 25, 2021

Hi,
I want to find a Hessian matrix for the loss function of the pre-trained neural network with respect to the parameters of the network. How can I use this method? Can someone please share an example? Thanks.

Hi,
I am looking for the same thing. Could you figure out how we can do it?