So I don't really think that it makes sense to allow such parameters. If you don't want to optimize some tensors, they're not parameters - they're fixed. You probably don't want to count them in. And if you really need to then
optimizer.SGD(filter(lambda p: p.requires_grad, model.parameters()), lr=1e-3) should do the trick.