Created
March 13, 2023 17:00
-
-
Save bshambaugh/557947afd4f9c7d6b22bcbf0b9e46f0a to your computer and use it in GitHub Desktop.
RNN shape?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I worked with this model during the EthDenver https://github.com/spro/char-rnn.pytorch . I did not manage to get the right shape for https://github.com/zkonduit/pyezkl/tree/main/examples/tutorial . (loading the model to convert to *.onnx) | |
I learned a bit about ML though. :P | |
You | |
11:48 AM | |
https://gist.github.com/bshambaugh/af01e7366ac7f43abd8295533703d990 (other scratch work) | |
.... | |
You | |
11:51 AM | |
Here is my colab python sketches: https://colab.research.google.com/drive/1mi2Rcehz-YWOeXGBNjCe-COR5XhnfPTB |
Use this for inspiration: https://github.com/hunter-z-hunter/hunter-z-hunter/blob/main/hunter.ipynb
https://github.com/hunter-z-hunter/hunter-z-hunter/blob/main/setup/trainer.py (maybe this is useful?)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jason, [2/24/23 1:01 PM]
https://github.com/zkonduit/ezkl
Jason, [2/26/23 3:34 PM]
https://github.com/nikhilbarhate99/Char-RNN-PyTorch/blob/master/CharRNN.py
Jason, [2/26/23 3:34 PM]
This might be doable
Jason, [2/26/23 3:50 PM]
The original is here: https://github.com/karpathy/char-rnn
Brent Shambaugh, [3/1/23 5:10 PM]
https://gist.github.com/karpathy/d4dee566867f8291f086 (minimal RNN)
Brent Shambaugh, [3/1/23 5:36 PM]
https://github.com/spro/char-rnn.pytorch (also try tis one
Brent Shambaugh, [3/1/23 11:50 PM]
https://github.com/zkonduit/ezkl/blob/4a444b249396b6a09e4c323143f6daf1b71b4c92/README.md#onnx-examples
onnx examples
This repository includes onnx example files as a submodule for testing out the cli.
If you want to add a model to examples/onnx, open a PR creating a new folder within examples/onnx with a descriptive model name. This folder should contain:
TODO: add associated python files in the onnx model directories.
It looks like the export function is failing because the shape of the input tensor is not matching the provided input_shape argument.
When calling the export function, you need to make sure that the input_array argument has the correct shape for the model input. In this case, the model input size is (batch_size, input_size), where batch_size is the number of input sequences and input_size is the length of each input sequence. So if you want to export a single input sequence with length 4, you should pass a 2D tensor with shape (1, 4).
Here's an example of how to do this:
lua
input_array = torch.tensor([[0, 1, 2, 3]])
hidden = rnn.init_hidden(1)
export(rnn, [1, 4], input_array, hidden)
This will export the model with an input tensor of shape (1, 4) and a batch size of 1. Note that you need to wrap the input values in a PyTorch tensor before passing them to the export function.
https://github.com/zkonduit/pyezkl/tree/main/examples/tutorial
As noted above this graph takes in 3 inputs and produces 2 outputs. The main function instantiates an instance of Circuit and saves it to an Onnx file.
def main():
torch_model = Circuit()
# Input to the model
shape = [3, 2, 2]
x = 0.1*torch.rand(1,shape, requires_grad=True)
y = 0.1torch.rand(1,shape, requires_grad=True)
z = 0.1torch.rand(1,*shape, requires_grad=True)
torch_out = torch_model(x, y, z)
# Export the model
torch.onnx.export(torch_model, # model being run
(x,y,z), # model input (or a tuple for multiple inputs)
"network.onnx", # where to save the model (can be a file or file-like object)
export_params=True, # store the trained parameter weights inside the model file
opset_version=10, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
input_names = ['input'], # the model's input names
output_names = ['output'], # the model's output names
dynamic_axes={'input' : {0 : 'batch_size'}, # variable length axes
'output' : {0 : 'batch_size'}})
if name == "main":
main()
Running the file generate an .onnx file. Note that this also create the required input json file, whereby we use the outputs of the pytorch model as the public inputs to the circuit.
Brent Shambaugh, [3/2/23 12:04 AM]
here is what I am working with loading the .onnx
Brent Shambaugh, [3/2/23 1:37 AM]
maybe this says something about shapes: https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/
Brent Shambaugh, [3/2/23 1:44 AM]
https://github.com/spro/char-rnn.pytorch/blob/master/model.py
Brent Shambaugh, [3/2/23 2:06 AM]
https://ppasumarthi-69210.medium.com/language-model-using-char-rnn-1df53f735880 (shapes?)
Brent Shambaugh, [3/2/23 2:11 AM]
https://stackoverflow.com/questions/61632584/understanding-input-shape-to-pytorch-lstm (# Size: [batch_size, seq_len, input_size])
Brent Shambaugh, [3/2/23 2:14 AM]
https://jvns.ca/blog/2020/11/30/implement-char-rnn-in-pytorch/ (another thing that might help find shapes) ... hint .... look at the training python code