Skip to content

Instantly share code, notes, and snippets.

@ruslo
Created December 23, 2018 14:11
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save ruslo/a07c9a615aed6bdcbff3c4c7ce2eec45 to your computer and use it in GitHub Desktop.
Save ruslo/a07c9a615aed6bdcbff3c4c7ce2eec45 to your computer and use it in GitHub Desktop.
/*!
* Copyright (c) 2017 by Contributors
* \brief Example code on load and run TVM module.s
* \file cpp_deploy_example.cc
*/
#include <cstdio>
#include <dlpack/dlpack.h>
#include <tvm/runtime/module.h>
#include <tvm/runtime/registry.h>
#include <tvm/runtime/packed_func.h>
void Verify(tvm::runtime::Module mod, std::string fname) {
// Get the function from the module.
tvm::runtime::PackedFunc f = mod.GetFunction(fname);
CHECK(f != nullptr);
// Allocate the DLPack data structures.
//
// Note that we use TVM runtime API to allocate the DLTensor in this example.
// TVM accept DLPack compatible DLTensors, so function can be invoked
// as long as we pass correct pointer to DLTensor array.
//
// For more information please refer to dlpack.
// One thing to notice is that DLPack contains alignment requirement for
// the data pointer and TVM takes advantage of that.
// If you plan to use your customized data container, please
// make sure the DLTensor you pass in meet the alignment requirement.
//
DLTensor* x;
DLTensor* y;
int ndim = 1;
int dtype_code = kDLFloat;
int dtype_bits = 32;
int dtype_lanes = 1;
int device_type = kDLOpenCL;
int device_id = 0;
const int size = 10;
int64_t shape[1] = {size};
TVMArrayAlloc(shape, ndim, dtype_code, dtype_bits, dtype_lanes,
device_type, device_id, &x);
TVMArrayAlloc(shape, ndim, dtype_code, dtype_bits, dtype_lanes,
device_type, device_id, &y);
std::vector<float> x_input(size);
std::vector<float> y_output(size);
for (int i = 0; i < size; ++i) {
x_input[i] = i;
}
TVMArrayCopyFromBytes(x, x_input.data(), size * sizeof(float));
// Invoke the function
// PackedFunc is a function that can be invoked via positional argument.
// The signature of the function is specified in tvm.build
f(x, y);
TVMArrayCopyToBytes(y, y_output.data(), size * sizeof(float));
// Print out the output
for (int i = 0; i < size; ++i) {
CHECK_EQ(y_output[i], i + 1.0f);
}
LOG(INFO) << "Finish verification...";
}
int main(void) {
tvm::runtime::Module mod_dylib =
tvm::runtime::Module::LoadFromFile("/full/path/to/lib/libtest_addone.so");
LOG(INFO) << "Verify dynamic loading from test_addone.so";
Verify(mod_dylib, "addone");
return 0;
}
@notebookdata
Copy link

Can you please share "libtest_addone.so" file i am still facing same problem.

@notebookdata
Copy link

I am still facing below error:

terminate called after throwing an instance of 'dmlc::Error'
what(): [13:32:15] cpp_deploy.cc:15: Check failed: f != nullptr

Stack trace returned 6 entries:
[bt] (0) ./testdeploy() [0x40394f]
[bt] (1) ./testdeploy() [0x403c5d]
[bt] (2) ./testdeploy() [0x402a25]
[bt] (3) ./testdeploy() [0x402fc2]
[bt] (4) /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7fc4b3d46830]
[bt] (5) ./testdeploy() [0x402789]

I followed steps to create module.
Installation:
https://docs.tvm.ai/install/from_source.html

Created module using:
https://docs.tvm.ai/tutorials/relay_quick_start.html#sphx-glr-tutorials-relay-quick-start-py

and tried to deploy module using above your code still getting error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment