Skip to content

Instantly share code, notes, and snippets.

@ruslo ruslo/cpp_deploy.cc
Created Dec 23, 2018

Embed
What would you like to do?
/*!
* Copyright (c) 2017 by Contributors
* \brief Example code on load and run TVM module.s
* \file cpp_deploy_example.cc
*/
#include <cstdio>
#include <dlpack/dlpack.h>
#include <tvm/runtime/module.h>
#include <tvm/runtime/registry.h>
#include <tvm/runtime/packed_func.h>
void Verify(tvm::runtime::Module mod, std::string fname) {
// Get the function from the module.
tvm::runtime::PackedFunc f = mod.GetFunction(fname);
CHECK(f != nullptr);
// Allocate the DLPack data structures.
//
// Note that we use TVM runtime API to allocate the DLTensor in this example.
// TVM accept DLPack compatible DLTensors, so function can be invoked
// as long as we pass correct pointer to DLTensor array.
//
// For more information please refer to dlpack.
// One thing to notice is that DLPack contains alignment requirement for
// the data pointer and TVM takes advantage of that.
// If you plan to use your customized data container, please
// make sure the DLTensor you pass in meet the alignment requirement.
//
DLTensor* x;
DLTensor* y;
int ndim = 1;
int dtype_code = kDLFloat;
int dtype_bits = 32;
int dtype_lanes = 1;
int device_type = kDLOpenCL;
int device_id = 0;
const int size = 10;
int64_t shape[1] = {size};
TVMArrayAlloc(shape, ndim, dtype_code, dtype_bits, dtype_lanes,
device_type, device_id, &x);
TVMArrayAlloc(shape, ndim, dtype_code, dtype_bits, dtype_lanes,
device_type, device_id, &y);
std::vector<float> x_input(size);
std::vector<float> y_output(size);
for (int i = 0; i < size; ++i) {
x_input[i] = i;
}
TVMArrayCopyFromBytes(x, x_input.data(), size * sizeof(float));
// Invoke the function
// PackedFunc is a function that can be invoked via positional argument.
// The signature of the function is specified in tvm.build
f(x, y);
TVMArrayCopyToBytes(y, y_output.data(), size * sizeof(float));
// Print out the output
for (int i = 0; i < size; ++i) {
CHECK_EQ(y_output[i], i + 1.0f);
}
LOG(INFO) << "Finish verification...";
}
int main(void) {
tvm::runtime::Module mod_dylib =
tvm::runtime::Module::LoadFromFile("/full/path/to/lib/libtest_addone.so");
LOG(INFO) << "Verify dynamic loading from test_addone.so";
Verify(mod_dylib, "addone");
return 0;
}
@myproject24

This comment has been minimized.

Copy link

commented Mar 21, 2019

Can you please share "libtest_addone.so" file i am still facing same problem.

@myproject24

This comment has been minimized.

Copy link

commented Mar 21, 2019

I am still facing below error:

terminate called after throwing an instance of 'dmlc::Error'
what(): [13:32:15] cpp_deploy.cc:15: Check failed: f != nullptr

Stack trace returned 6 entries:
[bt] (0) ./testdeploy() [0x40394f]
[bt] (1) ./testdeploy() [0x403c5d]
[bt] (2) ./testdeploy() [0x402a25]
[bt] (3) ./testdeploy() [0x402fc2]
[bt] (4) /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7fc4b3d46830]
[bt] (5) ./testdeploy() [0x402789]

I followed steps to create module.
Installation:
https://docs.tvm.ai/install/from_source.html

Created module using:
https://docs.tvm.ai/tutorials/relay_quick_start.html#sphx-glr-tutorials-relay-quick-start-py

and tried to deploy module using above your code still getting error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.