Ask Your Question
1

how to import torch model with nn.SpatialUpSamplingNearest

asked 2018-03-24 20:30:30 -0500

BigMao Chen gravatar image

Hi guys~~ I haved trained a torch model with layers of nn.SpatialUpSamplingNearest. But when I import it into opencv, error occured "Unknown nn class nn.SpatialUpSamplingNearest" and I checked opencv documentation and it seems that opencv 3.4.1 does not support this kind of layer.

I wonder what can I do to solve this problem? Thanks for your patience:)

edit retag flag offensive close merge delete

Comments

@BigMao Chen, Thanks! You're right, this layer isn't supported for now. There are two ways to solve the problem. First, add a new type condition into torch_importer. The second is to extend a PR https://github.com/opencv/opencv/pull... to ignore unknown layers but let user define custom ones (Caffe and TensorFlow importers support it there but it isn't complicated to do the same for Torch).

dkurt gravatar imagedkurt ( 2018-03-25 03:45:49 -0500 )edit

@dkurt thanks for your reply, because i am new in c++ and opencv, here I have some puzzles after reading your code, could you please give me some more instructions?

Are these all codes for a new custom layer in link text

Here are things I have done: 1~ In windows10 I use cmake to generate sln for opencv 2~ I follow your instruction to change codes in "modules/dnn/include/opencv2/dnn/dict.hpp" "modules/dnn/include/opencv2/dnn/dnn.inl.hpp" "modules/dnn/src/torch/torch_importer.cpp" 3~ I rebuild opencv in order to get new opencv_world341.lib and dll (rebuilding)

And after replace the old lib and dll with new one, how can I use the codes you provided in test_torch_importer.cpp?

BigMao Chen gravatar imageBigMao Chen ( 2018-03-25 20:49:41 -0500 )edit

@BigMao Chen, If you use git, you need to add a remote repository by git remote add dkurt https://github.com/dkurt/opencv.git. Then fetch a local branch by git fetch dkurt dnn_custom_layers. After that you need to checkout to it: git checkout dnn_custom_layers. Now you can build it. However you can just wait unless the pull request is merged.

dkurt gravatar imagedkurt ( 2018-03-25 23:47:45 -0500 )edit

@dkurt thanks for your instructions, I have download your opencv version and rebuid it. finally I import my model sucessfully. By the way, if I construct a model with concatTable() layer and import it in opencv, how many outputs will opencv generate for the input? I check the codes in concat_layer.cpp, and found in class ConcatLayerImpl the default set seems like one input for one output. because I have trained a model with concatTable layer and the forward step woks well in torch. but in opencv error appears "incorrect size of input array<inconsistent shape="" for="" concatlayerimpl="">, in cv::dnn::ConcatLayerlmpl::getMemoryShapes "

BigMao Chen gravatar imageBigMao Chen ( 2018-03-26 21:42:56 -0500 )edit

@BigMao Chen, please add more details. Add torch code and minimal working example of using OpenCV. It'd be OK if you started a new topic at http://answers.opencv.org. Thanks!

dkurt gravatar imagedkurt ( 2018-03-27 01:18:30 -0500 )edit

ok I will add a new topic~

BigMao Chen gravatar imageBigMao Chen ( 2018-03-27 01:27:40 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
2

answered 2018-03-25 04:56:24 -0500

dkurt gravatar image

@BigMao Chen, please take a look at a PR https://github.com/opencv/opencv/pull.... We've introduced a way to define custom layers for models from Torch, Caffe and TensorFlow. You may find a tutorial and a sample of how to implement this layer in a test_torch_importer.cpp file.

edit flag offensive delete link more

Comments

thanks a lot:) let me have a try first~~~

BigMao Chen gravatar imageBigMao Chen ( 2018-03-25 06:55:44 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-03-24 20:30:30 -0500

Seen: 119 times

Last updated: Mar 25 '18