Load and run tensorflow simple audio recognition demo

asked 2018-10-29 07:56:30 -0500

Anil Patel gravatar image

updated 2018-10-31 03:47:35 -0500

Hi All,

I want to load and run tensorflow simple audio recognition demo (.pb or .pbtxt) file using opencv dnn module. I know we can load tensorflow model (.pb file) with following command:

 cvNet = cv2.dnn.readNetFromTensorflow('speech_recognition_graph.pb')

But I don't know how I can give audio .wav file as input to this model. Can anyone know how I can generate BLOB data from an audio file? Or is there any other way to run above model?

For More Info: https://www.tensorflow.org/tutorials/...

Note: I only have OpenCV python package and do not have straight forward tensorflow python package.

Sample .pbtxt file : C:\fakepath\conv.jpg

Sample Python code to run audio demo : C:\fakepath\tensorflow_audio_run_example.png

edit retag flag offensive close merge delete


opencv is an image processing library not audio library. I have never try what you want to do but my idea is to load wav data (use portaudio) and write data in Mat with 2 rows and N columns (stereo) type CV_XX. If you are lucky may be all layers used speech_recognition_graph.pb will be known by dnn opencv module

LBerger gravatar imageLBerger ( 2018-10-29 08:13:42 -0500 )edit

seems, they are generating spectrogram images, like this:

but @Anil Patel -- we have absolutely no idea, what your tf model does or contains. to help you, we'd need to know the python code, that generated it, or a pbtxt file or similar. do you have a link, where it came from ?

berak gravatar imageberak ( 2018-10-29 08:31:12 -0500 )edit

if it is something like this youll have a good chance with opencv's dnn. however, if it is really trying to parse audio files from inside the tf network, the dnn won't have the resp. layers for this.

berak gravatar imageberak ( 2018-10-29 08:55:45 -0500 )edit

Thank you LBerger and Berak for your kind response. I directly used the model that https://www.tensorflow.org/tutorials/... generated. I can share my .pb or .pbtxt file if you want. In fact, I want to run this model in IMX platform where I don't have tensorflow python package and I have opencv package and few basic required packages in python like numpy, pandas etc. I am struggling with giving input to this tensorflow model loaded using readNetFromTensorflow.

Anil Patel gravatar imageAnil Patel ( 2018-10-30 00:57:10 -0500 )edit

please add the pbtxt file to your question

berak gravatar imageberak ( 2018-10-30 01:15:34 -0500 )edit

Yes Berak. I tried to add .pbtxt and .pb file but file is not supported so not able to load. I also tried to add .zip file but still it won't show any error and nothing happens. Is there any other way?

Anil Patel gravatar imageAnil Patel ( 2018-10-30 01:38:14 -0500 )edit

edit the question, and paste it there.

berak gravatar imageberak ( 2018-10-30 01:40:05 -0500 )edit

Sorry Berak for all trouble. I have edit question and add file after rename it to .jpg Please download it and rename it to .pbtxt Sorry for all inconvenience.

Anil Patel gravatar imageAnil Patel ( 2018-10-30 01:50:51 -0500 )edit

it's ok, thanks !

unfortunately, like i feared, you won't be able to run this model on opencv's dnn. there is no "FileReader" or "WavDecoder" here (to start with)

berak gravatar imageberak ( 2018-10-30 01:59:25 -0500 )edit

OK. Totally understand your point. So now I have to find any other way to run this model apart from opencv dnn model. It was a very good learning from you. Thank you very much for your time and effort.

Anil Patel gravatar imageAnil Patel ( 2018-10-30 02:05:36 -0500 )edit