Using SuperResolution with a Lepton 3.5 PureThermal 2

edit

I managed to get this working and the way to use your own buffer from an image, network feed, or any other source is to subclass cv::superres::FrameSource. There are two functions that must be over ridden. setFrame and nextFrame. nextFrame is the API the super resolution classes calls on the framesource to get whatever back buffer is required.

Since I am running super resolution with Cuda, my data must be uploaded to the GPU. And when the library calls nextFrame, I copy this gpu data into the destination buffer. Then super resolution takes over and uses the destination buffer it passes.

///
/// \brief The ThermalFrameSource class
///
class ThermalFrameSource : public cv::superres::FrameSource
{

public:
    ThermalFrameSource() : cv::superres::FrameSource()
    {

    }

    ///
    /// \brief ThermalFrameSource::setFrame
    /// \param frameToUse
    ///
    /// Note frameToUse is on the cpu, send to GPU
    virtual void setFrame( cv::Mat & frameToUse )
    {
        GpuFrame.upload(frameToUse);
    }

    ///
    /// \brief ThermalFrameSource::nextFrame
    /// \param frame
    ///
    /// Note this is on the gpu!!!
    virtual void nextFrame(cv::OutputArray frame)
    {
        // This is the line that matters here. In the cpu version you can copy 
        // any chunk of memory into the outputarray frame, even from static
        // buffers.
        GpuFrame.copyTo(frame);
    }

    virtual void reset()
    {

    }

private:
    cv::cuda::GpuMat GpuFrame;
    cv::Mat CustomFrame;
};

It is used by the following:

//Creation
SuperResolution = cv::superres::createSuperResolution_BTVL1_CUDA();
OpticalFlow     = cv::superres::createOptFlow_Farneback_CUDA();
SuperResolution->setOpticalFlow(OpticalFlow);
SuperResolution->setScale(4);
SuperResolution->setIterations(4);
SuperResolution->setTemporalAreaRadius(3);
FrameSource = new ThermalFrameSource();
SuperResolution->setInput(FrameSource);

//Usage. Note ThermalImage wraps a byte[120*160*3] array and is filled in
// with other parts of the code. Regardless of how it is filled, cv::Mat can use
// the memory pointer to pass to the custom FrameSource

        cv::Mat lowResThermal( 120, 160, CV_8UC3, ThermalImage->RawPixels());
        FrameSource->setFrame(lowResThermal);
        SuperResolution->nextFrame( SuperResThermal );
        uint8_t* resultingPixels = SuperResThermal.data;

To fill in my buffer (ThermalImage) from the camera, I use the libuvc API. Hope this helps others looking to use superresolution with a custom buffer.