Ask Your Question
3

Display IplImage in webbrowsers

asked 2013-02-06 03:10:44 -0600

Nertius gravatar image

updated 2020-11-30 03:34:31 -0600

Hi people,

I'm currently building a small personal project in c++ on a linux server and i am facing a problem i can't solve myself.

I would like to know if it was possible to display in a web browser an IplImage image.

I first thought i would be able to convert it in png and then encode it in base64 and finally display it with a simple cout<<

But i can't figure how to do so. So i would like to know if someone could guide me trough or if there is a simpler way of doing.

Many thanks for your time and help. Cheers to everyone ;)

PS: i precise that i'm not at all an expert in c++. I used to practice it a lot at school, but it's a long time ago now ;)

edit retag flag offensive close merge delete

Comments

Why don't you save the image in PNG and display it in html? (may be I don't understand the question...)

Mathieu Barnachon gravatar imageMathieu Barnachon ( 2013-02-06 04:12:38 -0600 )edit

Thank's for answering, but your trick won't do. The purpose of my program is very simple. The user will give me an array containing 0 and 1, and i will convert it in an image that can be displayed in web browsers. ;)

Nertius gravatar imageNertius ( 2013-02-06 06:40:05 -0600 )edit

4 answers

Sort by » oldest newest most voted
4

answered 2013-02-06 04:12:43 -0600

berak gravatar image
// convert the image to JPEG ( in memory! )

std::vector<uchar>outbuf;
std::vector<int> params;
params.push_back(CV_IMWRITE_JPEG_QUALITY);
params.push_back(100);
cv::imencode(".jpg", frame, outbuf, params);

then send the outbuf from your socket. i don't think you'll need additional base64

if you want to send an mjpeg stream, not a single image, you want to wrap that with a http-multipart form:

"Server: Mozarella/2.2\r\n"
"Accept-Range: bytes\r\n"
"Connection: close\r\n"
"Content-Type: multipart/x-mixed-replace; boundary=mjpegstream\r\n"
"\r\n"

then, for every frame:

sprintf(head,"--mjpegstream\r\nContent-Type: image/jpeg\r\nContent-Length: %lu\r\n\r\n",outlen);
write(sock,head,strlen(head));
write(sock,(char*)(&outbuf[0]),outlen);

good luck!

edit flag offensive delete link more

Comments

Thanks a lot for your answer, i was struggling for days. SO far i managed to make it work until imencode.

But after, i've never used such things in c++ :

sprintf(head,"--mjpegstream\r\nContent-Type: image/jpeg\r\nContent-Length: %lu\r\n\r\n",outlen); write(sock,head,strlen(head)); write(sock,(char*)(&outbuf[0]),outlen);

I'm not sure how to define and use head, outlen an sock ... could you please explain me ?

Thanks again, your help is very appreciated !!

Nertius gravatar imageNertius ( 2013-02-06 09:30:23 -0600 )edit
1

answered 2013-02-06 11:04:04 -0600

berak gravatar image

updated 2016-01-06 22:25:17 -0600

edit: here's the final version

//
// a single-threaded, multi client(using select), debug webserver - streaming out mjpg.
//  on win, _WIN32 has to be defined, must link against ws2_32.lib (socks on linux are for free)
//

//
// socket related abstractions:
//
#ifdef _WIN32  
    #include <winsock.h>
    #include <windows.h>
    #include <time.h>
    #define PORT        unsigned long
    #define ADDRPOINTER   int*
    struct _INIT_W32DATA
    {
       WSADATA w;
       _INIT_W32DATA() { WSAStartup( MAKEWORD( 2, 1 ), &w ); }
    } _init_once;
#else       /* ! win32 */
    #include <unistd.h>
    #include <sys/time.h>
    #include <sys/types.h>
    #include <sys/socket.h>
    #include <netdb.h>
    #include <netinet/in.h>
    #include <arpa/inet.h>
    #define PORT        unsigned short
    #define SOCKET    int
    #define HOSTENT  struct hostent
    #define SOCKADDR    struct sockaddr
    #define SOCKADDR_IN  struct sockaddr_in
    #define ADDRPOINTER  unsigned int*
    #define INVALID_SOCKET -1
    #define SOCKET_ERROR   -1
#endif /* _WIN32 */



#include <iostream>
using std::cerr;
using std::endl;

#include "opencv2/opencv.hpp"
using namespace cv;



class MJPGWriter
{
    SOCKET sock;
    fd_set master;
    int timeout; // master sock timeout, shutdown after timeout millis.
    int quality; // jpeg compression [1..100]

    int _write( int sock, char *s, int len ) 
    { 
        if ( len < 1 ) { len = strlen(s); }
        return ::send( sock, s, len, 0 );
    }

public:

    MJPGWriter(int port = 0) 
        : sock(INVALID_SOCKET) 
        , timeout(200000)
        , quality(30)
    {
        FD_ZERO( &master );
            if (port)
                open(port);
    }

    ~MJPGWriter() 
    {
        release();
    }

    bool release()
    {
        if ( sock != INVALID_SOCKET )
            ::shutdown( sock, 2 );
        sock = (INVALID_SOCKET);
        return false;
    }

    bool open( int port )
    {
        sock = ::socket (AF_INET, SOCK_STREAM, IPPROTO_TCP) ;

        SOCKADDR_IN address;       
        address.sin_addr.s_addr = INADDR_ANY;
        address.sin_family      = AF_INET;
        address.sin_port        = ::htons(port);
        if ( ::bind( sock, (SOCKADDR*) &address, sizeof(SOCKADDR_IN) ) == SOCKET_ERROR )
        {
            cerr << "error : couldn't bind sock "<<sock<<" to port "<<port<<"!" << endl;
            return release();
        }
        if ( ::listen( sock, 10 ) == SOCKET_ERROR )
        {
            cerr << "error : couldn't listen on sock "<<sock<<" on port "<<port<<" !" << endl;
            return release();
        }
        FD_SET( sock, &master );    
        return true;
    }

    bool isOpened() 
    {
        return sock != INVALID_SOCKET; 
    }

    bool write(const Mat & frame)
    {
        fd_set rread = master;
        struct timeval to = {0,timeout};
        SOCKET maxfd = sock+1;

        if ( ::select( maxfd, &rread, NULL, NULL, &to ) <= 0 )
            return true; // nothing broken, there's just noone listening

        std::vector<uchar>outbuf;
        std::vector<int> params;
        params.push_back(IMWRITE_JPEG_QUALITY);
        params.push_back(quality);
        cv::imencode(".jpg", frame, outbuf, params);
        int outlen = outbuf.size();

        #ifdef _WIN32 
        for ( unsigned i=0; i<rread.fd_count; i++ )
        {
            SOCKET s = rread.fd_array[i];    // fd_set on win is an array, while ...
        #else         
        for ( int s=0; s<maxfd; s++ )
        {
            if ( ! FD_ISSET(s,&rread) )      // ... on linux it's a bitmask ;)
                continue;
        #endif                   
            if ( s == sock ) // request on master socket, accept and send main header.
            {
                int         addrlen = sizeof(SOCKADDR);
                SOCKADDR_IN address = {0};     
                SOCKET      client  = ::accept( sock,  (SOCKADDR*)&address, &addrlen );
                if ( client == SOCKET_ERROR )
                {
                    cerr << "error : couldn't accept connection on sock " << sock<< " !" << endl;
                    return false;
                }
                maxfd=(maxfd>client?maxfd:client);
                FD_SET( client, &master );
                _write( client,"HTTP/1.0 200 OK\r\n",0);
                _write( client,
                    "Server: Mozarella/2.2\r\n"
                    "Accept-Range: bytes\r\n"
                    "Connection: close\r\n"
                    "Max-Age: 0\r\n"
                    "Expires: 0\r\n"
                    "Cache-Control: no-cache, private\r\n"
                    "Pragma: no-cache\r\n"
                    "Content-Type: multipart/x-mixed-replace; boundary=mjpegstream\r\n"
                    "\r\n",0);
                cerr << "new client " << client << endl;
            } 
            else // existing client, just stream pix
            {
                char head ...
(more)
edit flag offensive delete link more

Comments

Awesome !! thank you so much for this, i'll try to implement and understand it !!

Nertius gravatar imageNertius ( 2013-02-06 11:15:47 -0600 )edit
1

I think the header should also contains "HTTP/1.1 200 OK"

ssinfod gravatar imagessinfod ( 2015-12-23 14:40:09 -0600 )edit

^^ yes, true.

berak gravatar imageberak ( 2015-12-23 22:39:04 -0600 )edit

Hello berak, i'm trying your code ons OSX Yosemite. However when opening the page (e.g. with Chrome) the page keeps loading. I did some small debugging and it looks like your code is never sending the image with header to the browser.

cedricverst gravatar imagecedricverst ( 2016-02-15 05:57:30 -0600 )edit

hmm, tested with ff and vlc only ;(

also, this thing expects a second, third, etc. (seperate) request for the images, the 1st http request only returns the http multipart headers.

if you can fix/improve it, let me know ;)

berak gravatar imageberak ( 2016-02-15 06:15:29 -0600 )edit

I don't have enough knowledge of sockets.. So don't think I can help you with this. I really want to integrate your code in my project https://kerberos.io

cedricverst gravatar imagecedricverst ( 2016-02-15 06:28:57 -0600 )edit

PS: I also tried it with VLC, and the code doesn't work either. It only reacts on the first HTTP request, but blocks on the select afterwards.

cedricverst gravatar imagecedricverst ( 2016-02-15 06:32:21 -0600 )edit

ah, shame ;(

i'll try on linux again later, let's see.. (maybe if**ed up the select())

nice project / website, btw !

berak gravatar imageberak ( 2016-02-15 06:40:26 -0600 )edit

Thanks, maybe you can contribute ;)

cedricverst gravatar imagecedricverst ( 2016-02-15 06:43:44 -0600 )edit
1

Got it working, i've splitted your write method in two functions calls, and implemented an vector which contains all the client id's.

1) Use select to check if some new clients have subscribed, and if so create an descriptor for them.

2) In the second method send an image to all the subscribed sockets.

A crucial configration was the following:

setsockopt(client, SOL_SOCKET, SO_NOSIGPIPE, (void *)&set, sizeof(int));

This causes the program to not exit when trying to write to a disconnected client. It now works in all browsers + VLC. However Chrome does a strange thing, it always disconnects the first connection and keeps the second connection. (this is fixed by the option mentioned above).

cedricverst gravatar imagecedricverst ( 2016-02-15 08:59:47 -0600 )edit
0

answered 2013-02-06 09:57:01 -0600

berak gravatar image

oh i was somehow assuming, that you were writing a little webserver-program in c++ on your own, that's where the sockets (sock) come in. http://beej.us/guide/bgnet/ has some nice hints on that. the general flow:

  1. create a (master) socket
  2. bind it to a port
  3. listen on that port

    1. accept returns a client socket for each connection
    2. read / write on that client socket
int outlen = img.width * img.height * img.nChannels;  // bytecount
char head[512];  // make space for the header

sprintf(head,"--mjpegstream\r\nContent-Type: image/jpeg\r\nContent-Length: %lu\r\n\r\n",outlen);
write(sock,head,strlen(head));
write(sock,(char*)(&outbuf[0]),outlen);
edit flag offensive delete link more

Comments

I will try to understand this.

You were assuming write actually. I'm indeed triyng to write a small website in c++, i just don't how to do it yet.

My application is very simple. It's encoding qrcode. I already manage to have an array with 0 and 1 representing the black pixels and the white pixels of the qrcode.

Then, i wanted to use opencv's function to transform this array into a jpeg image and i managed to do so with you help and the function imencode. And now that my image is stored into outbuf i want this image to be printed into the webbrowser (firefox for me).

I know nothing about socket, and i have to say that what i read seems really difficult to me. I still wonder i there's another way.

Anyway, all of this seems kind of hard since i ...(more)

Nertius gravatar imageNertius ( 2013-02-06 10:45:48 -0600 )edit

hey, don't give up now, you've come quite far already ;) (and yes, that bsd-socket api seems to come straight from 1972)

berak gravatar imageberak ( 2013-02-06 10:52:45 -0600 )edit

Thank you for your support. It's very nice of you ;)

I actually can't see the purpose of the socket in my app. Usually, i'm in php where everthing's easy ;) and once you have the "data" of the image as i do width outbuf, you can do a simple : echo 'data:image/jpeg;base64,' . base64_encode($data);

I can't understand why it can't be done here and why we must go through sockets.

I'm doing this app only for fun, and i can't thank you enough for your help since no one around me knows c++ or opencv ;)

Nertius gravatar imageNertius ( 2013-02-06 11:04:57 -0600 )edit

oh, it seems, i got you totally wrong then. you don't want to write your own server, but interface with apache or similar, well ok ;)

there's still options: 1. just write it to disk: imwrite("myqr.png", img); and proceed in your usual php-way 2. make a 'cgi-app' from cpp, that does the imencode step, and then just cout<< img; call that instead of php ( needs to mess with apache's config though )

berak gravatar imageberak ( 2013-02-06 11:18:26 -0600 )edit

héhé, no problem, even though it was not usefull to my app, it was nice reading about something i don't know ;)

First solution wouldnt do. My goal is to send from a html script an url to encode in GET data. Then, i would like to use my cgi-bin c++ app to qr encode this url, and print it into the browser.

From the html script, it would look like it : <img src="http://www.*.com/cgi-bin/qrcode.cgi?url=http://www.yop.com" />

So the only solution possible is solution 2.

Nertius gravatar imageNertius ( 2013-02-06 11:29:17 -0600 )edit

you can still call system(cheesy_cpp_app_that_only_writes_qr_to_png_on_disk) from php, and then serve it ;)

berak gravatar imageberak ( 2013-02-06 11:37:17 -0600 )edit

nice website, btw

berak gravatar imageberak ( 2013-02-06 11:39:13 -0600 )edit

Hey, it's not actually my website lol, i was just writing random stuff and it turned out to be a real website ;)

Nertius gravatar imageNertius ( 2013-02-06 12:04:57 -0600 )edit

lots of misunderstanding leading to nice insights in this thread ;)

berak gravatar imageberak ( 2013-02-06 12:08:26 -0600 )edit

;)

Just here to say that i manage, with your precious help, to display the wanted qrcode in my webbrowser. So i guess we could just say, mission complete. It's actually pretty easy ... once you know how to do ;)

       cout &lt;&lt; "Content-Type:image/jpeg\r\n\r\n";
    std::vector&lt;uchar&gt;outbuf;
    std::vector&lt;int&gt; params;
    params.push_back(CV_IMWRITE_JPEG_QUALITY);
    params.push_back(100);
    imencode(".jpg", qrMat, outbuf, params);
    for(int i=0;i&lt;outbuf.size();i++)
        cout&lt;&lt;outbuf[i];

Here we are, thank's again mate, was nice learning from you ;)

Nertius gravatar imageNertius ( 2013-02-06 12:48:45 -0600 )edit
-1

answered 2015-12-23 15:03:29 -0600

ssinfod gravatar image

updated 2016-01-06 09:13:56 -0600

Here is a short example on how to convert IplImage to unsigned char and send it via a socket. (to simulate jpeg image sent from a web server). It is in C because I was not able to find any example for it...

CvCapture* camera;
IplImage* img_cam;
IplImage* img_web;
int params[2];
unsigned char* ptr;
unsigned char buf;
int i;
int buf_len;
char head[1024];

// Capture image and convert to jpg.
// I used cvCloneimage so I don't work on the original image.
img_cam = cvQueryFrame(camera);
img_web = cvCloneImage(img_cam);
params[0]= CV_IMWRITE_JPEG_QUALITY;
params[1]= 50; // Quality 0-100
CvMat* ei = cvEncodeImage(".jpg", img_web, params);
buf_len = ((ei->rows) * (ei->cols));
ptr = ei->data.ptr;
// Copy jpeg data from CvMat to unsigned char buffer.
for (i = 0 ; i < buf_len ; i++) {
    buf[i] = (*(ptr++));
}

...

// Send data via tcp socket.
SOCKET sock  = Accept(master); // See post above...
sprintf(head, "HTTP/1.1 200 OK\r\nServer: Built-in\r\nCache-Control: no-cache\r\nCache-Control: private\r\nContent-Type: image/jpeg\r\nContent-Length: %lu\r\nConnection: close\r\n\r\n", buf_len);
send(sock, head, strlen(head), MSG_NOSIGNAL);
send(sock, (uc*)(&buf[0]), buf_len, MSG_NOSIGNAL);
close(sock);
edit flag offensive delete link more

Comments

please do not use opencv's deprecated c-api for anything

berak gravatar imageberak ( 2015-12-23 22:37:52 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2013-02-06 03:10:44 -0600

Seen: 6,511 times

Last updated: Jan 06 '16