uniform LBP - mapping using lookup table

asked 2013-06-20 10:34:11 -0600

learner123
36 ●1 ●1 ●6

updated 2013-06-20 10:35:40 -0600

Hello, I have basic implemented Local Binary Pattern (LBP), without interpolation, using OpenCV and C++. Following is the code:

#include "opencv2/core/core.hpp"
#include "opencv2/contrib/contrib.hpp"
#include "opencv2/highgui/highgui.hpp"

using namespace cv;
using namespace std;

cv::Mat LBP(string src_image)   
{   
  cv::Mat temp_image = imread(src_image.c_str(), 1); 
  cv::Mat Image(temp_image.rows, temp_image.cols, CV_8UC1);
  cv::Mat lbp(temp_image.rows, temp_image.cols, CV_8UC1);

  if (temp_image.channels() == 3)   
    cvtColor(temp_image, Image, CV_BGR2GRAY);     

  imshow("src_image", Image); 

  int center = 0;   
  int center_lbp = 0;   

  for (int row = 1; row < Image.rows; row++)   
  {
    for (int col = 1; col < Image.cols; col++)   
    {   
      center = Image.at<int>(row, col);
      center_lbp = 0;   

      if ( center <= Image.at<int>(row-1, col-1) )   
    center_lbp += 1;   

      if ( center <= Image.at<int>(row-1, col)   )    
        center_lbp += 2;   

      if ( center <= Image.at<int>(row-1, col+1) )   
        center_lbp += 4;   

      if ( center <= Image.at<int>(row, col-1)   )   
        center_lbp += 8;   

      if ( center <= Image.at<int>(row, col+1)   )   
        center_lbp += 16;   

      if ( center <= Image.at<int>(row+1, col-1) )   
        center_lbp += 32;   

      if ( center <= Image.at<int>(row+1, col)   )   
        center_lbp += 64;

      if ( center <= Image.at<int>(row+1, col+1) )   
        center_lbp += 128;   

      cout << "center lbp value: " << center_lbp << endl;
      lbp.at<int>(row, col) = center_lbp;
    }
  }

    imshow("lbp_image", lbp);
    waitKey(0);   
    destroyAllWindows();

    return lbp;
} 

void histogram(cv::Mat image)
{
  int histSize = 256;
  float range[] = { 0, 256 } ;
  const float* histRange = { range }; 

  bool uniform = true; bool accumulate = false;

  Mat hist;

  calcHist( &image, 1, 0, Mat(), hist, 1, &histSize, &histRange, uniform, accumulate );

  int hist_w = 512; int hist_h = 400;
  int bin_w = cvRound( (double) hist_w/histSize );

  Mat histImage( hist_h, hist_w, CV_8UC3, Scalar( 0,0,0) );

  /// Normalize the result to [ 0, histImage.rows ]
  normalize(hist, hist, 0, histImage.rows, NORM_MINMAX, -1, Mat() );

  /// Draw 
  for( int i = 1; i < histSize; i++ )
  {
      line( histImage, Point( bin_w*(i-1), hist_h - cvRound(hist.at<float>(i-1)) ) ,
                       Point( bin_w*(i), hist_h - cvRound(hist.at<float>(i)) ),
                       Scalar( 255, 0, 0), 2, 8, 0  );
  }

  imshow("LBP image Histogram", histImage );
  waitKey(0);
}

//--------------------------------------------------------   
int main()   
{
  cv::Mat LBPimage = LBP("a.ppm");
  histogram(LBPimage);

  return 0;   
}

I now stuck at implementing uniform LBP. Can't figure out how to proceed. How to map using lookup table....can anybody please help?

thanks!

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2013-06-20 12:05:34 -0600

berak
32993 ●7 ●81 ●312

first of all, LBP features are mostly used with histograms, rarely by themselves, and uniform LBP is used for compressing the histograms from 256 bins to 59, which gives quite some speedup when comparing them later.

i hope, you read the paper

so, nice, you finished the first step of it ;) , steps 2 and 3 would be:

2: in your lbp function, calculate a histogram from the center_lbp values

Mat hist(1,256,CV_32F); // 256 bins, float for later interpolation

// later, inside the loop:
// increment the counter for each seen value, hist with 256 elements
hist.at<float>(0,center_lbp) += 1;  
// or, if uniform LBP, you'd use the lookup:
hist.at<float>(0,lookup[center_lbp]) += 1;  // hist with 59 elements

3: don't do that for the whole image, but divide it into , like 8x8 patches, collect a histogram for each small patch, and in the end, concat them to a big 1-d feature vector , the final product.

Mat spatial_histogram(Mat src, int grid_x, int grid_y, uchar lookup[256] ) {
    // calculate LBP patch size
    int width = src.cols/grid_x;
    int height = src.rows/grid_y;
    Mat result = Mat::zeros(0, 0, CV_32F);
    // iterate through grid
    for(int i = 0; i < grid_y; i++) {
        for(int j = 0; j < grid_x; j++) {
            Mat patch(src, Range(i*height,(i+1)*height), Range(j*width,(j+1)*width));
            int len = isUniform ? 59 : 256;
            Mat hist = Mat::zeros(1,len,CV_32F);
          // !! your code here (minus the img loading) :  lbp(patch,hist,lookup);
            result.push_back(hist);
        }
    }
    return result;
}

the uniform lookup itself is not too difficult.

a bitmask is 'uniform' if the number of transitions <= 2.
for an 8bit histogram, you'd need 256 slots, and only 58 of the 255 possible values are uniform.
a histogram for 'uniform' lbp features would have only 59 bins, one for each uniform value, and the last fo all others ( 58+noise )
you would make a lookup[256] table to translate from 8bit_value [0-255] -> 59 bins[0-58] in the histogram, run from 0 to 255, count the binary transitions for each number, and if it's uniform, set to an (increased) index, anything else gets set to 'noise' value

i'm actually getting lazy, for code look here

edit flag offensive delete link

Comments

thanks a lot! I would try it out right away...

learner123 ( 2013-06-21 05:35:28 -0600 )edit

@berak Could you please tell me where is the lookup table? thats where I'm stuck...how to create a lookup table...do i need to write down all 256 8-bit combinations and find out manually which ones correspond to uniform? you have used: " hist.at<float>(0,lookup[center_lbp]) += 1; " but where is this lookup[] ? Please help a little...I am new to all this...but I'm trying...I do want to learn it! thanks!

learner123 ( 2013-06-25 06:37:41 -0600 )edit

@berak hi, i moved further and the final histogram that I am getting is of size: 64x256. This is because, I divided my entire image into 64 blocks/patches. And each patch then gives me a 1x256 histogram. Becuase of the line: " result.push_back(hist); " each of 1x256 histogram for every patch is being pushed back into main histogram and thus forms 64x256 histogram. I wanted to ask:

---1) is this correct? the size of main histogram: 64x256

---2) currently my image gets divided into 64, "96x64 patch"....should they be smaller? my image size is: 768x512

I was expecting the main histogram to b a 1-D vector like "1xlarge_number"...but its coming out at 64x256. I gave x_grid = 8 and y_grid = 8

thanks a lot!

learner123 ( 2013-06-26 10:35:33 -0600 )edit

if you use uniform histograms(and lookups), it's 64 x 59(the saving there is the whole point), for the 'normal' version, 64 x 256, so, yes.
i don't know. if your patches are 96x64, your whole img is 768x512, right ? i never asked, what you are trying to recognize there, but for faces you'd usually crop the image to the interesting / face region, and resize that to like 100x100. don't worry too much about it now, this, and the number of patches is something to optimize later
"...but its coming out at 64x256" yes, that's true. if you need it in one row, do hist=hist.reshape(1,1); but again, it might not even be nessecary. whether you'd compare those histograms using norm() or compareHist(), it'll be allright, as long as both histograms have the same shape / num elements.

berak ( 2013-06-26 10:58:07 -0600 )edit

@berak alright! i would work further....thanks a lot! i am working with face images only...

learner123 ( 2013-06-26 11:09:14 -0600 )edit

yea, keep on truckin ;)

berak ( 2013-06-26 11:23:22 -0600 )edit

@berak I would go for reshaping hist because ultimately i have to input it to MLP....so a 1d-vector would help there....

learner123 ( 2013-06-27 03:59:23 -0600 )edit

@berak aahaann!!! on reshaping it...the size is 1x16384. and this is exactly the size returned by OpenCV's LBP function!!! so this means the concept of uniform patterns is not implemented there!!! working on things practically is so mch fun!! thanks to u!

learner123 ( 2013-06-27 04:14:47 -0600 )edit

@berak i have a doubt! the literature I read on LBP...all mentioned dividing image into 8x8 patches and you also suggested the same thing. But, here we set x_grid = 8 and y_grid = 8. So, the width is: src.cols/x_rid = 512/8 = 64 AND height is: src.rows/y_grid = 768/8 = 96 which means...the patch is size: 96x64 but wasn't it supposed to be 8x8 ??

sry if I'm missing out on smthng here...but its important to know this...

thanks again!

learner123 ( 2013-06-27 04:30:52 -0600 )edit

currently, the opencv lbp thing does not use uniform hists ( but maybe later ...)
no, 8x8 is the number of patches, the patch-size then depends on the image size

berak ( 2013-06-27 05:03:33 -0600 )edit

see more comments

answered 2013-06-25 07:00:17 -0600

berak
32993 ●7 ●81 ●312

"do i need to write down all 256 8-bit combinations and find out manually which ones correspond to uniform? "

well yes, you're supposed to .

but maybe we can shortcut it for now, here's a precomputed one for 8 neighbours:

const char lookup[256] = {
0, 1, 2, 3, 4, 58, 5, 6, 7, 58, 58, 58, 8, 58, 9, 10,
11, 58, 58, 58, 58, 58, 58, 58, 12, 58, 58, 58, 13, 58, 14, 15,
16, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58,
17, 58, 58, 58, 58, 58, 58, 58, 18, 58, 58, 58, 19, 58, 20, 21,
22, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58,
58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58,
23, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58,
24, 58, 58, 58, 58, 58, 58, 58, 25, 58, 58, 58, 26, 58, 27, 28,
29, 30, 58, 31, 58, 58, 58, 32, 58, 58, 58, 58, 58, 58, 58, 33,
58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 34,
58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58,
58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 35,
36, 37, 58, 38, 58, 58, 58, 39, 58, 58, 58, 58, 58, 58, 58, 40,
58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 41,
42, 43, 58, 44, 58, 58, 58, 45, 58, 58, 58, 58, 58, 58, 58, 46,
47, 48, 58, 49, 58, 58, 58, 50, 51, 52, 58, 53, 54, 55, 56, 57 };

and the code, that produced it:

#include <stdio.h>
#include <stdlib.h>
bool bit(unsigned b, unsigned i)
{
    return ((b & (1 << i)) != 0);
}

int main(int argc, char **argv)
{
    printf("const char lookup[256] = {");
    int numUniforms = 0;
    int numNeighbours = 8;
    int numSlots = 1 << numNeighbours;
    for ( int i=0; i<numSlots; i++ )
    {
        if ( i%16 == 0 ) printf("\n");

        int transitions = 0;
        for ( int j=0; j<numNeighbours-1; j++ )
            transitions += (bit(i,j) != bit(i,j+1));
        transitions += (bit(i,numNeighbours-1) != bit(i,0));

        if ( transitions <= 2 )
            printf("%d ",numUniforms++);
        else
            printf("58 ");
        if ( i<numSlots-1 ) 
            printf(",");
    }
    printf("};\n");
    return 0;
}

edit flag offensive delete link

Comments

@berak thanks a lot!! I'm able to understand things better now.... working on understanding rest of the code...

learner123 ( 2013-06-25 07:38:28 -0600 )edit

@berak in the line: int numSlots = 1 << numNeighbours; I tried to output numSlots value....it gives 256. How did that happen? I have never seen this kind of declaration before...could you please tell me? thank you!

learner123 ( 2013-06-25 07:51:09 -0600 )edit

got it! its giving value of 2^numSlots

learner123 ( 2013-06-25 07:59:55 -0600 )edit

that's a "shift" operator. if you shift 0001 one to the left, you get 0010 (or 2 decimal), if you do it again, you get 0100 (or 4 decimal)

it's the same as pow(2,8), or 2 **8 (in python)

sorry for the confusion, should have hardcoded 256 there (as the "58" / noise value is also hardcoded)

berak ( 2013-06-25 08:02:19 -0600 )edit

@berak could you please explain the logic behind this:

int transitions = 0;

for ( int j=0; j&lt;numNeighbours-1; j++ )

    transitions += (bit(i,j) != bit(i,j+1));

transitions += (bit(i,numNeighbours-1) != bit(i,0));

learner123 ( 2013-06-25 09:00:49 -0600 )edit

(bit(i,j) != bit(i,j+1));

compares bit j (in the number i) to bit j+1.

think of it as.: if ( bit_5 != bit_6 ) transition += 1; ( a bool can be converted to a 0/1 number in c++)

then compare the last to the first bit ( close the ring ):

transitions += (bit(i,numNeighbours-1) != bit(i,0));

berak ( 2013-06-25 09:13:30 -0600 )edit

hi, sorry if i am wrong, whether the look up table is correct for 8 neighbour pixels. if we are using 8 neighbour pixels then the integer 17 will be 00010001 so there are more than 2 transitions and it is not uniform. but if we are not considering the first 3 zeros 10001 it become uniform. as the values represents the intensity the correct uniform integers should be 0, 1, 2, 3, 4, 6, 7, 8, 12, 14, 15, 16, 24, 28, 30, 31, 32, 48, 56, 60, 62, 63, 64, 96, 112, 120, 124, 126, 127, 128, 129, 131, 135, 143, 159, 191, 192, 193, 195, 199, 207, 223, 224, 225, 227, 231, 239, 240, 241, 243, 247, 248, 249, 251, 252, 253, 254 and 255. is there any difference in both representations. which one we should use? please suggest.

aneesh07 ( 2018-09-19 23:52:34 -0600 )edit

@aneesh07 you're missing, that the goal of it is to come up with a shorter histogram (with 59 instead of 256 bins) so we need a mapping from "dense" to "sparse" space.

the lookup table is use for indexing into dense [0..256] space. it does NOT contain the uniform values.

berak ( 2018-09-20 00:14:24 -0600 )edit

thanks berak, So it means the uniform concept mentioned here and the uniform transitions of bits in the LBP concept are different right?.

aneesh07 ( 2018-09-20 00:43:06 -0600 )edit

your uniform values are correct. but it's not the same as the indices to those in the lookup table.

berak ( 2018-09-20 00:49:09 -0600 )edit

see more comments

uniform LBP - mapping using lookup table

2 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

uniform LBP - mapping using lookup table edit

2 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

uniform LBP - mapping using lookup table