Ask Your Question
1

How to remove line on music sheet

asked 2015-01-18 09:17:09 -0500

danielistyo gravatar image

updated 2015-08-24 13:26:14 -0500

I want to try to remove line/horizontal line on music sheet. Because it help me to recognize the notes. Can someone tell me step by step to remove line/horizontal line on music sheet?

before image description

after image description

edit retag flag offensive close merge delete

Comments

Start by adding an example image please.

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-18 09:28:52 -0500 )edit

I've added 2 images. Do you know the algorithm to solve it ? @StevenPuttemans

danielistyo gravatar imagedanielistyo ( 2015-01-18 09:47:09 -0500 )edit

5 answers

Sort by ยป oldest newest most voted
9

answered 2015-02-05 17:18:22 -0500

theodore gravatar image

Since I found the above challenge quite interesting I worked a bit on it. My approach is based mainly on morphological operations, and without applying any complex technique the result seems pretty much nice. Anyway, here is my approach:

#include <iostream>
#include <opencv2/opencv.hpp>

using namespace std;
using namespace cv;

int main()
{
    // Load source image
    string filename = "notes.png";
    Mat src = imread(filename);

    // Check if image is loaded fine
    if(!src.data)
        cerr << "Problem loading image!!!" << endl;

    // Show source image
    imshow("src", src);

image description

    // Transform source image to gray it is not
    Mat gray;

    if (src.channels() == 3)
    {
        cvtColor(src, gray, CV_BGR2GRAY);
    }
    else
    {
        gray = src;
    }

    // Show gray image
    imshow("gray", gray);

image description

    // Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
    Mat bw;
    adaptiveThreshold(~gray, bw, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 15, -2);

    // Show binary image
    imshow("binary", bw);

image description

    // Create the images that will use to extract the horizonta and vertical lines
    Mat horizontal = bw.clone();
    Mat vertical = bw.clone();

    // Specify size on horizontal axis
    int horizontalsize = horizontal.cols / 30;

    // Create structure element for extracting horizontal lines through morphology operations
    Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));

    // Apply morphology operations
    erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
    dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));

    // Show extracted horizontal lines
    imshow("horizontal", horizontal);

image description

    // Specify size on vertical axis
    int verticalsize = vertical.rows / 30;

    // Create structure element for extracting vertical lines through morphology operations
    Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));

    // Apply morphology operations
    erode(vertical, vertical, verticalStructure, Point(-1, -1));
    dilate(vertical, vertical, verticalStructure, Point(-1, -1));

    // Show extracted vertical lines
    imshow("vertical", vertical);

image description

    // Inverse vertical image
    bitwise_not(vertical, vertical);
    imshow("vertical_bit", vertical);

image description

    // Extract edges and smooth image according to the logic
    // 1. extract edges
    // 2. dilate(edges)
    // 3. src.copyTo(smooth)
    // 4. blur smooth img
    // 5. smooth.copyTo(src, edges)

    // Step 1
    Mat edges;
    adaptiveThreshold(vertical, edges, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 3, -2);
    imshow("edges", edges);

image description

    // Step 2
    Mat kernel = Mat::ones(2, 2, CV_8UC1);
    dilate(edges, edges, kernel);
    imshow("dilate", edges);

image description

    // Step 3
    Mat smooth;
    vertical.copyTo(smooth);

    // Step 4
    blur(smooth, smooth, Size(2, 2));

    // Step 5
    smooth.copyTo(vertical, edges);

    // Show final result
    imshow("smooth", vertical);

image description

    waitKey(0);
    return 0;
}

enjoy.

edit flag offensive delete link more

Comments

1

@theodore, again what a nice explanation! If you are up to it, feel free to again add this as a code sample? Maybe we should consider building up a coding by example github page with solutions like this. I know that @Haris was up for that in the past also. Will give it some thinking in the near future!

StevenPuttemans gravatar imageStevenPuttemans ( 2015-02-09 03:03:28 -0500 )edit
1

thanks @StevenPuttemans, when I find some time I will try to submit this example as a tutorial in github as well. The idea off creating an example github page seems quite nice though.

theodore gravatar imagetheodore ( 2015-02-09 04:24:18 -0500 )edit

waww nice @theodore.. Can your algorithm applied in java ?

danielistyo gravatar imagedanielistyo ( 2015-02-19 12:51:12 -0500 )edit
1

I do not see any reason that someone could not be able to port it in java.

theodore gravatar imagetheodore ( 2015-02-19 14:58:41 -0500 )edit
1

@theodore.. okay, I will try.

danielistyo gravatar imagedanielistyo ( 2015-02-19 22:03:42 -0500 )edit
1

@theodore

C++ : adaptiveThreshold(~gray, bw, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 15, -2);

JAVA : Imgproc.adaptiveThreshold(destination, binaryMat, 255,Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 15, -2);

why the result is different ? https://www.dropbox.com/s/22pp6e5j1t2...

danielistyo gravatar imagedanielistyo ( 2015-03-10 09:15:11 -0500 )edit
2

because you are not using the inverted image as a source, check out the ~ symbol before gray. I have stated that in the comment as well. It is equivalent to bitwise_nor(gray, gray); == ~gray if you understand that better. So, in your case you should either use ~destination or first apply bitwise_nor(destination, destination); and then use the destination into the adaptiveThreshold() function

theodore gravatar imagetheodore ( 2015-03-10 09:46:28 -0500 )edit
1

thank you @theodore. you're very helpfully

danielistyo gravatar imagedanielistyo ( 2015-03-10 09:56:24 -0500 )edit

I am trying to rewrite your code in python cv2. I am running into issues in this point:

adaptiveThreshHoldImage = cv2.adaptiveThreshold(~gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 15, -2)

# cv2.imshow('gray', adaptiveThreshHoldImage)

horizontal = adaptiveThreshHoldImage.copy()
vertical = adaptiveThreshHoldImage.copy()

(rows, cols) = adaptiveThreshHoldImage.shape

horizontalSize = cols/30
verticalSize = rows/30

horizontalStructure = cv2.getStructuringElement(cv2.MORPH_RECT,(horizontalSize, 1))

The adaptive image code runs fine but neither the erode nor the dilate work. Please help.

Khalian gravatar imageKhalian ( 2015-07-18 21:55:24 -0500 )edit
1

sorry for my delayed response but I was away from a computer for some time. Now regarding your question what do you mean by saying that nor erode and dilate is working? please be more specific. Though I think it is a python based problem while you are trying to port the code.

theodore gravatar imagetheodore ( 2015-08-03 14:30:42 -0500 )edit
2

answered 2015-01-21 09:28:19 -0500

essamzaky gravatar image

updated 2015-01-27 04:20:37 -0500

I found the following paper solving the problem you described http://www.ece.rutgers.edu/~kdana/Cap...

  1. first detect the lines , by using horizontal projection , and decide where is the peaks larger than 1/2 image width , this will be line in music sheet.

  2. find the points located in the detected lines then clear the vertical pixles located inside the line under the codition number of cleard pixles not larger than line font width

see page 34 ,35 in the page , the full source code is exist

edit flag offensive delete link more

Comments

Wait, I want to learn that code. I don't familiar with c++. But, thanks

danielistyo gravatar imagedanielistyo ( 2015-01-21 12:44:28 -0500 )edit
1

answered 2015-01-18 11:22:08 -0500

gfx gravatar image

updated 2015-01-18 14:17:35 -0500

Guanta gravatar image
//Scharr( src_gray, grad_x, ddepth, 1, 0, scale, delta, BORDER_DEFAULT );
Sobel( src_gray, grad_x, ddepth, 1 , 0 , 3 , scale, delta, BORDER_DEFAULT );
convertScaleAbs( grad_x, abs_grad_x );
/// Gradient Y
//Scharr( src_gray, grad_y, ddepth, 0, 1, scale, delta, BORDER_DEFAULT );
Sobel( src_gray, grad_y, ddepth, 0 , 1 , 3 , scale, delta, BORDER_DEFAULT );
convertScaleAbs( grad_y, abs_grad_y );
subtract(abs_grad_x, abs_grad_y, yourmask);

not sure ..... but now you have or xline mask or yline mask ...... your work now is subtract tese mask from your image......

edit flag offensive delete link more

Comments

2

wait.. I am learning your code. I am newbie. I can not do fast.

danielistyo gravatar imagedanielistyo ( 2015-01-19 09:22:03 -0500 )edit
0

answered 2015-01-19 02:47:42 -0500

use binary morphology. For example "cv::dilate" using a custom 3x3 structuring element containing a horizontal line.

edit flag offensive delete link more

Comments

And how will this approach not screw up every other note?

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-19 03:08:03 -0500 )edit
1

no, it's not working. because every pic has different pixel. I've tried it

danielistyo gravatar imagedanielistyo ( 2015-01-19 09:23:24 -0500 )edit
0

answered 2015-01-20 07:25:09 -0500

essamzaky gravatar image

Dear @danielistyo how did you generate the image 2 "after" the requierd image , is there any software solves thsi problem , i'm searching for the same algorithm or any other software

edit flag offensive delete link more

Comments

The second image is just realised by using a GUM tool I am guessing!

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-20 07:27:19 -0500 )edit

What is GUM tool , i googled about it but did not find it

essamzaky gravatar imageessamzaky ( 2015-01-20 08:44:53 -0500 )edit

Its called eraser, you have it in paint, photoshop, gimp, ...

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-20 09:00:02 -0500 )edit
1

Thanks @StevenPuttemans , I think i't not manual tool such as Eraser, it's an Automatic detection algorithim because there are some unnecessary regions such as the region in the right border in the image and some dots , if he did it manually he will remove this regions

essamzaky gravatar imageessamzaky ( 2015-01-21 02:58:51 -0500 )edit
2

@essamzaky I found that image in google not using software

danielistyo gravatar imagedanielistyo ( 2015-01-21 12:46:45 -0500 )edit
Login/Signup to Answer

Question Tools

5 followers

Stats

Asked: 2015-01-18 09:17:09 -0500

Seen: 7,874 times

Last updated: Feb 05 '15