What options exist in OpenCV to improve the thresholding algorithm used for contour detection of fish arches on sonar images?

asked 2020-07-27 04:10:09 -0500

Brendon Costa gravatar image

updated 2020-07-27 18:11:11 -0500

I am working on a fish detection algorithm for sonar images as a pet open source project using OpenCV and am looking for advice from someone with experience in computer vision about how I can improve its accuracy likely by improving the thresholding/segmentation algorithm in use.

Sonar images look a bit like below and the basic artifacts I want to find in them are:

  • Upside down horizontal arches that are likely fish
  • Cloud/blob/balls shape artifacts that are likely schools of bait fish

I would really like to extract contours of these cloud and fish-arch artifacts.

The example code below uses threshold() and findContours(). The results are reasonable in this case as it has been manually tuned for this image but does not work on other sonar images that may require different thresholds or a different thresholding algorithm.

I have tried OSTUs method and it doesn't really work very well for this use case. I think I need a thresholding/segmentation algorithm that uses contrast of localized blobs somehow, does such an algorithm exist in OpenCV or is there some other technique I should look more into?

Thanks, Brendon.

Original image searching for artifacts: image description

Example output:

image description

import numpy
import random
import cv2
import math


def IsContourUseful(contour):
    # I have a much more complex version of this in my real code
    # This is good enough for demo of the concept and easier to understand

    # Filter on area for all items
    area = cv2.contourArea(contour)
    if area < MIN_AREA:
        return False

    # Remove any contours close to the top
    for i in range(0, contour.shape[0]):
        if contour[i][0][1] <= 10:
            return False

    return True

def FindFishContoursInImageWithoutBottom(image, file_name_base):
    ret, thresh = cv2.threshold(image, MIN_THRESHOLD, 255, cv2.THRESH_BINARY)
    cv2.imwrite(file_name_base + 'thresholded.png', thresh)

    contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    contours = [c for c in contours if IsContourUseful(c)]
    print ('Found %s interesting contours' % (len(contours)))

    # Lets draw each contour with a diff colour so we can see them as separate items
    im_colour = cv2.cvtColor(image, cv2.COLOR_GRAY2RGB)
    um2 = cv2.UMat(im_colour)
    for contour in contours:
        colour = (random.randint(100,255), random.randint(100,255), random.randint(100,255))
        um2 = cv2.drawContours(um2, [contour], -1, colour, 1)
    cv2.imwrite(file_name_base + 'contours.png', um2)

    return contours

# Load png and make greyscale as that is what original sonar data looks like
file_name_base = 'fish_image_cropped_erased_bottom'
image = cv2.imread(file_name_base + '.png')
image = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)

FindFishContoursInImageWithoutBottom(image, file_name_base)

Some examples where thresholding failed to identify a large school of bait fish as they had lower intensity:

Overall as a human I can see there are features that exist in these example with the bait schools that the thresholding doesn't pick up as it is slightly lower intensity.

image description

image description

An example where it picked up a bunch of things when there was nothing there as had slightly higher global intensity:

There are some cases in this example with a number of false positives. I ... (more)

edit retag flag offensive close merge delete


could you add a few more pictures

sturkmen gravatar imagesturkmen ( 2020-07-27 13:07:30 -0500 )edit

I added a few more images and comments to the end of the question. Please let me know if there are any additional specific cases you might be interested in.

Brendon Costa gravatar imageBrendon Costa ( 2020-07-27 17:51:04 -0500 )edit

A trained dolphin could hold still perpendicular to things with a GoPro and an ArUco marker.

kpachinger gravatar imagekpachinger ( 2020-08-01 19:14:01 -0500 )edit

I tried to find a willing dolphin, but it kept eating the fish. Makes recognizing the empty space a bit easier :-)

I have done some experimentation with different filters, and found so far using bilateralFilter to remove initial noise but keep edge definitions, then morphologyEx(CLOSE) I can smooth out the contours a bit and often joins things that were incorrectly separated, and equalizeHist which helps make it less susceptible to variations in intensity before the threshold helps out quite a bit.

These seem all very primitive though, after reading more I am planning to see if SIFT might help for arch detection or try some region growing segmentation. I am just exploring techniques as I don't know the field very well.

Brendon Costa gravatar imageBrendon Costa ( 2020-08-02 18:02:30 -0500 )edit

The problem isn't practical so the answer is subjective. Tweak a threshold "algorithm" and you'll learn you need different (or multiple) approaches. Stereo radar, motion detection, cascade classifier... Do you need new hardware? Was your code device agnostic?

It's like shooting fish in a barrel. It's toxic and the Bass Pro executives would go hungry.

kpachinger gravatar imagekpachinger ( 2020-08-04 04:57:56 -0500 )edit

Thanks for feedback. I guess I will just play around see if I can get something that is good enough. I am working with different hardware and types of sonar that will require different classifiers. In the end I am ultimately looking for artifacts of interest not just fish (very subjective) and will have multiple data-sets over time/position. Actual fish is one thing I would "like" to detect, but interesting structure on the bottom, temperature gradients etc are all useful and I expect to generate an area of interest 2D geographical map from data collected over time (using an automated RC boat to collect data over a few years). I also plan a GUI to make it easy to have a person go through and tag things manually from the initial artifacts of interest detected so not entirely automated.

Brendon Costa gravatar imageBrendon Costa ( 2020-08-04 19:54:22 -0500 )edit

It takes a pearl of wisdom to find a sunken ship... -30.555,-62.573

kpachinger gravatar imagekpachinger ( 2020-08-06 05:45:53 -0500 )edit