Calculate hue distance from hue histogram?

edit

If I understand correctly, instead of using the real hue value of the pixels in your processing, you will use a Hue value relative to a reference. It's like rotating the hue circle. So you'll have H=0 not for red, but for the predominant color.

For you Href is the maxLoc (the position of the maximum value in the Hue histogram). You can check it by analyzing a predominantly green image to get a Href around 120°.

In a histogram, even if you have bins, you can associate the bin to a value. You can take the lower limit of the bin or (better) the middle value. So if you have 36 bins, each representing 10° of range, and maxLoc=2, than you can take Href = 2*10+5 = 25° or eventually Href = 2*10 = 20°. The first one is better, but I don't think this will have an impact on the results.

After you found the Href, you'll have to recalculate the H value of every pixel using the formula from the article.

Note: I don't know the article you are using, but to calculate distance between colors we often use the Lab color space. Thee, the perceptual distance between two colors is equal to the eucledian distance. With the hue distance you can have very distant values for similar colors around the neutral grey.