IMAGE SEGMENTATION — That’s what we’re talking about today. If you wonder what’s up with that, it’s basically the magic wand tool in photo editing softwares these days. However, don’t get your hopes up that we can make a magic-wand-like feature complete with the UI and all. Nope! We’re going full basic here.

So what is Image Segmentation? Like I said before, it’s like a magic wand tool. A region of interest is chosen and it becomes ready for more processing (yay!) Take for example this sample image in gray-scale.

Since it is in gray-scale, we know that the intensity values of each pixel ranges from 0 to 255 only. With this, we can get a histogram of the pixel intensity distribution to “see” the different regions in the image.

From the histogram, we can see that there is a lot of high intensity gray on the image. Looking back, the region on and near the peak of the histogram is the background of the image. If we filter the histogram to show all pixels with intensity values less than 150, then we will have a clear picture of the text alone!

[count, cells] = imhist(I, 256);
plot (cells, count);
A = I < 150;
imshow(A);

Well ain’t it easy? Well for a gray-scaled image, yes, however for colored images, things can get a  little bit difficult. Take for example our friendly dragon, Jakiro here.

If we straight up convert Jakiro into grayscale and run the same code on them, we obtain this histogram

–then this happens if we run the same filters.

OH GOD! WHAT HAVE WE DONE?!

Basically, what happened is that the intensity of each pixel does not tell color. That’s why the histogram is not very reliable here. Although, we can do something to isolate each color using…PARAMETRIC AND NON-PARAMETRIC PROBABILITY DISTRIBUTION ESTIMATION — whew, that’s a mouthful. But before the juicy technique, let’s talk about…

## NORMALIZED CHROMATICITY COORDINATES (NCC)

If you’re able to catch up with my blogs, then I’ll assume you know that colors are trichromatic. This just means that it can be expressed in three colors, usually in red, green, and blue or RGB. If we want to see all the combination of colors, then we’ll have a three-dimensional plot of colors. Imagine the hassle it would be to map every color in 3D! NCC or the normalized chromaticity coordinates simplifies this dilemma by expressing the 3D color space into 2D.

Per pixel, we normalize each color channel by the sum of all the RGB components such that

$r = \dfrac{R}{R+G+B} = \dfrac{R}{I}, g = \dfrac{G}{I}$, and $b = \dfrac{B}{I}$

$r + g + b = 1$

This allows us to express the normalized b values in terms of r and g where red is purely red when g is equal to zero, green is purely green when r is equal to zero and blue is purely blue when both r and g are equal to zero. This enables us to make the 2-dimensional color space instead of 3 as shown in the figure below

img = double(raw);
roi = double(region);

ri = img(:,:,1);
gi = img(:,:,2);
bi = img(:,:,3);
inti = ri + gi + bi;
inti(find(inti==0))=100000;
rin = ri./inti;
gin = gi./inti;

rr = roi(:,:,1);
gr = roi(:,:,2);
br = roi(:,:,3);
intr = rr + gr + br;
intr(find(intr==0))=100000;
r = rr./intr;
g = gr./intr;

Above is the code I used to get the NCC. Now that we have the NCC down, let’s proceed to the…

# Parametric Estimation

For the parametric distribution estimation method, we’re gonna first pick a region of interest (ROI), preferably of one color and get its histogram profile. When we normalize the histogram, then we have that color’s probability distribution function (PDF). From here, we’ll take the original image and compute the probability of each pixel being a member of the color of the ROI we picked. Since we have now define the color of the image in normalized reds and greens, the probability of each individual pixel with chromaticity r is given by

$p(r) = \dfrac{1}{\sigma_r \sqrt{2\pi}} exp \left( - \dfrac{(r-\mu_r)^2}{2\sigma_r^2} \right)$

where the $\mu_x$ and $\sigma_x$ are the mean and standard deviation of the ROI we chose. The same equation  is used for chromaticity g and the joint probability is obtained. For my case, this is the SciLab code I used.

sigr = stdev(r);
meanr = mean(r);
sigg = stdev(g);
meang = mean(g);

pr = (1/(sigr*sqrt(2*%pi)))*exp(-((rin – meanr).^2)/(2*sigr^2));
pg = (1/(sigr*sqrt(2*%pi)))*exp(-((gin – meang).^2)/(2*sigg^2));

prob = pr.*pg;

# Histogram Backprojection (Non-parametric  Estimation)

For the Non-parametric estimation, we won’t have to do math or probabilities here. What we basically will do is to label each point in the NCC a color and treat it as a database of colors instead. This enables for faster processing since the histogram values are looked up on the NCC.

To get a 2D histogram, we can convert the r and g values into integers and bin the values in a matrix. Here’s the code I used

BINS = 32;

rint= round(r*(BINS-1) + 1);
gint= round(g*(BINS-1) + 1);

colors = gint(:) + (rint(:)-1)*BINS;

hist = zeros(BINS,BINS);

for row = 1:BINS
for col = 1:(BINS-row+1)
hist(row,col) = length( find(colors==( ((col + (row-1)*BINS)))));
end;
end;

[x, y ,z] =size(raw);
seg = zeros(x,y);
for i = 1:x
for j = 1:y
r_segm = round((rin(i,j))*(BINS-1) + 1)
g_segm = round((gin(i,j))*(BINS-1) + 1)
segm(i,j) = hist(r_segm, g_segm)
end
end

# Images

Now that we both know the parametric and non-parametric approach to color segmentation, let’s try it out on Jakiro!.

Hmm!! Looks better now. We can see the individual head of the twin-headed dragon. Very cute ❤

For this activity, I guess a 10/10 for me is enough. It wouldn’t be possible without the help of Ma’am Jing’s lecture and some help of Carlo. I would also like to acknowledge Kuya Ron and his blog for enabling me to push further when I was stuck. I especially thank Trixia for always pushing me to finish my requirements and always being there to cheer me up. Thank you very much!