They said laptop is not an instrument. THEY SAID MAKING MUSIC WITH A LAPTOP IS EASY! WELL GUESS WHAT, THEY ARE WRONG! AND WE WILL PROVE IT TO THEM!!!

Rage comments aside, welcome to my blog again. Let’s play some music. In the previous blogs I introduced different image processing techniques now we can play some music! Yes you heard that right, we gonna play some music with image processing. Let’s just start with something simple, like Twinkle Twinkle Little Star.

score.png
Twinkle Twinkle Little Star music sheet

How are we gonna make play a music? We just have to identify the position of the note head and what the timing of the notes signifies. If you want to familiarize yourself with reading music sheets, then maybe visit this article right here. Since in this sheet, there’s only half note and a quarter note, we can use morphological operations here. But first let’s make the sheet a little bit better to read for the code. Like this,

sample.jpg
Music sheet (it’s longer now!)
cropped.jpg
Thresholded music sheet

Since it is in one line, I believe we can play the whole song in one run. Then, let’s segment the image. Since it is in black and white, we can easily segment via setting a threshold. Like in the image in the right. Since the whole sheet is long, I just cropped the thresholded image for you to see.

 

SE.jpg
S.E. used

Next up is the application of morphological operations to remove the staffs (the horizontal lines) since we are just happy enough to know the range of the vertical axis to determine the note of each staff. In this case, we’ll use this structuring element right here since the note heads are kind of ellipse like this. This enables us now to use SearchBlobs() function of IPD to pixel count the area of each blobs.

 

DISCLAIMER: Using SearchBlobs(), I actually got into trouble since the half notes were separated into two blobs. I circumvented this by adding the two halves by hard code. I’ll link my code down later.

Pushing forward, all we have to do now is to know the mean vertical location of each blob and compare the areas of the note heads. Of course, if the area is larger, then it’s the quarter note. Else, the half note. Here’s how I determined the notes and the timings. I defined functions since it is easier to loop by just calling each functions than running them inside the loop. It’s easier to debug too!

function n = note(f, t)
x = sin (2*%pi*f*t);
e = zeros(x);
len = length(e);
for i = i:length(x)
if i <= 500 then
e(i) = 0
elseif (i>500)&(i<=len-500) then
e(i) = 1
else
e(i) = 0
end
end
n = x.*e
endfunction;

function t = time(a)
if (a<60) then
t = soundsec(0.5)
else
t = soundsec(0.25)
end
endfunction

function f = frequency(y)

C = 261.63*2;
D = 293.66*2;
E = 329.63*2;
F = 349.23*2;
G = 392.00*2;
A = 440.00*2;
B = 493.88*2;
C# = 523.25*2;
D# = 587.33*2;
E# = 659.25*2;
F# = 698.46*2;
G# = 783.99*2;

if ((y>0)&(y<=5)) then
f = G#
elseif ((y>=6)&(y<=8)) then
f = F#
elseif ((y>=9)&(y<=12)) then
f = E#
elseif ((y>=13)&(y<=15)) then
f = D#
elseif ((y>=16)&(y<=19)) then
f = C#
elseif ((y>=20)&(y<=22)) then
f = B
elseif ((y>=23)&(y<=26)) then
f = A
elseif ((y>=27)&(y<=29)) then
f = G
elseif ((y>=30)&(y<=33)) then
f = F
elseif ((y>=34)&(y<=36)) then
f = E
elseif ((y>=37)&(y<=42)) then
f = D
end
endfunction

For the main code,

s = []
// Making music
for i = 1:length(data2(:,1))
f = frequency(data2(i,1))
t = time(data2(i,2))
s = [s note(f,t)]
end

sound(s)

Just note that the data2 variable there exist because of the hard code adjustments are made. The more important variable there is the s variable since that’s what we are playing.

So how does it sounds?

Hmmmm… sounds like something is off, don’t you think? That’s because the notes are continually playing at top amplitude. Let’s make a digital envelope, shall we?

WOAH!!!! DID WE JUST PLAY A SONG USING IT’S MUSIC SHEET?!
Yes we did, my friend. Yes we did.

Did we prove that we can make laptops make it’s own sound? Yes. Did we prove it’s not easy? Not really, with your knowledge it’s easy to make music out of image processing at this point! Well done! For this, I rate my self 10/10 for this one. Shout out again to Kuya Roland and Carlo who I asked for some ideas and Trixia for critizing my work.

agrees.png

 

Advertisements