Hey welcome to this new lecture! And here you'll learn how to detect faces and we'll
be using OpenCV with Python to detect one or more faces from an image.
So basically how does face detection works anyway? Well the idea is that
someone has created some Cascades which are basically some XML files such as
this one, and this XML file contains the information about the features that an
image of a face contains, so we're talking about a ratio of the shadows
of the eyes, and nose, and lips, and all these features, these pixel intensity
numbers are stored are recorded in this XML file which has been created by using
some images with faces as training samples. So basically you tell your
computer's ok these are, all these are faces and then you use software like
open CV to create such XML files. So these are called haarcascades and this
is the haarcascade for a frontal face object, and if you want other objects you
can find them in this link here so you've got full body and left eye
and lower body and so on. Alternatively you can use the resource
section of this lecture and you can download a zip file with all the XML
haarcascades there, but in this lecture we'll be focusing on frontal face, so
we'll be using this cascade to detect faces. The way it works now is that
we'll load the image in Python and then we will tell Python that this is a model
that you want to look for in the image and you want to find this model, so the
xml model in the image and what Python will do with it with the help of OpenCV,
it will start to search all the image using a window and then it will resize the image,
so it will decrease the image size and using the same window to detect for
for smaller faces and so on. We will go through that step by step. So let's write
the script that detects faces. Import cv2 is the first thing
you want to do, and then the next thing is you want to read
the cascade in Python, so let's call the variable face cascade cv2 and we have a
method called Cascade Classifier, so this will create a face cascade object
in Python and all we have to pass here is the path or the haarcascade. That's it.
And that will create a cascade classifier object, and now you can use
this cascade classifier object of the face feature to search for a face in your
image, so the next thing you might want to do is to load the image in Python.
The image that you want to search for a face. Let's say image equals, so you know that
we can load images in Python via the image read method of the OpenCV, and
you can just pass the photo dot jpg name here, so I'm passing the file path of
this image, so I'm not passing any second parameter here. That means I'm reading
the image as a color image, however a good idea is to use grayscale images
when searching for a face so I loaded the image here, but I'll be using the
grayscale version of the image when searching for a face in that image. That is
thought to produce high accuracy when were searching for faces because you know you
may notice that when you have very busy images with lots of features
OpenCV will not be 100 percent accurate so you may get faces that
OpenCV will miss out or you may get features that will be classified as faces.
Using the greyscale image though increases accuracy.
I could go ahead and pass a zero flag here so that I could read this image as
a grayscale, but I'd like to actually keep the original image as a color
version because I want to show the color version at the end but use the gray
image to pass it to the methods that will be searching for the face so I'll
create here a great image variable where I'll store the grayscale version of the
image so cv2 and we have a method called cvtColor and that takes this
argument, the original image of course and a flag, so an argument.
Call cv2 color BGR2GRAY, so what this means is that this will convert the BGR image,
so blue green red bands, it will convert it to a grayscale image. That's it, so if
you want to quickly show this gray image if you like cv2
imshow gray image there. We give a name for the window, gray and the image you
want to show. Then you need to pass the wait key parameter here. Let's say zero,
so we'll press any key and it will close the window cv2 destroy all windows.
So you learnt these methods in the previous lectures.
Let me execute this. So this is a grayscale version of the image.
And now we will use a method called detect multiscale and what this method
will do it will search for the cascade classifier so it will search for this
frontal face XML file in our image and it will return the coordinates of the
face in the image so for instance this is the image and what this method will
return is it will find the face and it'll say, so it will give you the number
of the row and the column of the upper left point of the face, so it will start
here and it will also give you the height of the face and the width of the face,
so we'd get a rectangle and then we would draw that rectangle in the image so
that's basically it, but you'll understand it better, and so we need to create let's
say a face is object variable where we'll store this X Y width and height values
so let's quickly grab that so we need to refer to the cascade classifier object
which is this one here, and what we want to do this with this object
is detected multiscale and you want to detect the
grey image and then you want to pass scale factor in there. You know that
when you have lots of parameters and they are too many to fit in a line, after
the coma you can just press enter and Python will read your new line as it was
one single line, so but be sure to just press Enter after the coma,
and so we have scale factor, and a good value to give for this
would be 1.05. Now what does this mean anyway? Well, let's consider this image.
What Python will do is that it will start from the original size of the
image and it will search it will create a window that will search for faces in
the image, so searching this area in this area in this area, and so once it does
that, then by giving a scale factor of one point zero five you're telling
Python to decrease the scale by five percent for the next face search.
So what Python will do is it will down scale the image by five percent and it
will search for bigger faces in the image so search again, search again and
then decrease it by five percent again the image and search for bigger faces
and so on until it goes to a final size. So that means a smaller value means
higher accuracy. If you give for example point five Python would decrease the
scale by 50 percent, so it will start from the original size and then it will
go 50 percent higher, and so you don't get much accuracy with that. The profit
with this number is the script will run quicker, so you'll have less passes to
the image for searching for a face. Zero point five is good, okay, and then you
have another parameter called minimum neighbors and that is usually set to
five and what this basically is, is that this tells Python how many neighbors to
search around the window, so you may want to experiment with these numbers
a little bit and see which gets the better result, but these two
are well accepted numbers. So let's do something now. Let's print
out faces and see what this is about, so what kinds of what kind of object this is.
And I could also print the type for faces, just like that. So I'll run the script
now and what the script will do is it will it will read this XML file and it will
load the image it will make the grayscale version of the of image and
then it will detect the coordinates of upper left corner of the face in the
image and the width and the height of the rectangle defining the face in the image.
And then it will print out the type of the faces and it would print the faces,
the actual faces object. So okay, press any key to exit this and so faces
is a Numpy array, a N dimensional array object and it is an array with four
values, so we have detected our face and these are the values basically defining
the face in in the image, so basically what we have here is, so this is 155
which would be the 155th column so this is the X, so the rectangle
should start somewhere here in the forehead and this should be 83, so row 83,
column 155, and then we have the width which is 382 and the height which is the same.
And so we have a rectangle in face. Now let's go ahead and draw that
rectangle in face, in the image, so we create the faces array and then what you
want to do is access all these values of this array. To do that we can use a
for loop, so for X Y width and height in faces.
Well, image so we are updating the image object that we have ready here, we have
loaded here, so we will update it by drawing a rectangle in the image and
again you need to pass the image object in here, and then you need to pass four more
arguments. The first argument would be the starting point of the rectangle so
that'll be x and y, so these coordinates of the face array. Great, and then the
next parameter is another tuple defining the coordinates of the other corner of
the image, so we've got the top left corner in here and we got the lowest
right corner in down here, so what that would be is X plus the width, and Y
plus the height. That's it. Yet another parameter this would be the color
that you want to give to this rectangle. So that comes as a BGR format, so you
want to pass value for the blue color, so let's say 0 for blue and let's have a
green rectangle so 255 for green, so we'll have a full green there and then 0
for red, and you can also pass another parameter which would be the width of
the rectangle let's say 3. Okay, and once you do that, so once you have updated
your image you may want to show your image window on the screen. So we use the
image show method but here we have a gray image so we want to pass the updated
image object, and we should be good to go. Save the script, go and try to run it.
And my system is not being able to find the face detector dot py. Yeah, I have
messed up something with the name here so let me change this. Dot py enter, so
nothing to do with the script and let me try again. Great, so I hope this is
what you were expecting, so the rectangle starts here and it has a width
and a height and it ends up in here. Sometimes though as you may already now
you may have images that are bigger than your screen resolution. In that case your
image would not fit in your screen, on your screen, so what you could do is
resize the image before showing it. So let me create a resized variable here,
and that would be equal to cv2 dot resize. And you want to pass the image, so which
contains the rectangle there and you want to resize the image to these two
values, so you need to set the resolution of the image here, and as you know you
can just put some values here. Let's say 500 by 500, but these values might
stretch out your image so you, a better solution would be to access the shape of
the image, so the resolution of the image. And this should be the width of your
image, so you already know this, I explained this so I'm just going through this
quickly and to make sure you get a good size you could divide it by three
and then again imgshape. You get the height which is the first value of the
tuple, and divided by three again and you also want to convert these to
integers because you might get float numbers and Python you will say that it
cannot resize an image to float numbers so that should be okay and then you want
to show the resized image. Quickly testing it, and I'm missing a
a bracket here, so this should be a tuple all these, but here I'm not starting
it ass a tuple. So bracket here and closes in here.
That should work now. Yeah, so this was my picture, but let's try a more challenging
picture now. This one here. So we've got two faces which are not very clear. This
is not a real frontal face and this guy here is he has his eyes closed and his
chin is not visible. And we've also got these two faces here in
the newspapers which, let me open the original image from here, which have
basically very low resolution and I think Python will not be able to detect
these two faces, so let me check this. Here's our face detector, and we want to
pass the name of the new image file. Save the script and go ahead and run it.
So we've got some back to reality here, and as you see this time Python and OpenCV
were able to detect this face, even though it's not in a very frontal position and
obviously it was also able to detect the hand of this guy, so Python reads it as
a face and what you can do in this case is to trick these values here, so the scale
factor and the minimum neighbors. Something you may also be interested to
know is the faces array here has two lists in this case, so this is the first
face the coordinates of the first face and then the second face which in this case
happens to be a hand, but anyway you get the idea.
So probably by maybe using a 1.1 scale you may be able to get rid of that hand.
So yeah, that's it. You can also try out to detect the face of this guy, but I don't
think you'll be able to do it, so just that you know that these techniques have
limitations, so it's a computer is not a human being, so it will always have some
some downsides in the accuracy. I hope you found this useful though,
and I'll see you in the next lecture.