Face Swap

Are you curious how the face swap filter works? Do you want to see what my final project for Computational Photography was? Keep reading and I’ll show both!

Step 1: Find the faces

First I used dlib.get_frontal_face_detector to find all the faces in the image.

Step 2: Find each face’s landmarks

Next I used dlib.shape_predictor to identify 68 important points in each of the faces from the previous step. This resulted in features such as the nose, eyes, eyebrows, lips, and jaw line.

Step 3: Turn the landmarks into a mesh

Once I had the landmarks, I used a technique called Delauney triangulation to create a series of triangles representing the mesh. I averaged the two meshes so that the triangles from each face would be made up of the same landmark points. I found that there were two ways to generate the mesh that worked with varying results. The first image is creating a mesh using all of the landmark points, and the second image shows using just the outer landmarks.

Step 4: Swap the triangles

Next, I used math to warp the source triangles to match their destination and I swapped each set of triangles. You can see the difference between using all or just the outer landmarks here.

Step 5: Blend

Then, I blended the faces so that the edges looked smoother. I created a mask by generating a convex hull from the points (cv2.convexHull).

And finally, I used cv2.seamlessClone to apply the masks and create these masterpieces.

Hope you learned something! Feel free to comment with any questions you have!