Lip Sync
Deciding a clip to lip sync:
I watch a lot of british comedy youtubers and would really enjoy recreating part of a sketch for the lip sync. I watched through some of my favourite youtube comedy sketch videos to find one that would be suitable for this task. I found a lot of them were either too long or had lots of different characters or settings - which would be difficult to convey in a lip sync with one character.
I decided on using the Jack and Dean sketch 'The Goblin'. While this has multiple characters, it only has one setting and I think I can make the conversation between the characters work by animating the reactions to the conversation when other characters are talking.
This is the sketch that I am going to create a lip sync for:
I am going to concentrate on animating the character played by Jack Howard. He plays an estate agent showing a couple around a potential house. I will animate his speech and his reactions to the couple talking.
Making Jack Howard in Fuse:
I am using Fuse to create a character model to fit Jack Howard. Using the creation options, I am putting together the outfit, hair, etc and then I am tweaking the facial features to try and make a character as close to Jack Howard as possible. Looking this close at a person makes me see details that I wouldn't usually notice - such as Jack's nose being slightly crooked - recreating small subtleties like this can really improve what a 3d model looks like.
Next I have uploaded my character model to Mixamo to rig it to a skeleton. This step needs to be done so that it can be animated with human movement.
Animating the face:
I started with animating the lips to match with the speech. I used a mouth shape reference image so I could make sure that the mouth shape would be right for the different sounds.
To start with I created a new keyframe for every sound in every word and closed his mouth at the end of every word. When I watched this back, there was way too much movement. It looked like he was speaking really fast and didn't fit with the audio. I deleted some keyframes and found that it looked better if I didn't close the mouth after every word and if I only kept mouth animations for the vowels and then constants that have emphasis on them.
I am finding animating by keyframes relatively simple, I just position the part that I am animating and press s to add a keyframe. It is very useful that it fills in the movement between two positions.
After animating the mouth movements to fit with the dialogue, I moved on to adding expressions and small movements. I went through each part that I could animate and added movement and expressions. Jack in the original video I was basing this off doesn't have a particularity expressive face, so I chose the expressions and movements that I felt fit with the audio, rather than trying to be true to the source material.
I also added in blinking occasionally to make the character model more lifelike.
Finished Lip Sync:
I am happy with how the lip sync turned out, but I have learnt through this that some movements need to be faster than others. I also found it difficult to make a 'shocked' face. I think the shocked face I created looks funny and fits with the comedy of the sketch, but it doesn't look very realistic - its an expression that definitely looks like it would be unnatural for a person to make in real life. I am happy with it though and I am proud of what I accomplished, even though I would improve some parts of it.