Changing facial features and body shape using blendshapes

✍ Last Updated : September 2, 2022

🚪 Prequisite Knowledge (Optional)

• Unity’s SkinnedMeshRenderer

❓ Key Question / Problem / Issue

How to populate a list of buttons that can change a 3d model feature using blendshapes

✅ Expected Output/Definition of Done

End user should be able to change facial features by clicking one of the buttons in a list of facial feature

🎁 Resulting Solution

Blendshape can be used to change a mesh shape without using armature/bone rig. It modify vertices position of a mesh in a skinnedmeshrenderer with each blendshape has their own weight, ranging from 0 to 100.

Blendshapes inside SkinnedMeshRenderer inspector

There are several ways on how blendshape can be applied, most common ones are using 1D slider, 2D slider, or using selection button.

To automate the generation of list of blendshapes that can be modified using slider, we’ll need to iterate through all active skinnedmeshrenderers and get the list of blendshape string from their shared mesh.

SkinnedMeshRenderer[] skinnedMeshRenderers = GetComponentsInChildren<SkinnedMeshRenderer>(true);
foreach(SkinnedMeshRenderer skinnedMeshRenderer in skinnedMeshRenderers)
	if(skinnedMeshRenderer.gameObject.activeInHierarchy)
		blendShapeHandler.RegisterBlendShape(skinnedMeshRenderer);

Since two or more SkinnedMeshRenderers might share the same blendshape name, and they are need to be synchronized (example : upperTorso’s stomach and lowerTorso’s stomach), we’ll only take only one blendshape name, so they only appear only once in the slider/button list. But that slider/button will have to be able to modify any SkinnedMeshRenderers that have said blendshape name.

Example:

The eyelash mesh and head both have eyes_4 blendshape, but the eyelash’s blendshape is not synced to the head’s blendshape

eyes_4 blendshape in both head and eyelash mesh are synced to 100

Note that, we only add the blendshapes that doesnt have the string “[anim]” or “[hidden]” since those will be used for different purpose, like blinking animation or any other animation.

public void RegisterBlendShape(SkinnedMeshRenderer skinnedMeshRenderer)
{
	Mesh mesh = skinnedMeshRenderer.sharedMesh;
	for(int i = 0; i < mesh.blendShapeCount; i++)
	{
		string blendName = mesh.GetBlendShapeName(i);
		BlendPair blendPair = blendPairs.Find(o => o.name == blendName);

		float value = 0;
		if(blendPair == null && blendName.IndexOf("[anim]") < 0 && blendName.IndexOf("[hidden]") < 0)
			blendPairs.Add(new BlendPair(){name = blendName, value = value});
	}
}

BlendPair is a class that holds the blendshape name and value.

public class BlendPair 
{
	public string name;
	public float  value;
}

To apply blendshape, iterate through each active skinnedMeshRenderer and call Unity’s SetBlendShapeWeight.

Note that unity’s SetBlendShapeWeight takes blendShape index instead of blendShape name. So we need to find the blendShape index first from the blendShape name using Mesh.GetBlendShapeIndex, with Mesh is skinnedMeshRenderer sharedMesh

public void SetBlendShapeWeight(List<SkinnedMeshRenderer> skinnedMeshRenderers, string blendName, float value)
{
	for(int i = 0; i < skinnedMeshRenderers.Count; i++)
		ApplyBlendShape(skinnedMeshRenderers[i], blendName, value);
}

public void ApplyBlendShape(SkinnedMeshRenderer skinnedMeshRenderer)
{
	for(int i = 0; i < blendPairs.Count; i++)
		ApplyBlendShape(skinnedMeshRenderer, blendPairs[i].name, blendPairs[i].value);
}

public void ApplyBlendShape(SkinnedMeshRenderer skinnedMeshRenderer, string blendName, float value)
{
	//Update blendPair value
	int blendPairIndex = blendPairs.FindIndex(o => o.name == blendName);
	if(blendPairIndex >= 0)
		blendPairs[blendPairIndex].value = value;

	if(skinnedMeshRenderer == null)
		return;

	Mesh mesh = skinnedMeshRenderer.sharedMesh;
	int blendShapeIndex = mesh.GetBlendShapeIndex(blendName);

	if(blendShapeIndex >= 0 && blendShapeIndex < mesh.blendShapeCount)
		skinnedMeshRenderer.SetBlendShapeWeight(blendShapeIndex, value);
}

This way, every skinnedMeshRenderer that has certain blendShape name will have the blendShape value updated with the same value, maintaining a synchronized blendshapes.