Practical Facial Rigging

In this post I'll be going over a face setup Im working on for a personal project. The idea behind this setup is to allow for quick and efficient facial rigging, by allowing a large level of control, from a large library of preset face shapes and individual face controls. This setup will also allow for easy transferability of the the rig across characters and with that, animation data across multiple face rigs.

Some of the concepts I'll be touching on were introduced by Jeremy Ernst for the Gears of War 3 game.

I mentioned a large library of preset face shapes. Yes, this means blendshapes, or morph targets, or whatever the software you use calls them. And lots of them. The idea is to make the facial animation easier and cleaner. By having many face poses broken down into separate groups, we can narrow down the animation of these poses to just one attribute control, instead of having to animate translations or rotations (and sometimes even scale depending on how little joints we are using on the face) to get the desired pose. Again, this keeps things a lot cleaner and simpler to animate (single animation curves in the graph editor).

This takes us next to how the poses are made. The pose library for the face will be built following the Facial Action Coding System. Individual poses will be sculpted to represent each action unit, and then used as blendshapes. Below are some links with more information if you want to learn more about it.

http://en.wikipedia.org/wiki/Facial_Action_Coding_System
http://www.cs.cmu.edu/~face/facs.htm
http://face-and-emotion.com/dataface/facs/description.jsp

To summarize, F.A.C.S outlines a series of action units that describe the face's range of motion through the contraction or relaxation of muscles in the face.

Although the current setup offers the animator a lot of intuitive control to drive the character's facial expressions, its still desirable to have individual controls to offset the preset poses. To allow for that, on top of the current setup there will also be a control rig, with individual controls for joints that are weighted on the face skin cluster. The interesting part however is to figure out how to have the controls follow the surface of the face while we are changing between different face poses. And this is where the concept of rivets comes in.

In essence, rivets allow you to pin something to something else, much like a point constraint, but at a component level. Meaning, the object will move with the point it is pinned to. Therefore, you can deform the object and the rivet will move to follow the deformation. Places for using rivets include surfaces (NURBS and Poly) and curves. Below I included a script for creating a locator pinned to a point on a surface or a point on a curve.

# import python modules
try:
    import maya.cmds as cmds
    import maya.OpenMaya as om
except Exception, e:
    print "Error while trying to import python modules."
    print "Exception: ", e

def rivetAtPointOnSurface():
    ''' Pin the object to selected points on the NURBS surface '''
    # get the selected components
    sel = cmds.ls(sl=True, fl=True)
    
    if not sel:
        cmds.error("Nothing selected")
        
    # get the surface name
    surface = sel[0].split(".")[0]
    # get the shape
    shape = cmds.listRelatives(surface, shapes=True)[0]
    
    # create rivet for each selected component
    for each in sel:
        # get the position to pin the object to on the surface
        pos = cmds.xform(each, q=True, ws=True, t=True)
        # define the point
        mPoint = om.MPoint(pos[0], pos[1], pos[2])
        # get the surface's dagPath
        selectionList = om.MSelectionList()
        selectionList.add(shape)
        mNode = om.MDagPath()
        selectionList.getDagPath(0,mNode)
    
        if cmds.objectType(shape) == "nurbsSurface":        
    
            surfaceFn = om.MFnNurbsSurface(mNode)
            util = om.MScriptUtil()
        
            uParamPtr = util.asDoublePtr()
            vParamPtr = util.asDoublePtr()
        
            if not surfaceFn.isPointOnSurface(mPoint):
                surfaceFn.getParamAtPoint(mPoint, uParamPtr, vParamPtr, False, om.MSpace.kObject, 0.001)
            else:
                mPoint = surfaceFn.closestPoint(mPoint, False, uParamPtr, vParamPtr, False, 0.001, om.MSpace.kObject)
                surfaceFn.getParamAtPoint(mPoint, uParamPtr, vParamPtr, False, om.MSpace.kObject, 0.001)
            
            # get the U and V paramters
            u = util.getDouble(uParamPtr)
            v = util.getDouble(vParamPtr)
    
            # create rivet locator
            rivet = cmds.spaceLocator(n="rivet")[0]
    
            # create point on surface node
            psi = cmds.createNode("pointOnSurfaceInfo", n="psi_" + surface)
            cmds.setAttr(psi + ".parameterU", u)
            cmds.setAttr(psi + ".parameterU", v)
            
            cmds.connectAttr(shape + ".ws", psi + ".is", f=True)
            cmds.connectAttr(psi + ".position", rivet + ".t", f=True)
            
            # constraint the rivet locator
            cmds.orientConstraint(surface, rivet, mo=True)
        else:
            cmds.error("Not a NURBS surface component")

def rivetAtPointOnCurve(curve):
    ''' Pin the object to the selected CVs '''
    # get the selected components
    sel = cmds.ls(sl=True, fl=True)
     
    if not sel:
        cmds.error("Nothing selected")
         
    # get the curve name
    curve = sel[0].split(".")[0]
    # get the shape
    shape = cmds.listRelatives(curve, shapes=True)[0]
     
    # create rivet for each selected component
    for each in sel:
        # get the position to pin the object to on the surface
        pos = cmds.xform(each, q=True, ws=True, t=True)
        # define the point
        mPoint = om.MPoint(pos[0], pos[1], pos[2])
        # get the surface's dagPath
        selectionList = om.MSelectionList()
        selectionList.add(shape)
        mNode = om.MDagPath()
        selectionList.getDagPath(0,mNode)
     
        if cmds.objectType(shape) == "nurbsCurve":       
     
            curveFn = om.MFnNurbsCurve(mNode)
            util = om.MScriptUtil()
         
            uParamPtr = util.asDoublePtr()
         
            if curveFn.isPointOnCurve(mPoint):
           curveFn.getParamAtPoint(mPoint, uParamPtr, 0.001, om.MSpace.kObject)
            else:
           mPoint = curveFn.closestPoint(mPoint, uParamPtr, 0.001, om.MSpace.kObject)
           curveFn.getParamAtPoint(mPoint, uParamPtr, 0.001, om.MSpace.kObject)

            # get the U paramter
            u = util.getDouble(uParamPtr)
     
            # create rivet locator
            rivet = cmds.spaceLocator(n="rivet")[0]
     
            # create pointOnCurveInfo node
            pci = cmds.createNode("pointOnCurveInfo", n="pci_" + curve)
            cmds.setAttr(pci + ".parameter", u)
             
            cmds.connectAttr(shape + ".ws", pci + ".ic", f=True)
            cmds.connectAttr(pci + ".position", rivet + ".t", f=True)
            # constraint the rivet locator
            cmds.orientConstraint(curve, rivet, mo=True)
        else:
            cmds.error("Not a NURBS Curve component")

Unfortunately with Maya's default nodes we can't attach an object to a point on a polygon. An idea by Michael Bazhutkin was to use the node curveFromMeshEdge to extract two curves from edges on a surface and use them to create a loft surface (a NURBS patch) and pin the rivet to the middle point on this surface.

def rivetAtPointOnMesh():
    ''' Pin the object to selected face on the NURBS surface '''
    # get the selected component
    sel = cmds.ls(sl=True, fl=True)

    if not sel:
        cmds.error("Nothing selected")

    # get the surface name
    surface = sel[0].split(".")[0]
    # get the shape
    shape = cmds.listRelatives(surface, shapes=True)[0]    
    # create rivet for each selected component
    for each in sel:
        # convert selection to edges
        cmds.polyListComponentConversion(each, ff=True, te=True)
    
        # create curves from edges
        edges = cmds.ls(sl=True, fl=True)

        if cmds.objectType(shape) == "mesh": 
    
            c1 = cmds.createNode("curveFromMeshEdge")
            cmds.setAttr(me1 + ".ihi", 1)
            cmds.setAttr(me1 + ".ei[0]", edges[0])
        
            c2 = cmds.createNode("curveFromMeshEdge")
            cmds.setAttr(me2 + ".ihi", 1)
            cmds.setAttr(me2 + ".ei[0]", edges[1])
            
            # create a lofted surface from curves
            loft = cmds.createNode("loft")
            cmds.setAttr(loft + ".ic", s=2)
            cmds.setAttr(loft + ".u", True)
            cmds.setAttr(loft + ".rsn", True)
            
            # create a pointOnSurfaceInfo node for center point on lofted surface
            psi = cmds.createNode("pointOnSurfaceInfo")
            cmds.setAttr(psi + ".turnOnPercentage", 1)
            cmds.setAttr(psi + ".parameterU", .5)
            cmds.setAttr(psi + ".parameterV", .5)
            
            cmds.connectAttr(loft + ".os", psi + ".is", f=True)
            cmds.connectAttr(c1 + ".oc", loft + ".ic[0]")
            cmds.connectAttr(c2 + ".oc", loft + ".ic[[1]")
            cmds.connectAttr(shape + ".w", c1 + ".im")
            cmds.connectAttr(shape + ".w", c2 + ".im")
            
            # create rivet locator
            rivet = cmds.spaceLocator(n="rivet")[0]
            cmds.connectAttr(psi + ".position", rivet + ".t", f=True)
            
            # constraint the rivet locator
            cmds.orientConstraint(surface, rivet, mo=True)
    
        else:
            cmds.error("Not a polygon mesh")

This works great, but doesn't let us create a rivet on a specific point on a polygonal mesh. For that we would need a plugin that works like the pointOnSurfaceInfo. But that's a subject for later.

Now that we have a way to track different positions on the face, we can use that information to move our controls. And yes, I did say track (with italics). Because in essence that is what we are doing, much like motion capture, we are tracking different positions on that face to be able to drive or controls, which are driving the face joints.

With that in mind, we just need to understand how all these things will be hooked up together. Let break this down.

To keep our face rig as clean as possible, we will have one blendshape target (Master Blendshape) piped into the final face mesh. All of the different blendshapes (from the F.A.C.S pose library) will be driving the Master Blendshape, and hence the final mesh. The rivet locators will be pinned to the Master Blendshape face. Because the rivets are only pinned to a position, we also need to constraint its rotation to that of the head so it follows it correctly. We can then drive the control's parent space with the rivet. We can do that with a parent constraint on a group above the control. And finally the joints are parent constrainted to the controls.

Cool. Now that we have covered the setup lets look at what we can further achieve with this setup.

Transferring Rigs

The idea is that you can use the same topology for various face meshes, and by applying the initial rig as a blendshape, we now have the rig working with a new face model.

Transferring Animation

Transferring animation data also becomes really easy. With the the pose library the animator will be animating attributes, and with the controls, offset transforms, which are relative and not based on world or parent space. Therefore we can transfer animation really easily between multiple face rigs with the same setup.

So that was an overview of the design. Hopefully that can inspire you to do something cool!