Frictional Game Wiki

<h1>Animation - Lipsync</h1> <div class="level1"> </div> <h3>Introduction</h3> <div class="level3"> <p> This tutorial was written for anybody working on animations that want to a character to automatically lipsync. Lipsyncing is used to move the mouth of a character to match the dialogue it is speaking. How the mouth should move is calculated in the code using the voice recording of the dialogue as well as a written transcript. This takes the line and converts it into phonemic characters. These are then converted into visemes which are used to move the mouth. The visemes has to be created by hand and that is what this tutorial will detail. </p> </div> <h3>Visemes</h3> <div class="level3"> <p> A viseme is a position that the mouth has when making a sound. We use a 3rd party imlementation from Annosoft. They have three different viseme mappings that can be used, each of different quality. The best quality comes from using 17 Visemes. </p> <p> These pages show which visemes are needed and how to model them. </p> <p> <a href="http://www.annosoft.com/docs/Visemes17.html" class="urlextern" title="http://www.annosoft.com/docs/Visemes17.html" rel="nofollow">http://www.annosoft.com/docs/Visemes17.html</a> </p> <p> <a href="http://www.annosoft.com/docs/Visemes17.html" class="urlextern" title="http://www.annosoft.com/docs/Visemes17.html" rel="nofollow">http://www.annosoft.com/docs/Visemes12.html</a> </p> <p> <a href="http://www.annosoft.com/docs/Visemes17.html" class="urlextern" title="http://www.annosoft.com/docs/Visemes17.html" rel="nofollow">http://www.annosoft.com/docs/Visemes9.html</a> </p> <p> The visemes should be saved in "entity/animations/visemes/" </p> <p> <b>Setting up</b> </p> <p> When the visemes are created they need to be setup in the ModelEditor. </p> <ul> <li class="level1"> Open the entitiy file in the ModelEditor</li> <li class="level2"> Click on Entity Settings > Animations</li> <li class="level2"> Add a new animation for each Viseme</li> <li class="level2"> </li> </ul> <p> <b>Material</b> </p> <p> Each material will require one or more the following textures: </p> <p> <b>Diffuse</b> <br/> Size: Normal. <br/> File format: dds, dxt5. <br/> Suffix: None. <br/> This is simply the color and is needed for pretty much all materials. </p> <p> <b>Normal map</b> <br/> Size: Normal. <br/> File format: dds, 3dc. <br/> Suffix: “nrm”. <br/> The normal map for the material and uses the standard format rgb = xyz and is in tangent space. Note that this needs to be converted to a format of XY only though (3dc)! </p> <p> <b>Specular map</b> <br/> Size: Half of diffuse <br/> File format: dds, dxt5. <br/> Suffix: “spec”. <br/> The specular map handles the specular intensity and power for each pixel. The intensity uses the R,G and B channel and the power in the Alpha channel. Intensity sets the amount of specularity (for each color channel) applied to a pixel, the higher value of the channel, the higher intensity. Power sets the sharpness of the specular highlight, a low value give a dull and diffuse look, while a high value give a wet/shiny look. </p> <p> <b>Alpha</b> <br/> Size: Normal <br/> File format: tga, single channel (greyscale). <br/> Suffix: “alpha”. <br/> Sets the alpha for each texture and for any lit surface this is a simply 0 or 1 decision per pixel since a lit pixel can never be half transparent, only fully visible or invisible. </p> <p> <b>Height map</b> <br/> Size: Half of diffuse <br/> File format: tga, single channel (greyscale). <br/> Suffix: “height”. <br/> Height map is more like a depth map actually. White = largest depth, Black = no depth and this is important to remember when creating it. </p> <p> <b>Illumination map</b> <br/> Size: Half of diffuse <br/> File format: dds, dxt1. <br/> Suffix: “illum”. <br/> This map will be added additively on top of the model when rendering and will not be affected by lighting. It is useful for creating glowing surfaces such as lightbulb, ember, etc. </p> <p> With size means the size of texture width and height. So if diffuse is 512×512 in size, then a map with half of the size of diffuse will be 256×256. Normal simply means that it has the same size of the diffuse texture. Also note that the size of map might be specified otherwise in the description of an object, and if so the specified size shall be used. </p> <p> All of the textures in a material (and the material itself!) must be named according the model which uses it. An example of naming is as follows: </p> <p> The model has the name of “ball01” and its material has a diffuse, normal map and specular map. The textures will then be named like this: </p> <p> ”ball01.dds”, the diffuse. <br/> “ball01_nrm.dds”, the normal map. “ball01_spec.dds”, the specular map. </p> <p> If the model has uses several materials then add some extra name after the name of the file when settings up the textures. Here is an example: </p> <p> The model has the name “frontdoor01” and have two submeshes: “frame” and “door”. Each of these submeshes have its own texture and therefore also material. The base name for each submesh material will be: “frontdoor01_frame” and “frontdoor01_door” respectively. This means that the textures will have the names: </p> <p> For frame: <br/> “frontdoor01_frame.dds”, the diffuse. <br/> “frontdoor01_frame_nrm.dds”, the normal map. <br/> “frontdoor01_frame_spec.dds”, the specular map </p> <p> For door: <br/> “frontdoor01_door.dds”, the diffuse. <br/> “frontdoor01_door_nrm.dds”, the normal map. <br/> “frontdoor01_door_spec.dds”, the specular map. </p> </div> <h3>Lipsync Tool</h3> <div class="level3"> <p> Can be used to edit the anno files. More to come… </p> </div>