266 Character Animation with Direct3D array of D3DVERTEXELEMENT9 objects. With these elements you control every aspect of how the bitstream from a mesh is interpreted. As a quick recap, the following function shows you how to get the vertex decla- ration from a mesh and access the different elements in it (very useful for debugging purposes). void PrintMeshDeclaration(ID3DXMesh* pMesh) { //Get vertex declaration D3DVERTEXELEMENT9 decl[MAX_FVF_DECL_SIZE]; pMesh->GetDeclaration(decl); //Loop through valid elements for(int i=0; i<MAX_FVF_DECL_SIZE; i++) { if(decl[i].Type != D3DDECLTYPE_UNUSED) { g_debug << "Offset: " << (int)decl[i].Offset << ", Type: " << (int)decl[i].Type << ", Usage: " << (int)decl[i].Usage << "\n"; } else break; } } This function prints the offset, type, and usage of all active elements in a vertex declaration. Sometimes, when you are building your own vertex formats, it can be very useful to know at what offset a certain element is stored (and what type it is); especially when you deal with different meshes from different sources and or formats. Remember that you’re already dealing with meshes containing different elements. In the bone hierarchy of the SkinnedMesh class, for example, you have static meshes containing position, normal, and texture coordinates. You also have the skinned meshes there as well, and on top of the position, normal, and texture coordinates, they also contain the bone index and bone weight components. So we need to be able to add components to any arbitrary vertex declaration. For this purpose I’ve implemented the AddTangentBinormal() function. This function is not much different from the PrintMeshDeclaration() function. It takes a mesh as input, extracts the current mesh declaration, and adds the tangent and the binormal elements to it. Then, it clones the original mesh by using the newly created vertex declaration. Lastly, it computes the tangents and the binormals for all the vertices in Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Chapter 12 Wrinkle Maps 267 the mesh using the D3DXComputeTangentFrame() function. Once this has been done it releases the old mesh and replaces it with the newly created mesh containing valid tangents and binormals: void AddTangentBinormal(ID3DXMesh** pMesh) { //Get vertex declaration from mesh D3DVERTEXELEMENT9 decl[MAX_FVF_DECL_SIZE]; (*pMesh)->GetDeclaration(decl); //Find the end index of the declaration int index = 0; while(decl[index].Type != D3DDECLTYPE_UNUSED) { index++; } //Get size of last element (in bytes) int size = 0; switch(decl[index - 1].Type) { case D3DDECLTYPE_FLOAT1: size = 4; break; case D3DDECLTYPE_FLOAT2: size = 8; break; case D3DDECLTYPE_FLOAT3: size = 12; break; case D3DDECLTYPE_FLOAT4: size = 16; break; case D3DDECLTYPE_D3DCOLOR: size = 4; break; Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 268 Character Animation with Direct3D case D3DDECLTYPE_UBYTE4: size = 4; break; default: //Unhandled declaration type }; //Create tangent element D3DVERTEXELEMENT9 tangent = { 0, decl[index - 1].Offset + size, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TANGENT, 0 }; //Create binormal element D3DVERTEXELEMENT9 binormal = { 0, tangent.Offset + 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BINORMAL, 0 }; //End element D3DVERTEXELEMENT9 endElement = D3DDECL_END(); //Add new elements to the old vertex declaration decl[index++] = tangent; decl[index++] = binormal; decl[index] = endElement; //Convert mesh to the new vertex declaration ID3DXMesh* pNewMesh = NULL; Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Chapter 12 Wrinkle Maps 269 if(FAILED((*pMesh)->CloneMesh( (*pMesh)->GetOptions(), decl, g_pDevice, &pNewMesh))) { //Failed to clone mesh return; } //Compute the tangents and binormals if(FAILED(D3DXComputeTangentFrame(pNewMesh, NULL))) { //Failed to compute tangents and binormals for new mesh return; } //Release old mesh (*pMesh)->Release(); //Assign new mesh to the mesh pointer *pMesh = pNewMesh; } As you can see, this function takes a pointer to a pointer to a mesh (or a double pointer). This means that we can actually reassign the pointer being sent in and replace what it is pointing to. Most of the resource-loading and mesh-handling functions in the D3DX library take a double pointer and operate in pretty much the same way as this function. The AddTangentBinormal() function very much reminds one of the ConvertToIndexedBlendedMesh() function defined in the ID3DXSkinInfo interface. What that function did was to add the bone weights and bone indices elements to a mesh in exactly the same way. It also filled the newly created elements with some sensible information (just like what is done with the D3DXComputeTangent- Frame() function). Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 270 Character Animation with Direct3D Sometimes you have data stored in a mesh using a certain vertex declaration that you want to change; however, the data is fine as it is and you just want to change the declaration. Well, instead of using the CloneMesh() function to create a copy, you can use the UpdateSemantics() function in the ID3DXBaseMesh class for this. So if you want to add new elements to the vertex declaration, use the CloneMesh() function, but if you just want to re-label an element (for example, switching the tangent and the binormal, or texture coordinate 1 with texture coordinate 2, etc.) use the UpdateSemantics() function. After you’ve sent whatever mesh you want normal mapped through this function you have a mesh ready to be normal mapped. I won’t dive into the math behind tangent and binormal calculations, but if you’re interested you can read more about that in [Lengyel01]. Next is the final piece of the puzzle: the shader. THE NORMAL MAPPING SHADER The shader code takes all that theory you’ve been reading about, as well as the pre- pared meshes, and outputs something that looks a lot better than what you’ve seen so far. In this chapter I have implemented normal mapping for the morphing meshes and the Face class. You should have little trouble, though, porting it to the skinned mesh shader yourself. After adding the tangent and the binormal to the vertex declaration of the base mesh in the Face class, the full vertex declaration of the Face class looks like the following: //Face Vertex Format D3DVERTEXELEMENT9 faceVertexDecl[] = { //1st Stream: Base Mesh {0, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 0}, {0, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL, 0}, {0, 24, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 0}, {0, 32, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TANGENT, 0}, {0, 44, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BINORMAL, 0}, Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Chapter 12 Wrinkle Maps 271 //2nd Stream {1, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 1}, {1, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL, 1}, {1, 24, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 1}, //3rd Stream {2, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 2}, {2, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL, 2}, {2, 24, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 2}, //4th Stream {3, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 3}, {3, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL, 3}, {3, 24, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 3}, //5th Stream {4, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 4}, {4, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL, 4}, {4, 24, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 4}, D3DDECL_END() }; Note the new tangent and binormal elements in the first stream (the base mesh stream). Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 272 Character Animation with Direct3D As an optimization we only add the tangent and binormal elements to the base mesh of the Face class. It would be more correct to add it to all the meshes in the Face class and then blend these (in the same manner you blend the normals). However, the results are still fine as long as you don’t perform deformations of ridiculous proportions. Next, you need the input structure to the vertex shader to match the vertex de- claration, like this: //Vertex Input struct VS_INPUT { //Stream 0: Base Mesh float4 pos0 : POSITION0; float3 norm0 : NORMAL0; float2 tex0 : TEXCOORD0; float3 tangent : TANGENT0; float3 binormal : BINORMAL0; //Stream 1: Morph Target 1 float4 pos1 : POSITION01; float3 norm1 : NORMAL1; //Stream 2: Morph Target 2 float4 pos2 : POSITION2; float3 norm2 : NORMAL2; //Stream 3: Morph Target 3 float4 pos3 : POSITION3; float3 norm3 : NORMAL3; //Stream 4: Morph Target 4 float4 pos4 : POSITION4; float3 norm4 : NORMAL4; }; Nothing surprising here; the new tangent and binormal vectors have been added to stream 0 just like in the vertex declaration. What is new, though, is the VS_OUTPUT structure (describing what comes out from the vertex shader and into the pixel shader): Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Chapter 12 Wrinkle Maps 273 //Vertex Output / Pixel Shader Input struct VS_OUTPUT { float4 position : POSITION0; float2 tex0 : TEXCOORD0; float3 lightVec : TEXCOORD1; }; Instead of the old shade float value that we used to send in to the pixel shader, we send in the light vector (in tangent space). This is the vector that gets interpolated (just like any other value you send into the pixel shader), as illustrated in Figure 12.5. Then, to transform information stored in the VS_INPUT structure to the VS_OUTPUT structure, the following vertex shader performs the morphing and the conversion of the light vector to tangent space: //Vertex Shader VS_OUTPUT morphNormalMapVS(VS_INPUT IN) { VS_OUTPUT OUT = (VS_OUTPUT)0; float4 position = IN.pos0; float3 normal = IN.norm0; //Blend Position position += (IN.pos1 - IN.pos0) * weights.r; position += (IN.pos2 - IN.pos0) * weights.g; position += (IN.pos3 - IN.pos0) * weights.b; position += (IN.pos4 - IN.pos0) * weights.a; //Blend Normal normal += (IN.norm1 - IN.norm0) * weights.r; normal += (IN.norm2 - IN.norm0) * weights.g; normal += (IN.norm3 - IN.norm0) * weights.b; normal += (IN.norm4 - IN.norm0) * weights.a; //Getting the position of the vertex in the world float4 posWorld = mul(position, matW); OUT.position = mul(posWorld, matVP); //Get normal, tangent, and binormal in world space normal = normalize(mul(normal, matW)); float3 tangent = normalize(mul(IN.tangent, matW)); float3 binormal = normalize(mul(IN.binormal, matW)); Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 274 Character Animation with Direct3D //Getting vertex -> light vector float3 light = normalize(lightPos - posWorld); //Calculating the binormal and setting //the tangent binormal and normal matrix float3x3 TBNMatrix = float3x3(tangent, binormal, normal); //Setting the lightVector OUT.lightVec = mul(TBNMatrix, light); OUT.tex0 = IN.tex0; return OUT; } It is very common that the binormal is actually left out of this whole process and then calculated on-the-fly in the vertex shader. This can end up saving a lot of memory—12 bytes per vertex, in fact. In large projects this can add up to a whole lot. The binormal can then be calculated as a cross-product between the normal and the tangent in the following manner: float3 binormal = normalize(cross(normal, tangent)); Once the position and normal of the face has been calculated, the direction from the light source to the vertex is calculated. This is fed into the TBN Matrix, which transforms the light vector to tangent space. This information, together with the texture coordinates (as usual) are stored in the VS_OUTPUT structure and sent onward to the pixel shader. //Pixel Shader float4 morphNormalMapPS(VS_OUTPUT IN) : COLOR0 { //Calculate the color and the normal float4 color = tex2D(DiffuseSampler, IN.tex0); //This is how you uncompress a normal map float3 normal = 2.0f * tex2D(NormalSampler, IN.tex0).rgb - 1.0f; //Normalize the light float3 light = normalize(IN.lightVec); Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Chapter 12 Wrinkle Maps 275 //Set the output color float shade = max(saturate(dot(normal, light)), 0.2f); return color * shade; } In the pixel shader, the diffuse color is first sampled from the diffuse map. Then the normal map is sampled using the same texture coordinate. For this pixel, the normal is calculated from the normal map color as described earlier and compared with the light vector sent from the vertex shader. The resulting dot product is then multiplied with the color pixel and drawn onscreen. Figure 12.6 shows a comparison between normal vertex lighting and the more advanced per- pixel normal map lighting scheme. As you can see, the normal mapped version has a lot more detail compared to the simpler vertex lighting scheme; this despite the fact that both faces have the exact same polygon count. In the normal map, I’ve added some scars and bumps to the head and tried to make the cheekbones and forehead more pronounced. Finally, here’s the code example for this somewhat complex and long chapter. FIGURE 12.6 Vertex lighting vs. normal mapping. Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. [...]... of a character and then copy this half, flip the copy, and merge it with the original half, thus producing the full character In essence this also means that the UV coordinates of both halves are the same, which is a big “no-no” when creating normal maps So no surfaces using tiled or mirrored UV coordinates ease purchase PDF Split-Merge on www.verypdf.com to remove this watermark 280 Character Animation. .. raise this value (which will be in the range of 0 to 1) with the shininess value of the surface The resulting value we multiply with the specular color and then add to the diffuse color of the pixel Voila! You’ve implemented specular highlights ease purchase PDF Split-Merge on www.verypdf.com to remove this watermark 284 Character Animation with Direct3D SPECULAR MAPS Different materials have different... detail Figure 12.8 shows the two meshes needed to create a normal map ease purchase PDF Split-Merge on www.verypdf.com to remove this watermark 278 Character Animation with Direct3D FIGURE 12.8 Meshes needed to create a normal map You are already familiar with the low-polygon mesh It may have a strict polygon limit (and other restrictions) depending on whatever game requirements you may have The high-polygon... example, of a character that has had its left side created as a mirror image of its right side This means that they use the same UV space in the diffuse and normal map This in turn means that when you light a pixel from the right shoulder it will work correctly However, when you light a pixel on the mirrored part of your character, this normal will also be mirrored, which leads to incorrect lighting With some... containing the high- and low-polygon version of the Soldier’s face (also together with the exported NMF files) You’ll find these files in the “Head Model” folder together with Example 12.1 There is also a great Photoshop plug-in tool from NVIDIA available here: http://developer.nvidia.com/object/photoshop_dds_plugins.html With this tool you can convert bump maps or height maps into normal maps This is... normal is pointing halfway between the incoming light and the incoming view direction, as shown in Figure 12.10 ease purchase PDF Split-Merge on www.verypdf.com to remove this watermark 282 Character Animation with Direct3D FIGURE 12.10 Specular highlight Figure 12.10 shows the halfway vector between the light source and the view direction Both the position of the light and the camera get sent to the... of a bowling ball with the rough surface of human skin Figure 12.11 shows some different specular highlights FIGURE 12.11 Specular highlights on different surfaces The images in Figure 12.11 show the same surface with an increasing amount of surface smoothness As you can see from this figure, having specular highlights gives you more information about the object you are looking at With specular highlights...276 Character Animation with Direct3D EXAMPLE 12.1 In this example, you’ll find the full code for loading the normal maps, converting the mesh, adding the tangents and the binormals, as well as the full shader code... creating normal maps So no surfaces using tiled or mirrored UV coordinates ease purchase PDF Split-Merge on www.verypdf.com to remove this watermark 280 Character Animation with Direct3D Just because a model cannot have a mesh with overlapping UV coordinates at the time the normal map is created doesn’t mean that it can’t have it at runtime This rule about having shared UV space (either tiled or mirrored... these programs have free trial versions that you can download and try out However, normal maps can also be created with the free Normal Mapper tool (including source code) from ATI, which you can download from here: http://www2.ati.com/developer/NormalMapper-3_2_2.zip This tool comes with an exporter to both 3D Studio Max and Maya that exports a model to the NMF format, which can then be used by the . 12.7 Normal mapping with animated light source. Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 278 Character Animation with Direct3D You are already familiar with the low-polygon. 266 Character Animation with Direct3D array of D3DVERTEXELEMENT9 objects. With these elements you control every aspect of how the bitstream. Split-Merge on www.verypdf.com to remove this watermark. 280 Character Animation with Direct3D Just because a model cannot have a mesh with overlapping UV coordinates at the time the normal map