Delta Engine Blog

Back to more useful programming stuff. Yesterday our modelers encountered a problem when exporting certain objects in 3D Studio Max. After some research I found out that the Tangent data isn't exported from 3D Studio and was rebuilt automatically with ComputeTangent in the engine. Usually this works fine, but several objects (not even very complex ones) had problems with the generated tangents. The texture mapping did fit perfectly, the normal map fits as well, all normals where fine and smoothed, but the TangentMatrix was messed up in the vertex shader.

First I thought there must be something wrong with the exporter and maybe there is just some export tangents option missing. I tested several exporters for the DirectX x file format: Panda exporter (the best exporter for .X files currently available), The Microsoft DirectX Exporter for 3DS Max (doesn't support much), XPorter (warning: Japanese site) and several other. None of them exports any tangent data (or binormals).

I tested developing my own exporter with help of the 3DS Max SDK (first a simple x file, than an ascii exporter and then the IGameExporter method), but there are no get tangent data functions available with the 3DS Max 6 SDK. I found some information in the net about the IGameMesh class in 3DS Max SDK 7 or 8 (8 isn't out yet), but I couldn't get any sample to compile. I also read that the GetTangent methods might return null and then I am where I was before. I also don't think the developer support from 3DS Max (discreet) is any good, it is very hard to get any information or downloads and most (80-90%) of the message board questions are just unanswered.

Well, back to my tangent problem. While searching for solutions for exporting tangent data in 3DS Max , I saw this article (NVMeshMender) from NVidia explaining how to generate tangent and binormal data for vertices. But after a short trip to FxComposer (btw: A new version 1.8 was released, check it out) I saw that NVMeshMender wasn't really the solution. As you can see on the following screenshot the 3DS Max exported mesh is still messed up (again, all position, normal, texture, etc. information is perfectly ok, only the generated tangent data is wrong).
Note: All models shown here are just test models and do not represent any final art. This is just a FxComposer screenshot:

So instead of spending more hours trying to implement some way to generate the tangent data in the 3DS Max exporter, I thought how could I generate the correct tangent data in my engine.

Here is the trick:

First check if the vertex declaration is wrong, make sure the used vertex shader declaration is used.
Generate texture coordinates and normals if they are missing (usually not for exported objects).
Weld (collapse) any vertices with the exact same data (position, normal, texture coordinates and tangent), this optimizes further processing and can improve rendering and minimizes the vertex buffer size.
Now if we didn't had any valid tangents, generate them in the following way:
- Clone the mesh, because we are going to reduce the vertices again and will kill texture coordinates to generate proper tangents.
- Weld (collapse) the vertices again, this time ignore any texture coordinates and put everything in one big smoothing group if exported model is smoothed (same normals). This won't change vertices with multiple different normals (e.g. a sharp box), only smooth surfaces!
- Now compute the tangent data with Mesh.ComputeTangent (in my example I didn't need the binormals, which are calculated in the vertex shader)
- And finally copy all generated tangents back to the original mesh with the untouched texture coordinates (this will duplicate tangents if texture coordinates are different for the same point at different faces).
Thats it, optimize the mesh and we are done!

And this is the difference between my method and just using ComputeTangent or NVMeshMender as before (2 simple screenshots from the engine):

Before (just computing tangents based on the vertices):

->

->

And after using my method with the compute tangent helper mesh:

And here is the source code for the main helper method. I use it to convert all meshes (no matter if generated in the engine or loaded from external files) to be compatible for the shader techniques.
Please note that some methods might not be available to you (e.g. the TangentVertex struct or my Graphics class, but these classes are not really used, replace them with your used vertex struct and your DirectX class).

/// <summary>
/// Generate normals and tangents if not present.
/// This method is very important for using shaders, most
/// shader techniques will expect the TangentVertex format!
/// </summary>
/// <param name="someMesh">Mesh we are going to manipulate</param>
public static void GenerateNormalsAndTangentsIfNotPresent(
  ref Mesh someMesh)
{
  if (someMesh == null)
    throw new ArgumentNullException("someMesh",
      "Can't generate normals and tangents without valid mesh.");
 
  // Quick check if vertex declaration of mesh already fits.
  VertexElement[] decl = someMesh.Declaration;
  if (TangentVertex.IsTangentVertexDeclaration(decl))
    // Everything looks fine, leave it that way.
    return;
  
  bool hadNormals = false;
  bool hadTangents = false;
  // Check the first couple of declaration usages
  for (int i = 0; i < 6 && i < decl.Length; i++)
  {
    if (decl[i].DeclarationUsage == DeclarationUsage.Normal)
    {
      hadNormals = true;
      break;
    } // if (decl[i].DeclarationUsage)
    if (decl[i].DeclarationUsage == DeclarationUsage.Tangent)
    {
      hadTangents = true;
      break;
    } // if (decl[i].DeclarationUsage)
  } // for (int)
 
  // Create new mesh and clone everything
  Mesh tempMesh = someMesh.Clone(
    someMesh.Options.Value,
    TangentVertex.VertexElements,
    Graphics.GetDirectXDevice());
  // Destroy current mesh, use the new one
  someMesh.Dispose();
  someMesh = tempMesh;
 
  // Check if we got texture coordinates, if not, generate them!
  bool gotMilkErmTexCoords = false;
  bool gotValidNormals = true;
  bool gotValidTangents = true;
  TangentVertex[] verts =
    (TangentVertex[])someMesh.LockVertexBuffer(
    typeof(TangentVertex),
    LockFlags.None,
    new int[1] { someMesh.NumberVertices });
  // Check all vertices
  for (int num = 0; num < verts.Length; num++)
  {
    // We only need at least 1 texture coordinate different from (0, 0)
    if (verts[num].u != 0.0f ||
      verts[num].v != 0.0f)
      gotMilkErmTexCoords = true;
 
    // All normals and tangents must be valid, else generate them below.
    if (verts[num].normal.IsZero())
      gotValidNormals = false;
    if (verts[num].tangent.IsZero())
      gotValidTangents = false;
 
    // If we found valid texture coordinates and no normals or tangents,
    // there isn't anything left to check here.
    if (gotMilkErmTexCoords == true &&
      gotValidNormals == false &&
      gotValidTangents == false)
      break;
  } // for (num, <, ++)
 
  // If declaration had normals, but we found no valid normals,
  // set hadNormals to false and generate valid normals (see below).
  if (gotValidNormals == false)
    hadNormals = false;
  // Same check for tangents
  if (gotValidTangents == false)
    hadTangents = false;
 
  // Generate dummy texture coordinates, not only useful for tangent
  // generation, but also unit tests display better visual meshes.
  if (gotMilkErmTexCoords == false)
  {
    for (int num = 0; num < verts.Length; num++)
    {
      // Similar stuff as in GenerateTextureCoordinates, very simple and
      // dummy way to generate texture coordinates from object position.
      // Usually only test objects don't have texture coordinates.
      verts[num].u = -0.75f + verts[num].pos.x / 2.0f;
      verts[num].v = +0.75f - verts[num].pos.y / 2.0f +
        verts[num].pos.z / 2.0f;
    } // for (num, <, ++)
  } // if (gotMilkErmTexCoords)
  someMesh.UnlockVertexBuffer();
 
  // Generate normals if this mesh hadn't any.
  if (hadNormals == false)
    someMesh.ComputeNormals();
 
  // Ok, first weld vertices which should be together anyway.
  // This optimizes rendering and enables us to do correct tangent
  // calculations below. For example a mesh using around 100 faces might
  // have around 300 vertices, if each face has its own 3 vertices. But
  // when collapsing same vertices together we can get this down to 100-150
  // vertices (which saves half of the bandwidth and vertex memory).
  WeldEpsilons weldEpsilons = new WeldEpsilons();
 
  // Position and normal should be the same (or nearly the same)
  weldEpsilons.Position = 0.0001f;
  weldEpsilons.Normal = 0.0001f;
 
  // Rest of the weldEpsilons values can stay 0, we don't use them
  // or if they are used (like texture coord or already generated tangent
  // data, they must be the same for vertices we want to collapse).
  someMesh.WeldVertices(
    // Don't collapse faces that are not smoothend together.
    WeldEpsilonsFlags.WeldPartialMatches,
    // Use the epsilon values defined above
    weldEpsilons,
    // Let WeldVertices generate the adjacency
    null);
 
  // Need to generate tangents because mesh doesn't provide any yet?
  if (hadTangents == false)
  {
    // Huston, we might have a problem!
    // If the vertices for a smoothend point exist several times the
    // DirectX ComputeTangent method is not able to threat them all the
    // same way (see Screenshots on my post on abi.exdream.com).
    // To circumvent this, we collapse all vertices in a cloned mesh
    // even if the texture coordinates don't fit. Then we copy the
    // generated tangents back to the original mesh vertices (duplicating
    // the tangents for vertices at the same point with the same normals
    // if required). This happens usually with models exported from 3DSMax.
 
    // Clone mesh just for tangent generation
    Mesh dummyTangentGenerationMesh = someMesh.Clone(
      someMesh.Options.Value,
      someMesh.Declaration,
      Graphics.GetDirectXDevice());
 
    // Reuse weldEpsilons, just change the TextureCoordinates, which we
    // don't care about anymore. TextureCoordinate expects 8 float values.
    weldEpsilons.TextureCoordinate =
      new float[] { 1, 1, 1, 1, 1, 1, 1, 1 };
    // Rest of the weldEpsilons values can stay 0, we don't use them.
    dummyTangentGenerationMesh.WeldVertices(
      // Don't collapse faces that are not smoothend together.
      WeldEpsilonsFlags.WeldPartialMatches,
      // Use the defined epsilon values
      weldEpsilons,
      // Let WeldVertices generate the adjacency
      null);
 
    // Compute tangents with texture channel 0,
    // tangents are in stream 0, binormals are not required
    // and wrapping doesn't help or work anyways (last 0 parameter).
    dummyTangentGenerationMesh.ComputeTangent(0, 0, D3DX.Default, 0);
 
    // Ok, time to copy the smoothly generated tangents back :)
    TangentVertex[] tangentVerts =
      (TangentVertex[])dummyTangentGenerationMesh.LockVertexBuffer(
      typeof(TangentVertex),
      LockFlags.None,
      new int[1] { dummyTangentGenerationMesh.NumberVertices });
    verts =
      (TangentVertex[])someMesh.LockVertexBuffer(
      typeof(TangentVertex),
      LockFlags.None,
      new int[1] { someMesh.NumberVertices });
    for (int num = 0; num < verts.Length; num++)
    {
      // Search for tangent vertex with the exact same position and normal.
      for (int tangentVertexNum = 0; tangentVertexNum <
        tangentVerts.Length; tangentVertexNum++)
        if (verts[num].pos == tangentVerts[tangentVertexNum].pos &&
          verts[num].normal == tangentVerts[tangentVertexNum].normal)
        {
          // Copy the tangent over
          verts[num].tangent = tangentVerts[tangentVertexNum].tangent;
          // No more checks required, proceed with next vertex
          break;
        } // for if (verts[num].pos)
    } // for (num)
    someMesh.UnlockVertexBuffer();
    dummyTangentGenerationMesh.UnlockVertexBuffer();
  } // if (hadTangents)
 
  // Finally optimize the mesh for the current graphics cards vertex cache.
  int[] adj = someMesh.ConvertPointRepsToAdjacency((GraphicsStream)null);
  someMesh.OptimizeInPlace(MeshFlags.OptimizeVertexCache, adj);
} // GenerateNormalsAndTangentsIfNotPresent(someMesh) 

Here are the basics for the TangentVertex struct:

public struct TangentVertex
{
  /// <summary>
  /// Position
  /// </summary>
  public Vector3 pos;
  /// <summary>
  /// Texture coordinates
  /// </summary>
  public float u, v;
  /// <summary>
  /// Normal
  /// </summary>
  public Vector3 normal;
  /// <summary>
  /// Tangent
  /// </summary>
  public Vector3 tangent;
  
  [constructors, helper methods, etc.]
} // struct TangentVertex 

And finally the unit test for the GenerateNormalsAndTangentsIfNotPresent method.

/// <summary>
/// Test generate normals and tangents if not present method.
/// </summary>
[Test]
public void TestGenerateNormalsAndTangentsIfNotPresent()
{
  // Create dummy DirectX device
  new Graphics().InitGraphics(DirectXHelper.BuildDummyDeviceSettings());
 
  // Create 3 meshes, 1 sphere and 2 boxes
  Mesh sphere = Mesh.Sphere(Graphics.GetDirectXDevice(), 1, 12, 12);
  Mesh box1 = Mesh.Box(Graphics.GetDirectXDevice(), 1, 1, 1);
  Mesh box2 = Mesh.Box(Graphics.GetDirectXDevice(), 1, 1, 1);
 
  // First test the sphere, we expect the same amount of vertices
  // for the resulting mesh and it should include normal and tangent
  // information.
  int sphereVerticesBefore = sphere.NumberVertices;
  GenerateNormalsAndTangentsIfNotPresent(ref sphere);
  Assert.AreEqual(sphereVerticesBefore, sphere.NumberVertices);
  // Get a tangent value and check if it is correct
  TangentVertex[] verts =
    (TangentVertex[])sphere.LockVertexBuffer(
    typeof(TangentVertex),
    LockFlags.None,
    new int[1] { sphere.NumberVertices });
  Assert.IsFalse(verts[0].normal.IsZero());
  Assert.IsFalse(verts[0].tangent.IsZero());
  sphere.UnlockVertexBuffer();
 
  // Check if declaration is correct
  Assert.IsTrue(TangentVertex.IsTangentVertexDeclaration(
    sphere.Declaration));
 
  // Now send box 1 to our method
  int box1VerticesBefore = box1.NumberVertices;
  GenerateNormalsAndTangentsIfNotPresent(ref box1);
  // Number of vertices shouldn't have changed (each face has its own
  // vertices, but they shouldn't be merged).
  Assert.AreEqual(box1VerticesBefore, box1.NumberVertices,
    "Box with each face having its own normals vertices count has " +
    "changed. The vertices at the edges shouldn't be merged.");
 
  // The first normal should be (0, 0, -1) (the bottom of the box).
  // The tangent for that should be (0, -1, 0)
  verts =
    (TangentVertex[])box1.LockVertexBuffer(
    typeof(TangentVertex),
    LockFlags.None,
    new int[1] { box1.NumberVertices });
  Assert.AreEqual(new Vector3(0, 0, -1), verts[0].normal);
  Assert.AreEqual(new Vector3(0, -1, 0), verts[0].tangent);
  box1.UnlockVertexBuffer();
 
  // Ok, it is time for box2. We are going to normalize it before
  // sending it to our normal and tangent generation method to force
  // this method to collapse all vertices with the same position and
  // normal!
  // ComputeNormals won't work because every mesh has its own vertices.
  // We will just set all normals based on the vector to the origin.
  box2 = box2.Clone(
    box2.Options.Value,
    CustomVertex.PositionNormal.Format,
    Graphics.GetDirectXDevice());
  CustomVertex.PositionNormal[] normalVerts =
    (CustomVertex.PositionNormal[])box2.LockVertexBuffer(
    typeof(CustomVertex.PositionNormal),
    LockFlags.None,
    new int[1] { box2.NumberVertices });
  for (int num = 0; num < verts.Length; num++)
  {
    normalVerts[num].Normal = -normalVerts[num].Position;
    normalVerts[num].Normal.Normalize();
  } // for (num)
  box2.UnlockVertexBuffer();
 
  int box2VerticesBefore = box2.NumberVertices;
  Assert.AreEqual(24, box2VerticesBefore);
  GenerateNormalsAndTangentsIfNotPresent(ref box2);
  // Number of vertices should have changed, vertices can be collapsed.
  Assert.IsFalse(box2VerticesBefore == box2.NumberVertices,
    "Box 2 has duplicate vertices (count=" + box2.NumberVertices +
    ") which can be collapsed!");
  // We should now have only 8 vertices.
  Assert.AreEqual(8, box2.NumberVertices);
} // TestGenerateNormalsAndTangentsIfNotPresent() 

Thats it. I hope after the introduction the source code is pretty self-explanatory. If you got any questions, just post a comment. I hope this helps if you have troubles with generating tangents, I'm just posting this because there is so few information about tangent generation around.