Normal map compression formats in UE4

Trying to optimize performance, the question of whether or not to use the alpha of a normal map for hiding a mask in order to minimize texture look-ups.  The other option would be to leave the normal itself and create a separate, lower resolution mask file.  In that case, the normal could be compressed as DXT1 to halve it's memory size.  

A little visual test on the accuracy of texture compression in UE4 using both 16-bit and 8-bit PSD's.  TC_Normal compression uses BC5, a successor to DXT5, for imported normal maps.  BC5 uses only the red and green channels of the texture, rebuilding the blue channel in the texture node while disregarding the alpha channel.  In order to be able to use the alpha channel (for a mask, roughness map, or any grey-scale texture), a compression like DXT5 (TC_Default in UE4) can be used—at the cost of clarity.  For both DXT5 and DXT1, the idea is to use these maps for larger normal changes on an environment mesh while layering detailed tiling normals on top to create finer detail anyways.  The cleaner way would be to use BC5 compression and create a separate mask texture (or compile multiple masks or grayscale textures into an RGBA mask texture).

Simplified:

  1. BC5 - highest quality, no alpha
  2. DXT5 - lower quality, with alpha, same memory size as BC5
  3. DXT1 - lower quality, no alpha, half the memory size of BC5/DXT5

A little post on converting the normal's color space from (0-1) to (-1-1) HERE

More reading on compression formats HERE and HERE

8 Bit BC5

8 Bit DXT5

8 Bit DXT1

16 Bit BC5

16 Bit DXT5

16 Bit DXT1

Bump Offset with tiling layers in Unreal Engine 4

Primary Heightmap

Bump offset can be a great way to get a better sense of depth in addition to normal maps (just short of tessellation), but with VR development it is almost essential (in fact, it looks better in VR than on a mono screen).  The stereo split native in VR headsets causes the normal map to look more like wallpaper when trying to create larger normal changes.  Smaller details do not suffer as much, especially at lower roughness and more direct lighting.  Michael Abrash, formerly the VR dev at Valve, now Chief Scientist at Oculus, spoke about it here and fielded some questions in the comments.  

Primary Normalmap

Secondary Normalmap

Tertiary Normalmap

To create that sense of perspective in larger normal details a bump offset, or parallax, shader is optimal.  The need to blend multiple albedo and normal layers, some tiling at a different rates, created a problem as some of the layers seemed to float above the initial bump offset layer.  I solved the problem by isolating the main features of the bump offset node, the camera vector and height texture, and adding UV tile coordinates in separately after the fact. 

Initially, the heightmap was tiled X2, and the same UV coord fed into the coordinate input of the bump offset to tile both of the primary textures (diffuse and normal) as well as the secondary normal.

The problem arose when the tertiary detail called for a higher tile (in this case X8).

By setting the bump offset's coordinate to (0,0), it frees the camera vector and tiled heightmap of having any additional UV information.  Adding the tiled UV coordinates (red lines) into the bump offset output (green lines) allows for multiple tileable textures, each with unique tile multipliers, to be affected accordingly by the primary heightmap offset.

Seen here in this wild animated GIF, each normal map blended, first without, and then with the bump offset.  May seem subtle when getting to the finer details, but makes a huge difference when seen in the headset.

Normal Map Blending in Unreal Engine 4

Cross_Normal.jpg

Blending normal maps can come in very handy in Unreal for layering tileable textures and getting an extra bit of detail without having to spend resources on larger textures.  After seeing the "Add" node being used in a few user level files, I decided to do some comparisons.  The results I found are similar to some of the mistakes made by simply overlaying normal maps in Photoshop instead of the proper method of multiplying the blue channel while only overlaying the red and green channels, or using programs like nDo or Combiner.  Those work for the purpose of adding one detail normal two another, but in terms of creating a live, scale-able normal blend in Unreal, a loss-less method for adding is needed (unless, of course, muting the detail of either normal map is the desired effect).

Circles_Normal.jpg

Unreal Engine 4 offers a BlendAngleCorrectNormal function, but it is a bit cumbersome for the blending of two normal maps already oriented correctly (green and red channels for each have corresponding directions).  I created a simple function that acts like the "Whiteout blend" described here.  Works well at maintaining all of the detail, rather than washing it out, and packaging it in a material function keeps things tidy.  BC5 compression ditches the blue and alpha channels of the map, the blue being re-created in the texture sample node, and changes the color depth (from 0 --> 1, to -1 --> 1).  For these examples I generated two normal maps from nDo and rendered screengrabs from Unreal 4, version 4.5.1:

Keep in mind, blending normal maps is never 100% correct, but is needed often.  It is best practice to bake a map correctly versus adding additional details after the fact.