Making of Varga by Paul Tosca
Now lets suppose we are looking at the objects directly from front , it will look something like in the image below.
At the top of the image you have a color scheme : high poly is white,the normals for high poly are cyan,low poly is magenta, the normals for low poly are yellow (since we have a quad plane the normals will have the same orientation everywhere on the surface),
Below the color codes you see the TS ( tangent space )triad :tangent(red left to right ), normal (blue down to up) and binormal (green and perpendicular to the screen pointing from you to the screen); notice in the figure above that all the faces from the high version are parallel with the green arrow ( binormal in tangent space ) ,excluding the triangles that will cap the cylinder and that will not be visible in the normal map anyway.
Now ,for every face in the high poly i can decompose the normal vectors ( cyan colorcoded ) in TS in their tangent( red colorcoded) and normal ( blue colorcoded ) components ,the binormal component will be zero for all of them since the faces are parallel with the binormal vector = green arrow hence the flat color in the green channel of the normal map.
The green interrupted guides were drawn to help visualize better the transitions between faces and to see how they correspond with the actual geometry.
The gray arrows pointing from low poly to high are the searching rays used during the computation of normal map.Now lets consider some points from the low poly (A,B,C,D,E,F,G) and see what actually happens.
As a side note , usually vectors are normalized( with unit length) and if you decompose a vector into TS components ( or other system coordinate )each component will have a real value between -1 and 1 however this will be converted to positive integer and stored into RGB channels of the normal map as follows:
-for tangent ( in the red channel ) negative values (-1,0) will be mapped linear to 0- 127 ,0 will be 128 (neutral) and positive values (0,1) will be mapped linear to 129-255
-same will apply for binormal in the green channel
-for the normal vector the values can be only positive (0,1) so they will be mapped to 128-255;most of the time the normal of the high will not deviate a lot in comparison with the normal vector in TS for the low poly hence the blue look of normal maps in TS ( the blue channel will be mostly white);because of this TS normal maps might be compressed since it uses fewer colors than OS (object space)/WS (world space) normal maps,also since TS vectors can't point backward the normal component will be always positive and when normalized it can be coded using only the tangent and binormal (the normal will be computed based on the other two)
For point D we shoot a ray and when intersecting the high version the normal will be that of face number 4 that will be parallel with the normal TS vector and following the dotted lines you can see that the red channel will be 128 since the tangent component will be zero, and the normal component will be 1 mapped to 255 (full white) in blue channel.
For point F , the searching ray will intersect face number 6 and we can see that the normal vector of this face (cyanish colorcoded) will be decomposed into two components : tangent component (red)with a positive value around 0.78 mapped to 228 in the red channel and binormal component (blue) with a positive value of 0.625 mapped to 208 in the blue channel.
For point B , the searching ray will intersect face number 2 and we can see that the normal vector of this face will be decomposed into tangent component (red)with a negative value around -0.78 mapped to something like 22 in the red channel and binormal component (blue) with a positive value of 0.625 mapped to 208 in the blue channel.
By the way ,you can see individual channels inside photoshop with ctrl+1 ,ctrl+2 ,ctrl+3 for red green and blue and ctrl+~ (that will be tilda ... left to 1)for full rgb, and you can copy them as follows : for red -> ctrl+1(go to red channel) , ctrl+a (select all) , ctrl+shift+c(copy visible) , ctrl+~(go to full rgb) and ctrl+v (paste), after this combination of shortcuts you will have the red channel of the current selected layer as a new layer on top.
If i rotate the cylinder 90degrees like below
the normal map will look something like this
and on separate channels
This time the red channel is flat at 128 ( like green channel was before ) since all the faces in the high are now parallel with the red arrow (and when the normals are decomposed the tangent components will be zero mapped to 128); the green channel ( binormal ) will look similar to the red channel before (but rotated 90 degrees CCW) and the blue channel looks the same as before but rotated 90 deg.