Ok mixing two normal maps in a shader is not that tricky, but what if they have different UVs and you only have a single TBN Matrix per vertex.
Only one of your normal maps will light correctly, unless they happen to have the same direction of UVs.
Has anyone had to do this before and come up with a neat cheap method?
Ideally without passing a TBN for each normal map…
Some ideas:
Minimal effort:
Try and keep the mapping roughly in the same direction (and look out for artists mirroring UVs which would be bad) and hope that its not so noticable.
Do the same as above and Weight the BTN based on the blending (in this case I’m using vertex colours to weight the amount normal maps blend). This is still not going to be perfect, but it would improve some situations.
Some how derive a new TBN or rotations for each normal map using the UVs and the first TBN
Bake 2d rotations offsets from the first TBN into vertex colours for the other channels. Rotate the tangent space normal map XY(RG colours)
I’m sorry, my art director would kill me if I gave out the details of our solutions to this problem, but I’ve done way more than a little work toward cheap solutions for this:
We found, however that with this solution, artists would author meshes and materials that didn’t actually require two sets of UVs.
This solution didn’t provide good results in the one case we really wanted.
+“other constraints.” There are some pretty heavy restrictions we had to use to make this work. It provided very good results and cost us no extra meshdata.
I didn’t try it, but I think a single channel addition to the vertex format doesn’t save you much compared to sending 3 channels to the GPU. I think they are expanded to the same block size. At the end most of these come down to 2 matrix ( or pseudo-matrix ) multiplies, so this is near identical to other methods.
Too meshdata heavy for us. We just couldn’t do it.
Pass in two tangents. If you need mirrored UVs, encode the sign somewhere clever, cross the tangents and normals to derive the binormals. If you have the interpolators free, I would highly recommend doing this in the vertex stage.
What you want to do is an expensive effect. Conceptually, you must get two spaces to the same space ( one matrix mul ) and then get that space into you lighting space ( another ). So, somehow you need to provide or neutralize all the relevant variables.
Good luck! I hope you figure out something even better than I did, because I’m still not quite happy.
Thanks for your reply Lithium, not enough interpolators is always the problem
Thinking out aloud…
My original idea about this was thinking of it as just an image processing problem of making two normal maps look the same as each other in tangent space. So if you rotate a normal map in photoshop 45 degrees to another normal map its wrong. So all you need to do is a 45 degree 2d rotation to the RG channels of the 45 degree normal map and you can make it correct.
So I thought I could rotate the RG in the pixel shader if I knew how far to rotate it compared to the normal map with the TBN matrix.
[QUOTE=mikiex;3004]So I thought I could rotate the RG in the pixel shader if I knew how far to rotate it compared to the normal map with the TBN matrix.
Would this work?[/QUOTE]
Absolutely, but you also need to know if it flipped. You could encode that in the sign of the rotation if you like.
It will cost you, I expect:
A meshdata channel, (Not Block)
An interpolator channel, (Not Block)
A few basic per-pixel ops,
Two per-pixel trig ops,
One TBN multiply.
That’s pretty good as far as these solutions go, the hardest part is going to be populating that data. I’d profile it though, and if you are ALU bound, you may want to explore what can be succinctly moved to the vertex shader.
The trig operation is the first (pseudo)Matrix Mult, though it is in 2D to get into a coherent Space, the second is the traditional TBN to get you into lighting space. Funny how these things reduce.
