Has anyone else tried to get a simple Direct X Shader to work in Max 2009? If you load any of the default .fx files included with Max 2009 (or create your own shader), the y value of TEXCOORD0 is not getting passed from the geometry to the shader. It appears to only work if you have more than one sampler using TEXCOORD0 or if you have multiple UV channels. For example, their colortest.fx fails miserably in Max 2009, but works in Max 9. Standardfx.fx works in both, but if you whittle it down to the bare minimum pixel shader that will work, you have to have the ambient occlusion sampler code, even if the ambient occlusion check box is off and they are both using the same UV set in the sampler:
…
if(g_TopDiffuseEnable)
TopCol = tex2D(g_TopSampler, diffuseUV);
No idea if it is related, but is it related to the offset in Y of texture coordinates (like if you are clamping instead of wrapping textures)? Probably not but I just thought I’d ask.
I’m sure Ben or Kees will see this and have an answer soon enough though.
Wow, I tried that earlier and it didn’t work, but I just forced the texture address to “wrap” again and that fixed it. It must have had an updating glitch when I refreshed the DX Materials earlier today. It appears that 3ds Max passes a negative Y value to TEXCOORD0 and that the default texture address is set to “clamp”.
You have to offset the texture coordinate in the Y direction by 1 (I can’t remember if it is positive or negative. So in the vertex shader you end up doing:
In.uvCoord.y += 1
or
In.uvCoord.y -= 1
or whatever, can’t remember if it is + or -.
Yeah it is a LAAAAAME glitch that everyone has had to debug.
In.texCoord.y += 1.0f; //this fixes Max’s V texcoord which is off by one
Out.texCoord = In.texCoord; //pass through texture coordinates from channel 1
ShaderFX will do this adjustment automatically for shaders inside max and leave it out for exported shaders, because you game engine will likely not need the same adjustment :).
So you probably want to do something similar for your own shaders.
There are plenty of other things that you might need to do inside a shader in the 3dsMax viewport that you do not need to do for your engine.
So I always keep one set of shaders for the max viewport and one for the engine.
With shaderfx that is pretty easy cause you just generate 2 sets of code from it, by hand you have a little bit more editting to do, but should be doable too.
I have a full 3ds Max material authoring pipeline that uses our game engine shaders exactly as they are in the engine. It required surprisingly few changes to make them work. I just created a simple preprocessor #define PS_MAXVERSION and then branched shader code where necessary for the differences in 3ds Max. Since we used a branching Uber shader, it was easy to setup a simple .fx file that just set the appropriate preprocessor constants and then included the main shader (and the 3ds Max branches had no performance effect in the engine because that branch was never defined by anything but 3ds Max).
We happened to explicitly state all of the texture addresses as wrap, so I never ran into this issue with that pipeline. It was only when I started to prototype new, simpler function-driven shaders that everything went haywire (plus, it all works as expected in Max 9, but not Max 2009).
The viewcube in max2009 uses a clamped shader, so if you dont specificy wrap in your shader you inherit the state from the viewcube shader… nice present from Autodesk
Turning off the viewcube will fix it, I’ve seen a similar problem in Maya too…
Ah yes, the texcoord thing. This is one of many “features” we’ve “enjoyed” as we’ve built a system to create shaders for use in 3ds Max. ShaderFX code is full of comments like:
– this crazy code is here to fix Max’s “support” for texture coordinates
I think we’ve become kinda calloused to this sort of thing though. Now we just put in the crazy work-arounds and move on.
Ben, isn’t working around Max bugs pretty similar to working around DirectX / SDK bugs? You figure out something that used to work isn’t working anymore, write some quick hack to fix it, comment it properly, send repro case to Autodesk/MS/Sony/ATI/Nvidia etc and hope for the best.
Meanwhile… Your game will ship with a lot of workaround hacks which might break when someone fixes the thing “properly”.
Oh wait, I have a few of Havok bugs I never got to submit since I know that if they fix it, my workaround will break and a lot of assets go broken… O:-)
All this code no-one inhouse wrote (and you have no source access to) having bugs is even nastier than having bugs in your own code.