Compiling Vertex Chameleon for Maya 2015

Tools Maya
1

Some of our Artists really like using Vertex Chameleon, but a compiled version was not released for Maya 2015. The company has released the source code so that you can compile it yourself. I compiled it, and a lot of the operations work, but operations involving vertex color layers are broken. Applying vertex colors to a model without layers works fine. But when layers are created, the vertex colors look corrupted, with black splotches all over the place.

Has anyone else tried compiling this for Maya 2015? The company no longer supports the product, so I’m not sure where else to look for help.

2

where’s the source? I couldn’t find it on the site, just the installers

3
4

i’ll take a look

5

After some investigation, it looks like something broke between Maya 2014 and 2015. I came up with a test case where I created a color set (using Maya’s Color Set Editor) and then applied vertex colors (using the Maya API function MFnMesh::setFaceVertexColors). It worked in Maya 2014 but the colors got messed up in Maya 2015.

Interesting to note that the colors only get messed up if there is a “createColorSet” node in the construction history. Will have to do some more digging.

6

Bug report submitted to Autodesk.

7

UPDATE: Autodesk just released Service Pack 6 for Maya 2015, and they fixed the bug! (MAYA-51689 Fixed the setFaceVertexColors API )

So anyone compiling Vertex Chameleon for Maya 2015 should no longer have to figure out a workaround for VertexPolyColourCommand.cpp. On a side note, there is one more fix that needs to be made to get the plug-in to run on 2015 64-bit. In the file RH_VCP_Base.mel, I had to change

if (endsWith($mayaVersion, "x64"))

to

if (`about -is64`)
8

[QUOTE=RFlannery;27334]UPDATE: Autodesk just released Service Pack 6 for Maya 2015, and they fixed the bug! (MAYA-51689 Fixed the setFaceVertexColors API )

So anyone compiling Vertex Chameleon for Maya 2015 should no longer have to figure out a workaround for VertexPolyColourCommand.cpp. On a side note, there is one more fix that needs to be made to get the plug-in to run on 2015 64-bit. In the file RH_VCP_Base.mel, I had to change

if (endsWith($mayaVersion, "x64"))

to

if (`about -is64`)

[/QUOTE]

I patched it with this change, and compiled the 64-bit version for Maya 2015 and built an installer. It’s just the 64 bit version for 2015.

https://www.dropbox.com/s/n9qqobayeh3tb5w/VertexChameleon-1.8.3-Setup.exe?dl=0

Hope that’s useful for others.

9

Anybody has been able to run it in the lastest Maya version?

10

Was interested in trying this out and as I do not have Maya 2015 installed and prefer to work in 2016 anyway, I had a quick go at compiling Vertex Chameleon for Maya 2016 on Windows. Managed to get it to a point where I can start a build and get 2k+ errors like " ‘min’: identifier not found " and other stuff that make me think my paths are still not correct. I have not touched anything C++/Visual Studio/building related in a while so I might be overlooking something very simple. I’m on Win10/VS2015 here, that might of course be a problem. Going to sleep and maybe giving this another try tomorrow.

If anyone has the .mll lying around, feel free to share! :laugh:

11

Hmmmm, got it built but having trouble with licensing. The plugin is free to use and the company give a license key in their manual. It’s this one:

contact@renderheads.com-(0 user license)-RenderHeads-RenderHeads-Vertex
Chameleon 1.x-702S-6600-6C22-4861-702S-6600-4K22-4861-0N66-4R0G-546U-5N6X

But when I try to enter I get the following in the Script Editor:

Error: line 1: Syntax error: unexpected end - at position 2 while parsing:
x-*

and also, more worryingly:

Error: line 0: Cannot find procedure “RHEnterLicense”.

The only reference in the mel scripts to “RHEnterLicense” is in RenderHeadsBase.mel in the code that is supposed to send the key over to the compiled plugin code and I’m afraid it’s missing the function because I built in some bogus way.

Here’s the .mll if someone else wants to take a look.
https://dl.dropboxusercontent.com/u/30047269/tech-artists/VertexChameleon2016.mll

Just install from here
http://renderheads.com/product/vertex-chameleon/
or from this link from earlier in the thread
https://www.dropbox.com/s/n9qqobayeh3tb5w/VertexChameleon-1.8.3-Setup.exe?dl=0
and then copy my .mll file over to
C:\Program Files (x86)\RenderHeads\VertexChameleon 1.0\plug-ins
Then open Maya, load it from Plugin Manager and use the command “RH_VertexChameleon;” to launch it.

Hmmm, I should document the build steps…

12

So, stuff I did to make it build:

Get the code from GitHub - RenderHeads/MayaPlugin-VertexChameleon: Advanced vertex colouring plugin for Autodesk Maya.

Download Maya 2016 devkit from apps.autodesk.com

Download VS2015, just for this :laugh: The VS solution got migrated to VS2015 on first open, of course.

I also downloaded the Win10 Windows Kits and pointed VS at those in Include Directories instead of the Win8.1 ones (um, shared, winrt), but I doubt that makes any difference

Make sure /include and /lib from the downloaded devkit are in C++ Additional Include Directories and Additional Library Directories (lib64 not needed; I wasusing $(MAYALIBBASE) as instructed by the github readme, but that does not matter).

Add MAYA2016 to Preprocessor Definitions

Add “#include <algorithm>” to MayaUtility.h to make min & max work

?? remove WIN32 from Preprocessor Definitions ??
^ this was the critical step in making the build go through, but I wonder if that’s also what makes the final .mll not work correctly…?

Change linker Output File from $(OutDir)VertexChameleon2016.mll to $(OutDir)MayaVertexChameleonPlugin.mll

After building rename the file to VertexChameleon2016.mll again

Hope I didnt forget anything

13

i was able to build and use without issues. I dont think i did anything with licenses at all, it just worked.

Here is the dropbox for 2015 and 2016: Dropbox

14

Thanks a lot, works!

Apparently I was using the scripts from an older version I had installed earlier, which made it ask for a license.

15

[QUOTE=TheMaxx;29356]i was able to build and use without issues. I dont think i did anything with licenses at all, it just worked.

Here is the dropbox for 2015 and 2016: Dropbox

Awesome! :):
Thank you so much!
Got the interface working but error message :sigh: when trying to work with it:

  1. Installed old version of vertex chamelean via their website
    on Windows 10 (x64) for Maya 2016.

  2. Overwrote all folders in the installation directory with TheMaxx’s dropbox-content.

  3. In maya 2016 put “RH_VertexChameleon” in mel.
    and got the interface but also an error message:
    “//Warning: file: C:/Program Files (x86)/RenderHeads/VertexChameleon 1.0/scripts/RH_VertexChameleon.mel line 631: The -bs/-borderStyle flag is obsolete and should not be used.”

  4. When trying to use functions from vertex chameleon interface it doesn’t work
    plus I get the following error message:
    “// Error: line 0: Cannot find procedure “RH_VertexPolyColour”.”

Did I overlook anything when installing?

16

[QUOTE=mercurius;29934]
3) In maya 2016 put “RH_VertexChameleon” in mel.
and got the interface but also an error message:
“//Warning: file: C:/Program Files (x86)/RenderHeads/VertexChameleon 1.0/scripts/RH_VertexChameleon.mel line 631: The -bs/-borderStyle flag is obsolete and should not be used.”
[/QUOTE]
I haven’t looked at the files in the dropbox location. But I know that when we updated the plugin for 2016, I had to go through all the MEL scripts, look for “-borderStyle” and delete those flags. It won’t break anything if you leave it in there, but you will keep getting that warning message over and over.
[QUOTE=mercurius;29934]
4) When trying to use functions from vertex chameleon interface it doesn’t work
plus I get the following error message:
“// Error: line 0: Cannot find procedure “RH_VertexPolyColour”.”

Did I overlook anything when installing?[/QUOTE]
“RH_VertexPolyColour” is a command defined by the plugin. If you go to Maya’s Plug-in Manager, is Vertex Chameleon actually loaded?

17

Also, in Maya 2016, the “Polygons” menu set was replaced by the “Modeling” menu set. So that will have to be fixed up in a few places:
“RH_VCP.res.mel” - line 57. Change “Add to Polygons Menu” to “Add to Modeling Menu”.
“RH_VertexChameleon.mel” - Replace “polygonsMenuSet” with “modelingMenuSet” wherever you find it.

This is the easiest thing to do, but it means that the scripts will be broken for Maya 2015 and earlier. If you want it to work with both versions, you will have to add some conditional statements. For example:

int $version = VCP_GetMayaVersionNumber();
string $polyMS = "polygonsMenuSet";//`findMenuSetFromLabel "Polygons"`;
if ($version >= 2016)
	$polyMS = "modelingMenuSet";//`findMenuSetFromLabel "Modeling"`;
18

[QUOTE=RFlannery;29936]
“RH_VertexPolyColour” is a command defined by the plugin. If you go to Maya’s Plug-in Manager, is Vertex Chameleon actually loaded?[/QUOTE]

Thanks for pointing at this!
Spot on.
The Maya plug-in manager has a drop down menu for
“C:/Program Files (x86)/RenderHeads/VertexChameleon 1.0/plug-ins”
However, no entries to underneath it indicating loaded/auto load.

How can I fix this?

  • thanks!
19

[QUOTE=mercurius;29938]The Maya plug-in manager has a drop down menu for
“C:/Program Files (x86)/RenderHeads/VertexChameleon 1.0/plug-ins”
However, no entries to underneath it indicating loaded/auto load.[/QUOTE]

Hmmm, that’s a strange one. Since the Plug-in Manager shows the path, I guess your MAYA_PLUG_IN_PATH is set correctly. Can you confirm that the .mll file from the dropbox download actually went into that folder?

20

[QUOTE=RFlannery;29939]Hmmm, that’s a strange one. Since the Plug-in Manager shows the path, I guess your MAYA_PLUG_IN_PATH is set correctly. Can you confirm that the .mll file from the dropbox download actually went into that folder?[/QUOTE]

The “VertexChamleon2016.mll” is in “C:/Program Files (x86)/RenderHeads/VertexC
hameleon 1.0/plug-ins/2016”

Maybe something went wrong with the way I installed it.
I installed the older version to Maya 2016 and overwrote the folders.