Maya, GitHub, and Script Paths for Mel and Python - How Would *You* Do It?

Tools Maya
,
1

I have a project on GitHub that I’m (slowly) populating with useful-to-me Mel and Python scripts for Maya (Windows only at this point) that I’d like to share. It’s GitHub - CausticPro/MayaVisualizerTools: Python modules, Mel scripts, and related data to help make Caustic Visualizer for Maya even better. though at the moment it’s quite unfinished. My intent is to have a repo that grows and changes and all people need do is occasionally sync, once the repo dirs are initially cloned and set up in their local Maya installations.

What I’d like to work out today is: what’s the easiest way for the most people to use the scripts in this repo easily – what’s the best way to set it up in Maya installation-wise so that it just… works?

METHOD ONE

I set this up and people complained about it.

I made a file called “VTSetup.py” and told people to put it in the directory with their existing userSetup.py, then within the userSetup.py have these two lines:

from VTSetup import *
VTSetup()

The VTSetup() command adds the (default location GitHub repo) ‘Python’ directory to the session’s sys.path and the (correctly reformatted) path to the end of mels’ MAYA_SCRIPT_PATH

User complaints (some contradict others, yes):
[ul]
[li]It’s too complicated, I am afraid of text editors and copying files around (and my lead TD doesn’t let me touch the userSetup file, which gets overwritten by P4 every time I launch Maya anyway)[/li][li]I don’t want to alter my userSetup.py, I like it pristine as God & I made it[/li][li]I use userSetup.mel so userSetup.py never executes, leave me alone[/li][li]I hate GIT so I downloaded the ZIP and expanded it in some path whose name I just made up randomly[/li][/ul]

METHOD TWO

Added “open_me_to_install.ma” which has a load-time script node that executes a similar script. The new script looks for instances of userSetup.mel, adds a little block of “putenv …” and “python("import sys;sys.path.append(…)")” commands to the end (if those lines are not already present). This has a few advantages:

[ul]
[li]it knows where the repo is on disk, based on the path of “open_me_to_install.ma”[/li][li]No user text editing, just open the file once and then close Maya.[/li][/ul]

It still makes me anxious (no users have complained yet, but give them time!):

[ul]
[li]If there’s no userSetup.mel it makes one, which would switch-off their userSetup.py if they had one of those (yeah, I may add code to append to the end of userSetup.py if found alone…)[/li][li]If you have multiple versions of Maya you need to open it for each of them (it bases the filename on the version in current Maya path name)[/li][li]I have to restart Maya after opening this file one time only for each version of Maya? Can’t you do it just once for all versions?[/li][/ul]

Now I know: there is no single way that will make everyone happy. Especially Maya users who seem to love complaining that devs did not read their minds, and then again tomorow because the dev yesterday didn’t predictively read them changing their mind today. But: I’m wondering if people have seen instances where adding (GitHub-based) directories of mel & python scripts has been done especially well. Anyone?

2

Have you looked at SundayPipeline?
http://www.3dg.dk/2011/08/23/sunday-pipeline-maya-sunday-install-script-source-code/

He has a really clever one liner install system.
It’s painless and works really well from a user perspective.

Also, for ease of use, you can have it check your online git repo every once in a while and download any updates to the folder too.

Honestly, the users you’re describing are highly unlikely to work in a git friendly way and the ones who would don’t need this kind of babysitting. SO you might as well meet the less tech savvy users all the way and give them the easiest system possible.

Also re userSetup.mel vs py, I was under the impression that both got sourced during run if they both existed? I know py gets run before MEL though…

3

Cute, though I guess it’s orthogonal to the “open_me_to_install.ma”" method, which still feels easiest to me – those four lines (which are actually a gateway to many more lines) could just be in the script node. His scheme also leaves the paths unaltered, hmm.

Maybe rename the ma file to “open_me_to_install_or_update.ma”?

IIRC, The Maya Python book (~page 131?) says userSetup.mel always takes precedence. Maybe it’s wrong, or I am? Easy to check I suppose when back at the office…

4

Pretty simple test, adding userSetup.mel and userSetup.py to …maya/2014-x64/scripts and having both print messages.

Mel prints, python doesn’t.

CORRECTION: WRONG! The python script prints its output to the output window, and the mel prints to the script editor history.

So: “Maya Python for Game and Film” is wrong in teh italicized call-out on page 131. Both will indeed run (which one is first I’m not sure)

5

I’d avoid having non-technical users try to work through github - as your own experience shows, it’s not likely to fit well with a lot of production artists’ preferences.

Complex installs are also a pain in the butt because they also mean complex un-installs – it’s a good idea to assume that people will be adding and deleting your stuff as needs change and they move between products.

OTOH if you’re githubbing, many of your users will be pretty technical - and they may want to be able to fix bugs and modify your code.

So, I think perhaps you really need two different modes of distribution: a source mode where people get all of your scripts in their own python environment and can work on them as individual files – and a packaged distribution that they can treat like a plugin and just turn on and off.

For the source distribution:

  1. Make sure it’s a single-rooted hierarchy so you don’t get in among other people’s folders. That way you can allow users some freedom to install it.
    1a) the likeliest problem here is stuff like the perforce api, which includes PYD files that must go into the maya install folder.
  2. you can create a usercustomize.py in the /maya/Python/Lib/site-packages if you want to avoid messing with usersetup. This runs before usersetup too, so if you want to spoof it you can. On the downside, on windows you need admin rights because the folder is in program files. However it’s a good one-stop location for shimming in your paths and any one-time initialization code.
  3. Use some simple code so that you can include non-code resources without knowing where the root is - use environment variables or config files instead of hard coded names, and have a single standard method for getting paths inside your distribution. You never know where people want to put stuff – half the time it seems crazy.

for end-user distribution:
Ultimately there’s going to have to a line of code somewhere that imports the root of your package. userSetup is as good a place as any, frankly, but code that goes in to usersetup should be just two lines: one to add your stuff to the path, and one to import your bootstrapping module. Do all the complex setup stuff in that imported module (and make sure you know what you want to do in there - it should be just the init module. Since you’ll need to have loose executable code in that module you don’t want to ever re-import it!)

Luckily the one-line root import works well with the single-rooted hierarchy setup you’d want for your coder users. At a very minimum you could send out a zip file which was identical to your source tree and the distribution version of your setup just puts that on the path and imports the setup module.

6

Side point: the end-user, zipped-up method of distribution is easy to morph into a Sunday -style download-then-run setup too.

7

[QUOTE=bjorke;21594]Pretty simple test, adding userSetup.mel and userSetup.py to …maya/2014-x64/scripts and having both print messages.

Mel prints, python doesn’t.

CORRECTION: WRONG! The python script prints its output to the output window, and the mel prints to the script editor history.

So: “Maya Python for Game and Film” is wrong in teh italicized call-out on page 131. Both will indeed run (which one is first I’m not sure)[/QUOTE]

The userSetup.py runs when Maya starts up and before it’s done loading. (I assume when it’s initializing its python interpreter)
The userSetup.mel runs after Maya is done loading up. It runs roughly the same time as when an evalDeferred command runs in the userSetup.py

Also, I definitely agree with Theodox on keeping the additions to the userSetup.py as simple as possible

8

I’d go for modules if you’re worried about people modifying userSetup. Something that isn’t documented and in fact according to one of the devs at Autodesk doesn’t happen, is that each module can itself have a userSetup.py in it’s internal scripts folder that runs when the path is added. This way a self contained package, distributed as a module can itself have a userSetup that set’s itself up. We did this for years at Eurocom and I’m starting to re-investigate this method of distribution for our internal pipeline at Crytek. It means that all you need to do to boot your setups is to add a module file and thats it, very clean and very self contained.

The other thing is that as of 2014 the exchange packs are all distributed as modules, but without the module file as Autodesk install them to a specific folder which in it’s self, is treated as the root of multiple modules. It kind of makes sense that if Autodesk are wanting third party packages distributed like this that we should follow suit.

Red9 is packaged by Autodesk from the source I give them, but all their installer is doing is just that, copying the folder to C:\ProgramData\Autodesk\ApplicationPlugins which in itself is pathed and managed when 2014 boots

9

In a development environment, where there may be several projects, each with sorted engines, tool requirements and differing versions of maya - and you want to deploy changes continually and rapidly along with content development in source control (like perforce), but everything is project or branch realtive there are alternative setups.

project root,

c:\depot\

and then a tools dir, something like:

c:\depot	ools\maya

in here, you can put your 'setup':
c:\depot	ools\maya\scripts\userSetup.py
* this file does all the loading, just as it would if it was in your user folder

Then in the root, you can set up a couple .bat files for your project
First, one that sets up env and such:

c:\depot\proj1_env.bat

And in there, set up things like:


:: Setup project environment
SET ROOT=C:\depot
SET CONTENT=C:\depot\proj1dev\content
SET GAME=C:\depot\proj1dev\game
SET TOOLS=C:\depot	ools

:: MAYA settings
set MAYA_MODULE_PATH=C:\Depot	ools\maya\modules
set MAYA_SCRIPT_PATH=C:\Depot	ools\maya; C:\Depot	ools\maya\scripts\mel;C:\Depot	ools\maya\scripts;
set MAYA_SHELF_PATH=C:\Depot	ools\maya\prefs\shelves
set PYTHONPATH=C:\Depot	ools\maya\scripts\python;C:\Depot	ools\maya;%PYTHONPATH%
:: Maya version for this project1
set PROJ1_MAYA_VERSION=2011

Then have another .bat file, for launching Maya

c:\depot\proj1_maya.bat

And that would have code like:


:: drive
PUSHD %~dp0

:: Keep changes local
SETLOCAL

:: Initialize env settings
CALL %~dp0proj1_env.bat

:: launch maya
start "" "%ProgramFiles(x86)%\Autodesk\Maya%PROJ1_MAYA_VERSION%\bin\Maya.exe" %*

:: Restore previous directory
POPD

ENDLOCAL

This method precludes the user ever needing to install or update anything, they can just sync the latest from source control and run the proper .bat file to launch maya.

10

[QUOTE=Mark-J;21618]I’d go for modules if you’re worried about people modifying userSetup. Something that isn’t documented and in fact according to one of the devs at Autodesk doesn’t happen, is that each module can itself have a userSetup.py in it’s internal scripts folder that runs when the path is added. This way a self contained package, distributed as a module can itself have a userSetup that set’s itself up. We did this for years at Eurocom and I’m starting to re-investigate this method of distribution for our internal pipeline at Crytek. It means that all you need to do to boot your setups is to add a module file and thats it, very clean and very self contained.

The other thing is that as of 2014 the exchange packs are all distributed as modules, but without the module file as Autodesk install them to a specific folder which in it’s self, is treated as the root of multiple modules. It kind of makes sense that if Autodesk are wanting third party packages distributed like this that we should follow suit.

Red9 is packaged by Autodesk from the source I give them, but all their installer is doing is just that, copying the folder to C:\ProgramData\Autodesk\ApplicationPlugins which in itself is pathed and managed when 2014 boots[/QUOTE]

I'd go for modules if you're worried about people modifying userSetup.

This is another reason I use the method I described, because then the user can modify ‘their’ userSetup.py file and other prefs, modules or plugins that resides in their user folder all they want … and their personal setup is separate from the project, yet both will get loaded.

11

After looking at MarkJ’s post I decided to revisit maya modules.

They are almost as irritating as I remembered ( funky custom files and lousy documentation!) but at least in my 2012 install they do work correctly and they do fire off their userSetup.py’s. So, a userSetup.py in modulefolder/scripts will get executed if any of the MAYA_MODULE_PATH locations have a .mod file that points to the module folder. It appears on my system, anyway, that the module userSetups load before the regular userSetup.py. This does seem like a nice way to distribute stuff without getting too deep into a user’s personal setup arrangements - and it’s easy to uninstall by just removing the mod file.

There’s one minor wrinkle. userSetup.py is getting executed as a script, not a module – so there is no file variable to use if you want to do your path modifications in there, rather than inside those icky module files. However, that’s easy to workaround by including a usercustomize.py alongside userSetup.py

Here’s a concrete example:

I have foilder for the environment like so:


X:\mayatools
X:\mayatools\scripts\
X:\mayatools\scripts\all_my_stuff.zip        # this is a zip file with my whole environment
X:\mayatools\scripts\usercustomize.py     # this runs at load. It just adds the all_my_stuff.zip using (site.addsitedir(os.path.dirname(__file__) + "/all_my_stuff.zip") .  No absolute paths needed.  You can't use this as a userSetup replacement: it loads BEFORE maya.cmds!
X:\mayatools\scripts\userSetup.py           # runs after maya initializes. It just does 'import name_of_my_root_package'   - all the actual bootstrapping is done in that module, which is in the zip file
X:\mayatools\icons
X:\mayatools\plugins
..etc

The .mod file lives in a maya modules path (mine is \maya\2012-x64\modules) and points at the mayatools folder:


+  mayatools 1.0  x:\mayatools

This is all just a test but it seems to be working; i’ll report back with further issues. The extra usercustomize is irritating but it’s needs no extra maintenance. It’s certainly nice to be able to toggle the whole thing on and off at will by removing the .mod file, and the freedom to place the actual working part anywhere (including a shard net drive).

The only issue I can see so far is the possibility of multiple environments side-by-side: each module doesn’t know about the others. The docs seem to say you can use relative paths to include common code - I wsan’t able to get it to work as described, however.

12

Yes I found that last week as I wanted to construct sub-paths based on file and as you say, as a script you can’t get it’s location. I ended just putting a blank module in the same folder with a stub function that just returned file, then in the userSetup importing and running that stub function to get it back. In 2014 on-wards there’s a ton of new support going in to modules via an xml file that runs all of the filepath additions and version dependency for you…looks something like this…

<Components>
	<RuntimeRequirements OS="win64|macOS|linux" Platform="Maya" SeriesMin="2013" />
	<MayaEnv expr="MAYA_SCRIPT_PATH+:=shelves" />
	<ComponentEntry ModuleName="./plug-ins/MayaCryExport22013_64.mll" AutoLoad="True"/>
</Components>
<Components>
	<RuntimeRequirements OS="win64|macOS|linux" Platform="Maya" SeriesMin="2012" />
	<MayaEnv expr="MAYA_SCRIPT_PATH+:=shelves" />
	<ComponentEntry ModuleName="./plug-ins/MayaCryExport22012_64.mll" AutoLoad="True"/>
</Components>
13

I’m glad to see Autodesk is putting a bit of love into modules…

http://docs.autodesk.com/MAYAUL/2014/ENU/Maya-API-Documentation/index.html?url=files/GUID-130A3F57-2A5D-4E56-B066-6B86F68EEA22.htm,topicNumber=d30e36711

We’ve succesfully used them for different maya versions with project specific tools (versioned too), common shared tools and 3rd party tool redistributions. These new additions (subfolders especially) will make managing this a whole lot easier, especially for pipelines spanning Maya Versions and projects.

14

That doc is only the tip of the iceberg, most of that stuff started to go in with 2013. The new xml management is gone through a little on Cyrille’s blog here:

http://around-the-corner.typepad.com/adn/2013/04/distributing-plug-insfiles-on-maya-and-3ds-max.html

although some of that info isn’t correct as none of it is supported under Maya 2014. This is all tied into the Exchange app support that they’re adding, makes sense for Autodesk to have a flexible single entry point for all supported Exchange packages.

15

ah, xml. What a great way to make simple information impossible to read. Score!

16

:wink: lol!

Have to agree but at least it’s a start!

17

I’m no advocate for xml, but do you have a preferred format for information like this?

18

Yaml is vastly more readable than the equivalent xml:

XML:


<?xml version="1.0"?>
<club>
  <players>
    <player id="kramnik"
            name="Vladimir Kramnik"
            rating="2700"
            status="GM" />
    <player id="fritz"
            name="Deep Fritz"
            rating="2700"
            status="Computer" />
    <player id="mertz"
            name="David Mertz"
            rating="1400"
            status="Amateur" />
  </players>
  <matches>
    <match>
        <Date>2002-10-04</Date>
        <White refid="fritz" />
        <Black refid="kramnik" />
        <Result>Draw</Result>
    </match>
    <match>
        <Date>2002-10-06</Date>
        <White refid="kramnik" />
        <Black refid="fritz" />
        <Result>White</Result>
    </match>
  </matches>
</club>

YAML:


---
players:
  Vladimir Kramnik: &kramnik
    rating: 2700
    status: GM
  Deep Fritz: &fritz
    rating: 2700
    status: Computer
  David Mertz: &mertz
    rating: 1400
    status: Amateur

matches:
  -
    Date: 2002-10-04
    White: *fritz
    Black: *kramnik
    Result: Draw
  -
    Date: 2002-10-06
    White: *kramnik
    Black: *fritz
    Result: White

example stolen from IBM Developer

19

I love configObj, just push a Python dict straight to an init style file, read it back directly as the original dict, a bit like json but vastly easier to read.

At the moment I’m re-writing the core of our Maya pipe and have a similar issue, I need to be able to read a config file to setup user paths etc at a project level. I looked at xml, but for the life of me I can’t bear the thought of writing a reader just to get the data back from an xml in the format I want so I’m looking at just having a Python module and just reading attrs directly from it… maybe a base class with attrs…whatever, but at least it’s far easier for any tech guy to open and understand.

I agree with Seth, xml is one of those things that most people write manually in the first place then spend an afternoon crafting a specific reader to extract what you want from it.

20

If you’re doing work that’s only python, the standard config module is the cheapest thing to deal with: it’s pretty straigthforward, it’s plaintext, and every python can read and write it. XML/YAML/JSON make sense if multiple languages have to get to the file. Standard library solutions are always best because (a) they’re incredibly well tested compared to anything we write and (b) they’re done!

For an opposing view, however, you might want to read RobG’s rant about Python and config files: Why bother with python and config files? | RobG3d