Unit testing DCC code?

As a matter of best practices, I build (and enforce that we build) as much of our code outside of Maya and Mobu as possible, so it can be unit tested and used by standalone pipelines. That works great.

My main area of trouble is unit testing the in-DCC code, you know, the stuff using mayacmds, pymel, especially mobu’s pyfbsdk. The issue is that I can hardly develop efficiently without unit tests, and I don’t have a good way of writing them , thus I cannot develop efficiently in maya or mobu.

I was wondering how everyone (who does unit testing) does this?

1 Like

Once upon a time I instrumented a bunch of our Maya Python code with PyUnit (junint) Python Unit Testing Framework

Unfortunately we have so many modules and teams using their own version of these modules that this was never much more than a proof of concept. Also, since so much of the Maya code that you write is data or workflow specific, you often can’t test much more than “does the module import”.

We’re using a fairly substantial amount of python on our current project that has nothing to do with Maya. I should see what they’re using for unit testing…

All of this is less of a priority since we have embedded QA on our team (FTW!), and if we make substantial changes, we can ask them to run through a bunch of checks for us. Also, our Maya tools go through an explicit publish before they are deployed to the team, and we can have people run out of the “latest” version in Perforce before they are deployed to catch problems. In cases of greatest instability such as our 2012 deployment, we do that work in a sandbox and have people who can test it set up a separate clientspec/workspace.

1 Like

Our setup is pretty simple, we do all of our unittesting from Wing using Nose. This is actually another reason i like the sitecustomize pattern, this way I know that the environment my unittests are running under is pretty much the same environment the production code is going to run under (except for Mobu, since at least Mobu 2010 or 2011, i forget which, didn’t do an import site, or somethign to taht effect, basically whatever version it was wouldn’t hook a sitecustomize).

So basically, sitecustomize, then take advantage of the setUp() and tearDown() methods to do all your DCC specific stuff like test scene setup, etc. For example:

import unittest
import pymel.core as pm

class TestANode(unittest.TestCase):
    
    def setUp(self):
        #this is also where you could open a file
        #say you had a unittest character or asset
        pm.newFile(force=True)
    
        #this is just an example
        self.test_node = pm.createNode('network')

So now all your tests can reference self.test_node or whatever. Pretty straightforward no-brainer pattern. You may not actually need a tearDown() in most cases, i’ll have to go back through some of my unittests and see what specific cases exist, i can make up a few off the top of my head, but i know i have some actual real cases. You can run this using commandline runners or scripts as well, as long as your interpreter is setup to play nice with Maya or whatever else. Hope that helps, this is super simple and i’m sure there are better ways to do it, but this’ll get you up and running pretty quick.

Seth, you can use pymel and maya cmds outside of the Maya process? I haven’t tried, I’ve only tried with mobu’s pyfbsdk, which won’t work.

So this weekend I’m going to work on mocking it.

[QUOTE=Rob Galanakis;11859]Seth, you can use pymel and maya cmds outside of the Maya process? I haven’t tried, I’ve only tried with mobu’s pyfbsdk, which won’t work.

So this weekend I’m going to work on mocking it.[/QUOTE]

I believe you can, using mayapy.exe is the simplest route, but you can get python.exe to play nice with some of the maya stuff, i think the docs mention how to do this.

This is how I got pymel to work outside of DCC using regular python.exe, which allows me to spawn Qt windows aswell, since mayapy.exe doesn’t:



import os
import sys

#====================================================================#
#THESE ARE THE MISSING STUFF WHEN RUNNING python.exe compared with mayapy.exe
#====================================================================#

os.environ["MAYA_LOCATION"] = "C:\Program Files\Autodesk\Maya2011"
os.environ["PYTHONHOME"]    = "C:\Program Files\Autodesk\Maya2011\Python"
os.environ["PATH"] = "C:\\Program Files\\Autodesk\\Maya2011\\bin;" + os.environ["PATH"]

sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\site-packages\setuptools-0.6c9-py2.6.egg")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\site-packages\pymel-1.0.0-py2.6.egg")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\site-packages\ipython-0.10.1-py2.6.egg")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\site-packages\ply-3.3-py2.6.egg")                         
sys.path.append("C:\Program Files\Autodesk\Maya2011\\bin\python26.zip")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\DLLs")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\plat-win")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\lib-tk")
sys.path.append("C:\Program Files\Autodesk\Maya2011\\bin")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python")
sys.path.append("C:\Program Files\Autodesk\Maya2011\Python\lib\site-packages")

# THIS 
import maya.standalone
maya.standalone.initialize(name='python')


# OR 
import pymel.core as pm

When doing test driven development, I usually lay my tests out first, make them fail on purpose before a single line of impl… and then start the implementation, feeding all sorts of possible input in the testcase just so I can cover what makes sense… Though I tend to minimize it only for places where it makes sense to unit test… best practises are best only in the right context… else you might be writing redundant code over and over… an example would be a custom sort algorithm where I know what I can expect out of it and then assert if something goes wrong etc…

Qt / PyQt has a test module aswell where you can test gui related stuff, though I haven’t had any use for it yet…

Using the above code, I can test DCC code outside of maya and see all the nice FAIL FAIL FAIL FAIL SUCCESS messages right inside eclipse with PyDev, no need to launch maya, no need to use mayapy.exe which blocks PyQt :slight_smile:

[QUOTE=Rob Galanakis;11859]Seth, you can use pymel and maya cmds outside of the Maya process? I haven’t tried, I’ve only tried with mobu’s pyfbsdk, which won’t work.

So this weekend I’m going to work on mocking it.[/QUOTE]

You can do the maya.standalone stuff. where you basically import Maya as a python module and you have an instance running on the background which you can do as you please, it’s really similar to mayabatch but in theory it’s live and you could do it from an interactive console.

I’ve done some unit testing with some sample files, say you have a tool that merges verts, you start with a cube and you know how many verts you have, then you run the tool and you should know how many verts you have left. Or start with a scene that has the pivot off base, run your center pivot tool and now you know where it is.

The only downfall to that is that you can’t test UI modules, so you need to separate your UI code from your main code (which you should be doing anyways)

[QUOTE=LoneWolf;11863]no need to use mayapy.exe which blocks PyQt :)[/QUOTE]

Hmm…this i did not know, altho i haven’t started diving into PyQt yet too much as it is. But this is cool, being able to unittest UI code using the same framework as all the other code almost makes a point for moving UI development into Qt. Thanks for sharing!

Yeah mayapy.exe blocks guis from Qt for some reason. Atleast in 2011, that’s where I work…

The message you will get when trying to create something out of the QtGui module in mayapy is something along the lines: QWidget: Cannot create a QWidget when no GUI is being used…

That’s why I use an external interpreter (python26.exe / python26w.exe), so I can run the same GUI tools outside of maya but still control an instance of maya… or do unit testing :slight_smile:

An example is, I made a batch exporter for all our assets. It also allows for queueing different tasks to be performed on each scene before exporting… mayapy wouldn’t allow me to spawn the GUI to queue the jobs / tasks to be performed using the GUI. To make this work I had to use an external interpreter, or else the GUI wouldn’t spawn and mayapy would just EXIT after the warning of no GUI support.

Rob, how did you go with mocking maya.cmds in the end? I may end up doing the same thing.

Yasin, I hit a snag when trying to import OpenMaya with python.exe. I’m assuming because its compiled, your SOL on that front? I’ll just corral those functions into a specific set of tests if so.

I do lots of maya.standalone tests for core functionality, combined with mocks in situations where the setup/teardown is too slow. I have had to give up on all but the most rudimentary tests of UI code, which is a shame - but if I have to choose, I test the iceberg below the waterline and let the bit that pokes up fend for itself.

yeah, we came to the conclusion that if we were able to get QTest to play nice, the tests would be too complicated to maintain. So we are testing UI stuff manually and most working code cane be tested by nose2.