Hi there. I am currently developing a big character autorig system for Maya here at Milford. I have put in some really neat features and have bipedal arms and legs ready but it’s becoming heavy.
After spending two weeks on optimization and making certain features optional it playbacks at 16 FPS with only two arm and two legs, full functionality and without skinning (it was down to 11 before optimisation). By removing certain features that I don’t really want to remove I can bring the FPS up to 21.
The problem is that I cannot find any bottlenecks, the amount of nodes in the scene with two arms and two legs are near to 6500 and that on it’s own slows Maya down a bit. Many a little makes a mickle and I fear that it will be almost unbearably slow when a spine, a heavy facial rig and the almost certain scenario of having multiple characters in the same scene joins the fray.
On it’s own this is way to slow, but I want to put it in comparison to other complex rigs out there. How fast are your character rigs? Have you faced similar obstacles? How did you solve it?
I’ve read the optimization thread here at the forum and it’s useful as a guideline on how to improve your speed, but I’ve already done the things mentioned there and feel like I’m stuck against a wall here.
If you need that level of complexity for the sake of deformation then generate multiple rig resolutions for animation. That way your animators can animate on a lighter version of the rig with only the basic deformations whilst at render/export time you can process it with the complex version of the rig.
I have already separated complex skinning, muscle deformation and some deformation specific controls from the animation rig. I’m only talking about animation rigs now, sorry if I was unclear
Clean scene with an animated curve in it plays back at 500 FPS.
After rig generation 4795 nodes have been created (yes I know its far from 6500 as I said in my first post but I just realized that I didn’t check the new count after I had optimized the rig) and the same curve now plays back at 110 FPS.
I have just found a possible bottleneck. I have 20 nodes in the scene that are custom nodes written in Python (three different nodes). Upon deletion I go from the previously mentioned 16 FPS rig playback to 19 FPS rig playback. I didn’t see such a large FPS increase when deleting these nodes when I were down on 11 FPS and therefore didn’t think they were a problem. The thing is that these nodes have a lot of incoming and outgoing connections and it may not be the fact that it’s Python and not C++ that makes them heavy. It might be the fact that they handle a lot of data.
The logical solution would be to rewrite the plug-ins in c++ instead, but they are basically just doing simple multiplication and I don’t think I would get a significant increase in FPS by rewriting them. The most interesting of these custom nodes utilizes Mayas MRampAttribute to remap the incoming connections (usually 16 in my case) on a range from 0-1 to cover the range of the graphical curve. It then multiplies each entry with the current curve value at that position in the range and outputs each multiplied value. Can the fact that a single node handles so many connections be a bottleneck?
I have run into performance issues when I have a single simple node with lots of input/output connections. Even when the multiplication is simple, the number of connections definitely impacted performance. These days, I prefer breaking complex connections apart into multiple, more straight-forward flows whenever possible (even if I’m duplicating any data).
I assume this has to do with Maya’s ‘Pull’ data flow through the connections, so that as only one thing changes, multiple nodes have to re-evaluate (possibly unnecessarily), and to compound the issue Maya only uses one CPU core for the node evaluations, etc. Not to mention that as a general performance rule, computers prefer to process many simple statements as opposed to a smaller number of complex ones.
So the short answer is yes, at least in my case I have had performance issues with nodes with many connections, and was able to improve performance (or stop locking up Maya :)) by using alternate methods and reducing the number of the input/output connections.
I’ve never profiled Maya, but I know in Max, the bottleneck for complexity was always getting and setting attrs on objects. We wrote a vertlet cloth simulation that ran at like .5 fps, I had to hack up the whole thing because I found out about 98% of the performance was spent getting/setting attrs from Max custom objects. I put everything into global arrays and we had realtime performance.
So I don’t know the actual performance implications here but it is something to look at. Try to batch together your calls to the Maya API and design your system so instead of lots of small calls you can use a few large ones. Ie, if you were doing a simulation, you’d want to pull all the maya data into some native python structures (which are going to be very fast), do all your calculations, and set them back en-masse. In fact I wonder if you were to write a C++ portion, if you just wrote some utility functions for the mass collection/assignment of attrs, if that would speed it up.
Thanks for your advice everyone, I’m going to try and split the node up later on and see if I get any increased performance. I will also create a node-switch network where the FPS heavy functionality can be toggled on and off by the animators. Hopefully this rig will be fast enough soon enough
Not sure what version of Maya you’re on but there’s a Heat Map Display option in the Hypergraph which uses the timerX call to profile the scene graph and provide some details. Maya 2013 has a new tool called the DG Profiler which is more spreadsheet based but can do multiple iterations over the DG and average the results to show you where the bottlenecks are.
Typically with rigs Maya spends the majority of its time in dirty propagation as rigs inherently have a lot of interdependencies (ik drives joints, joints drive constraints + skin, constraints drive other objects and so on and so forth). There are other things that can play a factor as well if you’re dealing with dense mesh deformers by not pruning the points from them (by default Maya assumes that all points are required on a deformer but if your blend shape only moves a lip then you’re better off using the Edit Membership tool to prune verts that aren’t part of the shape to increase evaluation speed).
For your custom nodes it’s hard to say without actually seeing the graph but if it’s dealing with a lot of connections then it might be good to run a timerX profile on it and see how often it’s getting hit or its inputs/outputs being flagged dirty.
Thanks for your advice Shawn. I’m working with 2012, know about the heatmap display and have used it to pinpoint some possible bottlenecks. I actually found a way to improve performance by fiddling a little with the node setup in regard to my custom nodes. I have 16 different inputs per custom node. They deal with multiplication, so I figured, why not try and create 16 multiplyDivide nodes instead, and only use my node to define the multiplier for each node. It was a success. Instead of having to remap and multiply 16 attributes through my custom node every time an attribute changed, it now only has to recalculate it a single time every time an animator change the look of the curve (if this is confusing, read a couple of posts above where I have explained a little bit more about the nodes in question). It is a little bit discouraging to know that 16 MDNs are faster than my single node when it basically just remaps and multiplies Anyway, the FPS went from 16 with all cool functionality on, to 18. And with some functionality off from 21 to 25. Progress is made
Yes that DG profiler really looks like a great tool! Looking forward to try it out. The heat map isn’t good enough to be usable all the way, but with this as a complimentary tool I might not have to write new debug tools
Hello again everyone. I am going to reuse my old thread to ask a question. I have decided to develop a system that enables animators to switch functionality on and off, as well as key that switch. This will allow them to use the heaviest functionality only when necessary. There is a lot of connections that needs to be rerouted when switching, and I want to make it as lightweight as possible. I thought about this a little bit last week, came up with different approaches, and would like some feedback on them:
A choice node for each attribute.
Multiply divide nodes to allow re-routing of three attributes per node
Condition nodes to allow re-routing of three attributes per node
A custom node solution to allow re-routing of all connections at the same time through a single custom switch node
Expression based disconnect-connect solution driven by a single keyable attribute
Scriptjob based disconnect-connect solution driven by an attributeChanged trigger on a single keyable attribute
I think a python custom node might be the best solution depending on the extent of the rerouting. I can see having a lot of condition/multDiv nodes being hard to manage beyond a certain number.
and again it really depends on the scale of the operation you need to do, but a custom node might also be easier to manage in a single area rather than trying to trace nodes through a chain during maintanance.
[QUOTE=dgovil;15982]I think a python custom node might be the best solution depending on the extent of the rerouting. I can see having a lot of condition/multDiv nodes being hard to manage beyond a certain number.
and again it really depends on the scale of the operation you need to do, but a custom node might also be easier to manage in a single area rather than trying to trace nodes through a chain during maintanance.
Just my 2c anyway.[/QUOTE]
I am leaning towards that to, but my experience with custom nodes that handles a lot of connections is not that great. It became very slow the last time I did that.
Anyway, the FPS went from 16 with all cool functionality on, to 18. And with some functionality off from 21 to 25. Progress is made