I am making an effort over the next few weeks or months to lift my game in regards to the reliability of the scripts and tools I produce.
I think my basic problem is that I don’t test them enough before releasing them to the artists, so I intend to be a lot more thorough, and maybe I don’t test them in quite the right way.
To that end, I thought it’d be interesting to hear from you all on the topic of testing. I would like to know if there are any common themes that come up, maybe there are best practises, maybe there are particular issues as regards tech-art.
While I particularly want to hear about how best to test tools,t he idea of testing software has wider implications beyond ‘does it work’. Maybe some of you have thoughts on these too. Things like the design of the tool, how it fits into the system of tools already in use at your company, do the artists like it or would they like the tools to be different somehow, and how the look and feel (and reliability) of the tools and systems at your company reflect on the team that produces them and so on and so on.
While it’s only one persons view, I agree with many points that are made: incremental releases via a suitable beta tester, the development of the UI over time (to accomodate basic to more advanced features of the tool) and datamining.
I personally aim to get the core functionality in place in its roughest form so the user can start saving time. This gives ample oppurtunity for useful feedback on improvements regarding functionality and addition of new features. That last point can be a bit of a double-edged sword as the tool is never finished, but that only encourages me to improve the structure of my future tools to allow for expansion.
And if you need more information regarding introducing new tools you’re developing/developed to the art team, then check out this great article from Bronwen Grimes: http://tech-artists.org/forum/showthread.php?t=226
As whw said, iteration is always better than finalization.
I have found that some well thought out smoke tests work great as well.
Basically find an artist or two to give you their workflow in terms of tool use, create a unit of work that establishes that workflow using the tool.
We are lucky that we have a three stage rollout.
We try to test as much in our development area, but being limited, it hurts for feedback and un-anticipated bug reports.
Then the core team, who hates when we break stuff, but, their extra level of responsibility is to test.
And lastly there is a beta group, with the same idea, but across more areas of discipline.
I think the thing to take away from that, have more than a single layer of an artist or two test. Start with the smaller group, and expand out. You can never get a full sample of how everyone will/won’t sue the tool correctly, but at least you can minimize any impact on the larger group.
‘’’
We try to test as much in our development area, but being limited, it hurts for feedback and un-anticipated bug reports.
‘’’ @amorano - This is something we experience for sure
I’ll echo Jason’s sentiment on unit testing. In addition to having a built-in safety net for your code, unit testing teaches you a bit about design as well. Worth investing if you aren’t already writing unit tests.
Not many posts, but I only put relevant stuff there. Quality over quantity.
I mentioned it in my latest PyMEL VirtualClass blogpost that Seth (djTomServo/@voMethod) gave me most of that data. Worth mentioning that Seth also helped get me started with the UnitTesting too. Just trying to pass it on . . .:nod: