I set up a system like that when I was back at Electronic Arts – the exporter would run, as a pre-export step, a SmokeTest framework and each project could drop modular little mell snippets for the smoke tests into their tools folder. Different teams had different specific problems.
I had a few standard buttons on each Smoke Test - like “Find” which would select anything that failed that particular test, and “Try to Fix” for things that were fixable by script. When an artist selected the failed test, a little description would show, too, which explained what went wrong, why it usually happened, and what the artist might do to fix it.
We had little tests for all sorts of stuff – I think one of the most used were actually to clean up after the old MJPolyTools edge and loop splitting scripts that everybody used then. The garbage would totally litter a scene, but not using the script would slow down the artists, and cleaning it up pre-export was pretty easy.
Here’s a screenshot of it’s early stage. Pretty simple, but the amount of time it saved was awesome.
http://www.alexhogan.com/folio/scripts/smokeTestBig.gif