Next time.... Next time we'll do it better!

Pipelines
1

Hi Guys, heres one for you, Its a genuine question and i’d like to hear your opinions, but its also something ive noticed on most (game) projects ive worked on and i’d like to hear if the same happens to you. Basically you finish a project, everyone is burnt out through the last few months of crunch, and in the last few weeks everyone is coming up with the terms ’ we’ll get it right from the start next time…’ or ‘lets hit the ground running’ or ‘we’ll learn from this’…ect.

Everyone has a couple of weeks holiday, refreshed and we start prototyping and researching our next project. The only thing is Ive noticed is the same bad habits start creeping in. People start creating their own directories and filenames for units, second hand rigs are used and so on.

So I guess I was just wondering, how early do you start putting your foot down and start getting some structure in a project?

Also how do you organise your file structure in this strange period of a project (not intergration, just basic housekeeping). Coming from Animation and Motion Capture last project this Tech role is a new thing for me, so any help on this kind of thing would be be great.

Also does this ‘new project’ fever happen to anyone else?..

Cheers Guys

Andrew.

2

[QUOTE=biggyninja;2874]Basically you finish a project, everyone is burnt out through the last few months of crunch, and in the last few weeks everyone is coming up with the terms ’ we’ll get it right from the start next time…’ or ‘lets hit the ground running’ or ‘we’ll learn from this’…ect. [/quote]
Oh this is easy. Whenever you hear ‘We’ll get it right from the start next time’, you should just smack that person. Just wallop 'em right upside the head. Because the ‘we’ll get it right from the start next time’ attitude is exactly why we didn’t it right this time. Feel free to smack yourself, too- we all do this in some ways.

Really you’ve identified the problem well but the solution is so blatantly obvious it is hard to see. There is no secret sauce, no magic structure or tool, no punishment or threat or reminder that can fix the problem you’ve identified. What we really need is a system that can adapt to this and change along with it. Trying to enforce some sort of rigidity or structure on a large scale is asking for trouble at every stage of the project. The system for anything, specifically content pipelines as you’re talking about, needs to be flexible and adaptive.

People need to be able to work how they want to work. The ‘pipeline issues’ must be taken care of automatically. No artist should ever have to name a material; any structural requirements need to be set up automatically (such as if your textures need to be in a folder up, this needs to be created automatically). Any system where you need to teach your artists the system, is going to fail. Because they are artists, and they don’t give a shit why we can’t use uppercase or have spaces in names. And they don’t give a shit so they won’t bother adhering to it- it is our asses that pay, not theirs. Make your life easier and make the artists’ lives easier by isolating them from any structural requirements you may make of them. Simplify it to a very dumb, single degree for what they have to adhere to; and build in a check to make sure they are adhering!

There is no knowing what is right from the start, because projects change, needs change, everything changes. Design your system for change, not based on a postmortem of what you’d have done differently based on your last project.

That is how you do it right next time! smacks self

3

Yeah, for the most part what Rob said, minus the smacking. Baseball bats are way more effective, for extra points, add a rusty nail.

But seriously, the big thing that learned from Halo 3 that continues to be reinforced to this day is: People are going to want things to change. Art is going to want to change models, animation is going to want new or different rigs for different cases right up to ship, and design reserves the right to change everything, to say nothing of publishing. Specific to tech art concerns, remember that the game artist is a both smart and devious breed. Worst case scenarie is you design your structure and they find ways around it that ultimately obviates all the work you’ve done. Bad TA, no whiskey.

The flipside is you don’t want to go too far down the path of “Here is our awesome and perfectly abstract system that can handle ANYTHING”. (no seriously, people have thought this is possible). All well and good theoretically, but what happens when you get to the point where production starts dictating what problems you need to solve, and in your quest for ultimate openness and abstraction you haven’t gotten to the part of the framework that contains the functionality for solving those problems? Well, you’re back to firing from the hip while your beautifully open framework decays by the side of the road and becomes the subject of drunken rants and laughing fits.

So i guess find the happy medium between structure and flexibility (thank you Zen Master Obvious, i know, i know). Don’t spend so much time thinking about HOW you can solve a problem that you’re left with inadequate time to actually solve it. Also, consider the edge cases, but don’t build your system with the idea that if it can handle the edge cases, the center should be a given.

These sound pretty obvious but…here we are production time, patching together production tools from the scraps of an R&D framework that didn’t heed the above advice…the horror:scared:

man…i was really trying to vocalize this image of a huge tools framework that was having little bits of it torn off by the realities of production but i couldn’t come up with a sufficiently slick way of putting it. descriptive_writing.Fail( True );

4

A - to the - Men

The only thing I’d add to that is “less whining, more worky”. It’s easy to say, oh if I could only rewrite these pipelines, oh if I could start over and rewrite all the shaders. Do or Do Not, there is no try.

In most of these cases you just have to stay the extra hour/weekend and pull the trigger on something. Do it on your own because no manager in their right mind would allow you to change pipelines mid production. But if you think you can do it, better and without making the project come to a halt, Just do it.

I had to rewrite every shader in a game to get the thing to run at 60, and rewrote complete import pipes because they sucked, so sometimes you don’t wait until next time, you fix it now and fix it so it’s easier to fix it again when the time comes

5

I can only agree with what has been said before.

Don’t bother trying to build the perfect pipeline, build a pipeline that is open to change instead.

Don’t build an exporter that is super efficient and clean, build one that is easy to change and ideally requires no input from artists at all.
Build one that is easy to get back into and change things a few months after you last seen it (not 20 functions nested like spaghetti into each other)

Don’t build an automatic rigging system that builds the best rigs ever, build one that is open to changing rigs (and hopefully preserve some of the animation already done). No animator is the same, no animator will always like the exact same rig. Give them a rig that works well for most, but isn’t so complex it cannot be changed.

Don’t bother trying to come up with the best possible nested folder structure, where everything is (seemingly) clearly organized together.
Come up with one that, if change happens, will still work, because one ‘small’ change in the game might completely mess up your complex nested structure (“hey, wouldn’t it be cool if we could randomly swap heads?”)

For example for textures, at one point they were stored nicely with the character and objects that used them. It might sound horrible to some, but these days all textures go into a single texture folder.
Yes that is a lot of files in one folder, but it is so obvious and simple to use, I very much prefer this now.

Sometimes over-organizing something is a lot worse then having a lot of files in a single folder.

6

I think if you look back a few years artists in games had far more to do with the technical construction and far more understanding because they required it. Smaller teams also meant more communication between programmers and artists. Games have also increased in technical complexity and maybe many artists are less interested in the how and why.

I agree automation is good, but I think its also important for the artists to be in touch with whats going on. We enforce naming conventions for textures and materials etc. I think even with that we are teaching people good practices. This whole disconnection of artists from technical aspects is something I worry about, which is why I try and pass as much knowlege to anyone interested.

We always structure our projects from the start and often use a very similar structure to the previous project (assuming it worked! :p:). The system we use is open to changes.

7

[QUOTE=mikiex;2880]We always structure our projects from the start and often use a very similar structure to the previous project (assuming it worked! :p:). The system we use is open to changes.[/QUOTE]

+1

We have a tool that creates the directory structure of a project and enough of the essential files (configs, packages etc) to run the level builder without errors, even if the level produced is meaningless. It’s a great starting point to quickly make prototypes and, because it’s all config driven, it’s easy to make a structural change later.

8

[QUOTE=mikiex;2880]I agree automation is good, but I think its also important for the artists to be in touch with whats going on. We enforce naming conventions for textures and materials etc. I think even with that we are teaching people good practices. This whole disconnection of artists from technical aspects is something I worry about, which is why I try and pass as much knowlege to anyone interested.[/QUOTE]

I wholly disagree and I think you are making a detrimental generalization here. There are a number of factors at play here:

First, what you are demanding artists who have no desire to be technical, to be technical, as a bare minimum requirement- this is a really bad thing to do. You are not teaching ‘good’ practices, you are teaching ‘some’ practices which happen to be ‘good’ for your pipeline but which should really be irrelevant or automatically handled.

Second, you are equating ‘technical’ with ‘procedural’, which it is completely not. The artists that learn the procedures aren’t becoming technical, they are learning procedures. There is nothing that translates into technical ability by making sure your files are named correctly. We aren’t technical artists because we are diligent filekeepers in this sense, it is because we understand and design the underlying logic requiring those procedures. Understanding this has absolutely no benefit to the artist and is useless- no, truly detrimental- information, because of the next point.

Third, you are tying your pipeline to trained knowledge on a very large and difficult scale. When you build a system and then require artists to learn that system, you are making a huge investment into that particular system and it is not open to change. It is extremely difficult to change that pipeline, because it is not changing code- it is retraining people AND changing code. A better pipe will remove the ‘people’ part of the equation entirely.

Actually, mikie, this is exactly what I run into when I talk to our Lead Char Artist- he was a large designer of our dynamic pipeline, and it sucks (I described it at GDC as a ‘clusterfuck of naming conventions’). When stuff is misnamed, he grumbles and blames people 6000 miles away for not following proper conventions and that people can’t check their work- then proceeds to fix it, often with mistakes. Really the issue is that he is asking an absurd and completely unnecessary task of artists- this needs to be handled automatically, pipeline things just need to be handled automatically, or you are always going to get these problems. I make them, he makes them, you make them; the only thing that won’t make human errors is good code, which once it works always works.

9

Hi guys, yeah, I agree with what you said about forcing artists to strictly follow a naming convention. I found that (especially with the younger artists) they became too nervous to work properly because they were too scared of breaking the build due to user error, and that nervousness made them even worse when it came to spelling because they wernt thinking straight. I was a big advocate of taking the pressure off the animators and automating the naming and export system, but unfortunately time was short and… well lets just say there was a lot of unhappy artists (me one of them). You cant force people to ‘not’ make a mistake. Thats why its a mistake (this is just from my point of view and experience Im not getting into an argument :))

Yeah, I really get what you guys are saying, some good stuff. I totally get what you are saying about giving artists freedom, just weve never had that luxury (which is why its been so painful). I think if you read into my post I said this is my first shot at the tech side because A) I was the only guy on the spot and B) because I went through so much pain in the past because of a lack of TA’s. Im not a TA whining at messy artists, god, Im not even a TA, im an animator who sets up fbx pipelines and has a basic grasp of maxscript who seems to be the only guy trying to sort things out :slight_smile:

I guess what I should be asking is ‘Any tips about setting up a project structure wise for a noob?’…

So yeah, same question, but more of a plea for help rather than a whinge (if thats what it came across as).

thanks guys.

(ps - most of my maxscript tests and meeting notes I do after work as it is, I work in games, I dont have any spare time :slight_smile: )

10

I wholly disagree and I think you are making a detrimental generalization here. There are a number of factors at play here:

First, what you are demanding artists who have no desire to be technical, to be technical, as a bare minimum requirement- this is a really bad thing to do. You are not teaching ‘good’ practices, you are teaching ‘some’ practices which happen to be ‘good’ for your pipeline but which should really be irrelevant or automatically handled.

First of all I was generalising, because I’m a generalist :slight_smile:

I’m talking about teaching all technical aspects of games and good practices that do translate to working for any company. Ok our naming convention may not be the same as other companies, but the fact there is one and it works well should rub off on the artist. Here I should also point out that the main reason for our names are readiblity and make it easy for anyone on the team to locate and understand what it is quickly.

Second, you are equating ‘technical’ with ‘procedural’, which it is completely not. The artists that learn the procedures aren’t becoming technical, they are learning procedures. There is nothing that translates into technical ability by making sure your files are named correctly. We aren’t technical artists because we are diligent filekeepers in this sense, it is because we understand and design the underlying logic requiring those procedures. Understanding this has absolutely no benefit to the artist and is useless- no, truly detrimental- information, because of the next point.

You wouldnt want artists to make filenames:

keeping readable by everyone
concise filenames
legal with all file systems
Using Pre,Pos Fixes “_” etc.
Not using Spaces
uncluttered
CamelCase
etc.

they would not benifit by using these practices? Maybe you wouldnt consider these technical?

Third, you are tying your pipeline to trained knowledge on a very large and difficult scale. When you build a system and then require artists to learn that system, you are making a huge investment into that particular system and it is not open to change. It is extremely difficult to change that pipeline, because it is not changing code- it is retraining people AND changing code. A better pipe will remove the ‘people’ part of the equation entirely.

I’ve not had any problems with people picking up our pipeline. I think to some extent we may of removed the people but it would be hard to know how it compares to you’re ideal.

Actually, mikie, this is exactly what I run into when I talk to our Lead Char Artist- he was a large designer of our dynamic pipeline, and it sucks (I described it at GDC as a ‘clusterfuck of naming conventions’). When stuff is misnamed, he grumbles and blames people 6000 miles away for not following proper conventions and that people can’t check their work- then proceeds to fix it, often with mistakes. Really the issue is that he is asking an absurd and completely unnecessary task of artists- this needs to be handled automatically, pipeline things just need to be handled automatically, or you are always going to get these problems. I make them, he makes them, you make them; the only thing that won’t make human errors is good code, which once it works always works.

Naming conventions don’t have to be anything to do with wether the pipline works or not.

One thing, even though I have worked in games for a good while and at a few places I dont think I know what the current state of other piplines are compared to the one I work with. I don’t know if we are ahead or behind, I wish there were a few detailed examples that I could use to compare.

11

[QUOTE=mikiex;2884]
You wouldnt want artists to make filenames:

keeping readable by everyone
concise filenames
legal with all file systems
Using Pre,Pos Fixes “_” etc.
Not using Spaces
uncluttered
CamelCase
etc.

they would not benifit by using these practices? Maybe you wouldnt consider these technical?[/quote]
The issue I have is with putting the onus on the artist to set it up this way- there needs to be a check in the system to make sure the names conform to standards, and at the same time that can educate the artists, by telling them what is wrong. And no, knowing how to name things is not technical.

I’ve not had any problems with people picking up our pipeline. I think to some extent we may of removed the people but it would be hard to know how it compares to you’re ideal.

I don’t know where you work, what games you’ve worked on, how big your teams were. But I can say for sure your pipeline is not as good, and your artists not as efficient, as they could be by creating a stronger pipeline by isolating them from it. There’s nothing subjective in that- an artist who never has to learn or worry about naming or organization will be ipso facto faster than one who does. A pipeline that relies on humans to check assets will ipso facto break more often than one where assets are checked and fixed automatically.

Naming conventions don’t have to be anything to do with wether the pipline works or not.

I agree- who or what enforces those conventions determines whether the pipe works or not, because that ripples down the entire thing.

One thing, even though I have worked in games for a good while and at a few places I dont think I know what the current state of other piplines are compared to the one I work with. I don’t know if we are ahead or behind, I wish there were a few detailed examples that I could use to compare.

We (myself and the Lead TA) are currently rewriting our entire content pipeline mid-production with hopefully zero impact to the artists. This was a big discussion during GDC- perhaps it would be worthwhile to blog the whole affair and get feedback and provide a case study. But I’d encourage people to write up descriptions of their pipelines as well, it is a very useful and interesting topic.

12

When I say naming conventions don’t have to be anything to do with whether a pipline works. What I mean is they can be just for readiblity and organising and actually have no affect on if the game works or not.

I’m interested for sure about how other people are doing things, Id like to know the steps and time an artist would take to get a textured cube into the game. Maybe there should be a number of these tests you could apply to any pipeline/engine to get an idea of performance.(Although you shouldnt cheat by totally automating just these tests :slight_smile: )

13

I’d be interested too, don’t have too much to add at this point, we do animation and we’re a small team, but I’d really like to learn from big pipelines and see what I can incorporate to our own.

Great thread!
-Johan

14

check out the following. Modular Procedural Rigging
No audio, but there are plenty of slides that give you a look at some pipeline strategies.

15

Cheers Harris, theres some really good stuff in there to do with rigging. Unfortunately we dont use maya, so a lot of the information was over my head, but I think I see what they are getting at.
A lot of the tools he mentioned such as layers, sharing animations and so on are really nailed down in Motionbuilder, its really easy to place anims from one skeleton to another, even if they are slightly different. Because motionbuilder uses its own rig, the naming conventions between two characters isnt an issue either.

Im starting to write up notes from all the comments on this thread, thanks a lot guys!

16

[QUOTE=mikiex;2888]
I’m interested for sure about how other people are doing things, Id like to know the steps and time an artist would take to get a textured cube into the game. Maybe there should be a number of these tests you could apply to any pipeline/engine to get an idea of performance.(Although you shouldnt cheat by totally automating just these tests :slight_smile: )[/QUOTE]

Our system now is almost completely automated, ie the artists don’t even setup the directory structure anymore, it’s all through tools. We have a content creation wizard that prompts the user as to what type of game asset they’re going to be creating which then creates all the relevant files, does source control management (adding new entries), creates the tree on disk, and even sets up the initial max or maya files. Really all the artist has to do is work in the file and press the export button. One of the things we’ve found is that our artists don’t really care to get into the minutia of file management, process oversight, etc, they Just Want To Make Content.

While this sounds rather draconian, the artist does have the option to setup their own assets manually and export without going through the tools pipeline, however, the amount of files that need to be created to register an asset with our pipeline is the sort of thing you do once then decide that, hmm, maybe there’s a reason for all those tools.

17

Interesting thread… I think the main point from any tech is don’t trust the artist / animator as far as you can throw them. If they can find a way to break pipelines or rigs they will, it’s all about keeping one step ahead of them and building into the systems enough flexibility so that when they do destroy things, you’ve got a way to fix.

Any pipeline built on naming convention is going to break at some point, that’s not to say naming convention is a bad thing… it does help artists organize and generally work in a cleaner manner, but don’t rely on it. Like Seth we run metaTags, or a mixture of metaData, message links and tagAttrs and never actually use any naming directly.

We’ve been continually developing our pipeline here for about 7years, similar toolset to Bungie by the look of it, hammered through 7 or 8 games so far. The biggest improvement was when we switched everything to referencing… all managed through our own referenceManagement setup that deals with namespaces and compatibility checking. The minute you remove animators abilities to re-parent rig nodes, everything becomes more stable. That said it did take a lot of work to get referencing up to speed…and many, many bug fixes from Autodesk!

Mark

18

[QUOTE=Mark-J;2910]The biggest improvement was when we switched everything to referencing… all managed through our own referenceManagement setup that deals with namespaces and compatibility checking. The minute you remove animators abilities to re-parent rig nodes, everything becomes more stable. That said it did take a lot of work to get referencing up to speed…and many, many bug fixes from Autodesk!

Mark[/QUOTE]

Hi mark, Im really interested in this referenceManagement (and the dealing with namespaces and compatibility checking). Is there any way you could explain this a bit further, Id really like to understand this a bit better (message me if you like).

I had a chat with the audio director today who also has a lot to do with the cinematics and was basically telling him since I joined this site and spoke to you guys how far behind we were in terms of both tools and technical artists. Weve really been caught out in the next gen era. Its something now were all scrambling to resolve.

Because we’ve done pretty good in the past with what we’ve had (Rome:Total War and Medieval2:Total War) weve assumed we could easily step into consoles, buts it turning out to be a sobering experience. The main thing is that weve recognized that now and doing something about it.

19

The jump to next gen consoles was an uphill one for most of us I think. Our first NextGen was ‘Pirates of the Caribbean’ and at the start all the artists thought they could run 20,000 poly characters with multi pass shaders etc… it didn’t take long to come to the conclusion that really NextGen is just PS2 with more lighting passes!

Anyway, referencing out of the box in Maya for large scale production is very average, the support is there, you just have to be very careful and methodical about it. It’s got better, 6.5 was terrible, 7 was Ok and was really where we decided to go down the referencing route, and on each release they’ve fixed more bugs. The problem is that there is practically no support for managing namespaces (2 mel commands and that’s it), and without proper namespace management you get into all sorts of issues.

Say for example you load in a character Jack (namespace Jack:) into the scene, now you have Jack:Rig, with JackRN (refNode). Now if you open the refEditor and change the namespace, you’ll be left with the refNode called Jack… More importantly if import an animation that’s already referencing the Jack rig, you’ll now have nested namespaces. Things soon get dirty and unmanageable when you have 20 rigs in a cutscene. There were also lots of bugs (some of which still exist) when you have multi references of the same rig in the scene.

Our loader is part of the CharacterManager (assetmanager for rigs and props). To load a rig we just browse to it and click. The namespace that rig comes in with is managed, if Jack already existed it would increment the namespace Jack_02 (it also resolves nested namespaces, renames the RefNodes and makes a single rig visibility layer all to match… very clean. On top of this it shows you a list of references already in the scene, the animator can just select from this list + a replacement rig and do the character switch. The compatibility does a cache of all important nodes on the rig / skeleton prior to reference switching, then does the same after and compares to two states with tolerance to let you know if the reference generated any changes to the state of the animation… a very useful function.

There are also a lot of workarounds you can do to help getting a referenced pipeline stable, especially if you’re working with characterSets and Trax like us.

20

I’m trying to automate stuff as mush as possible - our folders structures are created by a tool, our image resizing is done by a tool, our various lightbakes are done by a tool that sets up the correct channels etc.

Now, assuming that some textures need to be named a certain way, we can make error checkers to find the ones that are not. How can we develop a tool that will rename textures magically - assuming that the artists get the names almost right?