Folder structure and File naming conventions

One of my major roles at my present company is to maintain the file infrastructure of the pipeline. We’ve been having long discussions on what makes the best structure for specific projects and what naming conventions work best for both individual artists and new people being exposed to a project already underway.

There is usually two type of folder structure for a project: Top down and bottom up:
Top down is a methodology using the project name as the root of the folder structure, with applicable categories beneath it.
Bottom up structure is the opposite. Categories of folders broken down by project name.

I’d be interested in having a discussion with everyone over how they handle both their file structure and the conventions of naming files so that it is readable and documentable.

Obviously it varies immensely according to how a studio operates, but the big players usually have a rock solid method.

I tend to prefer a top-level folder named for the project, with sub-folders for the various types of data involved. I find that that makes a lot more sense to me, since if I’m looking for any given file I’m more likely to start by thinking about the project that it belongs to than to thinking (at least at first) about the kind of data.

For projects that involve other people, I’m a huge fan of SVN. At my last company we used it for everything, both code and art, and it saved us a lot of headaches. Highly recommended. If you’re on windows, Tortoise SVN makes it extremely painless to use SVN- check it out if you haven’t already.

Yep, we use TortoiseSVN at work, and I have it at home too, for keeping revision history of my files.
I’m not a big fan of “Save incremental”, I don’t like generating hundreds of files called “male_base_43.max” or whatever, instead I just keep a few files (maybe 01 to 05, each saved at a “significant” stage of the process), and version-control those so I have a roll-back history and backup.
TortoiseSVN 1.5 now supports partial checkouts, too, so it’s nicer for art source handling than it was before. In general it seems more optimised for code revisions than art storage, but it’s a nice alternative especially if you don’t want to shell out large amounts of money for AlienBrain or Perforce.

I also tend to prefer top-down folder structures. Just seems to make more sense to me, keeping track of a whole project and all it’s sub-assets, rather than having asset folders containing different files from different projects in sub-folders.

Small studio (3 man doing 3D), top down all the way!
Even when same textures used as in an older projects, copying them over to the current project. We also have strict naming conventions.
Each project has a code, and that code gets used in all our assets down to the object names in max. we had lot’s of inconsistencies not doing that, or archiving older projects and losing textures etc.

-Johan

This is an interesting topic. We’re still tweaking our structures almost every project. Here’s our current Perforce depot scheme, using “sr2” as an example project.

//gameX (root project folder)
[FONT=Courier New]//gameX/art (source art files not used directly by the game build, like .max, .psd, etc.)
[/FONT]//gameX/main (mainline branch, all game build files are under here.)
//gameX/main/code (all code for mainline)
//gameX/main/data/maps (game textures for mainline)
//gameX/branch1 (example Perforce branch)

[FONT=Verdana]This allows us to keep all files for a given project under one root folder, making it easier to deliver everything to other studios/outsourcers.[/FONT]

[FONT=Verdana]Art source files are kept outside the game build folders, but still inside the root project. We used to keep those in their own “\projectXart” root folder, but it’s generally more convenient to keep them together.[/FONT]

[FONT=Verdana]Branches are done alongside /main, each with their own /data, /code and other subdirs as needed.

[/FONT]

Yeah we follow pretty much the exact same as you guys at Volition except with the art being in a separate directory still (I guess cause the programmers don’t really want to sync sub directories or have art source files on their disk). We also use perforce and I’ve never had any troubles with it yet.

Great idea on the home source control MoP, I’ll definitely have to give that a spin once my new computer is built. For some reason I always put source control on a greater ‘multiple persons project’ scale, but this makes sense for sure.

[QUOTE=MoP;251]I’m not a big fan of “Save incremental”, [/QUOTE]

Yup i agree so much with that.
I HATE incremental save.
You get a TON of old files, no comments to figure out what file was what etc.
Just a big, unorganized mess.

So I also use SVN.

(I love SVN, especially after having used perforce which is so over complicated)

I made a small shortcut in 3dsMax (ctrl-s) that automatically checks-in my .max file everytime I save the max file.
That way I never have to use incremental save since SVN will have a history of all my max files. AND since SVN allows you to store a description with your check-in, it is ideal for finding stuff back.

I can so recommend that workflow for anybody who hates incremental saves as much as I do. Something so simple made my 3dsMax experience 20% less stressful (because we all know you have to save your max file 100x a day because max will crash)

One thing I got a little irked on when I came onto the MMO is, our folder structure was quite random, often misnamed, misplaced, etc (I guess this was due to being the Austin studio’s first project). There was really no strictness in enforcement. While I would have been a fan of kicking some butts and reorganizing things, the decision was ultimately to not enforce a strict structure. But in order for th exporters to find the right path, we needed to have predictable paths. The solution we had was to store a property in the User Property buffer, which was the unique character name. We then have a ‘path function’ that takes that character name, and performs a “case of” check on it. It returns an array of all the character’s needed paths and any additional info (such as the skeleton it uses), and we can choose the data that is needed. An added benefit is, that by having ALL project paths in a single function, it is easy to change them on the existing project, or to carry over the tools on future projects.

We use SVN mainly for code (not fully implemented yet), but since we have a small graphics team, I wrote a versioning script that keeps track of files and backs them up accordingly. It doesn’t handle deltas, but it auto-documents the content of the .max files to give a better idea of what is what.

Right now i’m working on one that does the same for After Effects, as we do most of our compositing work through that, and keeping track of renders and footage assets can be a real pain.

At Focus 360, we’ve three top down folder structures per project mapped to different shares. ( A hold over from when they were on three different servers, but the separation logically works… umm somewhat… ) We handle approximately 200 projects a year and a five digit project number is the first 5 characters in the filepath. Most artists talk about projects by #, not by client or project name. It allows quick access by typing the 5 digits on the num pad to get right to any project… The first two digits are the year the project was signed, the remaining three incremental for that year… ex. 08010 would be the tenth project signed in 2008.

The first share is the “Project” share where all the actual 3d and 2d work is saved. This folder structure includes the project number, client name and project name in the base folder. We are now using a 10TB isilon clustered server with an every two hours snapshot feature enabled on this share, which gives the artists plenty of versions to go back to, although not documented in anyway…

The second share is the “Elements” share which is composed of all the layers of an animation. This share has just the project number in the base folder name and is really only navigated to if you have an issue with a render. Our Shot Management Software, “Scene Pimp,” ( written by your’s truly ) manages all the files on this share. If you decide to remove a shot from the storyboard, S.P. will find all related rendered information across the shares and eliminate it for you.

The third share is “Frames” in which the base folder is once again just project numbers and the next level divides out first by “Scene Designation” and then “Final frames” and “Dailies.” Dailies are “lower” quality quicktime’s of the sequence’s for reviewing motion and timing. The Final folder holds the composited images.

Each shot name consists of a “Scene Designation” and a “Shot Designation” and folders out as such…So the working 3D folder and outputs for the first shot of a site animation look something like this below… All folders and file pathing are managed by Scene Pimp. Render completion emails notify of each render sequence and quicktime daily and arrive with a direct link so you can review them without navigating…

We have a simple utility to only access and show all the folders within a specific project after you designate which project you are working on…

\Projects\08002-La Playa-Vaquero\Production\3D
\Elements\08002\SITE\01
\Frames\08002\SITE\finals\01

Keith Morrison


full melt bubble hash

Our structure for assets is a little more spread out:
//ProjectName/
//ProjectName/000_Docs/
//ProjectName/010_Assets/
//ProjectName/010_Assets/010_Artwork/
//ProjectName/010_Assets/020_Animatics/
//ProjectName/010_Assets/030_Models/
//ProjectName/010_Assets/040_Ref/
//ProjectName/010_Assets/050_Comps/
//ProjectName/010_Assets/060_Audio/
//ProjectName/010_Assets/070_DVDs/
//ProjectName/020_Code/
//ProjectName/030_Promos/
//ProjectName/040_CD_Rel/

I’ve written a script that sets up each project, which is fairly straightforward, as well as various tools and shortcuts that immediately take you where you need to go, rather than having to dive through tonnes of folders.

I’m not exactly happy with it though. It’s a little too complex.

There’s not much on the Linux/VFX end of the spectrum, so I figured I’d lay down what we have at most of the places I’ve worked.

The general layout is something like:

/root/project/sequence/shot/dept/element/software/

This gets a little different everywhere I go, with the main differences being in the dept-element-software area. You want to keep your pathname as small as possible, but the bigger the show you get, the more you need those specifics. The idea is that you’re able to take any file and link it to specific database entries (for project, shot, etc) via some python/perl parser.

The trick is to lay it out so that it makes sense from the parser’s perspective, and then do all the sorting in an asset manager – as opposed to sorting at the file level. Smaller shops tend to have things like client output and publishes at a higher level (say, below sequence), which makes it damnably difficult to determine some of the more granular data that happens down the line.

Of course, for all this to work, you need an excellent asset manager, which can display all this data 17 ways from Sunday. And most places have bupkis. Which is where I come in.

More later.

I once wrote an article regarding files and path settings:

http://www.cgarena.com/freestuff/tutorials/max/projectSetup/

it’s ment to be for a smaller scale project like a music video or something…but worth a view maybe :wink:

with kind regards,
Anselm

Our folder system is pretty standard, but I do like to use a naming scheme that ensures proper sorting. We tend to end up with fewer folders that contain many files, instead of many folders which contain less files.

For example, in the case of a character, we put the raw model Maya ascii file, the texture maps and the master rig file all in one folder, and have just one subfolder for all the animation files.

//project/content/raw/characters/badguy/
//project/content/raw/characters/badguy/[COLOR=“Yellow”]badguy.ma
//project/content/raw/characters/badguy/badguyRig.ma
//project/content/raw/characters/badguy/badguy.tga
//project/content/raw/characters/badguy/anims/
//project/content/raw/characters/badguy/anims/badguy_shoot.ma
//project/content/raw/characters/badguy/anims/badguy_die.ma[/COLOR]

the exported files goto an intermediate location.

//project/intermediate/characters/

and finally the cooked files go in another location

//project/data/characters/

programmers who don’t want all the raw art can simply remove the //project/content/ folder from the clientspec in perforce. likewise, artists can remove the //project/code/ folder from their clientspec so we only get compiled code.

Environments get a little tricky when you have lots of textures and many variations on them.

//projects/content/raw/environments/
//projects/content/raw/environments/dungeon/
//projects/content/raw/environments/dungeon/[COLOR=“Yellow”]dungeon.ma
//projects/content/raw/environments/dungeon/textures
//projects/content/raw/environments/dungeon/textures/DUN_wallA1A.tga
//projects/content/raw/environments/dungeon/textures/DUN_wallA2A.tga
//projects/content/raw/environments/dungeon/textures/DUN_wallA3A.tga
//projects/content/raw/environments/dungeon/textures/DUN_wallA3B.tga
//projects/content/raw/environments/dungeon/textures/DUN_wallB1A.tga
//projects/content/raw/environments/dungeon/textures/DUN_wallB2A.tga
//projects/content/raw/environments/dungeon/textures/DUN_wallB3A.tga
//projects/content/raw/environments/dungeon/textures/source
//projects/content/raw/environments/dungeon/textures/source/DUN_wallA1A.psd
//projects/content/raw/environments/dungeon/textures/source/DUN_wallA2A.psd
//projects/content/raw/environments/dungeon/textures/source/DUN_wallA3A.psd
//projects/content/raw/environments/dungeon/textures/source/DUN_wallA3B.psd
//projects/content/raw/environments/dungeon/textures/source/DUN_wallB1A.psd
//projects/content/raw/environments/dungeon/textures/source/DUN_wallB2A.psd
//projects/content/raw/environments/dungeon/textures/source/DUN_wallB3A.psd
//projects/content/raw/environments/dungeon/props/
//projects/content/raw/environments/dungeon/props/DUN_torch.ma
//projects/content/raw/environments/dungeon/props/DUN_chain.ma
//projects/content/raw/environments/dungeon/props/textures/
//projects/content/raw/environments/dungeon/props/textures/DUN_torch.tga
//projects/content/raw/environments/dungeon/props/textures/DUN_chain.tga
//projects/content/raw/environments/dungeon/props/textures/source/
//projects/content/raw/environments/dungeon/props/textures/source/DUN_torch.psd
//projects/content/raw/environments/dungeon/props/textures/source/DUN_chain.psd[/COLOR]

The weird A1A suffix on all the texture files is there to make sure that variations off the same base texture all sort together when sorted by name.
For example:
DUN_WallA1A.tga is a typical 512x512 brick wall texture that tiles.
DUN_WallA1B.tga would be a 512x256 version of the same texture.
DUN_WallA2A.tga is the same 512x512 brick wall but with some cracks.
DUN_WallA2B.tga is the brick wall with cracks and also 512x256.
DUN_WallB1A.tga would be a 512x512 stone wall that tiles, and so on.

It’s a little crazy to get used to - but it reduces long, overly descriptive file names like, DUN_brickWall_512x256_02.tga or DUN_wallStone_window_01.tga etc.

Plus you can define what each character means, usually i stick with a base designation A-Z, a variation numeral 1-0, and a dimension A-Z, but you could easily use something like DUN_Wall_01AC.tga.

This might be getting a little off topic, but like Rob said, without strict naming conventions on files and folders, tools are harder to write and more prone to breaking or organizing data in the wrong places. When you’re working on a daily basis with the amount of files that modern games need, it because very important to know that your tools are handling your data properly.

On SR, we thought it would be better to create a tighter more strict naming convention in the beginning of the project and deal with the growing pains of it, then spend the entire project cleaning up messes or updating tools to adapt to a looser naming convention. I think people will eventually get used to it and appreciate it more as time goes on. It’s very frustrating trying to find a file\folder\data\object that is poorly organized.

I would say that the stricter the naming convention, the better your pipelines and systems work. But this is up for debate depending on your experiences.

[QUOTE=JasonChildress;594]

I would say that the stricter the naming convention, the better your pipelines and systems work. But this is up for debate depending on your experiences.[/QUOTE]

I agree, and usually the only thing stopping it from happening smoothly is legacy projects and artists who think about as structurally as string caught in a net of brambles.:):

And it is easy to get it wrong. We recently overhauled our entire folder structure because even though it was well documented, no one could find a damn thing!

[QUOTE=Rob Galanakis;350]One thing I got a little irked on when I came onto the MMO is, our folder structure was quite random, often misnamed, misplaced, etc (I guess this was due to being the Austin studio’s first project). There was really no strictness in enforcement. While I would have been a fan of kicking some butts and reorganizing things, the decision was ultimately to not enforce a strict structure. But in order for th exporters to find the right path, we needed to have predictable paths. The solution we had was to store a property in the User Property buffer, which was the unique character name. We then have a ‘path function’ that takes that character name, and performs a “case of” check on it. It returns an array of all the character’s needed paths and any additional info (such as the skeleton it uses), and we can choose the data that is needed. An added benefit is, that by having ALL project paths in a single function, it is easy to change them on the existing project, or to carry over the tools on future projects.[/QUOTE]

This is a problem I’m finding as well but I’m not sure I understand your solution. I’m kinda of an uber-noob:?: at this but I try to catch on as quick as I can. Mind expanding on your solution a little more? I would be really interested and appreciate it.

-Denis K.

Over time I’ve actually become fairly opposed to naming conventions for engine-specific purposes. At least as a general rule.

When designing a pipeline, I’d much rather leave the name of something free and editable by the content creator. “Name” being a filename, folder name, object or material name in a scene, etc.

Keeping content organized on the content front is hard enough without chewing up characters for pipeline purposes… I like to give the artists all the freedom I can there. I’d extend that to things like named selection sets or layers in 3ds Max… why let an exporter hijack those useful content tools?

By keeping the pipeline elements out of those areas, it forces me to put the “connective tissue” for the pipeline in another (better) place. If one object in your scene depends on another, it’s far better to make that connection via a GUID, node handle, or something else that’s much less likely to change than a string name any artist can modify, even accidentally. Names are fragile as a way to convey properties. With readily-available databases and scripted property interfaces there’s plenty of spots to stick your gory details these days.

While we’ve yet to fully free ourselves of this at Volition, I’m also disenchanted with use of certain “special” folders (or folder names) as a means of defining things in the pipelines. For example, a data building tool hard-coded to only search under “<project>\data extures” for bitmaps, etc. Like object/filenames, I think it’s ideal to let folder structures be a tool for content organization, rather than rigid conduits for the pipelines.

Within reason, of course… you still need certain content people in charge of managing that content organization/structure, or you end up with Lord of the Flies.

There’s some other exceptions, of course. File extensions (.png, .dae, etc.) are still a logical way of defining file types, mainly because Mr. Windows still says so.

[QUOTE=Adam Pletcher;704]Over time I’ve actually become fairly opposed to naming conventions for engine-specific purposes. At least as a general rule.

When designing a pipeline, I’d much rather leave the name of something free and editable by the content creator. “Name” being a filename, folder name, object or material name in a scene, etc.

Keeping content organized on the content front is hard enough without chewing up characters for pipeline purposes… I like to give the artists all the freedom I can there. I’d extend that to things like named selection sets or layers in 3ds Max… why let an exporter hijack those useful content tools?

By keeping the pipeline elements out of those areas, it forces me to put the “connective tissue” for the pipeline in another (better) place. If one object in your scene depends on another, it’s far better to make that connection via a GUID, node handle, or something else that’s much less likely to change than a string name any artist can modify, even accidentally. Names are fragile as a way to convey properties. With readily-available databases and scripted property interfaces there’s plenty of spots to stick your gory details these days.

While we’ve yet to fully free ourselves of this at Volition, I’m also disenchanted with use of certain “special” folders (or folder names) as a means of defining things in the pipelines. For example, a data building tool hard-coded to only search under “<project>\data extures” for bitmaps, etc. Like object/filenames, I think it’s ideal to let folder structures be a tool for content organization, rather than rigid conduits for the pipelines.

Within reason, of course… you still need certain content people in charge of managing that content organization/structure, or you end up with Lord of the Flies.

There’s some other exceptions, of course. File extensions (.png, .dae, etc.) are still a logical way of defining file types, mainly because Mr. Windows still says so.[/QUOTE]

I really like this method of thinking. It’s sort of like a tagging system with a little more structure.

I’m interested to see how pipelines develop once Gridiron Flow is released, as it takes on a similar philosophy of keeping track of files.

[QUOTE=dekorkh;701]This is a problem I’m finding as well but I’m not sure I understand your solution. I’m kinda of an uber-noob:?: at this but I try to catch on as quick as I can. Mind expanding on your solution a little more? I would be really interested and appreciate it.

-Denis K.[/QUOTE]

I’m only on the character/animation side of things, so I cannot say for how static assets are handled, but there’s no reason it can’t be expanded.

Each asset is basically tagged with some sort of unique identifier, and this is held in the scene- we hold the attribute in the User Property Buffer, but it can also be a Custom Attribute, app data, whatever, just some way to keep the data between Max sessions and attach it to the file (maybe the first numbers of the asset name, even).

When we need to do anything with the asset- whether exporting, auto-finding or auto-loading files, etc., we pass the unique ID into a “GetDataPaths” function.

This GetDataPaths function has a case of that contains all the possible unique ID’s and their relevant paths (an array or struct). We can then return the entire array or struct, or a single member (say, we only need the directory where animation exports go). We could change the entire folder structure, and only this single function would need to change, everything is centralized into only one hard-coded spot (and even the paths don’t need to be hard-coded if there is some consistency, be smart).

So, for example, say we have a character name of “Mario”, this is what all the code would look like:

The GetDataPaths function is something like:


fn GetDataPaths charName retMember:undefined =
(
	struct dataPaths (animExportDir, meshExportDir)
	charName = charName as name --case insenstive
	
	projDir = "C:\\proj\\"
	local retStruct = case charName of
	(
		#mario: dataPaths animExportDir:(projDir + "mario\\animexport\\") meshExportDir:(projDir + "mario\\meshExport\\")
		#luigi: dataPaths animExportDir:(projDir + "luigi\\animexport\\") meshExportDir:(projDir + "luigi\\meshExport\\")
	)
	
	if retMember == undefined then
		retStruct
	else
		getProperty retStruct retMember
)

And to actually use it:


--we need to find the anim export directory
local theCharName = getUserProp theNode "charName"	--assume our user prop is named "charName"
local theExportFolder = GetDataPaths theCharName retMember:#animExportDir
--if retMember is undefined or we don't supply it, function will return the entire struct, which is useful if we are going to use multiple data paths

exportFolder equals “C:\proj\mario\animexport\”

Hope that makes things more clear. Actually I have a confession to make, ours doesn’t work as nice (we use array instead of struct), but this would be much better :slight_smile: