[WIP] XNA based Engine + WPF based Editor

Coding
1

Hello fellow people =]

I’d like to show what I’ve been working on for the last couple of months. I call it Irradiance =]
You can find two screenies at the bottom of this post!

Why am I doing this?
I’ve always wanted to program a game engine way back since I was a kiddo modding games, and it feels like I’m finally on the right track.
The other reason is this is going to be a portfolio project (big one) and I hope that I’ll get to finish it before my
education comes to an end, cause by that time I need to be out on internship/placement. The goal here is to show
that I have some understanding regarding subjects about game engines as a Tech artist, but also that I can code tools outside of MEL scripting / Maya API.

How far have I got?
The screenshots might not show much but there is alot of stuff ready behind the scenes. Asset tracking, project creation and loading,
3D picking and selecting objects, deleting, parenting with transforms affecting children correctly, layers. Shaders are compiled and
stored as byte array inside a generated c# file duing the process. This file contains all the code needed to run the shader with the engine.
It implements IRadShader interface that contains stuff such as SetEngineParameters(world, view, proj) and SetLightParameters(lightArray)
and the logic inside those functions are done by scanning the .fx file and its available semantics and variable names during the
code generation, then the logic is generated aswell to match (such as properties setting shader constants).
I also have a modelmanager, shadermanager, scenegraph and hierarchy, lightmanager, texturemanager. The engine is component based similar to Unity3D.
I’ve optimized 90% of what I have written so far together with the lovely NProf profiling tool.

The Editor
The editor is coded with the lovely WPF framework from microsoft (Windows Presentation Foundation). This is basically like Silverlight
and it replaces Windowsforms. I’m using the M-V-VM Design approach to code the editor ( decoupling the UI and the
Logic to maintain the code easily and do unit testing). All the data is handled through Databinding in WPF.
The UI style is from Microsoft Expression Blend 3 Theme. Assigning stuff is handled by Reflection from .NET (such as components).
Building the game will be done with the MSBuild class that generates a Visual studio project in memory and compiles. My goal is
to make it as easy to use as possible. I’m not trying to compete with Sunburn or TorqueX, this is just for both learning purposes and
portfolio =)

The rendering pipeline
Note: OPTIONAL means can be set on or off!
Update active objects
Camera frustum culls octree nodes
Camera frustum culls objects that are left but outside screen (OPTIONAL)
Engine fills ZBuffer early on with Front to Back sorting (OPTIONAL)
Drawing with 3 possible settings(Did Occlusion Query culling but didn’t gain much performance so switched it off for now)
[ol]
[li]Batches renderstates and material states and loops inside the effect pass and does drawcalls
[/li][li]OR Uses the order left by the ZBuffer sorting
[/li][li]OR Uses the order from the scenegraph
[/li][/ol]
Handles Transparent objects back to front (not 100% finished yet)

I’m also doing 8 lights per pass in a for loop in the shader, I find this much better than one light per pass since I hate drawcalls and so does my GPU.

The future!
I’m working on integrating Hardware instancing with Shader Model 3 (extra vertex buffer holding the matrices) and
I’m also working on the property grid of the editor. Most of the logic for the propertygrid is done and stuff such as
scripts and materials will be assigned through the Reflection namespace in the .NET framework by instantiating and
adding it as an component. The scene class is almost done and I use the XNA binary intermediate serializer to save and load it,
and it handles the rest for creating and assigning what was saved last time.

Suggestions
I’m not studying Computer Graphics, everything I’ve learned is either from books/articles or from forums where people help out.
I started out my journey as a modeller when I was very young and I’m studying 3D art right now.
This means that what I’ve done might not be the best way / optimal way of doing stuff, even though I keep reading everyday.
So I’ll gladly hear out suggestions concerning anything related to this. Can be anything from what to concider on
the rendering pipeline to the layout of the editor. I’m also really curious if the game industry is using WPF, and if they are not, why may I ask?
I find it much better/easier and faster to use than anything from MFC to Winforms.

Screenshots

In Editor:

Built:

2

Wow - really nice! Great job!

3

Nice work, All the best.

4

Looking good.

5

Very ambitious project, but it looks like your off to a great start.

Good idea working on this during your school years. I find it pretty difficult sometimes to get time to work on personal projects.

Please keep us updated.

6

Keep up the good Work! thought i’d get my hands dirty with a game engine myself soon, really wanna play around with voxels

7

This is super cool, the only thing I’d suggest is, as great as an engine is, it is nothing until something’s been made with it. Everyone and their mother has written a game engine, very few have ever released any sort of finished product or demo, though.

8

Thanks for your kind words everyone :slight_smile:

I haven’t had time to work on this project for a while, but I do plan to continue with it soon. Reason I couldn’t work on it is because alot of school assignments / game projects / courses.
Other than that I choosed to specialize in tech art. So whatever freetime I get is either for coding. So this one will get some love very soon!

I’m at the stage where I can either write a collision detection library or continue with the material editor (right now it’s standalone and not integarted because I did it with Winform earlier on before I switched to WPF).

On the otherhand the new XNA 4.0 for windows phone sounds tempting and it would be cool to make something for it… Time will tell :slight_smile:

And yes Rob I would love to actually get something done with this engine and ship it to XBLA / Steam etc. Probably would bring some tears even since it would kinda be the first time a tool I’ve written is used throughout an entire project :slight_smile:

Anyhow I’ll try to update this thread when I get to a point where I can show more!

Take care all!

9

I feel like I should say, guns lots of guns…

But other than that its beginning to looks sweet.

good job

10

Update =}

Here is a 1080p video: http://www.youtube.com/watch?v=AIYTXATl9sQ

Propertygrid was fully coded and integrated some time ago. Now I’m focusing only on functionality and usability of the editor. Focus on graphics will come last once I get the rest working :slight_smile:

Next todo is assigning materials on objects in the scene by dragging and dropping onto a mesh similar to dragging and dropping models into the scene… (spawning a ray, searching for closest hit, once found get object and assign the dragged material to its material property )…

Hope you like it ^^

11

Material assignment through drag and drop functionality done :slight_smile:

Video: http://www.youtube.com/watch?v=BMLw4hEgIwI

I also redid the renderloop abit so it sorts models material wise and does no shader state switching or vertex, index buffer state switching unless needed.

This means every material is associated with a LIST of models that will use that specific material. It sorts that LIST model wise. The LIST is built during the frustum culling process, if the model passes the frustum test it gets added to a MultiDictionary which is basically a Dictionary with string as the key and List<Renderer> as a value. The key in this case is the material name and the List<Renderer> contains all the model renderer objects, which as I said before gets sorted before the camera renders it ^^

Before rendering starts it sets the vertex and the index buffer to the first object in that list and renders with the material. If it notices that current mesh has changed (becaue another object uses same material) it calls for a state switch on the graphics device. Otherwise it keeps rendering inside the effect pass without touching the graphicsdevice states.
Once the entire list has been rendered that was associated with the material, it goes on to the next material and calls for shader state switches and begins an new effect and pass and renders the rest of the models etc.

12

Support for the light bulbs have been added :slight_smile:

The shader now supports x amount of point lights where the “x” can be no greater than 8.

The lightmanager automatically handles any newly added “PointLight” objects derived from ILight. Objects are able to fetch the 8 or less closests lights based on an approximity radius (which considers the octree nodes first to eliminate brute force searching, though only useful if we have alot of lights) and use them in the shading pass. The shader is a ReliefShader that is applied to 1000 objects with a forloop doing the lighting in worldspace. If I did one light per pass I would have 8000 drawcalls, now I have 1000 drawcalls but a lil more expensive shader.

When I make simplified and optimized shaders this will run very fast. Relief shading is quite slow and yet I manage to almost keep 60 fps in the editor while recording, gfx card is 8800gt OC :slight_smile:

Video with 8 lights, sorry for bad quicktime quality / color funkyness: http://www.youtube.com/watch?v=N5g6d4AB2ig

13

Dam dam dam had time to work on the engine! Automatic reloading of assets implemented!

Video: http://www.youtube.com/watch?v=k63umbGLAUQ

The way I do this is every UIAssetViewModel object contains a DateTime XnbLastWriteDate. Xnb is the XNA binary format that built items are. That variable stores the date modified the xnb had when it was LOADED.

Now the editor automatically finds out what assets are old and what are new by checking the XnbLastWriteDate and comparing it with the source modified date. If source is newer it rebuilds the source at runtime into a new XNB. Since the new XNBs modified date doesn’t match the loadeds now, it means the XNB has been changed by someone replacing the old or rebuilt by the user / editor. So it reloads it.

I remove the asset from its manager. Delete all the unmanaged resources that was allocated and then readd it to the manager and call a refresh on the shader that uses this texture. The shaders now automatically refetch its textures from the TextureManager then which now contains the new texture! Leaving the old nowhere to be found, so it awaits its doom by the garbage collector :smiley:

This also works on models which is even simplier. I just raise a ReloadNeeded on the old model. Before rendering if ReloadNeeded has been set to True it just replaces itself with Engine.ModelManager.GetModel(this.modelName); and the old model is also nowhere to be found awaiting its doom by the garbage collector ^^
(unmanaged resources has already been taken care of such as the VertexBuffer, IndexBuffer and VertexDeclaration at this point, or else those would have been leaked. I take care of them at the same stage when I remove the model from the ModelManager Dictionary)

14

That looks really cool. I guess there is no undo for this because the file is overwritten en the previous resource will be eaten by the garbage collector?

And how about a huge amount of asset importing? Is this stable for the garbage collector?
(I’m srry if I ask stupid things :D, I don’t have much experience with engine programming.)

15

Hey :slight_smile:

Okay here is how the garbage collector works with managed and unmanaged resources. You need to know this to understand what’s happening :slight_smile:

In XNA the Model class is a managed type, basically a C# class just like any other.
What it contains however such as the Vertex Buffers, Index Buffers and Vertex Declaration is not managed. I think they are wrapped DX9 stuff.

If you have used XNA you know that to load a model you call the Load<Model>(…) method of the ContentManager. That ContentManager stores a reference in a dictionary with string, object pairs of that Model.

The reason the XNA team decided to do so is because if you would for instance not use the Model anymore and it would be unreachable from anywhere in the application, the garbage collector would think that it needs to be collected! But remember that unmanaged objects such as the buffers I listed before are not managed by the garbage collector. So if the garbage collector would collect the Model, the Unmanaged types would be unreachable and the resources they use would cause a Memory leak.

We have to dispose Unmanaged objects ourselves! Luckily the XNA team decided to keep a reference of the Model in the ContentManagers dictionary so even if you don’t use that Model anymore it’s not gonna get collected. If you ever need to unload and remove it to free up memory, you would call ContentManager.Unload() which will call Dispose() on all the loaded assets freeing the Unmanaged resources and then remove itself from the Dictionary the ContentManager implements so that the GC can finally collect it.

But it’s very limiting because calling Unload removes every single object you have loaded. So I wrote my own ContentManager which allows me to Unload Specific objects and Dispose their Unmanaged resources at the same time. So when I import something and it replaces an object, what it really does is it unloads the object from the ModelManager, it unloads the object from the ContentManager and then it disposes the unmanaged resources to free up the memory.

Enough about the garbage collector and unmanaged resources :stuck_out_tongue:

My point is, the Managed class Model doesn’t take much memory at all. The garbage collector sees it as a few bytes. What takes memory is the unmanaged stuff such as vertex data.

So in my engine, if you would import huge amount of assets like 200+, what the engine would do is import them directly if there is nothing to replace so it allocates the memory it needs (both for managed and unmanaged).

But if there is something to replace it will Dispose the unmanaged resources freeing up alot of memory leaving only the Managed object with a few bytes. Then it will replace itself with that Managed Model object in my ModelManagers dictionary making that Managed Model nowhere to be found and sooner or later be garbage collected by the collector returning me the few bytes it took, and few bytes is not heavy on the garbage collector :slight_smile:

Remember garbage collector doesn’t take care of the unmanaged resources, we free them ourselves by calling Dispose() that IDisposable interface implements.

And no, no undoing :smiley: Thats supposed to be done in the source file ^^

Edit: I declare this thread my blog xD

16

Thank you for the explanation. :slight_smile:

17

Np yw :stuck_out_tongue: Hope it was clear!

18

Very cool trick. Did you get the idea for the implementation from anywhere or was it your own? IE, it may make a cool downloadable component if you can abstract it away from other dependencies.

19

Hey Rob!

Anything to do with the refreshment of the assets are of my own ideas :slight_smile:

Though I’m not sure which part you ment me to abstract away, did you mean the CustomContentManager class of my own that allows removing and releasing specific objects instead of removing and releasing EVERYTHING that is loaded such as the default ContentManager does? If that’s what you ment then yes I could do that, because that’s the class I use to load and unload stuff :), it’s basically an extended ContentManager of the built in version.

If you ment something else such as runtime content building and scanning of xnbs/source files to decide wether an import is needed or not, that’s gonna be abit harder, as it’s both UI related and Engine related and WPF / M-V-VM related.

20

GPU particles implemented using the XNA sample from the education section.

Uses parametric algorithms in a vertex shader to calculate the position of every particle, where the parameters are stuff like TIME to solve the equation. PointSprite rendering so 1 vertex do describe it!

Using a DynamicVertexBuffer object because of how often the values are edited.

GPU particles removes the overhead on the CPU allowing tens of thousands particles to run entirely on the GPU. Though collision is not possible with this kind of system. Collisions could be added to the emitter object instead, and each time it collides you could spawn another emitter to give the illusion of particle bounces or something.

Rougly implemented the entire thing in 1 interface, 4 classes, 1 struct. The struct is the ParticleVertex:

ParticleVertex : Describes the vertex with pos, vel, col and time.
ParticleRenderer : Handles rendering by recieving a camera
ParticleEmitter : Handles adding of particles to the Particle System it holds
ParticleSystem : Low level stuff such as setting graphics states and updating of vertex buffers, implements IParticleSystem interface
ParticleSystemSettings : XNA Serializeable data

I had serious design issues with this one, couldn’t decide where each part should go etc. So instead of sitting and thinking for another day (had already thought on this before) I went and implemented what I had in mind, rougly, now I know what I want to change and what each class responsability should be, so it’s time for refactoring.

Also implemented Multithreaded loading through the Thread class. Loading every model and texture in a separate thread. Will extend it later so it will automatically allocate x number of threads where x is not greater than CPU cores.

Other than that I fired up CLR profiler (a MUST if you are a .NET developer)
and started profiling the memory allocations which I hadn’t done for a VERY long time. I noticed that a property that I had was constructing a new BoundingFrustum. The funny thing is that property was being queried everytime I was frustum culling an object, and I had 1000 objects. So everyframe I constructed 1000 BoundingFrustums xD. So the memory allocation was 1300mb after 30 seconds of running the application (though it’s also getting garbage collected so we never run out of mem).
Noticing that I went ahead and fixed it immidiately lol. No more allocations during the game loop :slight_smile: and the application is down to 30 mb of allocation.

The garbage collector is very happy and it gave me several FPS (12+ atleast!)
Get CLR profiler if you don’t have it already because the bottlenecks are always on places where you don’t think about.

Screenshot: