View in #engine_unreal on Slack
@ambrosiussen: Say I have a csv that contains actors I am going to spawn with a BP during runtime… And then during runtime I delete some of those actors. I then reload the level. How would you keep track of what got deleted so I don’t spawn it again?
To be more precise, I have several streaming levels that generate themselves on level load. So what I am trying to do is keep track of what got deleted when I last visited that particular streaming level, so I don’t spawn it the next time I load.
Simplest would be to store an ID (I have a unique string value per actor) in an array for the player blueprint / gamemode. Check against that when creating. Would be super fast in the beginning of the game… Not so much the more you delete
@Martin_Baadsgaard: @ambrosiussen AFAIK this would be a perfect usecase for a map
(as opposed to an array)
Maps are perfect for asking if a value is in the collection without an increasing cost with an increase in the collection size
@zuko: @ambrosiussen You shouldn’t need a unique ID. But I think @Martin_Baadsgaardis probably right and you could just use a reference or soft reference to the object directly and then store it in a format that makes sense for whatever you are doing.
Soft reference is probably going to be more efficient to store but slower to work on if you are needing to modify 1000’s of assets on the same frame. If you can get away from procedurally spawning these actors and instead having them all be in sublevels that align with the different levels you are already loading in that would ultimately be the easiest and most efficient way to do it. This way you go through unreal’s built in level streaming system and if you need the same frame to update visibility or collision or whatever on all objects (Since its an async process) you can call “FlushLevelStreaming” which forces everything to finish that frame.
@ambrosiussen: I’ll have to read up on what soft references are… As for the ID, I already have it from the datatable that the CSV generates. The reason I dont want to pregenerate levels and stream those in is because then I’d have to save all of those. Now I can generate thousands of CSV files and just pick a random one to spawn a level with. (I use Houdini to generate levels)
Thanks @Martin_Baadsgaard, will change to map right away 
@dhruv: Or does Unreal implement a bloom filter?
@Martin_Baadsgaard: "
Unordered map is an associative container that contains key-value pairs with unique keys. Search, insertion, and removal of elements have average constant-time complexity.
Internally, the elements are not sorted in any particular order, but organized into buckets. Which bucket an element is placed into depends entirely on the hash of its key. This allows fast access to individual elements, since once the hash is computed, it refers to the exact bucket the element is placed into.
"
TMap just wraps std::unordered_map
@dhruv: Ah yeah I forgot it did buckets as well in unordered map
Afaik it’s still not a O(1) if you have multiple items in a bucket but definitely much better performance than a flat hashmap or an array
@zuko: interesting
I’d be surprised yea if the cost didn’t go up at all with more items in it.
Hey so I am pretty happy to say I am actually gonna do a talk/stream thing about some of the cool stuff I’ve been doing over the past year. Basically this is how the manipulator tools I have teased in the past … evolved and were used in our actual project O.O. I also hope to release the manipulator tools very shortly as well now that all of our stuff is live so others can try it out on their stuff.
https://twitter.com/ChrisZukoArt/status/1297214532675407872
There’s a vid in the status link showing the rig playing in the engine from a sequencer since I don’t wanna bloat our server with vids.
@ambrosiussen: So I got the streaming and generation based on CSV files working… But now I have another I guess common issue
I stream my level async, which is fine. But then after it is done loading, I toggle its visibility on. That causes stutter for a bit
Is it not possible to change the visibility of a level async? (BP only)
(Currently each streaming level has ~800 actors… Maybe I have to make my zones smaller?)
@Martin_Baadsgaard: Is this maybe in the render thread that gets the hit?
What I am thinking is that when you change the visibility of objects, then optentially that can’t be async because it is tied to object drawing, but of course object drawing can just be delayed to next occuring frame. However, if there is data associated with it that needs to be loaded for the rendering thread, then you get a stutter
@ambrosiussen:
@Martin_Baadsgaard: What tool is that? Never seen that before…
It’s prettier than I’m used to 
I just wanted to do stat UnitGraph or something
@ambrosiussen: StartFPSChart and StopFPSChart yeah
Saves a CSV you can graph
@Martin_Baadsgaard: Ah
Well I guess that means that visibility is more than visibility 
Since the gamethread takes the hit
@ambrosiussen: Might just be the number of actors there then
@Martin_Baadsgaard: But then that GPU spike
I just realised that I do not know the difference between the two types of visibility
And if both are relevant at runtime
One being the eye icon / keyboard shortcut h and the other being the actual property
@ambrosiussen: Well let me clarify, I am creating a level instance, and then toggling its visibility once loaded
StreamingLevel->ShouldBeVisible
@Martin_Baadsgaard: Yea that might do more…
@ambrosiussen: Yeah. Thankfully the docs told me a lot /not
@Martin_Baadsgaard: SetShouldBeVisible() calls World->UpdateStreamingLevelShouldBeConsidered(this);
(just a note to myself there…)
Meh, that looks a bit too complicated for me to answer surely
It just sets a bool that puts the world in a state of “considering streamed levels”
But one would think that coulnd’t stall the system, since otherwise having a separate visibility toggle is kinda useless
@ambrosiussen: Okay the heck
FAndroidDeviceDetectionRunnable seems to stall for 333ms 
@zuko: That likely means that its waiting for something else to finish. In this case its probably the GPU when its handling visibility? Are you doing this stuff in a cooked build by chance?
@ambrosiussen: No, just PIE
@zuko: It should get much faster in a cooked build. But also in my experience usually if you get stalls during level streaming changes or visibility changes its typically because the actors have pretty complex construction scripts that involve initializing components or creating components on the fly. If you are spawning stuff on that same frame too… all of the components for all of those actors being initialized would likely be the slow down.
@Martin_Baadsgaard: So construction script gets called when streaming visibility gets toggled?
@ambrosiussen: They have no CS. Neither do they have a tick. They are just a static mesh and a collider, with a custom event that gets called from Player whenever something happens
@zuko: construction scripts only get called the first time when a level is added thankfully. But you might be able to get more useful info using the newer insights tool.
Begin Play however will get called every time you set a level to visible after being hidden.
The insights tool is what I used to nail down that component initialization / updates was causing the majority of our frame slowdowns during streaming.
@Martin_Baadsgaard: This stuff always confused the heck out of me
When what gets triggered…
@ambrosiussen: That’s a good tip! Are those insight tools supposed to replace the older profiling stuff?
@zuko: Understandable … its something I mix up all the time too but at the end of the day the rule is when loading an level that’s when all of the initialization / construction happens.
Begin Play is kind of an asshole … because of the way the world adds and removes objects when levels change their visibility but are still loaded in the world. They technically remove the object from having an owning world which kinda leaves it in a limbo state and assume if you want an object to return to an owning world “Change its visibility” then … it needs to reinit its begin play. 
Yea @ambrosiussen in 4.24 and 4.25 they keep adding more to it too.
@ambrosiussen: So I think I found the issue. I am using Create Instance to create level instances, load and display them. Perfectly fine, but it seems it takes an extra hit in comparison to just a regular Load Stream Level, even after the first time loading/creating it.
3x longer to be precise
@zuko: Huh that makes sense since an instance would have to be created in a way that allows others of the same level… Since a normal streaming level can only have one of its kind in a world. Good to know!
@ambrosiussen: But why do you pay 3x the cost?
First time you instance and then load i understand… But the times afterwards where it already is instanced and just needs to be loaded? It doesn’t seem to get removed according to Stat levels when you unload it
@Martin_Baadsgaard: I still have a slight suspicion it’s the caching/loading for the render thread that is the issue…
@ambrosiussen: Idk… Would have to debug it more
I got a pretty nice system working now. I export a CSV file from Houdini, I then load it into a little script I made in UE that asks you how big your streaming levels have to be. Once you hit OK, it will generate a persistent level with blueprint triggers to load a level, and then generates sublevels with all their content
Once its done, you can just hit play
Let me make a quick vid
https://www.youtube.com/watch?v=I7WyOjabQ1k
YouTube Video: Progress Day 8 - Motherload [Level Generation]
@zuko: Oh fun
I’d be curious as to where you go from there. My main wonder is if you will ever transfer the idea over to more of a procedural seed generation system vs exporting and working off of a csv from houdini. O.O Neat looking game so far though! Something I’d totally play.
@ambrosiussen: Well the levels don’t have to all be unique
I can for example generate 10 different versions, each cut up into 20 horizontal slices
Afterwards I can then just mix and match those layers to generate new ones
That’s a lot of variations
Besides, producing good looking procedural levels just from BP is HARD. You don’t have all the nice tools Houdini has
@zuko: Thats a good point yea nice