[MOD] Volumetric fog optmizations

[remisc]

This my attempt to improve performance impact that volumetric fog has in the game without getting rid of it.
Fog removal is easy and improves performance immensely but it also takes away from the
look of the sectors in a major way. I tried to alter the fog parameters so that the visible changes are minimized
or at least the look-and-feel of the fog stays as it was.

This is a result of a short investigation starting here: viewtopic.php?f=180&t=414524&start=720#p5197757
I basically found that volumetric fog is often taking more time rendering all the rest of the frame in some situations.
This is the case even when "volumetric fog" setting in graphics options is set to "off". (fog is never completely disabled)

The mod can be found either on github or packaged
Current version: v101

Compatibility:
6.10 - Confirmed
6.10 HF1 - Confirmed
6.20 - Confirmed
Expected to have all 4 DLCs installed

Example on fog in Heretic's End (vol fog set to "off"):
before: 38 fps
after: 87 fps
(Granted this is the sector that I spent most time trying different things)

There is one more thing that can spawn fog that I did not manage to adjust: nebulas tied to the resource gases.
Those are generated mainly using fogpattern_v2_macro and fogpattern_v3_macro.
In DLC sectors where there are 3 gases at once the v2 version can have a big impact (depending on the boundaries and falloff in a given sector).
v3 is used pretty much only in Turquoise Sea and is very pretty but also has very significant perfomrance cost.
(See the notes in github readme)


Note: Looking at the region definitions those parts seem to be worked on by the devs, so I can't guarantee how compatible this will be in the future,
Report if you start seeing diff errors in the logs.

[Realspace]

I use fogs largely in my xrsge mod for big sceneries too so I had to experiment with the same parameters to make them not so much fps eating.
I found out that the shape of fog also affects, spheres tend to be more dense than disk and at the same density they eats more fps. Changing sphere into disk allowed me to make the fogs lighter.
For the same form, density is of course paramount but it is not alone. Using larger sizes AND playing with seeds and min/max noise values, allows to have quite the same effect with less density, so less impact on fps. Game using larger blob of gas and closer, instead of smaller and more and spread out, they look good anyway.
Sphere can have max 4800 blob as size, while disks seem to be still usable with higher sizes of the blobs. This also makes the latter a better choice.
This is with vol. fog off in settings, that single feature literally kills my old RX580

Let's discuss the other parameters, did you see any change by altering the multiplier? How does it change the effect?

Gas res fogs (nebula) seem to be less fps-eating, they are not volumetric (old X3 way of showing), so I tend to use them to show where the resource is while vol. fog is larger and covers entire sectors. They can be set having low local density so will be less intrusive and not much fps-eating, still I did not figure out if doing so also changes the density of their resource not only the look.

[remisc]

I found out that the shape of fog also affects, spheres tend to be more dense than disk and at the same density they eats more fps. Changing sphere into disk allowed me to make the fogs lighter.

Yes, that is correct, the shape controlled with "<boundary>" and "<falloff>" does indeed have big impact, but ultimately I chose not to alter them for a few reasons:

• shape seems to also affect other things - like asteroid field sizes.
  I see that in some <volumetricfog> cases there are additional modifiers for offsets - so maybe it's possible to alter only the fog without affecting other features.
• As mentioned asteroid field sizes may be altered, but it may also affect gas regions - Hence altering where resources can be mined. (not sure on this one)
• I wanted to go through all the region definitions, and that meant that experimenting with all the parameters would simply be unfeasible time-wise. Testing in flight-school with very short load times helped but still it would be extremely tedious to compare, If only there was a way to alter it live in game, it would be easier.
• some shapes are clear creative choice from the level designer - so it I risk altering the character of a sector

Let's discuss the other parameters, did you see any change by altering the multiplier? How does it change the effect?

As far as I can tell it does not alter performance in any way. It just makes the fog brighter. I use it as a counter-weight to reducing density. It sort of feels like when density lowers the amount of shaded voxels the multiplier can make the remaining ones brighter - and the thinned-out fog starts to look a bit thicker.

Gas res fogs (nebula) seem to be less fps-eating,

Yes, but as far as I can tell that's dependent on the shape ( "<boundary>" and "<falloff>" ) and their amount. If a sector has 3 such nebulas combined with large shape it can have similar performance impact as volumetric fog. Also: I am not sure how resources are affected by those nebulas. Changes to shape may change the places where gas/ore are in the sector.
Not 100% sure on that one. I see some weirdness on that front but I didn't do much testing on it. I say this because:

• changing the shape changes the performance impact those nebulas have
• when the nebula of for example hydrogen is not present while the hydrogen <resource> is present I get an error message that corresponding hydrogen field was not found

Those 2 things combined point to shape altering the resources in the sector. Which Is undesirable from my point of view.

Although I have to admit that the gas nebula testing is very incomplete on my part - the flight-school region testing seems insufficient. Some nebulas don't show when I load them in, while the same nebulas when visited in their correct sectors do appear. I understand a given sector is comprised of potentially many <region> nodes connected together.
Correct, complete sector cloning into flight-school for eludes me. I can gather and clone all <regions> but when I tried that I feel I still am missing some things. Especially on the resources side.

[Realspace]

Use this little mod to make your tests, it works thousands times better: https://www.nexusmods.com/x4foundations/mods/772
Of course you have to edit the shiptest_gen_cluster.xml file to add your regions. Starting a new game is a matter of seconds.
Without this the massive work of redoing ALL sectors plus adding 5x more would have been impossible for me

[remisc]

Thanks, will check it out in the evening. Curious how it compares to the flight-school universe. I already had loadtimes of few seconds there.
Another thing that sped up testing for me was adding "<showfps>true</showfps>" and "<skipintro>true</skipintro>" to the config.xml.
Without those I'd have to make a few more inputs every time I launched the game.

[remisc]

The "Ship Test Site" mod is indeed easier to manipulate when it comes to changing more than just a region.
But even still I can't seem to clone a sector from normal galaxy correctly.
For example when trying to recreate Turquoise Sea
- I removed all connections from the shipttest cluster apart from shiptest_gen_sector00_cluster00_macro
- got a black void as expected
- I copied all connections from Cluster_50_macro cluster apart from highways into the shiptest one
- got what seems exactly like Turquoise Sea
- And performance in between them differs. It's actually better in normal universe than in the clone....

So yeah, I'll play with it some more another time when I get some motivation.

For now I see that 6.10 HF1 finally hit the GOG so I need to check if there were any changes.

[Realspace]

Performance is affected by the crowded space the shiptest creates when used on top of other mods and reading their god/job + vanilla jobs. In mine it assembled all added pirate ships and stations in the map's center, fps dropped. To test the real performance, cheat-jump away from the center. It should be consistent with the real game

[beko]

impressive difference in the described FPS. Will try that.

And THANK YOU for uploading this elsewhere and not only Steam Workshop and Nexus Mods.

[remisc]

@beko
Thanks for trying it.

Keep in mind that this only changes some zones within sectors - so it's not a universal gain in all sectors with fog.
I'd ideally want to test on true galaxy sectors instead of zone sandbox to get a better idea which sectors need more work, but that will have to wait for a while,
I want to actually play the game for a while and wait for 6.20 - there are changes to graphical pipeline for AMD on the latest beta. But since I am on linux GOG that beta is not available to me.

Additionally a thing I'd like to look at are smoke and fog on station - steam from ships and such - those also seem to cause slowdowns but I don't immedietely know where to look to change them.

Nonetheless if you feel like you find a sector that seems slow then let me know.
(Besides Turquoise Sea, Sanctuary of darkness and Asteroid belt - I know those could use better performance but I don't yet know how to meaningfully improve things without altering too much)

[Realspace]

Take a look at how I rewrote the regions in my xrsge mod. You can use the same approach for vanilla. F.i. in asteroid belt I removed a layer of fog. Ad oc made the procedures we discussed above to make it lighter. The result is a big fps increase.
I don't have the time nor the wish to export my regions for vanilla game but you can do it, will spare a lot of time.
While I use a lot of new regions, there are many vanilla regions I've redone and not used anymore but are still there.

[alt3rn1ty]

Good luck with this mod, it looks like you have put in a fantastic amount of research already, and obviously until Egosoft put the whole of X4 to bed there's a lot more to be done and checked for every update.

I dont use mods yet, still holding off due to waiting for how Ventures will eventually turn out.

But when I do this mod is now top of my bucket list.

I remember another forum member (Scoob) making a very long and well researched topic on the subject of the effect of different fogs in the game upsetting frame rates, what we do not know is how much of it Egosoft listened to and tried to amend. Obviously if you are still seeing them tweaking things occasionally its an ongoing process.

I really hope one day Egosoft would take the work done here onboard into the game themselves, or at least have it as a signed mod.

Really like the fact that not only is it getting done well, but also doesn't include any other subjective modifications to the game.

You probably know this already, but be wary of the fog effects in the VIG Windfall sectors, when those sectors first came out in the game they looked awful in places, and so Egosoft used fog effects to blend/cover up the bad bits without sacrificing the artistic feel of the place, similar to Sanctuary of Darkness. I still get really bad performance in the Windfall sectors, but that I mostly attribute to the ridiculous amount of fighters in there, I dare say though that fog is probably having a fair share of contributing.