GPU Geometry Setup killing animation rendering time - how to fix it?

[DT]

No software fix.  Working as intended. 

When rendering on a farm or multi CPU/GPU the geometry set up (tessellation) is typically processed then loaded into the local hardware RAM.

[Niko Planke]

Hey,
Sadly we will need to spend more time on investigating this before we are able give a satisfying answer.

We already identified one issue and KeyShot 11.1 will contain a fix for it, which should give better rendering speeds for animation renderings by reducing the amount of geometry setup required.

Still image renderings and animations using animated displacement will likely not be impacted by that fix.

As to the general slow down, we are still crunching the numbers and doing test renderings to get to the bottom of this.
Multiple factors  are at play in this, including Driver version, Dual GPU setups and KS 11 vs KS 10 and Scene setup.

I will provide an update as soon as we have something to share.
(Note: we are also investigating the cause of the differences with the benchmark Tool)

[Kuba Grabarczyk]

Hey Niko,

Thank you very much for your engagement, this is honestly my first time ever when anyone from Keyshot took my post seriously and took care about my issue. I appreciate that very much. If the Geometry setup time would be improved in any future version of Keyshot I will upgrade it immediately, this is currently my biggest bottleneck in the workflow. If you have any tips it could be improved by changing the GPU drivers, any hints will be appreciated too.

On the other hand, I am fully aware I am not using keyshot the way software was designed for. I use mainly very heavy and not optimized models (these are 3D CAD models of our customers converted to meshes in Blender) as well as quite long and complex deformable meshes. This is all imported via almebic into Keyshot and the files are usually around 10 GB per one. I mean, I use Keyshot mainly as an external rendering engine for Blender-made animations and I know this is not the main task keyshot was made for. Neverthless, the results are beautiful and very easy to achieve and this is the reason I am not using any better integrated render engine like CyclesX or others,

Thank you again for you support,
Kuba

[mafrieger]

From today (March 14th, 2022), Microsoft's Direct Storage Technology is available for Windows PC.
Since this reduce loading times for PC Games dramatically (SSD -> GPU),
maybe this is also a thing Keyshot guys could look into if there isn't any chance to benefit from

see:
https://devblogs.microsoft.com/directx/directstorage-api-available-on-pc/

background:
https://devblogs.microsoft.com/directx/directstorage-is-coming-to-pc/
https://www.youtube.com/watch?v=zolAIEH0n1c

[mafrieger]

Named usage scenarios of Direct Storage
looks pretty interesting to be used  in KS:

Scenarios:

1. Copying data from disk for GPU usage
a) Geometry
b) Textures

2. Copying data from disk for CPU usage
a) Audio

3. Decompressing data for GPU usage
a) Necessary for assets like textures (e.g., DEFLATE compression on top of BC formats)

source:
https://www.youtube.com/watch?v=zolAIEH0n1c&t=666s

[mafrieger]

btw:

from Nvidia site this seems already be supported:

For Consumer GPUs named RTX IO (RTX 30 Series)
https://www.nvidia.com/en-us/geforce/news/rtx-io-gpu-accelerated-storage-technology/

For Pro cards (Quadro)
this was already available in 2019, called GPUDirect Storage see https://developer.nvidia.com/blog/gpudirect-storage/

reused from:
https://forum.keyshot.com/index.php?topic=26584.msg112586#msg112586

[Kuba Grabarczyk]

Thank you for posting, it's indeed interesting.

Nevertheless I am quite sure it needs to be implemented inside keyshot. The loading time can be this long as it is now but crucial is to do it only once per animation, not per each frame. This feature is called "persistent data" in blender and was introduced quite recently, I believe it could be possible to code it also  Keyshot

[mafrieger]

sure it has to implemented in KS, but the needed API is now available.

The best option would be to have booth:
- Direct Storage to feed the GPU as fast as possible
- Persistent Data to not always have to refeed the GPU

with both, even animations with flexible bodys should be much faster...

[mafrieger]

and yes, I also fully vote for having support for Multi-GPU setups!!

[Niko Planke]

Hey,

we now also identified the issue you reported with the GPU slowdown  in KS 11.0.
The soon to be released version 11.1 should resolve the perfromance differences reported for GPU rendering(both in Dual and Single GPU setups).

Combined with the already mentioned fix for the "Geometry setup" you should be able to observe a significant perfromance increase compared to KS 11.0 and for some scenes KS 10.

Thank you for the colaboration and patience in this regard.
A little more patience will be needed until 11.1 will be available for download though.

In case you experience other issues feel free to reach out to the support team.

Be aware that I have no news on the issues related to driver version and the benchmark tool, but we are aware and working on it.

[Kuba Grabarczyk]

Hey Niko,

Thank you so much for the feedback, this is a great news! Looking forward to seeing Keyshot 11.1 released, as mentioned before, that will speed up my everyday work hugely. Again, thank you very much for open and constructive approach to work on the quite specific issue I had, appreciate it very much

Kuba

[Kuba Grabarczyk]

Hello everyone,

Today I performed one another test on my super heavy animation scene, testing the GPU performance improvements using just released Keyshot 11.1. The results are interesting, still not what I believe, we all could wish for but I just want to share the numbers, maybe someone will find my study useful.

Keyshot 10.2 (both GPUs) 47,95 s, GPU geometry setup: ~24 s, pure rendering time: 23,95 s
Keyshot 11.0 (both GPUs) 71,94 s ,GPU geometry setup: ~48 s, pure rendering time: 23,94 s
Keyshot 11.1 (both GPUs) 63,96 s, GPU geometry setup: ~43 s, pure rendering time: 23,96 s

Again, I am fully aware my scenes are very heavy (not retopologied deformable meshes, CAD models triangulated and not simplified well in Blender, lots of objects). So I do not demand fixing the issue, as I am probably in less than 1% of Keyshot users experiencing such issues. Nevertheless, just wanted to share the data. In the 11.1 release the results are better, however, the increase of GPU geometry setup time in comparison to Keyshot 10.2 is still significant and maybe a field for further improvements,

Kuba

[Will Gibbons]

Hello everyone,

Today I performed one another test on my super heavy animation scene, testing the GPU performance improvements using just released Keyshot 11.1. The results are interesting, still not what I believe, we all could wish for but I just want to share the numbers, maybe someone will find my study useful.

Keyshot 10.2 (both GPUs) 47,95 s, GPU geometry setup: ~24 s, pure rendering time: 23,95 s
Keyshot 11.0 (both GPUs) 71,94 s ,GPU geometry setup: ~48 s, pure rendering time: 23,94 s
Keyshot 11.1 (both GPUs) 63,96 s, GPU geometry setup: ~43 s, pure rendering time: 23,96 s

Again, I am fully aware my scenes are very heavy (not retopologied deformable meshes, CAD models triangulated and not simplified well in Blender, lots of objects). So I do not demand fixing the issue, as I am probably in less than 1% of Keyshot users experiencing such issues. Nevertheless, just wanted to share the data. In the 11.1 release the results are better, however, the increase of GPU geometry setup time in comparison to Keyshot 10.2 is still significant and maybe a field for further improvements,

Kuba

Interesting. Did Luxion ever look at this scene you're using for benchmarking here? It seems odd that it was handled better in KS 10.2. Perhaps it would be handy if they were able to see what was going on in this particular case.

[Kuba Grabarczyk]

I have sent the keyshot package with this scene to Niko as he asked for. Nevertheless it was just work-in-progress files packed and sent with all the names in Polish and quite messy objects organization. To find out what's going on there would be very hard. On the other hand, it happens not only to that scene, but to every single scene with complex objects and deformable meshes 

[mafrieger]

Today there are new Nvidia drivers available Version 512.15

Since they are talking about:

Luxion Keyshot 11 - support for 3D paint customization directly on model surfaces for more realism, optional material management and color options, and ability to record physics of object for keyframe animation

it should be worth a try...