KeyShot Forum

Technical discussions => General discussion => Topic started by: Kuba Grabarczyk on February 16, 2022, 11:29:51 AM

Title: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on February 16, 2022, 11:29:51 AM
Hello everyone,

Can anyone from Keyshot team say a few words about background of GPU geometry setup time when rendering in GPU mode?

I work mainly with animation with alembic files and deformable meshes imported from Blender 3.0. To be precise I use Keyshot 10.2 Pro. Since I have RTX 2070 and 3080Ti the rendering times are wonderful but for more complex scenes I can see that for each frame there is "GPU geometry setup" time which is sometimes even longer that rendering time (!)

Can anyone help how to shorten or somehow optimize it? This issue is literally doubling my rendering time which is killing my projects pipeline, help! I also notices when having really complex scenes GPU mode works very slowly in the real time view, especially any changes to geometry make it slowing down. So in the Keyshot viewport I used to work with CPU mode but eventually want to render the animation in GPU mode which is around 20 times faster in my case (but the geometry setup time is slowing it down...)

Since Blender 2.93, Blender and Cycles have something called "persistent data" which means it only loads geometry to GPU memory once per animation and not per frame. It speeds up the rendering time hugely. Are there any chances to implement such feature in Keyshot?

Any kind of help here will be very appreciated,

Best regards,
Kuba Grabarczyk
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: sc4r4b on February 16, 2022, 12:39:28 PM
It's the ABC file with deformable meshes.  From what I understand the abc is baking the deformed geometry.  So it's comparable to the GPU tessellating a new 3D model per frame.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on February 16, 2022, 12:52:23 PM
Do you have any ideas how to fix it? Any workarounds?
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: sc4r4b on February 16, 2022, 01:41:23 PM
All I can recommend at this time is lowering your animation frame rate.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Niko Planke on February 16, 2022, 01:51:38 PM
Hey Kuba,

Please contact Support at support@luxion.com, and provide them with a sample file and complete system specs?

I noticed that you have a dual GPU setup with two GPUs from different series.
Could you try to set the rendering to only happen on one of the GPUs and see if the "GPU geometry setup" still takes as long?
On an other note, we made improvements in KeyShot 11 for a similar issue,if you have the option please also check if the issue still exists in KS 11.

Best regards
Niko
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on February 17, 2022, 01:04:44 AM
Thank you very much for a reply, Niko! I have just carried out some test on a single still frame from my current project and the results are astonishing! I used very same rendering settings for the same frame changing only the usage of GPUs I have, look at that guys:

RTX 2070 Only: GPU Geometry Setup: 12 s, Rendering time: 77,41 s, Total time: 89,41 s
RTX 3080Ti Only: GPU Geometry Setup: 12 s, Rendering time: 25,96 s, Total time: 37,96 s
Both GPUs combined: GPU Geometry Setup: 24s, Rendering time: 20,97 s,Total time: 44,97 s

Honestely, this is the last thing I expected here. You can see clearly that Keyshot needs to perform GPU geometry setup separately for both cards. For complex scenes, this time is relevant. In my case the benefit of shortening rendering time by having both cards is smaller than cost of GPU Geometry Setup. This is really interesting and quite depressing for me as I have got worst performance by combining two cards, as I had expected to have a boost from that combination.

Unfortunately, I have no access to KS11 as I do not want to spend subscription cost monthly when paying 2k USD around year ago for full license... As far as I know I can't upgrade it right now. Nevertheless, would be great to know what exactly has been improved in KS11 on that field.

To sum up, for very complex scenes I'm gonna use my new RTX3080Ti only, which is quite sad, however, I still appreciate you suggestion very much Niko.

Kuba
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Niko Planke on February 17, 2022, 02:24:16 AM
Hey Kuba,

Thank you for trying.
I have created an issue in our system to track this.
We have an other customer experiencing this, maybe his solution works for you as well: https://forum.keyshot.com/index.php?topic=28389.0 (https://forum.keyshot.com/index.php?topic=28389.0)

Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on February 17, 2022, 01:08:00 PM
Thank you very much, Niko. If you find out any way to improve performance for two GPUs users with complex scenes, that will be very helpful. This is everyday issue I face in every single project I work with. It means for me waiting 10 hours instead of 5 for a 20s of 30fps 1080p animation, it really impacts my workflow significantly.

Thank you for linking the other topic. I tried option with rendering in background with no change, the rendering and GPU geometry setup times are same. I can agree the real-time view very likely would be faster but I am sorry, I will not be doing any coding to automate my mouse movements to workaround the issue, that is too much...
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mattjgerard on February 18, 2022, 06:55:14 AM
This doesn't fix the problem, but for now have you considered using a render farm? I don't have the length of project to warrant it, but would seem to be a decent option for faster turnarounds, at least until they figure out what is going on.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on February 18, 2022, 02:06:04 PM
I do not want too be sarcastic but possible workaround would be also changing the rendering software for one making real use of the users hardware.

As mentioned before, it is an issue i face in every single project I make and using the renderfarm is not a solution at all. Currently we produce 2-3 animations per month in the length of 1-5 minutes, 1080p, 30fps, around 1200 samples per frame. If luxion pays the invoice for rendering farm then yes, it sounds like a great idea, but I doubt the company is very eager to do that
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mattjgerard on February 21, 2022, 06:16:54 AM
There is no reason for Luxoin to pay the render farm invoices. You are hitting the limits of what the software is capable of, that's just the plain facts of it. Without knowing the specifics of your situation, there is actually no reason why a render farm would NOT solve your problem, and they are very reasonable in pricing, as long as you are pricing your services properly, it should be totally acheivable. There are a couple of Keyshot specific render farms on these forums that would gladly work with you on a test render to see how the service would work for you.

Now, I only render still images, I don't render animations very often, but why 1200 samples? I find that for 99% of our renders I rarely need to go above 512, and most of the time 256 will do just fine with some post production noise reduction. But i understand that your issue is more about the GPU load time.

which brings me to CPU rendering, is that an option? It is MUCH more stable and scaleable than GPU rendering, as you are discovering. GPU scaling doesn't work well unless you have identical cards on identical drivers, and even then, there is 2 memory banks to reload every frame rather than one for CPU rendering.

At my job we have a 64 thread machine on our network and pay the network rendering license to be able to send renders to it. Is that an option? I undersstand trying to fix workflow issues while the technical bugs get worked out. Would you be able to download a trial of KS11 and test one of your scenes on it to see if KS11 has fixed it? You don't have to buy the whole license, would be just to troubleshoot.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on February 22, 2022, 01:19:24 AM
That is true, I just feel I am the at limit of what Keyshot software offers. At the same time, I am aware I am not using Keyshot in the main field it was designed for. Still, would be great to get some improvements in the GPU Geometry Setup time.

Remember please, you can NOT use Keyshot inbuilt denoising or any similar stuff while working with animation - there is the risk each frame will be denoised differently and animation will not be smooth.

The number of samples depends on the scene, when you have some object partially fading out and want to show INTERIOR without the noise, 512 samples will not be sufficient for sure. 

I agree CPU rendering is much more stable and this is why we recently use it in the real-time view. Nevertheless, for the rendering is much slower, even though we have quite decent CPU. You can see the benchmark results below. I would rather not come back to making my rendering time 20x slower.

Speaking about rendering farms, that is true, it can be very useful. Saying what is cheap and what is expensive please keep in mind we may live in different countries. For example, in Poland the average salary is less than 1000 EUR and we are last but one in the ranking of business friendly countries in EU. That means, if I use render farm in every project, very likely my services would be to expensive for local companies.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mattjgerard on February 22, 2022, 07:06:29 AM
Kuba, you make some great points, and I appreciate the back and forth dialog.

This certainly is something that KeyShot should figure out. We do not use a render farm for nearly the same reason. Its just more cost efficient to host our own with the Network rendering plugin.

I wish keyshot could develop themselves as a plug in for various modeling packages, as Octane and VRay does for Cinema 4d and Blender, etc. That would be the perfect world in my view.

I hope support can get back to you on a solution, this is a problem that has existed from the first implementation of GPU rendering. Have you asked them for a KS11 trial license yet for testing? 
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on March 05, 2022, 02:04:49 PM
I finally found a little bit of time to make a quick comparison between KS10 and KS11 trial version hoping for some improvements.

I performed test on different but similar scene and different frame of animation. (this is frame no 1064 of the project I have just sent you, Niko) The results are astonishing again!!! Unfortunately for Keyshot 11 I received worse rendering time than for Keyshot 10 (!!)

Keyshot 10 (both GPUs) 47,95 s (GPU geometry setup around 24 s)
Keyshot 11 (both GPUs) 71,94 s  (GPU geometry setup around 48 s) (!!)

It looks like geometry setup time is two times slower per each card. Honestely this is exactly the opposite of what I hoped for... Neverthless just wanted to share, maybe some future generations will find my tests useful
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Will Gibbons on March 07, 2022, 05:00:21 PM
Quote from: Kuba Grabarczyk on March 05, 2022, 02:04:49 PM
I finally found a little bit of time to make a quick comparison between KS10 and KS11 trial version hoping for some improvements.

I performed test on different but similar scene and different frame of animation. (this is frame no 1064 of the project I have just sent you, Niko) The results are astonishing again!!! Unfortunately for Keyshot 11 I received worse rendering time than for Keyshot 10 (!!)

Keyshot 10 (both GPUs) 47,95 s (GPU geometry setup around 24 s)
Keyshot 11 (both GPUs) 71,94 s  (GPU geometry setup around 48 s) (!!)

It looks like geometry setup time is two times slower per each card. Honestely this is exactly the opposite of what I hoped for... Neverthless just wanted to share, maybe some future generations will find my tests useful

Ouch. That's really disappointing to hear! I hope it's something that can be addressed quickly!
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: DT on March 10, 2022, 05:53:10 PM
No software fix.  Working as intended.  ;)

When rendering on a farm or multi CPU/GPU the geometry set up (tessellation) is typically processed then loaded into the local hardware RAM.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Niko Planke on March 11, 2022, 12:05:25 AM
Hey,
Sadly we will need to spend more time on investigating this before we are able give a satisfying answer.

We already identified one issue and KeyShot 11.1 will contain a fix for it, which should give better rendering speeds for animation renderings by reducing the amount of geometry setup required.

Still image renderings and animations using animated displacement will likely not be impacted by that fix.

As to the general slow down, we are still crunching the numbers and doing test renderings to get to the bottom of this.
Multiple factors  are at play in this, including Driver version, Dual GPU setups and KS 11 vs KS 10 and Scene setup.

I will provide an update as soon as we have something to share.
(Note: we are also investigating the cause of the differences with the benchmark Tool)
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on March 11, 2022, 12:49:59 PM
Hey Niko,

Thank you very much for your engagement, this is honestly my first time ever when anyone from Keyshot took my post seriously and took care about my issue. I appreciate that very much. If the Geometry setup time would be improved in any future version of Keyshot I will upgrade it immediately, this is currently my biggest bottleneck in the workflow. If you have any tips it could be improved by changing the GPU drivers, any hints will be appreciated too.

On the other hand, I am fully aware I am not using keyshot the way software was designed for. I use mainly very heavy and not optimized models (these are 3D CAD models of our customers converted to meshes in Blender) as well as quite long and complex deformable meshes. This is all imported via almebic into Keyshot and the files are usually around 10 GB per one. I mean, I use Keyshot mainly as an external rendering engine for Blender-made animations and I know this is not the main task keyshot was made for. Neverthless, the results are beautiful and very easy to achieve and this is the reason I am not using any better integrated render engine like CyclesX or others,

Thank you again for you support,
Kuba
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mafrieger on March 15, 2022, 03:48:25 AM
From today (March 14th, 2022), Microsoft's Direct Storage Technology is available for Windows PC.
Since this reduce loading times for PC Games dramatically (SSD -> GPU),
maybe this is also a thing Keyshot guys could look into if there isn't any chance to benefit from

see:
https://devblogs.microsoft.com/directx/directstorage-api-available-on-pc/

background:
https://devblogs.microsoft.com/directx/directstorage-is-coming-to-pc/
https://www.youtube.com/watch?v=zolAIEH0n1c
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mafrieger on March 15, 2022, 03:55:50 AM
Named usage scenarios of Direct Storage
looks pretty interesting to be used  in KS:

Scenarios:

1. Copying data from disk for GPU usage
a) Geometry
b) Textures

2. Copying data from disk for CPU usage
a) Audio

3. Decompressing data for GPU usage
a) Necessary for assets like textures (e.g., DEFLATE compression on top of BC formats)

source:
https://www.youtube.com/watch?v=zolAIEH0n1c&t=666s

Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mafrieger on March 15, 2022, 04:12:23 AM
btw:

from Nvidia site this seems already be supported:

For Consumer GPUs named RTX IO (RTX 30 Series)
https://www.nvidia.com/en-us/geforce/news/rtx-io-gpu-accelerated-storage-technology/

For Pro cards (Quadro)
this was already available in 2019, called GPUDirect Storage see https://developer.nvidia.com/blog/gpudirect-storage/

reused from:
https://forum.keyshot.com/index.php?topic=26584.msg112586#msg112586
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on March 16, 2022, 02:30:26 PM
Thank you for posting, it's indeed interesting.

Nevertheless I am quite sure it needs to be implemented inside keyshot. The loading time can be this long as it is now but crucial is to do it only once per animation, not per each frame. This feature is called "persistent data" in blender and was introduced quite recently, I believe it could be possible to code it also  Keyshot
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mafrieger on March 18, 2022, 02:50:38 AM
sure it has to implemented in KS, but the needed API is now available.

The best option would be to have booth:
- Direct Storage to feed the GPU as fast as possible
- Persistent Data to not always have to refeed the GPU

with both, even animations with flexible bodys should be much faster...
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mafrieger on March 18, 2022, 02:52:50 AM
and yes, I also fully vote for having support for Multi-GPU setups!!
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Niko Planke on March 18, 2022, 07:04:07 AM
Hey,

we now also identified the issue you reported with the GPU slowdown  in KS 11.0.
The soon to be released version 11.1 should resolve the perfromance differences reported for GPU rendering(both in Dual and Single GPU setups).

Combined with the already mentioned fix for the "Geometry setup" you should be able to observe a significant perfromance increase compared to KS 11.0 and for some scenes KS 10.

Thank you for the colaboration and patience in this regard.
A little more patience will be needed until 11.1 will be available for download though.

In case you experience other issues feel free to reach out to the support team.

Be aware that I have no news on the issues related to driver version and the benchmark tool, but we are aware and working on it.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on March 18, 2022, 11:42:39 AM
Hey Niko,

Thank you so much for the feedback, this is a great news! Looking forward to seeing Keyshot 11.1 released, as mentioned before, that will speed up my everyday work hugely. Again, thank you very much for open and constructive approach to work on the quite specific issue I had, appreciate it very much

Kuba
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on March 22, 2022, 12:50:23 AM
Hello everyone,

Today I performed one another test on my super heavy animation scene, testing the GPU performance improvements using just released Keyshot 11.1. The results are interesting, still not what I believe, we all could wish for but I just want to share the numbers, maybe someone will find my study useful.

Keyshot 10.2 (both GPUs) 47,95 s, GPU geometry setup: ~24 s, pure rendering time: 23,95 s
Keyshot 11.0 (both GPUs) 71,94 s ,GPU geometry setup: ~48 s, pure rendering time: 23,94 s
Keyshot 11.1 (both GPUs) 63,96 s, GPU geometry setup: ~43 s, pure rendering time: 23,96 s

Again, I am fully aware my scenes are very heavy (not retopologied deformable meshes, CAD models triangulated and not simplified well in Blender, lots of objects). So I do not demand fixing the issue, as I am probably in less than 1% of Keyshot users experiencing such issues. Nevertheless, just wanted to share the data. In the 11.1 release the results are better, however, the increase of GPU geometry setup time in comparison to Keyshot 10.2 is still significant and maybe a field for further improvements,

Kuba
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Will Gibbons on March 22, 2022, 07:09:10 AM
Quote from: Kuba Grabarczyk on March 22, 2022, 12:50:23 AM
Hello everyone,

Today I performed one another test on my super heavy animation scene, testing the GPU performance improvements using just released Keyshot 11.1. The results are interesting, still not what I believe, we all could wish for but I just want to share the numbers, maybe someone will find my study useful.

Keyshot 10.2 (both GPUs) 47,95 s, GPU geometry setup: ~24 s, pure rendering time: 23,95 s
Keyshot 11.0 (both GPUs) 71,94 s ,GPU geometry setup: ~48 s, pure rendering time: 23,94 s
Keyshot 11.1 (both GPUs) 63,96 s, GPU geometry setup: ~43 s, pure rendering time: 23,96 s

Again, I am fully aware my scenes are very heavy (not retopologied deformable meshes, CAD models triangulated and not simplified well in Blender, lots of objects). So I do not demand fixing the issue, as I am probably in less than 1% of Keyshot users experiencing such issues. Nevertheless, just wanted to share the data. In the 11.1 release the results are better, however, the increase of GPU geometry setup time in comparison to Keyshot 10.2 is still significant and maybe a field for further improvements,

Kuba

Interesting. Did Luxion ever look at this scene you're using for benchmarking here? It seems odd that it was handled better in KS 10.2. Perhaps it would be handy if they were able to see what was going on in this particular case.
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: Kuba Grabarczyk on March 22, 2022, 07:22:37 AM
I have sent the keyshot package with this scene to Niko as he asked for. Nevertheless it was just work-in-progress files packed and sent with all the names in Polish and quite messy objects organization. To find out what's going on there would be very hard. On the other hand, it happens not only to that scene, but to every single scene with complex objects and deformable meshes 
Title: Re: GPU Geometry Setup killing animation rendering time - how to fix it?
Post by: mafrieger on March 23, 2022, 06:05:38 AM
Today there are new Nvidia drivers available Version 512.15

Since they are talking about:
QuoteLuxion Keyshot 11 - support for 3D paint customization directly on model surfaces for more realism, optional material management and color options, and ability to record physics of object for keyframe animation

it should be worth a try...