New to network rendering (questions)

[Benny L]

So I'm trying out network rendering and I've got a few questions hoping to get some answers, I'm very unfamiliar with it still so here goes:

1) Just clarifying, I see the network rendering split a project into different regions/tasks for each computer, but couldn't it combine all the cores and render it at once?
2) When it's sent to the network, and it assigned 3 tasks for 3 computers, so if computer#3 finished the task before other two, can it jump on the network and help out the other two computers?
3) If one of the computers on network shut off/got off the network, would the task fail/pause/send to other computers on the network? or the whole render would just fail?
4) While a project is being rendered by 3 computers, and I found a 4th computer to add to the network, could I assign it to the current render in progress project?
5) For some reason I can't get my main computer to behave like a Manager/worker at once (even though i see the option), I'd like it to manage and render at the same time, how do I go about it?

Thanks and any info is greatly appreciated thanks!

[Mario Stockinger]

1) Keyshot splits the project into tasks depending on the resolution i think. It doesn't matter if you have 1 Worker or 10. The Worker is rendering the current
    assigned task with all the cores that are enabled for this worker.
2) Each worker gets a task when a task is complete the worker gets the next open task. Depending of the prioritiy of the project it jumps to the next
    open task in the current project or to a new project.
3) IF a Worker is shut down the task the worker was working on fails and there is a retry of the task when all other task from the project are done.
4) When you add a additional worker to the pool then it gets the project from the manager and gets an open task assigned and is working on the project.
5) I've done a lot of testing with the Keyshot NW Renderer and i have to say it's very sensitive so i would not recommend a Manager / Worker on the same
    PC. The Rendering can generate a lot of CPU load and NW Traffic and as far as i testet the Manager / Worker is very sensitive to time outs.
 
What you can do is Monitor / Worker on the same PC. I do that too and that works. When you connect to the manager PC as admin you can do everything. (Manage Pools, delte jobs ect.)

[mattjgerard]

This is all good information. I have one comment, and this might not apply since we are running all 32 cores on 4 different virtual machines. I left 2 cores off the Master/Worker to allow for a couple of cores for the master tasks to run on. That way you shouldn't get timeouts due to the master tasks waiting for CPU time. When you run the configurator for the master/worker you can set how many cores the NW render can use, so just drop one or two off it, and see how it runs.  Your results may vary

[Mario Stockinger]

Our Manager is a VM , the Keyshot PC is also used for other tasks so we decided to set up a VM for the Manager.

Some of our render projects have > 5GB bip File size. We hat to set the workers to direct connection because with the default settings we had a lot problems because of the file size.

[Benny L]

1) Keyshot splits the project into tasks depending on the resolution i think. It doesn't matter if you have 1 Worker or 10. The Worker is rendering the current
    assigned task with all the cores that are enabled for this worker.
2) Each worker gets a task when a task is complete the worker gets the next open task. Depending of the prioritiy of the project it jumps to the next
    open task in the current project or to a new project.
3) IF a Worker is shut down the task the worker was working on fails and there is a retry of the task when all other task from the project are done.
4) When you add a additional worker to the pool then it gets the project from the manager and gets an open task assigned and is working on the project.
5) I've done a lot of testing with the Keyshot NW Renderer and i have to say it's very sensitive so i would not recommend a Manager / Worker on the same
    PC. The Rendering can generate a lot of CPU load and NW Traffic and as far as i testet the Manager / Worker is very sensitive to time outs.
 
What you can do is Monitor / Worker on the same PC. I do that too and that works. When you connect to the manager PC as admin you can do everything. (Manage Pools, delte jobs ect.)

Great info, thanks!! Can you clarify the last statement on "what you can do is monitor / worker on the same PC" , didn't you just say it's sensitive to do so on #4?

Our Manager is a VM , the Keyshot PC is also used for other tasks so we decided to set up a VM for the Manager.

Some of our render projects have > 5GB bip File size. We hat to set the workers to direct connection because with the default settings we had a lot problems because of the file size.

Are you saying you have a VM like parallel desktop / fusion to run another OS and set that as Manager? 

[Benny L]

Our Manager is a VM , the Keyshot PC is also used for other tasks so we decided to set up a VM for the Manager.

Some of our render projects have > 5GB bip File size. We hat to set the workers to direct connection because with the default settings we had a lot problems because of the file size.

On a separate question, I believe you do this at work?  I'm testing it at my workplace currently using my colleague's computers. However, the worker computers aren't able to detect my manager computer. I'm suspecting it's my company's secured wifi or firewalls, because it works flawlessly at my home network, any thoughts?

[Mario Stockinger]

Yes we have a render farm and our Manager is running on an ESX Server Cluster with  about 30 other Servers.... No VM Ware Player ect...  :-)
I also have a VM Worker PC i use when one of the ESX Server has enough resources left.....

Our Keyshot PC is a Dual CPU Xeon System with 2x 14 Cores + HT and 128 GB Ram.

All workers must be able to ping the Manager. You can test that - if you cant ping the  manager then a firewall blocks the connections or the ip Range / Subnet  is not routet in the  company lan.  (Oru Workers / Manager / Monitors are on different IP ranges (192.168.30.x , 192.168.20.x , 192.6.x.x)
Auto Detect only works within the same IP Range.

If the Manager is on a different Subnet than your Worker then the auto detection will not work in most cases.
So in the Worker Setup you can set the Manager IP and test the connection.

We also set the Workers to direct connection - in the Worker setup - direct connection you must replace 127.0.0.1 with the IP Adress of the worker PC.
So the Workers can communicate without the Manager.

The Monitor is just the Frontend that connects to the Manager. Thats no Performace Problem.  We have issues when we are rendering an animation (different toppic) but i guess this  are a few bugs in the NW Renderer Software.

KS6.3 NW Renderer was way more stable thant the actual Version of KS7 NW Renderer.

[mattjgerard]

Are you saying you have a VM like parallel desktop / fusion to run another OS and set that as Manager?

I have no idea what the IT dept is using, but I know that they are running all linux servers, then create VMs as needed for different apps they run on the servers. I have 4 windows 10 VM's that I can log into and view as they are working, KSRender01 is my master/worker, then 02, 03, and 04 are the slaves. Each have 8 cores running, for a total of 32. I know I am doing a slightly wrong thing by running my master as a worker. When that machine is rendering, the CPU load goes to 100% and might be taking away some efficiency of the master tasks and slowing that down, but we run pretty simple scenes, and not many at a time, so there really isn't much to manage. The whole thing renders slightly slower than my main workstation which is a HP640 12c24thread machine. BUt the ability to offload that render allows me to keep working.

I know just enough about servers and Virtual machines to get myself into trouble. I will say that the IT guy freaked when I sent the first job and he started getting emergency alerts when his server load pegged to 100%! He said he's never seen that happen before

We are now spec'ing out a dedicated machine for rendering so we don't hurt the performance of the rest of the server tasks