Local KeyShot rendering - "FIND A KEYSHOT QUEUE NEAR YOU"

[DriesV]

I just thought of something that I would consider an extremely valuable and useful service.
The idea came from doing tests with 'Remote network rendering'. I actually got this working with the current KeyShot network rendering implementation.

Maybe you guys/girls know about www.3dhubs.com.
Local 3D Printing
"FIND A 3D PRINTER NEAR YOU"
It's an awesome webportal where you can find 3D printers in your neighbourhood and meet the people operating them. At its core, the site is a market place (supply and demand) where model makers can order 3D prints at a local 3D printer. It's the culmination of social manufacturing!

Now let's stretch this idea a bit further...into KeyShot territory.

www.keyshothubs.com
Local KeyShot rendering
"FIND A KEYSHOT QUEUE NEAR YOU" (It even rhymes! )
The idea is to have a network of KeyShot network rendering setups, with rendering capacity for anyone around the world (using KeyShot ofcourse...) to use. Anyone who needs a render faster than his/her own system would be able to produce, would be able to browse through all available public network rendering setups and choose an appropriate "farm" based on: amount of cores, cost as $/(core*hour), priority, location...

I think this would rock SO HARD!
It could be the culmination of social visualization...

Dries

[NDenekamp]

This sounds like a great idea.

There is of course regular render farm business. but your concept would be more locally sourced, on the fly kind of thing. However would it still require one to have a network render licence (which is limited to number of cores depending on package) in order to submit jobs this way? having such a licence suggest a person likely already has multiple machines to run as slaves.

With Renderbay for example, you submit a packaged .bip file, and they load it up to their system and assign the number of GHz you've paid for. With 3dhubs.com it seems you also upload your STL file to be printed and the website then sends it to the person who operates your choses printer with instructions for material etc. They then load it into their printer and hit go.

Perhaps then the Keyshot portal could work in a similar way to allow anyone who has a Keyshot licence (even regular non-pro non-network render licences) to upload a packaged .bip and specify output size and quality (sample level?).

This is of course not as automated and requires the user executing the render to manually load up the file and queue it, and also does not allow a job to be distributed between more than one networked setup.. It would also mean slower turn-around, something which would be key as if it takes too long via this route, I may as well leave my lower end machine running overnight.. etc.

Maybe you already have a better insight into how this would work. I've never used networked render before so I can speak from personal experience.

n

[DriesV]

However it would still require one to have a network render licence (which is limited to number of cores depending on package) in order to submit jobs this way. This suggest a person who has such a licence likely already has multiple machines to run as slaves.

You DO NOT need a network render license to be able to submit jobs! This is key to the whole concept. You ONLY need a license for the master system, in this case the remote network rendering setup. (Slaves don't require a network render license either.)
Currently a user who wants to submit jobs needs to install the network rendering software and configure it as 'queue only' (thus not requiring any license). That's it.

Maybe you already have a better insight into how this would work. I've never used networked render before so I can speak from personal experience.

I think it would work really well in a scenario where you need your own dedicated render queue for a certain amount of "render time". E.g. you get total control over a remote render queue with 32 cores for 8 hours. You then pay for the use of 32 cores for 8 hours.
It would be much more flexible than having to send ksp files to a farm and waiting for the result to be sent back. If the render is not meeting expectations and you need to rerender a frame, then you can just drop another job into the queue. The way KS NR works now, you can see render progress live and renders are downloaded locally automatically. This is a huge benefit.

Because it would be community-driven (another key feature!) the cost to render could be very low, compared to traditional render farms, that are operated to turn a profit.

Niels, drop me a PM and I'll let you try it out.

Dries

[NDenekamp]

Currently a user who wants to submit jobs needs to install the network rendering software and configure it as 'queue only' (thus not requiring any license). That's it.

I see, I wasn't aware of this option. Normally I just see 'add to queue' and 'send to network' greyed out under render modes. Thats making more sense now

I know you are still testing this, but does that type of setup allow for a job to be submitted to multiple geographically separated slaves? And can you apply a restriction on how many of your machines cores I could use? (you may want to be able to use your machine for other things while doing network rendering in the background)

Also, does this mean that people without network licence, but with the network render slave software installed and a beefy computer can contribute by making cores available?

[DriesV]

does that type of setup allow for a job to be submitted to multiple geographically separated slaves?

Yes, slaves can connect to the master system from different locations. However, a poor network connection between slave and master can seriously impair rendering performance (I tested this too...). I would keep master and slaves within the same physical location.

And can you apply a restriction on how many of your machines cores I could use?

Yes, number of cores for NR can be specified in increments of 1.

does this mean that people without network licence, but with the network render slave software installed and a beefy computer can contribute by making cores available?

Theoretically yes, but this will require a beefy & stable network connection between master and slaves as well! Maybe you want to try this too? LOL

Suddenly the KeyShot NR license model makes much more sense now.

Dries

[DriesV]

UPDATE: Just did some succesful testing with remote slave machines!

The hard numbers:

12 local cores >>> 19m 50s
12 local cores + 20 remote cores >>> 8m 23s

Dries

[NDenekamp]

Nice, thats nearly linear..! ~11% under an estimation for 32 local cores

All cores being equal in this test?

I believe you have 32 cores total right? maybe you can do a test to compare how fast they do.


Side note, Where do I find this network queue instal? Unless i'm word blind – which does happen from time to time – I don't see it on the Keyshot website.


Niels

[DriesV]

All cores being equal in this test?

Nope...
12 local cores = i7 3930K @ 4.3 GHz
20 remote cores = Dual Xeon E5-2680 (out of a total of 32)
The i7 has the fastest cores.

I believe you have 32 cores total right?

My NR license is for 32 cores.

Where do I find this network queue instal?

The installer indeed isn't available for download on the KeyShot website. I'm not sure I'm allowed to post download links here. Maybe someone from Luxion can step in to clarify?

Dries

[NDenekamp]

Right, the i7 in your machine delivers about 1.5x FPS/thread compared to Xeon E5-2680.

Bit confusing how Keyshot refers to Threads as Cores.. (Maybe to be fair on the AMD folk who live without hyper threading?)

What I mean with the 32 cores was if you could run a local test with 32 cores, and then a remote network test with an equivalent amount of compute power to see what the impact of the internet connection is on remote render time.

[DriesV]

The discussion is diverting a bit...
The impact of the network connection will depend on many things: KeyShot scene size (the master must distribute the job to all slaves), line quality etc. This is why I wouldn't use remote slaves in the first place. Too much hassle, too many variables.

Dries

[djorzgul]

I went through the thread and learned few things. Very useful. Thanks for that

Also, the "find a keyshot node near you" idea is super cool, I hope v5 will bring some news regarding that.

One theoretical question ( for now, but can become reality soon), if for example I buy/rent 20 network render licenses, would it be possible to use Amazon ec2 cloud and rent 20 cores to ? Reading the documentation and search a bit I'd say it is, but I am not sure if I missed something.
I am in the middle of project that would greatly benefit from that. Of course, I am aware that cloud based approach depends on many things ( scene size, bandwidth..) and that can be slower than local network, but still... it would much more interesting to setup working cloud based farm than to buy more hardware and make great looking renders from a local park