Network Render Machine/Farm

[3D Off the Page]

Has anyone tried setting up an enclosure of blade servers for a network render machine or technically farm?

I have the parts on the way and I'm curious if anyone has had luck setting this up. 

Here is what I'm planning:
  HP C3000 enclosure
  HP BL460c blades 2 x quad 2.66 (6x)
  16gb per blade
  72gb 15k sff hdd 1x per blade
  Windows 7 Pro

All told with shipping $2,478.00 for 96 cores.  Add on the $1,440 for the render licenses and I'm still hoping to render with 96 cores for less than my dual six core machine.  Plus I still will have room for two more blades.

[DriesV]

...
HP BL460c blades 2 x quad 2.66 (6x)
...
All told with shipping $2,478.00 for 96 cores.
...

What Xeons are in those blades?

Dries

[3D Off the Page]

Those have the E5430 processor.

[DriesV]

Xeon E5430s don't support hyper-threading, as they're built on the Core 2 architecture and were released back in 2007. So you're not getting 96 logical cores, but just 48 if my calculations are correct (6 nodes x 2 CPUs x 4 phys. cores (no HT) = 48 cores). This can be an important consideration for ordering a suitable network rendering license.

I think a comparison to a modern solution is appropriate:
I expect a current Xeon E3-1240 v2 (single socket 4 physical cores) machine to perform around 35-40% better than one of those blades.
A farm of 5 E3-1240 v2 machines would be a fair bit faster (15%?) than your complete blade setup.
You would have 40 logical cores in that case (5 nodes x 4 phys. cores x 2 (HT!) = 40 cores).
The newer Intel architectures are vastly more efficient than Core 2.

Personally I wouldn't deploy Core 2 technology servers for network rendering in a production environment. The architecture is way too aged to really be an efficient rendering solution today.
Also be aware that power consumption per blade will be more than twice that of an E3-1240 v2 machine.
Just counting CPUs here (so not exhaustive):
*E3-1240 v2 has max TDP of 69W. That makes 345W for 5 CPUs (5 nodes).
*E5430 has max TDP of 80W. This means 160W for a dual socket node. That makes 960W for 12 CPUs (6 nodes).
I know what electric bill I would be more happy to pay.

Just my 2cts.

Dries

[DriesV]

Having said that, just under $2.5k for that piece of kit is still a pretty good deal.

Dries

[3D Off the Page]

Thanks for the info, I didn't realize that those processors didn't support ht.

I'm still going to look into blades for a rendering solution.  Those were only $50 each without drives or ram.  I can still use the drives.

[3D Off the Page]

Hey DriesV, now I'm shifting my focus on 2x x5570's X 4 blades to start.  I was looking at x5670's as well but the cost per core went from $33 to $83!!!

Thanks again for pointing out the non-ht compatibility for those other blades.

[DriesV]

Okay, time for a new extrapolation.
This time based on real-world rendering performance scales (cinebench 11.5 scores; abundant on the net and relative cb scores are a very accurate estimate for relative KeyShot performance)...

A dual X5570 node scores 10.5.
4 of such blade nodes would perform in the lines of a score of 42.

An E3-1240 v2 node scores 7.19.
6 of such nodes would perform in the lines of a score of 43.14.
So this setup would be a tiny bit faster (3%) than those 4 blade nodes.

Performance-wise this new blade array is clearly more capable than the first.

Now let's look at power consumption at the CPU level.
*X5570 has max TDP of 95W. This means 190W for a dual socket node. That makes 760W for 4 nodes.
*E3-1240 v2 has max TDP of 69W. That makes 414W for 6 nodes.
Still almost double of an E3-1240 v2 setup, but much better than your first blade array.
Note: a blade server has many optimizations for cooling and power supply that will make it more energy-efficient than my simple calculation shows.

In all, I think that new blade setup (for the right price - what are you paying?) makes for a very compelling alternative to a modern E3-1240 v2 setup.

Btw, with that blade setup a 64 core network rendering license is actually perfectly suited.

Dries

[DriesV]

OR... You could go the i7 3770K (nearly identical to E3-1240 v2, but with OC potential) route that scores 9.35 in cinebench when OC'd to 4.6GHz.
5 of such nodes would perform in the lines of a score of 46.75.
That's actually +10% faster than your latest blade setup!

You can build/buy such a system with high-quality components, silent water cooling, 16GB RAM, 120GB SSD and Windows 8 Pro for around 830 euros (VAT excl.).
That's 4150 euros for the complete NR setup. How does that compare to your blade setup?

EDIT: prices adapted for higher end components.

Dries

[3D Off the Page]

The new blade setup is approx $3,500 usd.  That includes win7 pro.

The blades are dual x5570's with 16gb ram and a single 73gb 15k drive.

[3D Off the Page]

Plus, with the C3000 enclosure I can add 4 more blades for $550 each without hdd or os.  Add $175 for those two items.

Steve