Alpha Channel Cutout Problems.

Started by ksall, July 22, 2010, 05:28:25 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

ksall

I am trying to render a scene where I will replace the background in photoshop.
I go ahead and use a default backplate (or solid color) and render the scene as a .tiff with alpha.
Afterwards I open in PS and use the alpha as a selection, invert and delete to get my background transparent.  When I insert my new background, it becomes apparent that something is "off".

It seems that the alpha did not cut cleanly, and my rendered models are slightly "rimmed" with the backplate color (or solid color that I selected for background).

Has anyone else come across this?  Where it seems like the backplate has bled onto the edges of the models?  Is this a Keyshot issue or am I doin't something wrong in PS?

Please advise.

Speedster

I've run into this also, so I don't use alpha anymore.  Instead I always render a clown pass if I'm going to drop out the background.  Very neat and clean sharp cutouts.  Take just a bit longer, but worth it.  Also, instead of backing out of my materials after the clown, I make sure to save the hero shot bip, proceed with the clown, then close without saving.  Just reopen again and off to the races!
Bill G

m2tts

By "clown" you mean assigning total black and total white to any parts you want to mask right? I do this all the time to make selecting individual parts easier in PhotoShop. I take the black and white image, after being changed from an rgb to grayscale, and drag the gray channel from it to the open render file. Instant alpha channels for individual parts in the final PhotoShop rendering. Making the black and white pass takes about ten seconds to render.

guest84672

Quote from: ksall on July 22, 2010, 05:28:25 PM
I am trying to render a scene where I will replace the background in photoshop.
I go ahead and use a default backplate (or solid color) and render the scene as a .tiff with alpha.
Afterwards I open in PS and use the alpha as a selection, invert and delete to get my background transparent.  When I insert my new background, it becomes apparent that something is "off".

It seems that the alpha did not cut cleanly, and my rendered models are slightly "rimmed" with the backplate color (or solid color that I selected for background).

Has anyone else come across this?  Where it seems like the backplate has bled onto the edges of the models?  Is this a Keyshot issue or am I doin't something wrong in PS?

Please advise.

If the cutout is not "clean", then set "pixel filter size" to 1 in the render settings. PFS adds a blur to the image.

Thomas

ksall

Hi Thomas,

Setting the PFS to 1.00 still leave a bit of blurred residue.
Is there any way to omit this blur completely?  The slider does not allow an input below 1.00 for PFS, but what I would need (I guess) is a PFS of zero.

Using the "clown" render procedure doesn't seem to help either, as the blur is incident on the models and not the background.

Any help on this matter would be greatly appreciated.

Thanks.

guest84672

Shouldn't be the case. Also, make sure that depth of field is not enabled.

m2tts

#6
How are you making the selection of the alpha channel in PS? Make sure you're making your selection using the selection menu "load selection", or pick it using the ctrl key right from the channel tab. This will ensure that the selection will get those edges transparent. Don't use the magic wand to do it as that will not select all the pixels you need it to.  Is the background you're rendering against black or white; or is it some color? Anti-aliasing is supposed to blend color of edges with the background to keep the edges looking smooth.

JohnG

Quote from: ksall on July 22, 2010, 05:28:25 PM
It seems that the alpha did not cut cleanly, and my rendered models are slightly "rimmed" with the backplate color (or solid color that I selected for background).

This is a well known issue when trying to composite cg elements, particularly in photoshop. The reason it happens is a bit complex, basically renderers (including Keyshot) have to multiply the cg element by its alpha in order to get the correct edge transparency. These renders are said to be 'premultiplied'.
However this multiplication step is also part of the compositing process, the compositing software has to multiply the foreground (and background) by the alpha in order to make the composite. Unfortunately photoshop has no way of knowing that the cg image has already been multiplied by the alpha so this double multiplication results in artifacts including dark edge pixels.

Compositing software used in film and video such as Nuke or Digital Fusion have ways to handle this problem but photoshop doesn't as it was never designed for cgi compositing. There are some involved ways to alter photoshop's normal way of compositing to get around this problem, but probably the simplest way is to try and remove the edge pixels.

Try Layer/Matting to remove white or black edge pixels.

A levels adjustment on the layer mask is another common method to choke or contract the edge of the mask.

Another option would be the maximum filter which is designed for adjusting the edges of masks
Filter/Other/maximum

jbeau

Make sure your pixel filtering and AA does not change between passes. This will help with the premult, but there will still be issues with white colors on black backgrounds + Alpha.  I wish there was some alternate AA settings as even the highest has issues with thin geometry and transparent areas. Alphas + high AA in KS with premult are broken... You will almost always see AA fringe. You can use a simple chocker on your alpha with a .5 pixel and then roto areas that don't need it. A clown pass = more then 3 flat RGB colors; not recommended when flat colors are touching. Pure non-touching red, green, blue flat colors make a successful key.


JeffM

In Photoshop, Layer>Matting>Defringe is usually all it needs.

BigBlack

Hi Guys,
I've the same issue. There'r some new solutions?