Using the stencil-buffer in a post-process?

Hi all,

(Cross-post from the forum - didn’t get any response there!)

I’m trying to do a graphical effect where a couple of objects in the scene write to the stencil buffer, and then a post-fx reads the stencil buffer to do a full-screen effect.

The part where the objects write to the stencil buffer is definitely working.

Let’s say I just want to write green pixels where the stencil buffer has been written to earlier. Here’s my shader:

Shader “Green” {
SubShader {
Tags { “RenderType”=“Opaque” “Queue”=“Geometry+1”}
Pass {
Stencil {
Ref 1
Comp equal
Pass keep
ZFail keep
}

		CGPROGRAM
		#pragma vertex vert
		#pragma fragment frag
		struct appdata {
			float4 vertex : POSITION;
		};
		struct v2f {
			float4 pos : SV_POSITION;
		};
		v2f vert(appdata v) {
			v2f o;
			o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
			return o;
		}
		half4 frag(v2f i) : COLOR {
			return half4(0,1,0,1);
		}
		ENDCG
	}
} 

}

If I have my OnRenderImage function blitting straight from source to dest using this shader, everything works as expected - I get solid green on screen only where I wrote to the stencil:

void OnRenderImage (RenderTexture source, RenderTexture dest) 
{
	Graphics.Blit (source, dest, greenMat);	
     }

But, if I try to use an intermediate buffer, it seems to ignore the Stencil section of the pass, and just write green to every pixel:

void OnRenderImage (RenderTexture source, RenderTexture dest) 
{
	RenderTexture buffer = RenderTexture.GetTemporary(source.width, source.height, 0);

	Graphics.Blit (source, buffer, greenMat);
	Graphics.Blit(buffer, dest);

     }

This just results in a completely green screen.

I’ve noticed that there’s a RenderTexture.SupportsStencil method, which returns false for my buffer, but I can’t find any documentation on why, or how to change it. Is that even relevant? I only want to use the main screen’s stencil buffer.

This is on PC.

Any help would be much appreciated,
Rob.

Even though this question is two and a half years old, there is nowhere on the internet with the correct answer. I spent a long time figuring it out.

This should allow you to use the stencil buffer in a post-process effect.

Recently,I make a post effect of sss by blur the character skin in post effect,but need stencil buffer,I did many way tries,The actual truth is that antialiasing or other post effect before mine cleared the stencil buffer ,if you want to get the correct stencil buffer you could disable AA or other post effects. Though I found that the cam.clearStencilAfterLightingPass is always false,and use the [ImageEffectOpaque] attribute did help nothing. By the way , I have to tell the truth ,The qiankanglai did the right ways.

public Shader shader;
private Material mat;

    void OnEnable()
    {
        mat = new Material(shader);
    }

    //'It Just Works' -- Todd Howard
    void OnRenderImage(RenderTexture source, RenderTexture destination)
    {
        RenderTexture buffer = RenderTexture.GetTemporary(source.width, source.height, 24);
        Graphics.Blit(source, buffer, mat);  
        Graphics.Blit(buffer, destination);
        RenderTexture.ReleaseTemporary(buffer);
    }

It seems stencil is cleared before OnRenderImage. I came up with a method to use this for post process:

  • Draw in OnPostRender
  • Blit with the origin depth buffer & new color buffer

I think you should try only drawing the buffer color buffer + dest depth buffer.

ps. I’ve posted more details and a fastBloom with stencil buffer in my blog

Thank for qklxtlx’s reply, which enlightens me a lot.
The key point is to use same stencil buffer when using Graphic.Blit. For your case, just replace line 5 with code below:`

Graphics.SetRenderTarget(buffer.colorBuffer, source.depthBuffer);

Graphics.Blit(source, material);

To be more sepecific, when you use RenderTarget.GetTempory(int width, int height) to get a RenderTarget, that target don’t contain the stencil buffer, becouse the depth bit is set to 0 in default. Use RenderTarget.GetTempory(int width, int height, 24) to get a target with depth/stencil buffer. (It’s like create depth buffer using D24S8 formate in D3D I guess)