In order to do the stencil thing, you could create your own shader that takes in the original texture and the stencil texture, and outputs rgb from the original texture and alpha of the stencil texture.
something like this
Shader "Custom/CustomDiffuse"
{
Properties
{
_MainTex("Main Texture (rgb)", 2D) = "white" {}
_Color ("Main Color", Color) = (1,1,1,1)
_Stencil("Stencil Texture (a)", 2D) = "white" {}
}
Subshader
{
Tags
{
"Queue"="Transparent"
"IgnoreProjector"="False"
"RenderType"="Transparent"
}
CGPROGRAM
#pragma surface surf Lambert alpha
#pragma target 2.0
struct Input {
float2 uv_MainTex;
};
half4 _Color;
sampler2D _MainTex;
sampler2D _Stencil;
void surf (Input IN, inout SurfaceOutput o)
{
o.Albedo = tex2D(_MainTex, IN.uv_MainTex) * _Color;
o.Alpha = tex2D(_Stencil, IN.uv_MainTex).a;
}
ENDCG
}
}
Regarding the getpixel/setpixel, if you have unity pro, you could paint your changes to a RenderTexture. That way, the stencil generation is done on the GPU, which is usually a lot faster. The easiest is probably to generate your stencil texture first and then apply that stencil to your texture, either as a shader (as mentioned above) or premultiplied into the texture using another RenderTexture.
To create your stencil, you could create a small brush texture that is opaque in the center and transparent at the edges and use Graphics.DrawTexture to draw int onto the rendertexture at the desired location. The following code is one way of using graphics.blit and graphics.drawtexture to create a stencil (note that I haven’t actually tested this, so there might be some bugs).
void AddToStencil(Texture2D stencil, Texture2D brush, Vector2 brushPosition, int brushSizePixels)
{
//Create temporary render texture
int width = stencil.width;
int height = stencil.height;
RenderTexture rt = RenderTexture.GetTemporary(width, height, 0, RenderTextureFormat.ARGB32);
//Copy existing stencil to render texture (blit sets the active RenderTexture)
Graphics.Blit(stencil, rt);
//Apply brush
RenderTexture.active = rt;
float bs2 = brushSizePixels / 2f;
Graphics.DrawTexture(new Rect(brushPosition.x - bs2, brushPosition.y - bs2, brushSizePixels, brushSizePixels), brush);
//Read texture back to stencil
stencil.ReadPixels(new Rect(0, 0, width, height), 0, 0, true);
stencil.Apply();
RenderTexture.active = null;
rt.Release();
}
Depending on how many of these stencils you want, and how much GPU memory you have, you could also consider having the stencil in a permanent RenderTexture, and save the first blit and the ReadPixels at the cost of having a permanently allocated RenderTexture.
Note that the stencil texture is now inverse, so in the shader above, you would change the alpha to be
o.Alpha = 1 - tex2D(_Stencil, IN.uv_MainTex).a;