WebCamTexture flipped and rotated when applied to texture iOS

I have script that applies a webcam texture to a guitexture.

The video texture is applied and updates fi e but the image is flipped and rotated 90 degrees.

Even if I rotate the game object -90 or 90 the video image is still flipped and rotated the same.

The image is fine in the editor preview.

Is there something I am missing or a setting for iPhone camera I am missing?

Guys the solution I used was to apply a transform matrix which is processed in the shader.

Just use what you need from here:

You can just add “matrix [_Rotation]” to your shader pass in your existing shader if you don’t want to build your own shader at run-time as they did in this example.

The full transformation matrix is built for you so you don’t need to understand how the 4x4 Matrix works internally, just plug in your offset, scale and rotation values. Unity - Scripting API: Matrix4x4.TRS

The real problem I have at the moment is HOW DO YOU DETECT THE RESOLUTION OF THE CAMERA??? You can’t scale the resulting image to give the correct aspect ratio without knowing this making the feature fairly useless. Is there something I missed here? I need to build for iOS and Android, have only just hit this problem so will post back if I find a solution.

It is interesting that no one else is commenting on this issue. It makes me think I am just plain doing something wrong.

I posted a bug with unity but have not heard anything back from them.

I asked for help from unity and they said it would be something I was doing wrong but they would send the solution in a day or 2 that was a week ago:)

I would think that the expected behaviour of this feature is to be the same across devices.

In the mean time I am manually manipulating the pixel data to flip and rotate the data from the camera on iOS.

Unfortunately this is quite a lot of processing on something that is repeated pretty much every update.
,It is interesting that th

I’m seeing exactly the same thing. However, if you’re simply rendering with the texture, don’t manually manipulate the pixel data, instead construct your own orthographic camera, and a custom plane mesh with different UVs on the various platforms.

However, I agree with your general observation that handling of webcams isn’t quite as cross-platform as you might like.

Code attached below which I’m currently using to generate a 2D plane with identical positions but different UVs which works on iOS and Mac (haven’t yet tested windows or android):

	videoPlaneMesh = new Mesh ();
	
	videoPlaneMesh.vertices = new Vector3[] { new Vector3 (-1, -1, 0), new Vector3 (1, -1, 0), new Vector3 (-1, 1, 0), new Vector3 (1, 1, 0) };
	
	//
	// We have to use different UVs for OpenCV versus unity Webcam, because the y-axes of the image we get from the camera
	// are opposite.
	//
	if (useUnityWebcam)
	{
#if UNITY_IPHONE && !UNITY_EDITOR
		// This version is for iPad, and seems to permute X and Z
		videoPlaneMesh.uv = new Vector2[] { new Vector2 (1, 1), new Vector2 (1, 0), new Vector2 (0, 1), new Vector2 (0, 0) };
#else
		videoPlaneMesh.uv = new Vector2[] { new Vector2 (0, 0), new Vector2 (1, 0), new Vector2 (0, 1), new Vector2 (1, 1) };
#endif //UNITY_IPHONE

The solution I used was to apply a transform matrix which is processed in the shader.

Just use what you need from here:

You can just add “matrix [_Rotation]” to your shader pass in your existing shader if you don’t want to build your own shader at run-time as they did in this example.

The full transformation matrix is built for you so you don’t need to understand how the 4x4 Matrix works internally, just plug in your offset, scale and rotation values. Unity - Scripting API: Matrix4x4.TRS

The real problem I have at the moment is HOW DO YOU DETECT THE RESOLUTION OF THE CAMERA??? You can’t scale the resulting image to give the correct aspect ratio without knowing this making the feature fairly useless. Is there something I missed here? I need to build for iOS and Android, have only just hit this problem so will post back if I find a solution.

Hi Iamdain,
I’m having the same problem but am stuck in trying to create the correct inputs for the TRS function. What scale did you use to get the proper flip of the rawdata from the iOS device?
Thanks in advance for your help!
-Amy

Great ideas here!

I am new to Unity IOS. I have tried to do the matrix bit to the best of my abilities with no luck. Iamdain could you please explain your solution in more detail? From a newbies point of view maybe?

I found next solution.

  1. Create new shader:

    Shader “Custome/Web Camera Shader”
    {
    Properties
    {
    _MainTex ( “Main Texture”, 2D ) = “white” {}
    }

     SubShader 
     {
     	Pass
     	{
     		CGPROGRAM
     		
     		#pragma vertex vert
     		#pragma fragment frag
     		
     		uniform sampler2D _MainTex;
     		uniform float4x4 _Rotation;
     		
     		struct vertexInput
     		{
     			float4 vertex : POSITION;
     			float4 texcoord : TEXCOORD0;
     		};
     		
     		struct vertexOutput
     		{
     			float4 pos : SV_POSITION;
     			half2 uv : TEXCOORD0;
     		};
     		
     		
     		vertexOutput vert(vertexInput v)
     		{
     			vertexOutput o;
     			
     			float4 newPosition = mul( UNITY_MATRIX_MVP, mul(_Rotation, v.vertex) );
     			
     			o.pos = newPosition;
     			o.uv = v.texcoord;
     			
     			return o;
     		}
     		
     		float4 frag(vertexOutput i) : COLOR
     		{
     			return tex2D( _MainTex, i.uv );
     		}
     		
     		ENDCG
     	}
     }
     Fallback "Diffuse"
    

    }

  2. Apply shader to material of a plane you want to display WebCamTexture

  3. Set matrix for the material (in Start() method):

    Quaternion rotation = Quaternion.Euler (0, 90, 0);
    Matrix4x4 rotationMatrix = Matrix4x4.TRS (Vector3.zero, rotation, new Vector3(1, 1, 1));
    gameObject.renderer.material.SetMatrix (“_Rotation”, rotationMatrix);

  4. Set WebCamTexture to material texture like this (in Update() method):

    cameraScreen.renderer.material.mainTexture = webCamera.texture;

Just rotate the camera to 90 degrees along the z axis(the camera is which is rendering the webcamtexture gameobject).