tl;dr : This issue is fixed in Unity 3.5.2.

Hello,

I'm trying to display a swimming pool as the 3D augmentation of an augmented reality iOS application, using the QCAR SDK by Qualcomm. I use the Daylight Water prefab of the Water Pro package to render water.

At this point, the app is set up as suggested by a QCAR developer on this thread:

• The ARCamera clears the background with a transparent black color.
• A depth mask shader is used to hide the outer walls of the pool.
• A background plane, child of ARCamera, is textured with the video. It uses the simple Unlit/Texture shader, with only one change to put the plane in the Background render queue. This is done to prevent the depth mask from masking the video.

A project demonstrating the bug is linked later in this post. For anyone willing to try this, a marker needs to be printed and filmed at for the transformation to appear. The marker is available at these two locations:

• UnityProject/Assets/Editor/QCAR/ForPrint/target_stones_A4.pdf
• UnityProject/Assets/Editor/QCAR/ImageTargetTextures/StonesAndChips/stones_scaled.jpg

I run the app on the iPad 2 and point the camera at the marker. The augmentation appears and the water rendering works for a few seconds. Then it feels like the water reflection is not updated anymore, and an OpenGL error is displayed each frame in Xcode's console:

• OpenGLES error 0x0506 in /UnityProject/iOSbuild/Classes/iPhone_GlesSupport.cpp:204

In pseudo-code, this is the call to:

glDiscardFramebufferEXT(GL_FRAMEBUFFER_OES, 2, {GL_DEPTH_ATTACHMENT_OES, GL_STENCIL_ATTACHMENT_OES})

that returns:

INVALID_FRAMEBUFFER_OPERATION_EXT

in the Unity code function:

void PreparePresentSurfaceGLES(EAGLSurfaceDesc* surface).

I'm using Unity 3.4.2f3 and Xcode 4.2. The QCAR developer tested the app on Unity 3.5 beta and the bug also shows up. I can't reproduce it on an iPod Touch though.

Sometimes a memory warning is issued before this OpenGL error appears, but not at every run: there is a leak issue to resolve as well (it shows up when profiling with Instruments). But at this point I'm not sure that it's related to the water bug because the leaks show up without any 3D augmentation.

2012 february 27th Edit:

Qualcomm has just released a new SDK and Unity extension (v1.5.9 for iOS), now branded as Vuforia. I used their updated "Background Texture Access" sample but the bug is still there.

If "Depth Only" is selected in the inspector as the Clear Flags for the ARCamera, the water reflection is not cleared from the start (ie as soon as the 3D augmentation shows up): I have added a capsule on top of the water and a trail effect appears as you move the iPad around. Selecting "Solid Color" instead fixes this. But in both cases, reflection and refraction stop being updated after a few seconds.

Here is the sample ReadMe file:

The BackgroundTextureAccess sample shows two approaches to rendering the camera image in Unity:

1) Using a separate orthographic camera, render the camera image with a special shader to produce a negative greyscale effect that warps the image in response to touch events. Note that this shader only works with OpenGL ES 2.0 selected. This is the default mode.

2) Using the ARCamera, render the camera image as a plane at the far end of the perspective frustum. To enable this mode, deactivate the BackgroundCamera (and its child object) and activate the VideoBackground child of the ARCamera.

I've used the second approach in the sample linked in this post.

2012 february 28th Edit:

I've updated the sample with the latest Unity release (v3.5.0f5). Unfortunately it didn't solve the bug. The Unity iOS C++ code has changed a bit: the OpenGL error is now located at line 224:

• OpenGLES error 0x0506 in /UnityProject/iOSbuild/Classes/iPhone_GlesSupport.cpp:224

2012 march 01st Edit:

I've profiled the above sample with the Activity Monitor template of Xcode's Instruments. The memory footprint starts at about 32Mb, but steadily increases by about 1Mb per minute.

If I disable the Vuforia-related game objects from the scene and replace them with a Unity camera pointing at the water, the app's memory footprint is about 25Mb but doesn't increase over time, as expected.

If I disable the water 3D augmentation or replace it with a "big" 3D scene (like the level geometry of the 3D platform game tutorial), the memory footprint still increases, but only by about 0.13Mb per minute.

Conclusion: there seems to be an issue with Vuforia regardless of the augmentation, but using the Water Pro package as the augmentation makes the issue worse.

2012 april 26th Edit:

I've updated the sample with the latest Unity release (v3.5.1f2). It's temporarily available here. The bug is still there, but it might be fixed in the next release. See this thread.

Memory-related information in this post should probably be disregarded, it might just be that garbage collection is still magic to me... See this other thread.

Thank you for reading this far. :)