x


Water Pro reflection bug in augmented reality iOS app using Vuforia (QCAR) SDK

tl;dr : This issue is fixed in Unity 3.5.2.

Hello,

I'm trying to display a swimming pool as the 3D augmentation of an augmented reality iOS application, using the QCAR SDK by Qualcomm. I use the Daylight Water prefab of the Water Pro package to render water.

At this point, the app is set up as suggested by a QCAR developer on this thread:

  • The ARCamera clears the background with a transparent black color.
  • A depth mask shader is used to hide the outer walls of the pool.
  • A background plane, child of ARCamera, is textured with the video. It uses the simple Unlit/Texture shader, with only one change to put the plane in the Background render queue. This is done to prevent the depth mask from masking the video.

A project demonstrating the bug is linked later in this post. For anyone willing to try this, a marker needs to be printed and filmed at for the transformation to appear. The marker is available at these two locations:

  • UnityProject/Assets/Editor/QCAR/ForPrint/target_stones_A4.pdf
  • UnityProject/Assets/Editor/QCAR/ImageTargetTextures/StonesAndChips/stones_scaled.jpg

I run the app on the iPad 2 and point the camera at the marker. The augmentation appears and the water rendering works for a few seconds. Then it feels like the water reflection is not updated anymore, and an OpenGL error is displayed each frame in Xcode's console:

  • OpenGLES error 0x0506 in /UnityProject/iOSbuild/Classes/iPhone_GlesSupport.cpp:204

In pseudo-code, this is the call to:

glDiscardFramebufferEXT(GL_FRAMEBUFFER_OES, 2, {GL_DEPTH_ATTACHMENT_OES, GL_STENCIL_ATTACHMENT_OES})

that returns:

INVALID_FRAMEBUFFER_OPERATION_EXT

in the Unity code function:

void PreparePresentSurfaceGLES(EAGLSurfaceDesc* surface).

I'm using Unity 3.4.2f3 and Xcode 4.2. The QCAR developer tested the app on Unity 3.5 beta and the bug also shows up. I can't reproduce it on an iPod Touch though.

Sometimes a memory warning is issued before this OpenGL error appears, but not at every run: there is a leak issue to resolve as well (it shows up when profiling with Instruments). But at this point I'm not sure that it's related to the water bug because the leaks show up without any 3D augmentation.


2012 february 27th Edit:

Qualcomm has just released a new SDK and Unity extension (v1.5.9 for iOS), now branded as Vuforia. I used their updated "Background Texture Access" sample but the bug is still there.

If "Depth Only" is selected in the inspector as the Clear Flags for the ARCamera, the water reflection is not cleared from the start (ie as soon as the 3D augmentation shows up): I have added a capsule on top of the water and a trail effect appears as you move the iPad around. Selecting "Solid Color" instead fixes this. But in both cases, reflection and refraction stop being updated after a few seconds.

Here is the sample ReadMe file:

The BackgroundTextureAccess sample shows two approaches to rendering the camera image in Unity:

1) Using a separate orthographic camera, render the camera image with a special shader to produce a negative greyscale effect that warps the image in response to touch events. Note that this shader only works with OpenGL ES 2.0 selected. This is the default mode.

2) Using the ARCamera, render the camera image as a plane at the far end of the perspective frustum. To enable this mode, deactivate the BackgroundCamera (and its child object) and activate the VideoBackground child of the ARCamera.

I've used the second approach in the sample linked in this post.


2012 february 28th Edit:

I've updated the sample with the latest Unity release (v3.5.0f5). Unfortunately it didn't solve the bug. The Unity iOS C++ code has changed a bit: the OpenGL error is now located at line 224:

  • OpenGLES error 0x0506 in /UnityProject/iOSbuild/Classes/iPhone_GlesSupport.cpp:224

2012 march 01st Edit:

I've profiled the above sample with the Activity Monitor template of Xcode's Instruments. The memory footprint starts at about 32Mb, but steadily increases by about 1Mb per minute.

If I disable the Vuforia-related game objects from the scene and replace them with a Unity camera pointing at the water, the app's memory footprint is about 25Mb but doesn't increase over time, as expected.

If I disable the water 3D augmentation or replace it with a "big" 3D scene (like the level geometry of the 3D platform game tutorial), the memory footprint still increases, but only by about 0.13Mb per minute.

Conclusion: there seems to be an issue with Vuforia regardless of the augmentation, but using the Water Pro package as the augmentation makes the issue worse.


2012 april 26th Edit:

I've updated the sample with the latest Unity release (v3.5.1f2). It's temporarily available here. The bug is still there, but it might be fixed in the next release. See this thread.

Memory-related information in this post should probably be disregarded, it might just be that garbage collection is still magic to me... See this other thread.

Thank you for reading this far. :)

more ▼

asked Feb 21, 2012 at 10:21 PM

devGuillaume gravatar image

devGuillaume
173 9 10 15

Hi There,

same error I got after after 20 min of simulation of my application need help.

Feb 23, 2012 at 03:34 PM gap

Would it be possible to share your application?

Feb 24, 2012 at 09:47 AM devGuillaume

Gidday,

I'm having exactly the same issue, after about 30 seconds there's a barrage of errors and all of the reflective stuff falls over. One of the ways I solved the "image blurring" was to force a Skybox as the background for the reflection camera - it works relatively well. Here I was thinking I was alone with this issue!

Mar 02, 2012 at 12:43 AM MRKane

Hi MRKane,

I may sound thick but could you be more specific please? :) Does the "image blurring" refer to the reflective stuff falling over, or to some other problem? Is the "reflection camera" the ARCamera? By "forcing a Skybox as the background", do you mean adding a skybox to the scene through Edit -> Render Settings, setting the ARCamera Clear Flags to Skybox, or something else?

Thank you.

Mar 02, 2012 at 08:54 AM devGuillaume

Gidday,

I've had more time to get into the problem here (but there's no solution yet so don't get your hopes up). When using the ARCamera there's no clearing going on. You talk about it on the 27th edit. By tweaking the part of the code that checks for a skybox, and forcing one (sorry I don't have my script here, but it was along the lines of internally assigning a SkyBox - will update in a day or two) you can get rid of the trails without loosing the AR backgrounds - it puts the skybox there instead.

I've still not solved the problem of it stopping for no apparent reason. I've tried putting in script to clear the image (as it necessary on the xBox) but that doesn't solve the issue either.

I'm pretty sure it's not your scene that's causing the issue...as mine is only a medium-poly boat!

Mar 04, 2012 at 08:47 AM MRKane
show all comments (comments are locked)
10|3000 characters needed characters left

1 answer: sort voted first

This issue is fixed in Unity 3.5.2. See this post for details.

more ▼

answered May 28, 2012 at 08:58 AM

devGuillaume gravatar image

devGuillaume
173 9 10 15

(comments are locked)
10|3000 characters needed characters left

How does this have -1 answers?

more ▼

answered Feb 28, 2012 at 09:59 PM

dibonaj gravatar image

dibonaj
268 40 35 36

(comments are locked)
10|3000 characters needed characters left
Your answer
toggle preview:

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Topics:

x2905
x456
x27

asked: Feb 21, 2012 at 10:21 PM

Seen: 3798 times

Last Updated: Jul 14 at 04:28 PM