Unity/Kinect with Infer.net

Hello,

I’m working on a project using Unity with Microsoft Kinect (using OpenNI/NITE) and Microsoft infer .net machine learning tool to create some body gesture recognition using Kinect. I used the sample project with NITE as a start for unity and it works fine.
The problem is when I tried to add the Infer .net dlls.

I am using Unity 3.2 pro. I copied all the dlls from the infer.net folder to the assets folder in my project. it first complained about some dlls from system (I don’t know why ?) so I copied them to the assets as well (System.deplyment.dll , system.windows.forms.dll).

Finally the compiler was ok with all the classes and types. I compile in VS2010 and everything is ok. but when I play the program in Unity it gives me this error :

    InvalidProgramException: Errors found when compiling algorithm: 
MicrosoftResearch.Infer.ModelCompiler.CompileWithoutParams (ITypeDeclaration itd, System.Reflection.MethodBase method, MicrosoftResearch.Transforms.AttributeRegistry`2 inputAttributes)

and the log is

platform assembly: C:\Program Files (x86)\Unity\Editor\Data\Mono\lib\mono\2.0\Mono.Security.dll (this message is harmless)
Platform assembly: C:\Program Files (x86)\Unity\Editor\Data\Mono\lib\mono\2.0\System.Runtime.Serialization.Formatters.Soap.dll (this message is harmless)
Platform assembly: C:\Program Files (x86)\Unity\Editor\Data\Mono\lib\mono\2.0\Mono.Posix.dll (this message is harmless)
Platform assembly: C:\Program Files (x86)\Unity\Editor\Data\Mono\lib\mono\2.0\Microsoft.JScript.dll (this message is harmless)
Compilation failed with 3 error(s)(0,0) : error CS0006: cannot find metadata file `data-0AF40020'
(0,0) : error CS0006: cannot find metadata file `data-18E90020'
(0,0) : error CS0006: cannot find metadata file `data-12CB0020'
InvalidProgramException: Errors found when compiling algorithm: 
  at MicrosoftResearch.Infer.ModelCompiler.CompileWithoutParams (ITypeDeclaration itd, System.Reflection.MethodBase method, MicrosoftResearch.Transforms.AttributeRegistry`2 inputAttributes) [0x00000] in <filename unknown>:0   at MicrosoftResearch.Infer.InferenceEngine.Compile () [0x00000] in <filename unknown>:0   at MicrosoftResearch.Infer.InferenceEngine.BuildAndCompile (Boolean inferOnlySpecifiedVars, IEnumerable`1 vars) [0x00000] in <filename unknown>:0   at MicrosoftResearch.Infer.InferenceEngine.GetCompiledInferenceAlgorithm (Boolean inferOnlySpecifiedVars, IVariable var) [0x00000] in <filename unknown>:0   at MicrosoftResearch.Infer.InferenceEngine.InferAll (Boolean inferOnlySpecifiedVars, IVariable var) [0x00000] in <filename unknown>:0   at MicrosoftResearch.Infer.InferenceEngine.Infer[VectorGaussian] (IVariable var) [0x00000] in <filename unknown>:0   at BPM.InferW (System.Collections.Generic.List`1[] xValuesData) [0x0005e] in G:\study\kinect project\Unity kinect\Assets\scripts\BPM.cs:252   at BPM.Train (System.Collections.Generic.List`1[] xValuesData) [0x00069] in G:\study\kinect project\Unity kinect\Assets\scripts\BPM.cs:231   at BPMtest.Test_BPM (Int32 nClass, Int32 totalFeatures, Double noisePrec, System.String fileName, MicrosoftResearch.Infer.Maths.Vector[] testData) [0x00048] in G:\study\kinect project\Unity kinect\Assets\scripts\BPMtest.cs:58   at BPMtest.Start () [0x000c1] in G:\study\kinect project\Unity kinect\Assets\scripts\BPMtest.cs:38 

Any Ideas ?

The error unhappily does not say much as such. You won’t be able to avoid to ask on the infer related pages (after checking its docs) on why it would fire that exception ie what could lead there.

Once you know that we can likely point you to the root on the unity side that lead there

Oh I missed that part that you really copied the Windows.Forms.

You can’t use Windows.forms with Unity at all and as such also no library that has a relationship to it.
Not only is this library you copied for MS.NET and not even mono, but also it has various deep system dependencies which won’t work out with unity.

Question is also why you need it cause Windows Forms is for the Windowing system etc, thats not a “just to be used” assembly aspect, which would imply that one of the libraries or an aspect of it is meant to show windows or dialogs. perhaps there is a library to “just use”?