Using numpy in Unity

I’m working on a scientific project and at the moment we are in the process of integrating our Unity codebase with our python codebase. Python is used extensively for the back-end scientific work in the project and it would be impossible to rewrite the the entire base in C# for unity. Essentially what we are trying to do is take emg data from a Myo armband and perform Linear Discriminant Analysis (LDA) feature extraction and classification on the data with the Numpy python package.

At the moment, I have been able to get regular python scripts working and interacting with the hardware in the project through Unity. I’m using IronPython hosted on Microsoft’s Dynamic Language Runtime environment in C#. This is what an example C# script looks like:

using UnityEngine;
using System;
using System.Threading;
using Microsoft.Scripting.Hosting;
using IronPython.Hosting;
using IronPython.Runtime;
using IronPython.Runtime.Exceptions;
using IronPython.Runtime.Operations;


public class PythonScriptRunner : MonoBehaviour {

    private PythonScriptRunner() {
        //constructor to allow an instance of this object to be passed to the python file
        //the static variable runThread is to be used to control running/quitting the python thread 
    }

    public string scriptPath = "Assets\\Scripts\\Python\	est.py";
    public GameObject myo = null;
    private ThalmicMyo thalmicMyo;

    private Thread thread;
    public static bool runThread;


    void Start() {
        //start myo band
        thalmicMyo = myo.GetComponent<ThalmicMyo>();

        Debug.Log("Starting Python Script Thread from " + scriptPath);
        runThread = true;
        thread = new Thread(RunScript);
        thread.Start();
    }

    // Run in separate thread to prevent freezing the game
    private void RunScript ()
    {

        // create the engine  
        var ScriptEngine = Python.CreateEngine();
        var ScriptScope = ScriptEngine.CreateScope();
        // load the assemblies for unity, using the types of GameObject  
        // so we don't have to hardcoded paths  
        ScriptEngine.Runtime.LoadAssembly(typeof(GameObject).Assembly);
        var ScriptSource = ScriptEngine.CreateScriptSourceFromFile(scriptPath);

        //set local variables in the python session
        ScriptScope.SetVariable("myo", myo);
        ScriptScope.SetVariable("thalmicMyo", thalmicMyo);
        ScriptScope.SetVariable("PythonScriptRunner", new PythonScriptRunner());
        //self reference object for sending stop running command to python script with global runThread var


        try
        {
            //run python script
            ScriptSource.Execute(ScriptScope);
            Debug.Log("Python script ran succesfully");
        }
        catch(Exception exception) {
            //basic error output for python script. Doesn't always say what line the error is on.
            Debug.LogError(exception);
            Debug.LogError(ScriptEngine.GetService<ExceptionOperations>().FormatException(exception));

        }
    }

    void OnApplicationQuit()
    {
        // If the thread is still running, we should shut it down,
        // otherwise it can prevent the game from exiting correctly.
        runThread = false;
        Debug.Log("Attempting to close thread.");
        thread.Join();
        Debug.Log("Successfully closed thread.");

    }
}

The python file that this script runs would ideally be the main driver that controls the analysis of the myo data using LDA classification. My question is, is it possible to pull numpy and any of its dependencies into Unity and use in this same way? Perhaps there is some portable way to access the code. Also, eventually we will be using the scikit-learn python package to generate training data which will be used in the LDA classification, so is it also possible to use the scikit-learn package in the same way as the numpy package? Thanks for any help.

I have not been able to verify this but this link might be able to assist.

My reasoning is that while you’re unlikely to find Unity3d specific help for this (yet), it’s likely to be something people working with IronPython have encountered and achieved - and in theory if you get it working there, it shouldn’t be impossibly hard to get it working in Unity after that.

@sonic122995
@runonthespot

Hi, different question: if I have a 3-d model of a human in numpy, can I make that a model in Unity? Apologies if the word “model” is not Unity-specific jargon; I’m new to Unity and C#. Thanks!,@sonic122995
@runonthespot

This may be a different question than yours, but I’m curious if you’ve made any headway on this problem:

I have a 3-d Numpy array representation of a human being, and I would like to turn that into a Unity model. Apologies if my terminology is incorrect; I’m new to Unity and C#. Is there an easy way to do this?