Serialisation... What can, what can't?

So I’ve been creating my own editor window over the past week now. Apart from my head feeling like its going to explode, I have pretty much doubled my knowledge with c#. I’ve been all over the place, reading information from various sources. The biggest pain for me was seriali(z)sation. This took me the longest to get my head around. I do get it, I understand what happens and why it happnes. But there are some unity specific things that I’m still struggling to grasp:

Is there two types of serialisation? One when scripts are recompiled, and one when you enter/exit play mode? Let’s just say there are for the purpose of my example. Type 1 being when scripts are recompiled and type 2 being when you enter/exit play mode.

Firstly, with monobehaviors, private variables aren’t serialised. Makes sense, you’d only really want your public variables to return to their original state when you exit play mode. However, I run a test which made me think there is two types. Let’s say you create a private int member in a script. Give it a value of 0, and debug that every frame. Click play, and it will debug 0. Then go back into your script and give the variable a value of 1 where it is declared. Then go back into unity and after in recompiles, it will still debug the number 0. Then, if you unplay and replay, the number 1 will now be debugged. This is why I believe there to be two types of serialisation. The private variable is being serialised on recompilation but not upon play/exit. Adding [SerializeField] then makes the variable be serialized in both cases. So is this actually the case, or is there something I am missing?

My second question is about editor scripts, static variables and delegates. So with custom editor scripts, private variables are serialised as if they have the [SerializeField] attribute attached to them by default. This is handy because you can then use private variables with your GUI without having to serialse every single one of them. But static variables are not serialised at all. Is this not possible at all? It would be really useful if it was as I have to read from disk every time I open my editor window to restore the last used settings for my tool. If I could just store the settings in static variables and write them to disk when you save the scene, it would be much easier. But because static variables arent serialised, they would be reset whenever there is a recompile or play mode is entered/exited. Surely unity internally serialises static variables. For example, project settings are static variables and they survive serialisation and I’m pretty sure they arent written to disk ever time you close a settings tab.

It’s also a pain that Dictionarys, delegates etc are not serialised. For my tool, I have a dictionary that could potentially have a lot of data in it. And the way this data is collected is by looping through every active gameobject. Now I’ve got an efficient solution so I only need to build this Dictionary once, and then add and remove items as needed. However, because Dictionarys aren’t serialised, I have to rebuild the Dictionary every time there is a recompile or play/exit. The same goes for delegates, I’m using SceneView.onSceneGUIDelegate to change the scene view with my custom window and this has to be re added everytime there is a recompile or a blah blah blah

So basically, is there any way to serialise static variables, delegates, dictionarys etc? It would make sense to be able to at least serialise static variables, surely this is how you keep settings loaded in memory without having to keep reading/writing to disk?

Finally, I just want to say that [InitializeOnLoad] is very misleading. It should be named [InitializeEveryTimeYouRecompileOrEnterSlashExitPlayMode]

Thank you so much to anyone that made it to here!!!
And double thanks to anyone that answers!!!

So many questions bundled in one. You should have divided them to small precise questions. So don’t expect a short answer.

Type 1 being when scripts are
recompiled and type 2 being when you
enter/exit play mode.

There’s no such thing as ‘2 types of serializations’ - It’s just that serialization occur in multiple situations, some are editor-only and some are run-time. When you enter/exit play mode, or when you make a modification to one of your scripts and save it an assembly reload happens

What happens when an assembly is
reloaded? When you enter / exit play
mode or change a script Unity has to
reload the mono assemblies, that is
the dll’s associated with Unity.

On the user side this is a 3 step process:

1- Pull all the serializable data out of managed land, creating an internal representation of the data on the C++ side of Unity.
2- Destroy all memory / information associated with the managed side of Unity, and reload the assemblies.
3- Reserialize the data that was saved in C++ back into managed land.

From Unity’s serialization blog post. For more information refer to Tim Cooper’s Unite video on the subject.

What can, what can’t?

From the SerializeField documentation reference:

The serialization system used can do
the following:

  • CAN serialize public nonstatic fields (of serializable types)
  • CAN serialize nonpublic nonstatic fields marked with the
    [SerializeField] attribute.
  • CANNOT serialize static fields.
  • CANNOT serialize properties.

Your field will only serialize if it
is of a type that Unity can serialize:

Serializable types are:

  • All classes inheriting from UnityEngine.Object, for example
    GameObject, Component, MonoBehaviour,
    Texture2D, AnimationClip… - All basic
    data types like int, string, float,
    bool. - Some built-in types like
    Vector2, Vector3, Vector4, Quaternion,
    Matrix4x4, Color, Rect, LayerMask… -
    Arrays of a serializable type
  • List of a serializable type (new in Unity2.6)
  • Enums

I encourage you to get ILSpy, and check out the Unity.SerializationLogic.dll in the \Unity\Editor\Data\Managed you’ll see things for yourself and understand them much better.

For example, here’s how Unity determines if a collection type is supported or not:

public static bool IsSupportedCollection(TypeReference typeReference)
{
	return (typeReference is ArrayType || CecilUtils.IsGenericList(typeReference)) && (!typeReference.IsArray || ((ArrayType)typeReference).Rank <= 1) && UnitySerializationLogic.IsTypeSerializable(CecilUtils.ElementTypeOfCollection(typeReference));
}

Which means: Only 1-dimensional arrays/lists whose element type is serializable are supported by the serialization system.

Moving on, what determines a serializable type?

private static bool IsTypeSerializable(TypeReference typeReference)
{
	return !typeReference.IsAssignableTo("UnityScript.Lang.Array") && (UnitySerializationLogic.IsSerializablePrimitive(typeReference) || typeReference.IsEnum() || UnitySerializationLogic.IsUnityEngineObject(typeReference) || UnityEngineTypePredicates.IsAnimationCurve(typeReference) || UnityEngineTypePredicates.IsGradient(typeReference) || UnityEngineTypePredicates.IsRectOffset(typeReference) || UnityEngineTypePredicates.IsGUIStyle(typeReference) || UnitySerializationLogic.ShouldImplementIDeserializable(typeReference));
}

public static bool WillUnitySerialize(FieldDefinition fieldDefinition, TypeResolver typeResolver)
{
	return fieldDefinition != null && !fieldDefinition.IsStatic && !UnitySerializationLogic.IsConst(fieldDefinition) && !fieldDefinition.IsNotSerialized && !fieldDefinition.IsInitOnly && !UnitySerializationLogic.ShouldNotTryToResolve(fieldDefinition.FieldType) && UnitySerializationLogic.IsFieldTypeSerializable(typeResolver.Resolve(fieldDefinition.FieldType)) && !UnitySerializationLogic.IsDelegate(typeResolver.Resolve(fieldDefinition.FieldType)) && (fieldDefinition.IsPublic || UnitySerializationLogic.HasSerializeFieldAttribute(fieldDefinition) || UnitySerializationLogic.ShouldHaveHadAllFieldsPublic(fieldDefinition)) && !(fieldDefinition.FullName == "UnityScript.Lang.Array");
}

As you can see, the “What can, what can’t” is clearly stated in plain text.


Now for serializing things that Unity can’t:

For delegates, see this answer. For dictionaries, you can use the new ISerializationCallbackReciever. If you’re willing to pay, checkout Full Inspector. It uses custom serialization for pretty much everything (pretty much, the best way to go about serialization is to make your own serialization system using protobuf or such)

Here’s a simple way I use to serialize some types that Unity can’t (using BinaryFormatter, but you could modify it easily to use a dif/better serializer):

public static class SerializationHelper
{
	/// <summary>
	/// Serializes 'value' to a string, using BinaryFormatter
	/// </summary>
	public static string SerializeToString<T>(T value)
	{
		using (var stream = new MemoryStream())
		{
			(new BinaryFormatter()).Serialize(stream, value);
			stream.Flush();
			return Convert.ToBase64String(stream.ToArray());
		}
	}

	/// <summary>
	/// Deserializes an object of type T from the string 'data'
	/// </summary>
	public static T DeserializeFromString<T>(string data)
	{
		byte[] bytes = Convert.FromBase64String(data);
		using (var stream = new MemoryStream(bytes))
			return (T)(new BinaryFormatter()).Deserialize(stream);
	}
}

/// <summary>
/// A wrapper to serialize classes/structs that Unity can't
/// </summary>
public abstract class SerializedClass<T>
{
	[SerializeField]
	private string serializedData = string.Empty;

	protected T _value;

	public SerializedClass() { }

	public SerializedClass(T value)
	{
		Value = value;
	}

	public bool IsNull { get { return this == null || Value == null; } }

	public T Value
	{
		get
		{
			if (_value == null)
				_value = Deserialize();
			return _value;
		}
		set
		{
			if (_value == null || !_value.Equals(value))
			{
				_value = value;
				Serialize();
			}
		}
	}

	public string Data { get { return serializedData; } }

	public virtual void Serialize()
	{
		serializedData = _value == null ?
			string.Empty : SerializationHelper.SerializeToString<T>(_value);
	}

	protected virtual T Deserialize()
	{
		return string.IsNullOrEmpty(serializedData) ?
			default(T) : SerializationHelper.DeserializeFromString<T>(serializedData);
	}
}

Then you could easily have serializable stuff, like System.Type:

[Serializable]
public class SerializedType : SerializedClass<Type>
{
}

Dictionaries (not the best approach I know…):

[Serializable]
public class SerializedStringBoolDictionary : SerializedClass<Dictionary<string, bool>>
{
	// it's important to remember not to create an empty constructor and new up a dictionary
	// because unity will call that constructor upon deserialization and so you'll end up overwriting your data

	public SerializedStringBoolDictionary(Dictionary<string, bool> dic)
		: base(dic)
	{
	}

	public bool TryGetValue(string key, out bool value)
	{
		return Value.TryGetValue(key, out value);
	}

	public bool this[string key]
	{
		get { return Value[key]; }
		set
		{
			Value[key] = value;
			Serialize();
		}
	}
}

Usage:

SerializedStringBoolDictionary dic = new SerializedStringBoolDictionary(Dictionary);

Now for serializing static state, I rarely require this but if I do, it’d be for Settings objects which are usually ScriptableObjects I usually follow this pattern:

[Serializable]
public class Settings : ScriptableObject
{
    [SerializeField]
    private float floatValue;

    [SerializeField]
    private string stringValue;

    public float FloatValue { get { return floatValue; } }
    public float StringValue { get { return stringValue; } }

    public static float sFloatValue { get { return Instance.FloatValue; } }
    public static string sStringValue { get { return Instance.StringValue; } }

    private static Settings instance;
    private static Settings Instance
    {
        get
        {
            return LazyLoadScriptableAsset(ref instance, settingsPath, true);
        }
    }

    public static T LazyLoadScriptableAsset<T>(ref T value, string path, bool log) where T : ScriptableObject
    {
			if (value == null)
			{
				path = path.Replace('/', '\\'); // Normalize the path, could be the other way around too
				value = AssetDatabase.LoadAssetAtPath(path, typeof(T)) as T;
				if (value == null)
				{
					if (log) Debug.Log("No asset of type `" + typeof(T) + "` was found - creating new");
					value = CreateScriptableObjectAsset<T>(path);
				}
			}
			return value;
    }

    private static T CreateScriptableObjectAsset<T>(string path) where T : ScriptableObject
    {
			T so = CreateInstance<T>();
			if (Path.GetExtension(path) != ".asset")
				path += ".asset";
			AssetDatabase.CreateAsset(so, path);
			return so;
		}
    }
}

You know have a Settings object that could have its value persist and serialize correctly with static access (if required)

I’m never a fan of static/global access, use this solution as a last resort/worse cast solution. Most the times you don’t need statics, why not just have normal Settings class that you could instantiate and pass around? Statics discourage code re-usability, encourage dependency hiding and makes your code harder to maintain.


As for InitializeOnLoad It’s kind of vague I agree, instead of “Load”, I think it should be “AssemblyReload”

Some good observations there. Lets deal with them one at a time.

Give it a value of 0, and debug that every frame. Click play, and it will debug 0. Then go back into your script and give the variable a value of 1 where it is declared. Then go back into unity and after in recompiles, it will still debug the number 0. Then, if you unplay and replay, the number 1 will now be debugged.

What you’re experiencing is the fact that Unity cannot ‘hotswap’ code. You need to exit playmode and restart to see code changes reflected in the running player. There are a few exceptions to this, but as a general rule you should never expect a code change to manifest itself until after restarting the player. There are many cases where very strange behavior will result. Finally, you’re not seeing the result of serialization, but rather member variable initialization. When you explicitly set a member variable’s value it will be initialized to that value whenever new instances of that script are instantiated (unless its static).

So with custom editor scripts, private variables are serialised as if they have the [SerializeField] attribute attached to them by default.

This is just not true. I don’t know exactly how you figured this, but what you’re likely experiencing is that the editor script never lost scope, and so it’s members were never reset. The discrepancy between these and the static variables is an interesting phenomenon to note as well, as it doesn’t make any sense in a general scenario. I think you have some code running that is doing things that you might not be aware of.

But static variables are not serialised at all. Is this not possible at all? It would be really useful if it was as I have to read from disk every time I open my editor window to restore the last used settings for my tool.

If you’re reading them from disk then you, basically, should be able to answer your own question. (Yes, any data is serializable, and you’re successfully doing so.)

For example, project settings are static variables and they survive serialisation and I’m pretty sure they arent written to disk ever time you close a settings tab.

What gives you this idea?

It’s also a pain that Dictionarys, delegates etc are not serialised.

You can serialize dictionaries, but think about why delegates cannot be for a moment. They’re not data (well, ok, they are, but…) they’re running instances of classes. You can serialize the state of an object, but you cannot serialize the object itself.

Finally, I’m not sure you and I are really thinking the same thing when we use the word ‘serialize’, so lets discuss that one real quick. When I think ‘serialize’, I think actually writing an objects state to disk using my a mechanism of my own devising. You can also leave serialization to Unity by marking fields as serializable, and by using Editor/PlayerPrefs. When you ask,

…is there any way to serialise static variables, delegates, dictionarys etc?

Yes, of course (as I pointed out to you about dictionaries already, but no delegate invocation lists…what would be the point?). I’ll tell you what though. Others may give you answers as well, but this question is done. If you have some points of clarification that you’d like addressed, by all means, add comment to my (and others’) answer(s). However, now you should be at the point where you can ask concise questions targeting specific elements of your editor script(s).

Happy coding.