Adaptive soundtrack/dynamic score

What's the best way to implement an adaptive soundtrack/dynamic score in Unity?

The way I approach it in the tutorial of my game Traces of Illumination is to have short loops (4 bars) that I can exchange. Basically, each "step" in the tutorial has it's own musical "style" and it starts with something very simple (just a beat) and adds different instruments and variations (and effects) while the tutorial progresses. So, I have one loop "only drums", and then another loop "drums + bassline", and another one "drums + bassline + some synth pad", another one "variation of drums + bassline + some synth pad" and so on.

Since Unity 2.6 this works without any clicking or stutters for me (before 2.6, I needed to play an extra sound when switching from one loop to another to mask any "clicking" sounds that occured due to the inaccuracy of the underlying sound-engine, that "masking" is no longer necessary; I still do have a very short crossfade-phase, so I don't just "switch" but fade very quickly from one track to the other within a split second).

One thing you need is the exact tempo of the song-parts; I use that for synchronizing certain events in the game (e.g. the tutorial instructions appearing or enemies spawning) with the beat of the music. That way, you can even write your own very rudimentary "sample-sequencer" so that you can, for instance, have drum-rolls leading into another section of your song (those would be added "on top" much like in the approach I outline below). In this approach, I'm usually only playing one fully pre-arranged track at a time, and I'm switching tracks only at the beginning/end of 4 bars (which is my "loop-length"). So, the music doesn't react immediately but usually with a little delay.

An alternative or more advanced approach I don't have practical experience with, yet (but that's should certainly work as well or even better, depending on your specific needs): Have a couple of loops of different tracks (e.g. instruments, or variations of the same theme) that you can layer. E.g. you could have the "rhythm-layer" (or a couple of versions of that), and a "bassline layer" and some "melody layers", or even "effect layers". Those would be "designed" (well, composed, produced) in a way that you can create an actual arrangement during runtime in the game. IMHO, that gives you quite a bit more flexibility because you can fade layers in and out as you wish. It's nice to have the extra-flexibility of this approach but it very likely is more of a challenge for the processor (depending on whether you use compressed audio and on how many tracks you want to play simultaneously). Also, implementing the relevant "sound framework" would be more complex, and then using it - while giving you much more possibilities - also naturally will be more challenging (you may end up writing your own editor for this to be really convenient to work with).

Obviously, you'll either need to be the one creating the music or need to work very closely with someone who understands these kinds of approaches. And, in any case, you'll need to do some serious custom programming (which requires both programming skills and some musical skills).

It's difficult to get it exactly right as you don't have accurate control over buffering. The result is that you tend to get little stutters when jumping in the sound track. For the best quality you probably are best of making a specialized plugin that handles the music track so you can have accurate control over buffers.

As my experimenting with audio tracks is from a while ago, things could have improved but at least the behavior was that reading back the current time offset of an audio stream you'll get the position in the audio stream that's written to the buffer instead of what's currently playing

To get more reliable behavior consider using uncompressed audio or use decompress on load. Use two audio streams and simple fade-in/fade-out coroutines to nicely adapt to the new audio point (so, you first set volume to 0 for second audio stream, set track and position, start to play, then fade out first track and simultaneously fade in second).

This trick doesn't work on iphone (unless you work uncompressed) as iphone only support 1 compressed audio stream playing.