Whichever way you do this, you probably want to use a gradually changing set of values rather than affecting the movement by random numbers each frame, as this will usually result in random jittery movement as opposed to smooth loops and curls. Both my examples below use the Sin function to give this kind of smoothly changing movement.

Also I'm assuming you only want to modulate the movement in the x-z plane as is usual with many games.

One way you could approach this is to modulate the target position that your 'drunk guy' is heading for. For example, in FixedUpdate, you could calculate a dummy target position which is offset from your real target position by an amount proportional to the current distance from the target. This will give you quite a uniform left-to-right swerving which reduces as it approaches the target.

// adjust these to taste:
float swerveAmount = 0.5f;
float swerveSpeed = 2; 

void FixedUpdate()
{
    Vector3 delta = target - transform.position;
    Vector3 deltaP = new Vector3(-delta.y,0,delta.x);
    Vector3 offset = deltaP * swerveAmount * Mathf.Sin(Time.time * swerveSpeed);
    Vector3 swerveTarget = target + deltaP;

    // then apply forces to head towards 'swerveTarget'!
}

Another method which would give you more random looping and curling might be to use two objects:

1 - A hidden dummy object which moves straight for the target, as you've suggested that you can already do (if you want to use forces, this could be a rigidbody with no collider or renderer). We'll call this the 'seeker'.

2 - A separate 'drunk guy' rigidbody which is visible, which always heads for a point which is offset from the 'seeker' object by a varying amount.

For example, on the 'drunk guy' object, you would have something like this:

// adjust these to taste:
float swerveAmount = 3f;
float swerveSpeed = 2; 

// distance from target at which swerving starts to reduce
float thresholdDist = 20;  

void FixedUpdate()
{
    float dist = (target.position - seeker.position).magnitude;
    float reduce = Mathf.Clamp01(dist / thresholdDist);
    float offsetX = Mathf.Sin(Time.time * swerveSpeed) * swerveAmount * reduce;
    float offsetZ = Mathf.Cos(Time.time * swerveSpeed) * swerveAmount * reduce;
    Vector3 offset = new Vector3(offsetX, 0, offsetZ); 
    Vector3 swerveTarget = seeker.position + offset;

    // then apply forces to head towards 'swerveTarget'!
}

(these are illustrative untested code snippets. if you encounter errors, please leave a comment and I'll fix them!)

You could apply a random force that diminishes as he gets closer.

float DiminishingForce(float distance)
{
    return Mathf.Clamp(0.0f, MAX_FORCE, distance * FORCE_RATIO);
}

Just multiply your random force by that. Of course, you'll need to define MAX_FORCE and FORCE_RATIO.