GameObject facing hit.point doesn't always work

I’ve got a script that makes a character turn to face the hit.point then move towards it while playing a walk animation. This all works great, most the time. Sometimes, when it is commanded to walk somewhere, it will walk backwards or diagonally. This usually happens if you press different areas quickly or if you press close behind the character when its stationary.
The piece of relevant code is as follows:

private var pointToGo : Vector3 = Vector3.zero

function Update () 
{
       if(Input.GetMouseButtonDown(0))
       {
            ray = Camera.main.ScreenPointToRay(Input.mousePosition);
            if(Physics.Raycast(ray, hit)){
            
                if(hit.transform.name == "NavBar")
                {
           Zombie.transform.rotation = Quaternion.LookRotation(transform.position - hit.point);

I’ve put the function before the move to hit.point and the animations so I’m not sure why it is doing this.
Anyone know the answer?

Check to make sure that you are hitting NavBar (debug the name of what youve hit) if its just for moving perhaps ignore all layers with the raycast except NavBar. You could also make this block of code its own method incase you need to use it in something else. Tried to leave this as a comment, phones not seeing the option. Let me kmow if this is the issue if not I’ll try to help out more.