Why does my vector pan my character slightly to the side?

I’m modifying the default FirstPersonController script to have a jetpack, and I’ve got it mostly down. However, there’s an annoying bug where if I fly at an angle to the x or z axes it’ll make me strafe slightly to the side.

I think it’s because I clamp the x and z distances. Please check my code and tell me what’s wrong.

I did notice that when I set the FlyModifier to a very small number like 0.1, I did stop the strafing, but it took forever to start building momentum.

    // always move along the camera forward as it is the direction that it being aimed at
    Vector3 desiredMove = transform.forward*m_Input.y + transform.right*m_Input.x;

    // get a normal for the surface that is being touched to move along it
    RaycastHit hitInfo;
    Physics.SphereCast(transform.position, m_CharacterController.radius, Vector3.down, out hitInfo,
                       m_CharacterController.height/2f);
    desiredMove = Vector3.ProjectOnPlane(desiredMove, hitInfo.normal).normalized;

    //If just a regular jump, do regular movement, otherwise add momentum
    if (!m_Flying) {          
        
        //On the ground
        if (m_CharacterController.isGrounded) {
            m_MoveDir.x = desiredMove.x*speed;
            m_MoveDir.z = desiredMove.z*speed;    
        } else { //Falling through the air
            m_MoveDir.x += desiredMove.x*speed*m_flyModifier*Time.deltaTime;
            m_MoveDir.z += desiredMove.z*speed*m_flyModifier*Time.deltaTime;
    
            m_MoveDir.x = Vector2.ClampMagnitude(new Vector2(m_MoveDir.x, m_MoveDir.z), m_RunSpeed).x;
            m_MoveDir.z = Vector2.ClampMagnitude(new Vector2(m_MoveDir.x, m_MoveDir.z), m_RunSpeed).y;             
        }
    //Jetpacking                              
    } else {
        
        m_MoveDir.x += desiredMove.x*speed*m_flyModifier*Time.deltaTime;
        m_MoveDir.z += desiredMove.z*speed*m_flyModifier*Time.deltaTime;
        
        m_MoveDir.x = Vector2.ClampMagnitude(new Vector2(m_MoveDir.x, m_MoveDir.z), m_RunSpeed).x;
        m_MoveDir.z = Vector2.ClampMagnitude(new Vector2(m_MoveDir.x, m_MoveDir.z), m_RunSpeed).y;

    }

I figured it out. I had these updating one after another:

m_MoveDir.x = Vector2.ClampMagnitude(new Vector2(m_MoveDir.x, m_MoveDir.z), m_RunSpeed).x;
m_MoveDir.z = Vector2.ClampMagnitude(new Vector2(m_MoveDir.x, m_MoveDir.z), m_RunSpeed).y;

And the X value was different when the Z value was calculating.