How do I stop my car object from moving irregularly?

I tried a car racing game for 2 players. The car is a rigidbody with four wheel colliders for wheels. The game utilize hlapi of unity for networking. The game works fine till either car reaches a high speed, then it skips back to a previous position and then again to current position instantly along the track as if glitching, this happens frequently at high speeds. I tried this with one car only and the problem persists. When I increase network update frequency to max, the car glitches after a much higher speed. Im unable to figure out what causes this. This effect happens only when game is not Host and Client only. How do I figure out this problem?

My newbie opinion is you are updating non local objects positions routhly, without interpolation.

Here is the first part of a good tutorials to start dealing with interpolation (and more generally unet concepts) :
tutorial