If I add a rigidbody to the wall (not just a collider) and enable its Is Kinematic property, the jump height next to that specific wall remains constant, avoiding the issue.
There is still a big WTF hovering over my head, but I remember having read somewhere in the Unity documentation that any object that interacts with physics should have a rigidbody, so I guess this is a “by design” issue.
The Walls collider are tall thus when u jump(While hugging the wall) the Collider on the player still remains in contact thus Forces adds up making the jump more higher.
WORKAROUND
Simply make bool variable and make it true when in air and false on landing. and call the jump only when variable is false.
,Anyone facing this issue can just add bool variable which is true,when GameObject is in the air and false,when it lands.