Just posting it here because I understand how annoying is this and also its not very well documented anywhere but atleast the users will know next time.
Solution:
"Press the insert button on keyboard to switch from Select and Replace to Cursor mode"
Discovered by accident:D