For Invites sent by you: You can cancel an invite if the same is not accepted by your match. Once the invite is accepted, it cannot be cancelled. Click on the 3 dots on the top right of the profile and click "Cancel Connect" to cancel an invite.
For Invites received by you: You can decline an invite after accepting it if you change your mind later.