Xbox - How To Find A Missed Invite And Accept It.
Don't worry if you miss the invite notification. You can still join your Rival match. You can find the invite in the Invitations tab.
To find the “Invitations” tab, press the glowing Xbox logo button. Next, use the left stick to scroll right until you reach the "Multiplayer" menu.
Use the left stick to navigate down to the "Invitations" menu, and press "A". On this menu, you will see your latest invites. Press "A" on the invitation, and you will be automatically taken to your game.