If you are the away team you will need to accept an invite instead of sending one. The invite will appear along the bottom of the screen. To accept the invite hold down the glowing Xbox logo button. Once you hold it down for long enough you will automatically be taken to the team select screen.
(:21) Don't worry if you miss the invite notification, you can still join your match. You can find the missed invite in the notification tab. To find the notifications tab, press the glowing Xbox logo button.
(:36) Next, use the left stick to scroll down until you reach the bell icon. Press "A" On this icon, you will see your latest notifications and invites. Press "A" on the invitation, and you will be automatically taken to your game