what shell I do when push teh button to agree to the offer to play once more and the oponent left ?

I think I have to push the x button to be able to play an other game, but than there is the risk to be banned by the system. But I have no idea, sometime they came back an play sometime they play with some other and there is no other chance than pressing the x button or do I have a different option ?
best regards,
Sigi

Seeks created in the lobby is punished if you don't wait 15 seconds first, you can abort rematches without waiting.

Thank´s