There was issue with certain endpoints that would lead to dropping
nearly every other RTP packet from the callee.

The scenario is as follows: the caller (bridged through FreeSWITCH)
offers codecs A and B with payload types X and Y, respectively.
The callee answers with codec B and payload type X.

The fix is to use the sender's expected payload type when checking
against the received RTP packet, rather than the receiver's.