The very best first thing to do when troubleshooting delivery performance and queue backlogs is to see what errors the remote site is returning back, or internal notifications that PowerMTA is returning based on your config settings, since these almost always make it clear what the issue is. With this, what results when running the command;
pmta show topq --errors yahooin.queue/*