If the non-performing specialist was a third party, the existing servicer could either transfer the servicing rights once a loan hits a certain delinquency trigger, or simply have the loans subserviced – of course with high levels of accountability – on a fee-for-services basis until the delinquency is resolved.
One structure along these general lines has recently been proposed by Fannie Mae, Freddie Mac, and Ginnie Mae. While many details would need to be worked out and possible implications thought through, I believe it is a promising start. Another structural change that would help would be a limit on the extent to which servicers have to advance principal and interest on non-performing loans.