We are pleased to see the FHA proposing to introduce more statistical sophistication into its Quality Assurance Process (QAP) and to see the MBA responding with reasonable critiques. There are a number of items under discussion which have been long-standing issues in the industry, including what defines a loan manufacturing defect, what are appropriate tolerance and severity levels for defects, and what are appropriate remedies. “Loan quality” must have a standardized definition to be useful. But the item that caught our attention was the discussion of statistical sampling.
FHA is proposing the following in its solicitation of information:
“Statistical sampling. FHA is also considering whether to establish a process to review a statistically significant random sample of loans for each mortgagee within a prescribed time frame after loan endorsement. Lenders would receive feedback on findings within an established timeframe. FHA would use the statistical sample, to estimate the defect rate on each lender’s overall FHA portfolio and then extrapolate the origination defect rate to all lender originations during the sampled time period, and thus have the lender compensate FHA for the estimated total risk to FHA resulting from the lender’s origination processes.The purpose of this process would be to increase the efficiency of FHA’s post-endorsement review process. HUD invites comment on the use of and optimal methodology for a statistically significant random sample, including the nature of the loans that should be included or excluded from the sample.”
The MBA has responded with this:
“Most importantly, MBA has serious concerns about the impact of a sampling methodology on independent mortgage bankers and community banks and the number of lenders participating in the FHA program. While larger lenders may be able to originate enough loans to generate statistically significant sample sizes, many smaller lenders would be challenged in this regard. It is unclear how HUD would address this situation and what, if any, allowances would be made for small lenders. Moreover, depending on the structure of the penalty system, paying an upfront percentage could have a much greater impact on smaller lenders than larger lenders. The possibility of sampling bias that results in “overpaying” for smaller lenders has potentially devastating consequences reducing competition and increasing the price for consumers. Companies could be forced out of business or cease originating FHA loans.”
Given the number of lenders we have seen who report only on the number of “findings” in their reviews, with no mention of defect rates or sampling method or population counts, it is encouraging to hear influential industry players talking about sample sizes and valid inferences to populations and statistical significance (albeit in a slightly different context.) If nothing else, it reminds us that the loan audits that take up so much of our time represent a small fraction of the loans we originate (or service). And that what matters is the quality of the entire origination (or servicing) pool, not just the samples we draw (which are simply proxies for the population.)
We say let the discussion continue. The more informed lenders are about what constitutes loan quality, the better they can do their jobs.