Multi-level Modeling (and Thinking) in the Development and Validation of Health Care Quality Measures for Substance Abuse Disorder Treatment
Process health care quality measures are often used to guide the treatment of individual patients as well as to characterize facility-level performance. Although there may be congruence between the patient- and facility-level associations between process quality measures and clinical outcomes, these relationships may differ in magnitude and even in direction. For example, patients who receive a certain process of care may in fact have better outcomes than other patients, yet facilities with higher rates of delivering the process of care may have no better or perhaps worse case mix-adjusted outcomes than facilities with lower rates. This apparently paradoxical phenomenon, related to the concept of the ecological fallacy, has been largely ignored in the design, construction, validation, and interpretation of quality measures. In this presentation, the development and validation of a quality measure for substance use disorder (SUD) care will illustrate this and other multi-level conceptual and statistical issues that must be addressed when developing and validating health care quality measures.