Application Of Power Analysis To Determine The Optimal Reporting Timeframe For Use In A Statewide Trauma Quality Reporting System
*Naveen Fatima Sangji, *Anne H Cain-Nielsen, *Jill L Jakubus, *Judy N Mikhail, *Alisha Lussiez, *Pooja Neiman, *John R Montgomery, *Bryant W Oliphant, *John W Scott, Mark R Hemmila
University of Michigan, Ann Arbor, MI
Introduction: Meaningful reporting of quality metrics relies on being able to detect a statistical difference when true differences in performance exist. Larger groups and longer timeframes can produce higher rates of statistical differences. However, data that is perceived as old is less relevant when attempting to enact change in the clinical setting. Hence, selection of timeframes must strike a balance between being too small (more type II errors) and too long (stale data). We explored the use of power analysis to optimize selection of timeframes for trauma quality reporting. Methods: Using Level 3 trauma center data, we tested for differences in four outcomes (mortality; mortality/hospice; major complications; transfer to higher level of care ≤ 12 hours) within four cohorts of patients (all patients; patients >65 years; patients with hip fractures; and all patients excluding those transferred to a higher level of care). Using bootstrapping, we calculated power for rejecting the null hypothesis that no difference exists amongst the centers for four different time frames: the entire period (1/1/2017 through 8/31/2020), and 2-year, 1.5-year, and 1-year periods. Using the entire sample from each site, we simulated randomly generated datasets. Each simulated dataset was tested for whether a difference was observed from the average. Power was calculated as the percentage of simulated datasets where a difference was observed. This process was repeated for each assumption. Results: Power calculations for each of the four cohorts during different time frames are shown in the Table. The optimal timeframe for generating quality reports to assess whether a single site’s outcomes are different from the overall average was determined to be 2 years for our collaborative of 22 Level 3 trauma centers based on an 80% cutoff. Conclusions: Power analysis with simulated datasets allows for testing of different timeframes for data used to assess outcome differences. This methodology allows an optimal timeframe to be selected that balances the ability to detect differences with avoidance of data reporting not being timely.
Back to 2021 Abstracts