John Kevin Ratliff, MD, FAANS, FACS (middle)
Professor and Vice Chair of Neurosurgery, Departmental Quality Officer and Co-Director of the Spine and Peripheral Nerve Surgery Division, Stanford University Medical Center
Anand Veeravagu, MD (right)
Assistant Professor of Neurosurgery, Assistant Professor Orthopedic Surgery and Director of Minimally Invasive NeuroSpine Surgery, Stanford University Medical Center
The health care policy landscape is fraught with uncertainty, today more than ever. However, health care quality metrics, which have proliferated in number and scope in recent years, are unlikely to disappear. Of course quality matters. The trouble is finding the right way to assess it. Complication rates are too often a focal point of these metrics, particularly for evaluating surgical outcomes. However, simply identifying the incidence of complications is inadequate. Context is required. Take for example, that the following factors have all been shown to influence complication rates:
In an ideal world, surgeons would possess a nuanced understanding of the specific contributors to complications rates before entering the operating room; thus enabling improved patient counseling, surgeon decision-making and optimization of outcomes.
Neurosurgery decided to take on this challenge and developed a predictive model of complication risk that is specific to spine surgery — the Risk Assessment Tool (RAT). The RAT computes a predicted probability of complication development based on patient demographics, comorbidities, and procedure-level factors (e.g. use of instrumentation). We took a big-data approach and leveraged a longitudinal national database of more than 270,000 spine surgery patients to develop the model. The RAT achieved an area under the curve (AUC) of 0.70 for predicting adverse events, markedly exceeding the predictive capability of the Charlson comorbidity index (CCI).
The next step was to perform a prospective validation of the RAT, where we recruited 246 patients and compared the predictive ability of the RAT, CCI and the American College of Surgeons National Surgery Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator. We found highly comparable performance of the RAT and the ACS NSQIP calculator (AUC: 0.669 and 0.670, respectively), but concluded that the RAT had a significantly lower error score when performing prediction. The RAT has been packaged into a simple-to-use iOS app, available for free download and the hope is to further refine the model by gathering input from other spine surgeons.
Tools and metrics that do not properly contextualize neurosurgical patients and procedures are unlikely to assess outcomes in our field accurately. Our experience with the development and validation of a predictive model of spine surgery complications has taught us that neurosurgeons must take a proactive approach to understanding and assessing the metrics used to ensure neurosurgical quality. In doing so, we are better positioned to inform policy makers and payers, as well as counsel patients using the highest quality evidence available.