Vascular Surgery Studies Utilizing ACS-NSQIP Demonstrate Low to Moderate Adherence to Current Reporting Standards
Walker R Ueland1, Amin Mirzaie1, Amanda Delgado2, Katherine Lambert3, Scott A Berceli1, Salvatore T Scali1, Martin R Back1, Thomas S Huber1, Gilbert R Upchurch, Jr.1, Samir K Shah1
1University of Florida, Gainesville, FL;2Augusta University Medical Center, Augusta, GA;3Duke University Medical Center, Durham, NC
Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database is a multi-institutional registry that contains data on patient demographics and outcomes for a variety of surgical specialties. ACS-NSQIP serves as an increasingly common database for health services and outcomes research in vascular surgery. Quality of data reporting is essential to research transparency and rigor, as there are published standards for data reporting. To date, these standards have never been used to assess vascular surgery studies using ACS-NSQIP. The goal of the present study is to appraise the methodologic quality of vascular surgery research that utilizes the ACS-NSQIP database.
Methods: PubMed was queried for ACS-NSQIP research articles in vascular surgery. Exclusion criteria included qualitative and linked studies. We used two guidelines to appraise the reporting quality of each study: the Reporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement and the Journal of American Medical Association-Surgical Section (JAMA-Surgery) checklist. Eight items were excluded from these checklists for inapplicability to the ACS-NSQIP database.
Results: We screened 3,418 articles based on ACS-NSQIP and identified 192 that fit inclusion criteria. We used the RECORD and JAMA-Surgery checklists to evaluate 159 vascular articles after exclusion criteria (Table 1). The median score for the RECORD checklist was five out of eight. The most commonly missed items were: 1) providing information on data cleaning methods (90%) and 2) referencing validation studies of CPT codes (99%). The median score for the JAMA-Surgery checklist was three out of seven. The most missed items were: 1) including a flow chart to visualize study population selection (88%) and 2) identifying/addressing competing risks (99%).
Conclusions: With increasing use of observational registries for surgical research, rigor in reporting of data is essential. Our results suggest that vascular surgery studies using the ACS-NSQIP database demonstrate low to moderate adherence to current reporting standards. Our study offers insight into improving the quality of research by providing information on data cleaning methods, referencing validation studies for CPT codes, utilizing a flow chart to visualize study population selection, and identifying/addressing competing risks.
Table I: The fulfillment of RECORD statement and JAMA-Surgery checklist items of the study sample.
|The type of data or name of dataset is specified in title/abstract
|Specified outcomes and exposures of interest
|Method of population selection, with inclusion and exclusion criteria specified
|Defined geographic region and timeframe in title/abstract
|Discussed the implication of data not available or collected to answer research question
|Defined codes used to select population
|Provided data cleaning methodology
|Described any validation of codes
|Clear take home message that addresses advancement of knowledge
|Statement ensuring IRB compliance
|Discussed implications of missing data
|Clearly identified potential confounders
|Solid research question and hypothesis
|Clearly defined inclusion criteria, exclusion criteria, and included a flow chart to visualize study population selection
|Identified and addressed competing risks
Back to 2023 Display Posters