The purpose of a long term x-ray analysis reliability study is to further demonstrate the sustained continued ability of NUCCA certification examiners to agree on x-ray analysis. The NUCCA certification program provides an opportunity for collecting data when submitted radiographs are examined for consistency in marking by two different blinded examiners. This ongoing study maintains and increases transparency of the certification process which provides a demonstrated significance and high standard for the certified NUCCA doctor and encouragement for those in the process. Most importantly, it provides a data collection opportunity in demonstrating inter-examiner reliability of the NUCCA radiograph marking and analysis procedure. In addition, it provides an infrastructure to scientifically test new marking and analysis procedures with ease.
A database created with certification records from several years allows for establishing an undisputable record of achievement indicating that Upper Cervical practitioners can analyze films with complete agreement. Statistics used for demonstrating such excellence require many sets of films examined by no more than three assessors over a long study period. The Chair of the Certification Committee is the authority over the two Board Certified examiners assessing the submitted film sets. It is the Chair’s analysis that serves as the “Gold Standard” by which the examiners must adhere in their analysis of the films. The Certification Chair approves the selection of no less than three potential Board Certified examiners and ensures they are trained to accomplish the task with moderate agreement before the final trial begins.
To develop a study foundation and test the data collection protocol while adding further significance to the ongoing certification study, an inter-examiner reliability study is underway. This provides baseline data for comparison in tracking ongoing improvements. A five year time line is planned during which a protocol for sustainability is developed then instituted to maintain the infrastructure to research new analysis methods.
The first project year develops and refines a sound data collection protocol to assure data quality and to troubleshoot difficulties as they arise. A manuscript will be written documenting the first year project development and refining the research procedure report of the reliability outcomes of the first year pilot study. Year one and two are a development phase in which the procedure is perfected. Data is collected continuously for a minimum of one year. Upon conclusion of the first year of data collection, sometime early in year two, analysis of the study procedure and data collected will yield a paper for the indexed literature. Needed changes will be introduced with consideration to include Part Two Certification into the investigational process. Feasibility and the procedure of this are explored in year one of the development. Years four and five involve a two year data collection and sustainability phase. At the end of five years, the pooled dataset is analyzed and reliability of the examiners reported for the indexed literature.
As radiographs are submitted for review, the NUCCA Certification Chair inspects incoming films to determine acceptability for further examination and inclusion into certification study. The Chair analyzes the film set, sending the data sheet to the study coordinator for database entry. This record becomes the ‘standard’ by which examiners must agree in their analysis. All pre-existing lines are removed to maintain blinding for the examiners who analyze the films according to the established protocol. The administrator is blinded to the identity of the certification candidate. The examiners have the option to reject films based on their experience at which time they are returned to the Certification Chair who returns them to the certification candidate.
The administrator sends the films to examiner one who records the findings on a data sheet, returning films and completed sheet to the coordinator. A copy of the datasheet is also sent to the Certification Director to compare against the ‘standard’, then determining if the film set passes for certification. The administrator inspects the returned film sets for identifiers, removed if found, and sent to examiner two according to a pre-established randomization rubric. All data from the examiner datasheets is double entered, verified into the study database by contracted data entry personnel. As a Web Based Data Entry Platform is developed, data entry can occur via that resource. Once both examiners have reviewed the film set, the film set and copies of the data sheet are sent to the certification examiner who determines if radiographs have passed the certification requirement.
With our ongoing dedication to the excellence of the care we continue to provide our patients, we continue to refine, study and validate our clinical and practice procedures.
