Is proctoring mandatory when starting a TAVI program?

The rapid evolution of transcatheter aortic valve replacement (TAVI) therapy in recent years has resulted in its widespread popularization as an alternative to conventional aortic valve replacement (AVR) in highrisk patients. As worldwide experience with this modality increases, more and more patients are being offered this alternative to open surgery for the treatment of severe, symptomatic aortic stenosis. It has been estimated that in the next few years 25,000-30,000 procedures will be performed annually in the United States alone, a number that will assuredly increase (1). This migration towards catheter-based AVR is being characterized by the development of new materials, the need to acquire new skills, and the emergence of new medical field. A successful TAVI procedure requires a merging of different skills including personnel experienced with guide-wire skills and fluoroscopic imaging, as well as a vast knowledge of open vascular approaches, cardiac structures, and aortic root anatomy. Therefore, cardiothoracic surgeons and interventional cardiologists have been confronted with the need to gain experience in these novel approaches to aortic valve disease while maintaining high quality results.

Although TAVI is currently being offered in many centers worldwide, there still exists the misguided perception that starting a TAVI program involves a long and steep learning curve. However, more recent evidence with TAVI programs suggests otherwise as demonstrated by several institutions (2-4). Despite these studies showing decreased rates of complications and mortality with the acquisition of individual and institutional proficiency, there still exist several difficulties in translating conclusions into practice. Standardized guidelines for safe initiation of a TAVI program and clear methods to negotiate this learning curve have not yet been reported. In an effort to reduce the impact of this learning process, numerous educational methods focused on TAVI skills and procedures are offered by industry to medical centers desiring a TAVI program. These methods include dry laboratory sessions, training devices, animal models, and perioperative video training. Simulation, albeit necessary and informative at early stages, is not adequate to develop the higher level procedural and technical skills needed for clinical practice, and thus, a proctored setting is always mandatory.

In our opinion, acquisition of state-of-the-art technology and procedures, like TAVI, must be a gradual process based on a comprehensive educational experience that should pass through a structured training with two different levels of supervision: the preceptorship and proctorship. Although the terms preceptor and proctor have often been used interchangeably, proctoring and preceptoring are two different forms of supervision. For attainment of new skills, an initial period of supervision by a preceptor is essential to establish a foundation of TAVI skills and knowledge. During this part of the learning process, the preceptor scrubs with the student for the procedure with the aim of guiding and assisting the trainee. At the same time, the preceptor actively transfers skills to the trainee by providing real-time feedback of trainee performance. It is important to note that in the preceptoring phase, the preceptor is primarily responsible for the procedure and can readily take over if a situation demands. The preceptorship can be performed using various models like mini-fellowship and mini-residency.

hen the preceptor believes that the trainee has gained enough procedural proficiency, proctoring phase may begin. The proctoring phase is characterized by the presence of an observer (proctor) who is responsible for the assessment of the skills and knowledge of the trainee during the initial learning curve. However, the proctoring phase differs from the preceptor phase in that the TAVI trainee retains overall responsibility for the care of the patient. The proctor is responsible for observing trainee performance and competence in performing the procedure safely and effectively and provides expert-level critique specific to each case. Although the proctoring phase has a lower overall level of supervision, the proctor should always be immediately available on site at the institution for troubleshooting problems that may arise. These recommendations may result in the privileging of the TAVI trainee in performing the procedure without supervision or the proctor may recommend further training with the hope of eventually gaining enough experience to work independently.

After training commences, it is important that the TAVI team undertake rigorous performance monitoring of their results as an effort to recognize and counteract higher rates of post-procedure adverse outcomes. A broad range of statistical tools can be used to monitor a center’s overall TAVI outcomes as well as individuals performing the procedure. In our opinion, the appropriate statistical analysis to undertake is a time-series analysis such as cumulative sum (CUSUM) charting. Developed during World War II as a quality control test for ammunition production lines, CUSUM is a visual analytical method that allows one to determine whether a production process is “in control” or “out of control” (5). The pioneering use of CUSUM analysis in cardiac surgery dates back to de Leval and colleagues, who used it to monitor surgical outcomes in paediatric cardiac surgery (6). Since then, several cardiac surgeons have recognized and implemented the utility of the CUSUM methodology to assess their results (7).

When viewed graphically, the CUSUM analysis easily identifies and provides a clear view of changes in the failure rate for a given procedure: the graph moves upward if the failure rate increases and downward if it decreases. Moreover, a priori construction of 80% alert and 95% alarm lines enables the team to determine whether the outcomes of innovative procedures are acceptable or need to be improved. Specifically, a curve crossing the upper boundary is interpreted as an increased failure rate to an unacceptably high level, whereas a curve that crosses the lower boundary denotes that the complication rate is less than or equal to the accepted failure rate. A curve falling between the boundary lines indicates a lack of statistical significance and further monitoring may be indicated. Advantages of CUSUM charts include incorporating time as a “hidden variable” while avoiding the problem of repeated significance testing. In addition, this method is sufficiently sensitive and simple and does not require complex statistical software.

During the past decade at our institution there has been a dramatic shift toward minimally invasive approaches for mitral and aortic valve surgery. Control charts have been routinely used to monitor the performance of surgeons implementing these new techniques with excellent results (8,9), and thus, applying the same methodology to assess TAVI results was endorsed with enthusiasm (Figure 1). Moreover, since 2010, we have started a TAVI quality improvement program, for which CUSUm charts plan an integral and central role. The great strength of CUSUM charts lies in their ability to quickly and easily identify a trend. If the trend suggests that the production process is “out of control”, it is mandatory to initiate an in-depth review to determine the cause. Sudeen improvements in the trend are also notable as the team should thoroughly investigate and report the underlying cause to aid other institutions in achieving similar results.

Figure 1 An example of an institutional CUSUM curve for TAVI procedures. The CUSUM curve represents the consecutive performance of an individual or a team displayed as a line chart with the X-axis representing the consecutive series of procedures and the Y-axis representing the CUSUM score. CUSUM is defined as SN = (Xi -p0), where Xi=0 for success and Xi=1 for failure. The graph starts at 0, but is incremented by 1-p0 for a failure and decremented by p0 for a success. For the purpose of TAVI procedures, p0, or the “acceptable failure rate” was set at 10%. In addition, 80% “alert” and 95% “alarm” boundary lines were calculated, using a “target failure rate” of 10%. As evidenced in the figure, our institutional CUSUM curve showed a small positive slope at the beginning of our experience reflecting the impact of the learning curve. Subsequently, after a period of good performance, we experienced a period of less than expected results as reflected by the presence of a positive slop of the curve from procedure number 37 to 47. This cluster of failures prompted an internal review and retraining program was instituted for all the members of the TAVI team

Nevertheless, the precise way in which control charts are set up needs to be considered carefully. For example, in our center the occurrence of a surgical failure in each TAVI procedure accounted for a 0.9 (upward) inflection in the curves, whereas each success led to a 0.1 (downward) inflection. After a careful literature review of major TAVI series revealed a major complication rate of 10-20%, we targeted a 10% rate of failure as an acceptable value for our center’s results (10). However, the development of specifically dedicated risk-scores for TAVI procedures will allow the use of risk-adjusted CUSUM charts, which have been proved to be more sensitive than unadjusted curves (5).

The failure of a TAVI procedure may be due to several factors, many of which are not directly related to the operative technique itself. By allowing the early identification of negative trends, the CUSUM charts may prompt internal audits aimed at identifying the causes of failure and preventing further adverse outcomes. Safe initiation of a TAVI program requires a standardized training program fit with different levels of supervision as outlined above. Although supervision is a modality by which competency can be evaluated, the learning curve goes on after completing any form of training, and is probably lifelong. The CUSUM failure method is a unique analytical tool that accounts for time and personnel experience and allows for a prospective, real-time monitoring and audit of TAVI team performance.

Acknowledgements

Disclosure: The authors declare no conflict of interest.

References

1. Block PC. Lessons from the learning curve. JACC Cardiovasc Interv 2012;5:80-1.
2. Gurvitch R, Tay EL, Wijesinghe N, et al. Transcatheter aortic valve implantation: lessons from the learning curve of the first 270 high-risk patients. Catheter Cardiovasc Interv 2011;78:977-84.
3. Alli OO, Booker JD, Lennon RJ, et al. Transcatheter aortic valve implantation: assessing the learning curve. JACC Cardiovasc Interv 2012;5:72-9.
4. Kempfert J, Rastan A, Holzhey D, et al. Transapical aortic valve implantation: analysis of risk factors and learning experience in 299 patients. Circulation 2011;124:S124-9.
5. Rogers CA, Reeves BC, Caputo M, et al. Control chart methods for monitoring cardiac surgical performance and their interpretation. J Thorac Cardiovasc Surg 2004;128:811-9.
6. de Leval MR, François K, Bull C, et al. Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switch operations. J Thorac Cardiovasc Surg 1994;107:914-23; discussion 923-4.
7. Caputo M, Reeves BC, Rogers CA, et al. Monitoring the performance of residents during training in offpump coronary surgery. J Thorac Cardiovasc Surg 2004;128:907-15.
8. Murzi M, Cerillo AG, Bevilacqua S, et al. Enhancing departmental quality control in minimally invasive mitral valve surgery: a single-institution experience. Eur J Cardiothorac Surg 2012. [Epub ahead of print].
9. Murzi M, Cerillo AG, Bevilacqua S, et al. Traversing the learning curve in minimally invasive heart valve surgery: a cumulative analysis of an individual surgeon’s experience with a right minithoracotomy approach for aortic valve replacement. Eur J Cardiothorac Surg 2012;41:1242-6.
10. Wendler O, Walther T, Schroefel H, et al. The SOURCE Registry: what is the learning curve in trans-apical aortic valve implantation? Eur J Cardiothorac Surg 2011;39:853- 9; discussion 859-60.