Competence by Design

Competence by Design is a medical education initiative instituted by the Royal College of Physicians and Surgeons of Canada which is intended to improve the training of residents in specialty disciplines. Since 2017, the post-graduate medical education training of residents in Canada has transitioned to Competence by Design in a staggered approach. 

The post-graduate medical education training of residents in diagnostic radiology transitioned to Competence by Design in July 2022 for the incoming cohort. Current PGY-3 to PGY-5 residents will continue to be evaluated and assessed as previously.

The transition to Competence by Design in diagnostic radiology post-graduate medical education represents a paradigm shift from traditional time-based post-graduate medical education training to outcomes-based post-graduate medical education training. Outcomes in Competence by Design are delineated by an organizational framework of competencies. To demonstrate competence residents will be required to complete workplace-based assessments known as Entrustable Professional Activities (EPAs). 

Outcomes in Competence by Design are delineated by an organizational framework of competencies. A competency is an “observable ability” related to a specific task. Multiple required competencies of the specialty arising from diagnostic radiology PGME training are outlined by the Royal College of Physicians and Surgeons of Canada. For example, a required competency of the specialty arising from diagnostic radiology PGME training is to “perform imaging studies and image-guided procedures in a skillful and safe manner”. 

Given that a competency is an “observable ability” it may be “measured and assessed to ensure [its] acquisition”. To demonstrate the attainment of competencies, in Competence by Design residents complete Entrustable Professional Activities (EPAs) which are “essential tasks” of a specialty that a resident may be “trusted to perform”. The completion of EPAs comprises the foundation of assessment in Competence by Design. In contrast to summative assessments utilized in time-based traditional PGME training, emphasis on these multiple low-stakes assessments in Competence by Design is intended to foster learning and strengthen performance improvement. 

The document of required EPAs as outlined by the Royal College of Physicians and Surgeons of Canada is cumbersome and spans multiple pages. Thus our department has developed infographics summarizing the EPAs required at each stage of training. Importantly, rotations on which required EPAs may be potentially completed have been mapped within the document. Posters summarizing these EPAs will be posted within the VGH and SPH radiology departments to better help facilitate this transition. 

Multiple “discrete skills” are needed to perform EPAs, which are known as CanMEDS Milestones. CanMEDS Milestones encompass the CanMEDS roles, which include medical expert, scholar, leader, communicator, collaborator, health advocate and professional. 

For example, an EPA which residents must complete is “providing diagnostic radiology reports which guide patient management” and this EPA incorporates various CanMEDS Milestones pertaining to the medical expert role, such as “summarizing findings of clinical relevance” and “providing recommendations for further imaging and/or management” and various CanMEDS Milestones pertaining to the communicator role, such as “providing clear, concise, accurate and grammatically correct reports”.

CanMEDS Milestones relevant to each EPA are delineated on electronic EPA completion forms in Entrada. When completing an EPA, faculty, fellows, and senior residents will be asked to denote whether each CanMEDS Milestone has been “achieved” or is “in progress”.

CanMEDS Milestones should be used as a guide in providing narrative feedback to residents during the completion of an EPA. When providing narrative feedback to residents during an EPA, please consider ensuring that the feedback provided pertains to one or more of the listed CanMEDS Milestones. 

As part of EPA completion, faculty, fellows and senior residents shall be required to complete an Entrustment Scale for each EPA. An Entrustment Scale denotes the assessor’s confidence in the ability of the resident to perform the activity outlined by the EPA. 

The Entrustment Scale that has been chosen to be utilized nationally for diagnostic radiology as well as institutionally across all disciplines is the Ottawa-Score (“O-Score”). The scale is based on the surgical medical education literature and it is heavily procedurally based. In the months prior to our welcoming Competence by Design in our department, the Residency Program Committee discussed the applicability of using such a scale within the diagnostic radiology context. We anticipate that such discussions will remain ongoing. 

It is important that faculty, fellows and senior residents who will complete EPAs be aware that: 

1. Residents who receive a rating of 4 out of 5 or 5 out of 5 on the entrustment scale will be considered by the Competence Committee to have successfully completed the EPA. Please note that a given EPA may be required to be successfully completed on more than one occasion.

2. Entrustment decisions should be based on the resident’s stage of training (ex. Transition to Discipline). 

One of the main educational benefits of Competence by Design is increased opportunities for feedback during post-graduate medical education training which is facilitated through the completion of EPAs. In addition to CanMEDS Milestones and Entrustment Scales, narrative feedback serves as a core component of EPAs. 

During the completion of EPAs, faculty, fellows, and senior residents should emphasize narrative feedback and try to facilitate an interactive feedback discussion. Given the integral role of feedback in Competence by Design, it is important that narrative feedback provided to residents is of high quality and incorporates specific and actionable feedback which is timely provided. Upon the completion of each EPA faculty should ensure that feedback provided includes both feedback for reinforcement (“positive feedback”) and feedback for improvement (“negative feedback”) to normalize EPAs as opportunities to foster learning and strengthen performance improvement. 

Overall, the integral role of feedback in Competence by Design emphasizes “assessment for learning” as opposed to “learning for assessment”. Thus the implementation of Competence by Design is intended to shift the culture of assessment in post-graduate medical education training.

Did you know? During post-graduate medical education program accreditation the Royal College of Physicians and Surgeons reviews completed EPAs. Given the integral role of feedback in Competence by Design, the quality of feedback provided to residents during the completion of EPAs is evaluated by the accreditation committee. 

In Competence by Design dedicated faculty who comprise a Competence Committee iteratively evaluate assessment data and collectively provide recommendations on the progression of each resident through the stages of training. A Competence Committee thus facilitates a collective decision making process using collated assessment data of a resident’s performance.

While EPAs will form the foundation of assessment data evaluated by the Competence Committee, in the initial stages of Competence by Design the Competence Committee will continue to take in to consideration other assessment data, including end of rotation evaluations and performance on standardized examinations. Over time in the upcoming years it is intended by the Royal College of Physicians and Surgeons of Canada that EPAs will replace end of rotation evaluations. 

Through delegating resident evaluation to a Competence Committee it is intended that daily faculty to resident interactions can focus solely on feedback and performance improvement.

In addition to the completion of Entrustable Professional Activities (EPAs), additional opportunities to foster learning and strengthen performance improvement in Competence by Design involve the deployment of Academic Coaches. 

Academic Coaches are dedicated faculty who develop a “longitudinal relationship” with a resident to provide “coaching over time”. Residents may meet iteratively with a dedicated Academic Coach to discuss their performance and to set “performance goals”.

While the completion of EPAs focusses on “coaching in the moment”, through building a longitudinal relationship with a resident, Academic Coaches focus on helping a resident to become a competent and independent physician throughout their post-graduate medical education training. 

More details regarding the deployment of Academic Coaches in our department will be shared in the upcoming months. 

Multiple positive educational outcomes are intended to arise from the transition to Competence by Design, including the direct delineation of the required competencies of the specialty, facilitation of modified learning and assessment opportunities for residents, and most importantly increased opportunities for the provision of feedback and coaching. 

The implementation of Competence by Design in other specialties has demonstrated that while the initial months of the transition to Competence by Design may focus primarily on simply completing required EPAs, with time focus shifts to the provision of feedback and coaching and a greater understanding and appreciation for the rationale of this educational transition is manifested. 

We continue to encourage those within our department to share their perspectives on and experiences with Competence by Design so that its implementation in our department can be continually improved.

Diagnostic Radiology Entrustable Professional Activities Summary Infographic

Strategies for Entrustable Professional Activities Completion Infographic

Resources are best viewed when downloaded.