Wendy is a second year internal medicine resident currently struggling to feel “competent” as an outpatient primary care provider. Over the course of her internship year, her competence and confidence with inpatient care grew. However, her outpatient clinics were infrequent in comparison and she feels like she hasn’t learned to be a “good primary care doctor.” She shares her frustrations with her mentor—in part fueled by the breadth of skills required, the lack of benchmarks to measure her progress, and the lack of validated assessment tools. Her mentor attempts to help Wendy develop an individualized learning plan with formative assessments.
Medical education has recently undergone a transformation from process-oriented measures of success to more outcomes-oriented assessments that gauge what a learner can actually do and not only to what they have been exposed. Although the notion of competencies is not new, more recent conceptualizations move far beyond simple listings of skills and include multidimensional, dynamic, contextual, and developmental elements. This movement has been particularly transformative in the realm of graduate medical education (GME) with the advent of the Accreditation Council for Graduate Medical Education (ACGME) competencies and has become increasingly common as medical schools begin to embrace competency-based medical education (CBME) and translate the GME competencies for medical students “further upstream.” Learning objectives and related competencies are now more frequently articulated and tend to be more commonly linked to specific assessment tools that measure achievement of that specific competency. However, the practice of medicine is inherently complex and often requires subtle, nuanced skills that are both difficult to teach and challenging to quantify. In particular, competencies that fall within the realm of the Behavioral and Social Sciences (BSS) have proven particularly difficult to articulate, teach, and assess.
This chapter will first provide a brief overview of CBME that articulates common advantages and limitations of this approach. We then offer an update on the status of BSS curricular thinking within medical education, including the recent work of the Association of American Medical Colleges (AAMC) BSS Expert Panel, the BSS Curriculum Consortium (funded by National Institutes of Health [NIH]), and the behavioral science subcommittee of the MCAT 5th Revision (MR5) Committee (charged with creating the 5th version of the Medical College Admissions Test). This chapter presents a careful melding of the concepts and processes of CBME with the content of BSS by offering a synthesis of the innovative work emerging on both undergraduate medical education (UME) and GME levels. Specific assessment methods and evaluation tools for BSS competencies are described in Chapter 43.
Pedagogical Building Blocks for CBME
The Royal College of Physicians and Surgeons of Canada define competency-based education and training (CBET) as “an outcomes-based approach to the design, implementation, assessment and evaluation of an education program using an organizing framework of competencies.” CBET and CBME are intended to focus on outcomes, emphasize abilities, de-emphasize time-based training, and promote greater learner-centeredness. Epstein defines professional competence as “the habitual and judicious use of knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served”. More simply, Ten Cate defines competence as “the threshold level in the development of expertise that permits unsupervised practice”.
A number of classical pedagogical and learning theories have helped medical education move from an expert-centric, apprenticeship model to a more evolved, evidence-based approach emphasizing adult learning and professional development. Medicine is perhaps unique in the number of disciplines, the quantity of knowledge, and the integrative capacity required to achieve even minimal competence. Miller’s Pyramid offers a useful rubric in considering the progression of skill acquisition moving from Knows to Knows How to Shows How to Does in a passing resemblance to the classic See One, Do One, Teach One. Dreyfus and Dreyfus present a continuum ranging from novice at one end, then progressing to advanced, competent, proficient, and expert. It is notable that in the Dreyfus model “competent” does not lie at the end of the continuum but precedes proficient and expert implying that competence is a stage, an important benchmark, on the way to becoming an expert.
In each of these important models and definitions it is clear that competence is thought to progress in a developmental fashion in response to direct (and indirect) intervention from the educational system and the learning environment. Recognizing this developmental aspect to competence, an international group developed a set of definitions to help clarify the confusion often engendered by the terms competencies, competence, and competent. Competencies are best viewed as “abilities.” Competence and competent represent states of being that can be further subdivided into three categories:
Competent: Possessing the required abilities in all domains at a specified stage of medical education or practice.
Dyscompetent: Relatively lacking in one or more domains of required abilities at a specified stage of medical education or practice.
Incompetent: Lacking the required abilities in all domains in a certain context at a defined state of medical education or practice.
Competence is acquired by the learner and not necessarily imparted by the faculty. Peers, patients, allied health professionals, the learner himself or herself, and others may be considered teachers capable of assisting a learner in acquiring competence. Once competent, learning and growth continue well into the professional years and perhaps for the rest of one’s career. Competence is multidimensional and includes knowledge, procedural skills, metacognition, emotional management, social relationships, and communication. To best grasp how such breadth can be meaningfully captured and assessed to assist the faculty and learner in our opening vignette, it is helpful to review the history and evolution of CBME.
Competency-based medical education is not a new concept and represents an amalgam of educational theories, with origins in the work of Thorndike and Dewey. The earliest conception of competency-based training actually arose in the United States during the 1920s as educational reform became linked to industrial and business models of work that centered on clear specification of outcomes. However, the more recent conception of CBME had much of its genesis in the teacher education reform movement of the 1960s. This interest was spurred by a US Office of Education National Center for Education research grant program in 1968 to 10 universities to develop and implement new teacher training models that focused on student achievement (outcomes). Elam laid down a series of principles and characteristics of CBET in 1971 (see Table 41-1). In 2002, Carraccio and colleagues noted that some sectors in medical education explored competency-based models in the 1970s, but except for one study, no comparisons between competency-based and the traditional structure/process-based curricula were undertaken. The World Health Organization (WHO) commissioned a paper published in 1978 that strongly recommended the adoption of competency-based educational models for medical education, noting, “The intended output of a competency-based programme is a health professional who can practise medicine at a defined level of proficiency, in accord with local conditions, to meet local needs.”
Table 41-1.The Competency-based education and training (CBET). ||Download (.pdf) Table 41-1. The Competency-based education and training (CBET).
|CBET Principles ||CBET Characteristics |
Competencies are role derived, specified in behavioral terms, and made public
Assessment criteria are competency based and specify what constitutes mastery level of achievement
Assessment requires performance as the prime evidence but also takes knowledge into account
Individual learners progress at rates dependent on demonstrated competency
The instructional program facilitates development and evaluation of the specific competencies
Learning is individualized
Feedback to the learner is critical
Emphasis is more on the exit criteria than on the admission criteria
CBET requires a systematic program (approach)
Training is modularized
Both the learner and the program have accountability
In the few studies within medical fields that have investigated competency-based models, there appear to be some benefits to trainees in the CBET model. For example, Johns Hopkins has employed competency-based approaches in its neurosurgery program, with success in accelerating competency in specific surgeries.
In 2002, the ACGME Outcome Project changed the accreditation focus from a program’s process and structure (e.g., rotations, written curricula) to actual learner/program outcomes. Programs had to objectively document that their residents achieved competence in six general dimensions of practice—Medical Knowledge, Patient Care, Communication and Interpersonal Skills, Professionalism, Practice-Based Learning and Improvement, and Systems-Based Practice. In phase 1 of the Outcome Project, programs defined objectives to demonstrate learning in the competencies. In phase 2, they integrated the competencies into their curricula and expanded their evaluation systems to assess actual performance. In phase 3, programs are required to use aggregate performance data for curriculum reform—that is, examine important linkages between quality of patient care and education in the competencies. Phase 4 began in July 2011 and focuses on identification of benchmark programs and dissemination/adoption of emerging models of educational excellence. Each of the ACGME competencies are listed in Table 41-2 and paired with recommended assessment tools. Given the broad nature of these competencies, a substantial burden was placed on programs to define objectives and develop curricula.
Table 41-2.ACGME competencies and assessment tools. ||Download (.pdf) Table 41-2. ACGME competencies and assessment tools.
|ACGME Competency ||Recommended Assessment Tools |
|Medical knowledge ||MCQ exam, Oral exam |
|Patient care ||Standardized patient, objective structured clinical examination (OSCE), patient survey, multisource feedback (a.k.a. 360 degree assessment) |
|Communication and interpersonal skills ||Standardized patient, OSCE, patient survey |
|Professionalism ||OSCE, patient survey, multisource feedback |
|Practice-based learning and improvement ||Portfolio, medical record review |
|Systems-based practice ||MCQ exam, patient survey, multisource feedback |
In addition to the ACGME, a number of international competency frameworks have been developed to guide UME and GME curricula and assessment. Each provides a useful contrast to the ACGME approach, but like ACGME, is not specific to the social and behavioral sciences. Table 41-3 provides a side-by-side comparison of these different competency frameworks. Competencies have been grouped by their approximate equivalence to the ACGME categories.
Table 41-3.Medical competency frameworks. ||Download (.pdf) Table 41-3. Medical competency frameworks.
|ACGME (United States) ||General Medical Council (Great Britain) ||Scottish Doctor (Scotland) ||CanMEDS Framework (Canada) |
|Six broad competency areas ||Seven outcome categories for provisionally registered doctors ||Twelve learning outcomes domains ||Based on six physician roles and the integrative role of the medical expert |
|Medical knowledge ||• Good clinical care || |
• Basic, social, and clinical sciences and underlying principles
• Medical informatics
|• Scholar |
|Patient care || |
• Good clinical care
• Clinical skills
• Practical procedures
• Patient investigation
• Patient management
• Decision making skills and clinical reasoning and judgment
• Health promotion and disease prevention
• Health advocate
• Medical expert
|Interpersonal and communication skills || |
• Relationships with patients working with colleagues
• Clinical skills
• Health promotion and disease prevention
|Professionalism || |
• Health (self-care)
• Probity (integrity)
• Attitudes, ethical understanding and legal responsibilities
• Role of the doctor in the health service
• Personal development
|Practice-based learning and improvement || |
• Maintaining good medical practice
| || |
|Systems-based practice || |
• Working with colleagues
• Teaching and training, appraising and assessing
• Patient management
• Role of the doctor in the health service
• Medical expert (integration of all roles)
The American Board of Internal Medicine (ABIM) and the ACGME have sponsored the development of milestones for internal medicine residency training using the Dreyfus model of skill acquisition as the frame of reference for the milestones. They also have viewed competence as stage dependent to some degree. For example, what represents competence in interpersonal communication and skills changes as the resident progresses through the training program. The task force, therefore, used a hybrid of the Dreyfus and international group definition of competence to calibrate the milestones with the expectation that residents achieve, at a minimum, the defined level of competence before completion of residency training. The task force also included nonphysicians and trainees to ensure key developmental milestones were not missed and were stage appropriate. By the time a learner reaches competence, he or she has already progressed from simply applying rules to facts and features without context (novice) to considering the specific features of concrete situations (advanced beginner). The competent learner considers both context-free and situational elements but also hierarchically organizes and reduces them to a smaller set on which to base a decision. In addition, he or she becomes more intimately involved in the process and feels more responsible for the outcome. In the next stage, proficiency, learners solve problems with an intuition that usually derives from some time in independent practice. Thus, although it is expected that some residents will achieve proficiency in some competencies, the task force decided “proficiency” was the desired goal. They defined competence, however, in sufficiently robust terms that a resident deemed “competent” at graduation would be able to practice medicine effectively in an unsupervised clinical environment, a critical “entrustment” decision all training programs ultimately must make.
CBME has been embraced by UME as it moves away from the classic Flexner model of classroom-based basic science in years 1–2 with little clinical experience until year 3. Many medical schools have adopted the ACGME competency framework for both conceptual reasons and expediency. Presumably all graduating students will continue on to GME, and so will need to be familiar with the ACGME competencies. Conceptually, it is compelling to think of a full continuum of benchmarked competencies beginning with the first day of medical school and ending with achievement of competencies in residency. Of course, true CBME argues for a move away from time-based education, which could mean that some students (or residents) will achieve competence in less time whereas others may take longer. As with GME, UME BSS competencies are subsumed under the same rubric but perhaps less well articulated and more challenging to assess. Benchmarks for UME BSS competencies have not been developed, but helpful guidelines and tools have been provided by the Institute of Medicine (IOM) and AAMC (described below).
More recently, the concept of “entrustable professional activities” (EPA’s) has emerged as a more practical and granular approach related to competency assessment. In medical training, a teacher essentially wants to train and “entrust” a learner to perform a particular activity such as performing a cardiac examination or diagnosing depression. Once “entrusted,” the learner may continue to deepen that particular skill but can do so as a self-directed learner. Additional activities are added throughout training and potentially throughout an entire medical career. Although newer models of CBME have added contextual and multidimensional factors, the EPA approach does this in a more robust, holistic, systematic, and measurable way that yields practical outcomes.
Promises and Pitfalls of CBME
CBME represents a dramatic shift in the way we think about medical training and evaluation of medical learners at all levels. However, CBME is not without its critics. CBME, and particularly Milestones, struggle with the issue of granularity, or how atomistic one must be to reduce a complex skill into its constituent elements. Such reductions run the risk of becoming long, exhaustive lists of overlapping and sometimes redundant skills that still manage to be incomplete and may fail to capture the more subtle skills required in a complex interaction.
Particular “doctoring” skills, such as empathy or respect, and important skills like self-care or reflective capacity may prove particularly hard to benchmark and assess. Some have argued that the substantial assessment burden is amplified by the lack of psychometrically valid instruments and the lack of appropriately trained faculty evaluators. Beyond the obvious objections to the time, money, and resources required for CBME, some have argued that CBME focuses on the lowest common denominator rather than promoting learner excellence. As the level of learner advances, competencies become more specialized, requiring ever greater levels of detail and specificity by program directors and evaluators. Proponents argue that despite these challenges, medicine has and always will be required to “entrust” its trainees to deliver medical care. Greater attention to specific, contextualized competencies makes this inevitable process more valid, reliable, and safe.
Creating a BSS Competency Framework
Despite the substantive struggles bound to arise from CBME, the challenge of articulating a particular skill set, describing its developmental progression, and developing assessment methodologies has important implications for the field. Recent educational research has supported the reduction of complex skill sets into constituent foundational skills, and CBME consensus conferences have affirmed the relative value of CBME in contrast to past guiding approaches. Moreover, CBME holds special promise for nonbiomedical disciplines often overlooked in medical curricula or thought to be too difficult to assess. BSS leaders have been required to articulate, operationalize, and justify BSS competencies thought essential to the practice of medicine. These initial efforts can be found in the work of the NIH BSS Curriculum Consortium, the IOM BSS Report Committee, and the AAMC BSS Expert Panel (see below).
Few would argue that BSS do not have a place within medical education and practice, but BSS is not always in sync with the culture of medicine which is still dominated by biomedicine. The BSS umbrella includes specific clinical content (medical knowledge) such as understanding the relationship between smoking and cardiovascular disease. However, BSS also includes methods of inquiry (qualitative or survey research) and introduces a philosophy of science and knowledge that includes the interdependence of variables, nonlinear causal models, tolerance of ambiguity, and social contextual factors. BSS includes meta-processes such as habits of mind, reflective capacity, self-regulation, professional identity development, and emotional management. And, finally, BSS speaks of the sociopolitical aspects of medicine, including medicine’s “contract” with society, social justice, health care equality, ethics, and morality. In ACGME competency terms, BSS falls within all six competency domains but is most heavily represented in the Professionalism, Communication and Interpersonal Skills, and Systems-Based Practice categories.
Although lists of key BSS topics and more focused BSS competencies are available, they are not universally accepted and are paradoxically seen as both bloated and incomplete. Rather than focusing on the exclusion or inclusion of favored topics, these lists can serve as important starting points to develop and evaluate BSS curricula and competencies. To date, work to further define and list BSS topics and competencies has included the foundational work of the Association of Behavioral Science and Medical Education (ABSAME, the IOM BSS report (2004), the NIH BSS Curriculum Consortium, and more recently, the MR5 group and the AAMC BSS report (2011). Highlights from these seminal reports are presented below, and links to the full reports are provided at the end of the chapter.
ABSAME seeks to develop and promote the inclusion of the behavioral sciences in medical education. As part of that process, this AAMC-affiliated group developed one of the earliest and most comprehensive BSS content lists. Broad content categories included “Health, Illness, Sickness, and Disease,” “Biological Components of Human Behavior,” “Individual Behavior,” “Interpersonal and Individual/Social Behavior,” “Culture, Society, Institutions, and Organizations,” “Epidemiology,” “Clinical Reasoning,” and “Psychopathology.”
In 2004, the IOM released the report, Social and Behavioral Sciences in Medical Education. After a comprehensive review of BSS curricula across US medical schools, this report included five recommendations to move the field forward and presented a listing of six essential BSS content areas and 20 high-priority topics (see Table 41-4). These core content areas and topic listings have since been corroborated with a national faculty survey.
Table 41-4.IOM content areas and high-priority topics. ||Download (.pdf) Table 41-4. IOM content areas and high-priority topics.
|Core Content Area ||High-Priority Topics |
|1. Mind–body interactions in health and disease || |
Biological mediators between psychological and social factors and health
Psychological, social, and behavioral factors in chronic disease
Psychological and social aspects of human development that influence disease and illness
Psychosocial aspects of pain
|2. Patient behavior || |
|3. Physician role and behavior || |
Ethical guidelines for professional behavior
Personal values, attitudes, and biases as they influence patient care
Social accountability and responsibility
Work in health care teams and organizations
Use of and linkage with community resources to enhance patient care
|4. Physician–patient interactions || |
|5. Social and cultural issues in health care || |
|6. Health policy and economics || |
Overview of the US health care system
Economic incentives affecting patients’ health-related behaviors
Costs, cost-effectiveness, and physician responses to financial incentives
In response to one of the report recommendations, NIH funded nine medical schools to develop and evaluate BSS curricular innovations. Innovations have included greater explication of the IOM content areas, improved pedagogical methods, and more valid and reliable assessment tools for BSS competencies.
The MR5 Committee addressed the issue of BSS competencies even further upstream by specifying what BSS concepts, knowledge, and skills would be assessed for all medical school applicants taking the new MCAT starting in 2015. By setting the assessment standards, the MR5 committee essentially provided a road map to medical school advisors and applicants regarding what they should learn before applying to medical school. Table 41-5 lists the preliminary five core content areas and associated topics. As would be expected, these BSS areas represent the basic science of BSS, in contrast to the clinical applications of BSS described in the IOM report.
Table 41-5.MCAT BSS content. ||Download (.pdf) Table 41-5. MCAT BSS content.
|Foundational Concept 1 (ways that individuals perceive, think about, and react to the world) ||Foundational Concept 2 (factors that influence behavior and behavior change) ||Foundational Concept 3 (factors that influence how we think about ourselves and others) ||Foundational Concept 4 (ways in which cultural and social differences influence well-being) ||Foundational Concept 5 (ways in which social stratification affects access to resources and well-being) |
A: Sensing the environment
1) Sensory processing
4) Other senses
A: Individual influences on behavior
1) Biological bases of behavior
3) Psychological disorders
1) Self-concept and identity
2) Formation of identity
A: Understanding social structure
1) Components of social structure
2) Theoretical approaches
3) Specific social institutions
A: Social inequality
1) Spatial inequality
2) Social class
3) Health disparities
4) Healthcare disparities
B: Making sense of the environment
B: Social processes that influence human behavior
2) How the presence of others affects individual behavior
3) Group processes
B: Social thinking
1) Attributing behavior to persons or situations
2) Prejudice and bias
3) Processes related to stereotypes
B: Demographic characteristics and process
1) Demographic shifts and social change
2) Demographic structure of society
C: Responding to the world
C: Attitude and Behavioral change
1) Associative learning
2) Observational learning
3) Attitude and Behavioral change
C: Social interactions
1) Self-presentation and interacting with others
2) Social behavior
Elements of social interaction
| || |
The report from the AAMC BSS Expert Panel, Behavioral and Social Sciences Foundations for Future Physicians, defines BSS within the context of medicine, illustrates the ways in which both BSS content and process influence teaching and clinical care, and provides a short list of sample BSS competencies. More importantly, this report provides a robust curricular tool that guides educators and learners in identifying and elaborating BSS content that arises in any clinical case. This tool, the BSS Matrix, uses the IOM content areas (see Table 41-4) and the CanMEDS professional roles (see Table 41-6) as complementary rubrics to reduce complex clinical encounters into their foundational BSS competency units for further teaching and/or evaluation (see Figure 41-1).
Table 41-6.CanMEDS physician competency framework. ||Download (.pdf) Table 41-6. CanMEDS physician competency framework.
|As Professionals, physicians are committed to the health and well-being of individuals and society through ethical practice, profession-led regulation, and high personal standards of behavior. |
|As Communicators, physicians effectively facilitate the doctor–patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter. |
|As Collaborators, physicians effectively work within a health care team to achieve optimal patient care. |
|As Managers, physicians are integral participants in health care organizations, organizing sustainable practices, making decisions about allocating resources, and contributing to the effectiveness of the health care system. |
|As Health Advocates, physicians responsibly use their expertise and influence to advance the health and well-being of individual patients, communities, and populations. |
|As Scholars, physicians demonstrate a lifelong commitment to reflective learning, as well as the creation, dissemination, application, and translation of medical knowledge. |
|As Medical Experts, physicians integrate all of the CanMEDS Roles, applying medical knowledge, clinical skills, and professional attitudes in their provision of patient-centered care. Medical Expert is the central physician Role in the CanMEDS framework. |
The AAMC BSS matrix. Sample case: A 27-year-old man with atrial fibrillation following binge drinking episodes x2.
We resume our opening example to illustrate how the resident and preceptor could use the BSS Matrix to promote BSS competence.
Preceptor: Wendy, thank you for bringing this up. It might take a long time and a lot of patience before you can really start to feel competent as a primary care provider. There are a lot of skills involved and they are probably best taught while immersed in the context of caring for real patients. Do you have a case you can share with me? We’ll use the AAMC BSS Matrix to identify some key learning issues for your development plan.
Resident: Well, my last patient was a 27-year-old man coming to see me after his second trip to the ED for a-fib following a night of heavy drinking. He was pretty shaken up by the experience and had no idea that binge drinking and atrial fibrillation could be related. He needed an INR check and warfarin adjustment but I wanted to counsel him about binge drinking. I’m not sure if I did a very good job of it.
Preceptor: Terrific case and lots of important behavioral and counseling issues we can discuss. Let’s map it onto the BSS Matrix and see what roles and what content jump out at us. First, identify what roles you might be required to fill as his primary care provider. Next, think about the BSS core content you need to know and the competencies you need to possess. Let’s fill in the cells with learning questions for this case together. (See Figure 41-1)
Preceptor: Great list! Now I’d like to ask you to circle the cell where you feel most competent already and the one to two cells where you feel least competent. Maybe we can start our conversation there and develop a learning approach to build your competence. [The conversation continues. .]
As can be deduced from the above examples and the various BSS competency and content lists, the process of developing CBME for BSS requires a high level of faculty and learner engagement, time, resources, and, most importantly, an institutional or organizational culture that supports this material. However, it is important to remember that even if those necessary conditions are not present, organizational transformation does occur and can be stimulated from both the bottom-up and top-down. In Chapter 42, the authors carefully detail the necessary and sufficient conditions to support BSS education and the steps to promote cultural shifts that ensure both the quality and sustainability of the resulting product.
Although BSS competencies may “set the bar,” it is the assessment toward achieving those competencies that allows us to “entrust” the learner with those clinical activities. Assessments may be formative (i.e., used to shape learning plans) or they may be summative (i.e., providing a grade or determining attainment of a particular skill level). Within the realm of CBME, most assessments measure skills or abilities rather than attitudes, since behavior and attitudes tend to be only weakly correlated. Direct observations of the learner are the most commonly used assessment methodology, but observations should occur frequently and require observer training and standardization. In Chapter 43, Carney and Milan describe a full menu of assessment and evaluation approaches, including the relative advantages and disadvantages of each approach. As they argue, regardless of the selected methodology, BSS competencies can and should be assessed frequently and in conjunction with other more biomedically based competencies.
Given the international convergence of competency-based models of medical education, ranging from medical school through medical residencies, it appears that CBME is here to stay for the foreseeable future. Although broad and including general “meta-skills” along with foundational scientific knowledge, the BSS fit within current CBME frameworks and can be measured with existing tools. As learners struggle to develop competence and teachers stretch to accommodate their needs, tools such as the AAMC BSS Matrix can identify the complex, interactive learning issues embedded in each case. Moreover, as the construct of EPAs further develops, perhaps BSS knowledge and skills can be seen as foundational to the practice of medicine in partnership with the biomedical sciences.