MD 2000

Progress in Integrating MD2000 into the Medical Curriculum (2.2/6)

Student Concerns: Students felt strongly that MD2000 needs to be better integrated into the curriculum with more faculty support and fewer “extra assignments”, and made a number of specific suggestions for improvement.

Response: While good progress has been make in implementing our competency-based curriculum, much still remains to be done. Below are noted some of the major benchmarks we have reached as well as a discussion of the remaining challenges that still need to be addressed.

Course Certification: All preclinical and clinical courses had been certified to address one or more of the abilities by the end of academic year 1999-2000.

Performance-based Assessment in Preclinical Courses: Performance-based assessment tools are employed in Bio 181–Human Morphology prosection; Bio 371– Medical Interviewing (direct observation); and Bio 373/374–Introduction to Clinical Medicine (direct observation by a preceptor and standardized patient physical examination). This year in Bio 158–Medical Microbiology and Bio 372–Epidemiology students read and report on research articles that promote critical analysis and lifelong learning skills.

While we are making progress in employing performance-based assessments that measure more accurately students’ basic science skills and their beginning level of clinical competence, we need additional forms of such assessments at the preclinical level. As course leaders begin to use more widely innovative technology, new performance-based assessment instruments will likely be incorporated into the curriculum.

Performance-based Assessment in Core Clerkship and Other Required Clinical Courses: In all of the core clerkships, as well as the sub-internships and the longitudinal ambulatory clerkships, there is now a common evaluation instrument employed to assess student competency in the abilities at the intermediate and the advanced levels. In the medicine, family medicine, pediatrics, and obstetrics and gynecology clerkships, there are Objective Structured Clinical Examinations (OSCEs) that assess student clinical competence. The fourth-year OSCE also addresses the MD2000 competencies. In addition, the completion of a basic clinical skills procedural log is now required of all graduating medical students.

We need additional performance-based measures in the core clerkships and the sub-internships. Toward that end, the restructured eight-week surgery clerkship may add an OSCE in order to assess clinical skills. We will also meet with sub-internship directors to discuss the development of an OSCE in this four week advanced course.

Lastly, with the new graduation competency requirements instituted by the Accreditation Council for Graduate Medical Education (ACGME), residents’ directors will also need to employ performance-based assessment measures in assessing the residents’ clinical competence. Joint assessment projects involving both residents and medical students will help us expand our use of innovative evaluation approaches.

MD2000 Faculty-Development Activities: To educate the faculty and residents in our new competency-based curriculum, the medical school currently offers these workshop activities:

1. The curriculum affairs office holds two annual faculty-development workshops that are open to all faculty. These workshops center on issues related to our competency-based curriculum.

2. Each September, the Office of Curriculum Affairs (OCA) hosts a new faculty orientation meeting in which we discuss our competency-based curriculum.

3. Each January, the curriculum affairs office holds a workshop for preclinical course leaders on improving instruction and evaluation.

4. In the spring 2001, OCA staff will meet with residents and resident directors to discuss the challenges of teaching medical students and how the office may be helpful in enabling clinical faculty and staff to understand better our competency-based curriculum. Staff will also discuss with residents effective ways to evaluate our students’ clinical competence.

The curriculum affairs staff is also in the process of developing an assessment handbook for faculty that will detail how preclinical courses and clerkships, the sub-internships and the longitudinal ambulatory clerkships are addressing the abilities. This material will be placed in course syllabi for students’ review. Moreover, in the handbook there will be a compendium of other assessment tools that course leaders might incorporate in their courses for assessing the nine competencies.

The report recommends that the implementation of the curriculum be suspended until competency requirements have been fully defined. Requirements for obtaining each of the nine abilities have been substantially defined and made operational. However, some requirements will necessarily have to be changed in light of evaluation data and recommendations by students and faculty to strengthen or improve the assessment process. Starting in academic year 2001–02, all MD2000 requirements will remain stable and none will change during “midstream.” We anticipate no major new requirements being introduced in the next several years.

Another recommendation of the report is that the criteria for advanced-level competency need to be realistic. The criteria for obtaining the advanced level in Ability VII–The Social and Community Contexts of Health Care and Ability VIII–Moral Reasoning and Clinical Ethics have been clarified and make more realistic to obtain. For instance, last academic year, no students received advanced-level competency in Abil-ity VII. This year, so far, three students have been awarded Ability VII advanced-level competency credit.

The Ability VIII–Moral Reasoning and Clinical Ethics assessment committee will present to the Medical Curriculum Committee in February new guidelines for students gaining advanced level competency credit. The criteria in the other abilities are realistic and many students satisfy these competencies at the advanced level.

Students voiced concern in the report about the ways in which students had to demonstrate competence. In particular, objections were raised about having to write three to five page essays. The responsibility for assessing student competence rests with the course leaders. Those who require a written assignment in order to evaluate whether students have obtained a competency have been informed of students’ concerns and have agreed to explore ways to develop alternative assessment strategies that are more related to day-to-day clinical activities.

Assess MD2000 (3.0/6)

Student Concerns: Students requested an assessment of the benefit of MD2000 to their medical education.

Response: For the last three years, we have employed an outside evaluator to assess the effectiveness of our competency-based curriculum. These findings have been conveyed to the dean, the provost and an institutional assessment committee composed of leading educational evaluators. All major recommendations have been acted upon. The evaluation reports include the views of students, faculty, and administrators. These reports are available to students upon request. During the spring semester, we plan to interview recent graduates of the program who are now in the first year of their residency. This data will be presented to the Institutional Assessment Committee at its June meeting.

We also receive yearly formal feedback from graduating fourth-year students via the medical school graduation questionnaire and through our individual consultations with fourth-year students. Recently, at the suggestion of students, we were invited to attend first-year and second-year class meetings each semester at which MD2000 concerns can be discussed. So far, we have implemented each student suggestion brought up in these meetings. These meetings with the first-year and second-year classes will be conducted on a regular basis each year.

It should be noted that student views of the effectiveness of our competency-based curriculum have varied widely as the innovation has been implemented over the last five years. We hope that by addressing the concerns and recommendations of students in this report that we will improve the quality of our curriculum innovation.

Timeframe: Ongoing evaluation and assessment of our competency-based curriculum.

Guidance on How to Meet Requirements (3.2/6):

Student Concerns: Students requested more guidance on how students can meet MD2000 requirements, and noted that the requirements for a given ability do not seem to be consistent and often vary depending on which administrator students speak to.

Response: For the past two years, we have been developing an MD2000 advising system that we will expand next year, particularly in light of student concerns and recommendations contained in the Medical School Student Senate’s report.

At this time, MD2000 handbooks are provided to PLME students, first-year and second-year students and third- and fourth-year students. Curriculum affairs staff meets individually with all fourth-year medical students to help them plan their program to meet the competency requirements at the intermediate and the advanced levels. This academic year, staff met with incoming first-year students, particularly postbaccalaureate students, and with Brown–Dartmouth students and other transfer students who are starting their third year. Two meetings are also held with the first-year class, first during orientation and then later at a class meeting. Similar meetings will be conducted next year. At these meetings, ability assessment committee chairs will address students’ questions on the meaning and importance of their respective abilities.

A hard copy of student progress in meeting the abilities is now being sent to first-year and second-year students at the end of each semester. Copies of competency attainment charts are sent to each third- and fourth-year student at appropriate times during the year, for example, after quarter II or III in the third year, at the start of the fourth year and in February or early March of the fourth year.

Make the Process for Retroactive Credit Less Burdensome (3.7/6):

Student Concerns: Students were moderately concerned that the process to apply for retroactive credit should be made less burdensome, especially for postbaccalaureate students.

Response: Obtaining retroactive credits is normally necessary for only two abilities: Ability VII–The Social and Community Contexts of Health Care and Ability VIII–Moral Reasoning and Clinical Ethics. Recently, The Ability VII assessment committee simplified the process by which students may gain retroactive competency credit at the beginning level for previous community service or research. Particularly, post-baccalaureate and EIP students will now only have to discuss their experiences with Office of Curriculum Affairs staff and present, only when necessary, documentation of their successful completion of the community service or research projects.

Likewise, Ability VIII–Moral Reasoning and Clinical Ethics has developed a simplified process that reduces paper work. Students need only submit to the assessment committee a course syllabus of a past ethics course to receive competency credit. At colleges and universities where students are part of the Early Identification Program, we have certified ethics courses that may be used to gain Ability VIII competency credit at the beginning level.

Standardize Competencies Across Electives (3.8/6)

Student Concerns: Students were concerned that faculty were not applying the same standards in certifying competency across the curriculum. Students were also concerned that not all faculty were adequately informed about the competency-based curriculum.

Response: Initially, when implementing MD2000, there was a consensus among assessment committee members that it was very important that course leaders have discretion in determining which ability(s) they wish to address in their electives. Balancing the needs of a school wide curriculum with the rights of the faculty to determine what and how they will teach is a challenge in every medical school, and one from which Brown is not immune. We will continue our ongoing efforts to monitor and discuss this matter with the faculty.

As the assessment committees review courses, they will be asked to address the standardization issue, particularly in regard to the various community-service/research projects that students undertake for Ability VII–The Social and Community Context of Health Care competency credit.

Also, some elective course leaders certify a student for an ability at either the intermediate or advanced level–– given the student’s performance and/or the number of weeks the student spends in the elective. (For example, it is hard to assess a student’s level of performance at the advanced level if s/he takes the elective for only one week.) Until the Medical Student Senate report, we were unaware that “students who are enrolled in the same clerkship or elective are often granted different level of competencies. The Office of Curriculum Affairs will email third-and fourth-year students asking them to identify such instances. We will then contact the course leaders and work out a more equitable and systematic certification process.

In several clerkships which certify for Ability VII–The Social and Community Contexts of Health Care, students do not automatically gain these competency credits by completing the normal clerkship activities. This ability is optional in these clerkships and competency credit is given depending on whether a student completes a special project. This is a temporary arrangement until the new community health clerkship is in place in academic year 2001–02. This clerkship will then certify for Ability VII.

In the past, the medical school has held information sessions with department chairs and faculty, but this has not been done on a regular basis in the last two years. We will implement the suggestion that we hold regular information sessions to make the faculty more knowledgeable in our competency-based curriculum. Curriculum affairs staff will attend course faculty meetings as well as attend department meetings. In addition, the director of the Office of Curriculum Affairs will now sit on the Graduate Medical Education Committee composed of Brown Medical School residency directors. Through these monthly meetings, he will be able to disseminate information on MD2000 as well as engage in joint planning with residency directors in addressing similar competency-based curriculum issues.

In regard to students gaining information as to which abilities are addressed in a course, the OCA staff has included the information in the elective catalog that is in hard cover and on line through the web. The curriculum office also has distributed bookmarks that list the competencies addressed in the preclinical courses, the core clerkships, the sub-internships and the longitudinal ambulatory clerkships. The MD2000 handbook for the fourth year also contains a list of all the electives and the advanced competencies they are certified to give.

In March 2001, medical school administrators will attend a workshop on developing uniform answers to substantive MD2000 questions and concerns. The Office of Curriculum Affairs will also develop written responses to substantive questions that will be published in the various MD2000 handbooks.

MD2000 Web Program (4.8/6):

Student Concerns: Students asked that the MD2000 web program, MedPlan, be made more accessible to students.

Response: MedPlan is now accessible from the computer cluster on Macs and PCs. It can also be accessed on any Brown computer and by spring 2001 it will be accessible on the web. Directions on how to access MedPlan are found in the computer cluster and will be sent to all students. Directions are also included in many Office of Student Affairs and Office of Curriculum Affairs publications. The medical student handbooks provide information on what abilities require independent work and which abilities are met by successfully completing a course.

Timeframe: MedPlan will be online by April 1, 2001.