Technology Enhanced Learning Task Force

 

Final Report

 

 

University of Minnesota

Medical School

 

 

 

 

January 30, 2005

 

 

 

Stuart M. Speedie, PhD

Chair


Table of Contents

Executive Summary. 3

Introduction. 5

Charge to the Task Force. 5

Membership List 6

Deliberative Process. 6

Summary Review of the Literature. 7

Recommendations. 9

Recommendation: Learning Spaces (Rank 1) 9

Recommendation: Multi-Site Access (Rank 2) 13

Recommendation: Simulations (Rank 2) 16

Recommendation: Blended Courses ( Rank 2) 18

Recommendation: Ad Hoc Communications (Rank 3) 20

Recommendation:  Electronic Medical Records ( Rank 3) 22

Recommendation: Computer-based Testing (Rank 3) 24

Recommendation: Performance Tracking ( Rank 3) 26

Recommendation: Parity ( Rank  4) 27

Recommendation: eLearning Preparation ( Rank 4) 30

Recommendation: eLearning Resource (Rank 4) 31

Summary. 32

Appendix A: TEL Task Force White Paper 33

Appendix B: Literature Review.. 42

Appendix C: TEL Best Practices at the University of Minnesota Medical School 43

Appendix D: Technology Enhanced Learning Methods. 52

 


Executive Summary

In October 2004 Dean Powell directed Senior Associate Dean Kathleen Watson and Stuart Speedie, Director of Education Informatics, to convene a task force to consider the utility and costs of additional investments in technology enhanced learning (TEL) for undergraduate, graduate and continuing medical education.  The motivation for this work arose from several sources. The University of Minnesota’s planning priorities for the Biennial Budget placed additional emphasis on technology enhanced learning. There was also a desire to further explore ways by which vertical integration of medical education could be achieved, how these technologies could be employed to address the issues raised by the latest LCME report concerning the relationships between the Twin Cities and Duluth campuses and how TEL might play a role in addressing the larger issues facing medical education in the context of today’s health care system.

Charge to the Task Force

The task force received the following charge from Dean Powell with respect to their considerations of technology enhanced learning:  The group was to make recommendations for use of information technology that would:

 

  1. Ensure comparability of learning experiences for students across sites in the Medical School with special emphasis upon the Duluth Campus, but also consideration of other sites, such as RPAP and the Twin Cities community sites for clinical medicine.
  2. Identify the equipment and technologic infrastructure required to facilitate vertical integration of teaching and off-schedule, self-directed learning for undergraduate and graduate medical education.
  3. Identify the equipment and technologic infrastructure required to facilitate rigorous, reflective outcomes-based evaluation of student performance.
  4. Review the medical educational literature and national (e.g. AAMC) task forces on the best practices in effective technology-enhanced learning and evaluation methods. 
  5. Provide examples of best practices in Technology-Enhanced Learning currently being used in UME, GME and CME at the University of Minnesota.
  6. Create matrices of instructional and evaluation methods in Technology-enhanced learning, as a reference for faculty.
  7. Make recommendations for personnel needed to create and maintain new technologies.
  8. Estimate the costs per student associated with the equipment, technologic infrastructure and personnel.

 

Below are the recommendations of the task force that address items 1,2, 3 of the report.  The report also addresses cost issues within each of the recommendation discussions.  A matrix summarizing the TEL findings from the literature is contained in Appendix  B.  A list of TEL best practices at the Twin Cities and Duluth campuses is provided in Appendix C.  Appendix D provides a table listing the TEL strategies used for instruction and assessment and briefly discusses the applicability of each.

Task Force Recommendations

The following recommendations are put forth by the TEL task force in response to the charge to the group.  Each row provides label and a one sentence summary of the entire recommendation.  And the ranked order of importance in meeting that charge.  Multiple recommendations may be listed at the same rank indicating that the task force considered them to be of equal importance.

 

 

Rank

Label

Recommendation Summary

1

Learning Spaces

Create/adopt a common, standard, secure, permanently available student-centered on-line learning space

2

Multi-site Access

Provide multi-site, multi-media access to clinical education activities such as seminars, Grand Rounds, Morning Reports, and clerkship didactic presentations that are accessible from anywhere and at any time with a network connection. 

2

Simulations

Increase the use of technology-based simulations and simulators for teaching and evaluating procedural skills.

2

Blended Courses

Work to evolve both basic science and clinical courses with significant lecture content to a more blended approach consisting of a mixture of face-to-face group meetings combined with self-study modules, case-based learning, narrated slide presentations and other on-line activities that the student can complete on their own schedule.

3

Ad Hoc Communications

Develop/adopt means for faculty and students to communicate easily and frequently on an ad hoc basis with each other regardless of site to facilitate advising and student group communications.

3

Electronic Health Record

Introduce the electronic medical record as a teaching tool from the very beginning of the curriculum and work to assure that students have appropriate access to electronic records during their clerkships.

3

Computer-based Testing

Adopt and implement computer-based testing capabilities.

3

Performance Tracking

Develop/adopt a unified, standard student performance tracking system that incorporates all forms of evaluation including classroom tests, faculty evaluations and procedural skills and competencies checklists.

4

Parity

Create parity in learning opportunities for medical students regardless of their personal financial ability to afford the equipment to take advantage of e-learning technologies.

4

eLearning Preparation

Prepare trainees for life-long e-learning by defining competencies in and teaching students e-learning methods.

4

eLearning Resources

 Serve as an e-learning resource for teaching faculty (e.g.  RPAP preceptors, residents, attendings), practicing physicians and health care systems in Minnesota.

Costs

The estimated costs of implementing these recommendations fall into two categories and do not take into account existing Medical School resources that might be available.  Investments are one-time costs incurred for the purpose of the startup of the recommended action.  Continuing costs are those necessary for maintaining the functions of the recommendations once they have been implemented.  Both categories include personnel costs, but those in the investment category are restricted to a defined period of time not exceeding three years.  The total investment costs for all recommendations per enrolled medical student (not counting residents) over a three year implementation period  range from $496 to $2,676.  In addition, the annual continuing costs range from $658 to $1,713 per medical student.  The number of FTEs of various classes required to implement all recommendations is estimated to be from 8.15 to 11.5 for continuing operations.  The investment costs incorporated a range of 5.5 to 9.0 FTE years of effort.


Introduction

Information technology is playing an increasing prominent role in higher education.  Even when administrative uses such as on-line course registration are excluded, the uses of computer-based applications for teaching and learning are disseminating with considerable speed throughout our college and university systems.  This approach, now becoming known as technology enhanced learning or TEL has become a priority initiative at the University of Minnesota.  Administrative units have been created at the University Provost’s level and within the Academic Health Center.  This prioritization is reflected in the interest and activities of the faculty, many of whom have embraced components of technology enhanced learning for their teaching responsibilities.  In Fall 2004, 1,054 courses at the University of Minnesota have a WebCT website.  Each of the schools in the Academic Health Center is making investments in a variety of TEL approaches from using interactive videoconferencing to creating a virtual classroom involving students in the Twin Cities and Duluth to using simulators to teach clinical skills.

 

At the same time students have come to realize and are now demanding information technology tools to assist them in their studies.  Surveys of incoming medical students reveal that essentially 100% either have a computer or have ready access to one.  By the time students enter their clinical years approximately 80% have purchased a personal digital assistant.  Laptop computers are appearing in classrooms and students are increasingly dependent on the electronic resources made available by the Medical School and the Bio-Medical Library.

 

In July of 2000, the Medical School created the Education Informatics group in the Office of Medical Education on the Twin Cities campus to focus on TEL support for medical education in response to the 1999 LCME Report.  This group has worked successfully with faculty and students to provide educational resources that primarily support undergraduate medical education.  The Duluth campus has undertaken a similar initiative with equal if not greater success in deploying TEL applications for their students.  In several respects the Duluth campus has assumed a leading role in classroom use of tablet PCs, in developing self instructional modules, and in web-based testing.  In addition, several departments with specific course responsibilities have undertaking efforts to incorporate TEL into their courses and clerkships.

 

In October 2004 Dean Powell directed Senior Associate Dean Kathleen Watson and Director of Education Informatics Stuart Speedie to convene a task force for the purpose of considering the utility and costs of additional investments in technology enhanced learning for medical education and extending those investments to the graduate and continuing medical education.  The motivation for this work arose out of the University of Minnesota’s planning priorities that placed additional emphasis on technology enhanced learning. It was also motivated by a desire to further explore ways by which vertical integration of medical education could be achieved, how these technologies could be employed to address the issues raised by the latest LCME report concerning the relationships between the Twin Cities and Duluth campuses and how TEL might play a role in addressing the larger issues facing medical education in the context of today’s health care system.

Charge to the Task Force

The Task force was charged with making recommendations regarding technology enhanced learning that would:

 

  1. Ensure comparability of learning experiences for students across sites in the Medical School, with special emphasis upon the Duluth Campus, but also consideration of other sites, such as RPAP and the Twin Cities community sites for clinical medicine
  2. Identify the equipment and technologic infrastructure required to facilitate vertical integration of teaching and off-schedule, self-directed learning for undergraduate and graduate medical education.
  3. Identify the equipment and technologic infrastructure required to facilitate rigorous, reflective outcomes-based evaluation of student performance.
  4. Review the medical educational literature and national (eg AAMC) task forces on the best practices in effective technology-enhanced learning and evaluation methods. 
  5. Provide examples of best practices in Technology-Enhanced Learning currently being used in UME, GME and CME at the University of Minnesota.
  6. Create matrices of instructional and evaluation methods in technology-enhanced learning, as a reference for faculty.
  7. Make recommendations for personnel needed to create and maintain new technologies.
  8. Estimate the costs per student associated with the equipment, technologic infrastructure and personnel.

Membership List

The following representatives from the Medical School and the University of Minnesota agreed to serve on the Task Force:

Bryan Armitage 

Student, Twin Cities Campus

Bradley Benson, MD

Faculty, Medicine (pediatrics)

Adam Boettcher

Student, Twin Cities Campus

Joe Clinton, MD

Chair, Dept. of Emergency Medicine

Krista Gallagher

Medical Education Webmaster

Glenn Giesler, PhD

Faculty, Neuroscience

Gwen Halaas, MD

RPAP Director, Family Medicine and Community Health

Richard Hoffman, PhD

Associate Dean, Duluth Campus

Andrew Calvin

Student, Twin Cities Campus

Linda Perkowski, PhD

Associate Dean, Twin Cities Campus

Edward Ratner, MD

Faculty, Dept. of Medicine

Kenneth Roberts, PhD

Faculty, Dept. of Urologic Surgery

Tom Ronay

Student, Twin Cities Campus

Janet Shanedling

Director of Educational Development, Academic Health Center

Theodore Thompson, MD

Director of Clinical Education

Paul Tuite, MD

Faculty, Neurology

Jennifer Welsh, MD

Faculty, Family Medicine and Community Health

Eric Celeste

[insert title], University Libraries

Linda Jorn

Director, University of Minnesota Digital Media Center

Ramsey Peterson

Student, Duluth Campus

Mark Summers

TEL Specialist, Duluth Campus

Deliberative Process

The Table 1 below documents the deliberative process that the task force undertook in order to arrive at its recommendations.  Prior to the first meeting of the task force, Stuart Speedie developed a White Paper on technology enhanced learning to inform the members of the current state of TEL in the medical school, describe the current findings from the literature concerning TEL, and to lay the groundwork for the deliberations of the group (See Appendix A).  Early in the process the group decided that it could most effectively accomplish its task if it focused the majority of its efforts on the first three charges.  Subgroups and/or the task force leadership responded to the balance of the items.  In the initial discussion it was recognized that technology enhanced learning is a group of enabling technologies rather than a defining strategy, i.e. TEL is not an educational end in itself.  Accordingly the group decided that the best approach their assignment required an examination of and recommendations concerning educational strategies that could be facilitated by technology and that would contribute to a better educational environment for both students and faculty.   The table below details the activities of the task force.

 

Date

Activity

October 14, 2004

Organizational Meeting

November 13, 2004

Workshop on TEL

Outside Speaker/TEL Expert

Development of Initial Recommendations

December 2, 2004

Review and discussion of initial recommendations

December 3-12, 2004

Electronic rating of draft recommendations with respect to overall educational importance.

December 14, 2004

Review and discussion of revised recommendations.

December 15-January 12, 2004

Electronic rating of draft recommendations with respect to their importance to meeting the charge of the task force.

January 13, 2005

Final review and approval of the Recommendations.

January 29, 2005

review and approval of Final Report

February 1, 2005

Deliver final report to the Dean

Table 1.  Task Force Calendar of Activities.

Summary Review of the Literature

Stuart Speedie and Edward Ratner undertook a review of the literature in technology enhanced learning (also known as computer-based learning, computer assisted teaching, distance education, among many such terms).  This review covered both the medical education literature and the general higher education literature from 1970 to present.  The complete review is located in Appendix B.  Here we summarize the findings:

 

  • TEL works. It is possible to create technology enhanced learning applications like those described above that are capable of achieving their stated objectives.  They work reliably and students can use them for learning, evaluation and communication.  Students can and do make use of and learn from TEL materials and strategies.

 

  • Computer-based independent learning is at least equivalent to standard teaching methods in producing student learning.  Repeated well-designed studies have demonstrated this finding.  In addition a few studies have found that TEL based independent learning methods produce greater learning than traditional teaching methods.

 

  • Learning can be faster with TEL materials.  Of the several studies that have examined the issue of time to achievement in TEL-based independent learning, most have demonstrated learning may take longer using traditional teaching methods.  This likely due to the fact that students in traditional teaching are controlled by the instructor’s pace, but in TEL-based independent learning they can proceed at their own pace.  If this pace is quicker than that set by the instructor, faster learning is likely to result.

 

  • Given a choice, students generally prefer face to face teaching over recorded presentations but then learn equally well from both.  While students can use and learn from TEL based independent learning, they report a continuing preference for a live teacher.  This may well be due as much to the fact that their entire educational experience is within the conventional system and they have not yet developed the skills to learn independently in a TEL environment.  There is also the very real advantage that the learning environment in which teaching is done by a live instructor can be more flexible and responsive to students than a TEL environment.  However evidence from the literature also indicates that they learn just as well with either approach.

 

  • TEL methods require a larger initial commitment to preparation than lectures but may reduce faculty time commitments in the long run.  Development of TEL materials is often reported to require more time and resources than the lectures even with the advent of PowerPoint presentations.  In many ways the development of TEL materials is equivalent to writing a textbook – except that the textbook is in an electronic format and is in a more complex format than the typical printed textbook.  In addition those TEL strategies that involve electronic communications with students either through email, discussion boards or chat rooms may require more instructor time than traditional office hours.  However, once TEL materials are developed students can review and learn from them without the presence of the instructor which frees the instructor for other activities.

 

  • TEL materials are relatively expensive to develop.  Creation of TEL-based learning materials is often beyond the expertise of the typical teacher and usually requires the time and skills of educational technology specialists.  These same specialists are also often needed to maintain and update the materials.  In addition special purpose hardware is sometimes requires such as when creating video clips or conducting videoconferences.  All these are costs over and above the normal costs of instruction.

 

  • Simulations are an effective method of teaching clinical skills.  Numerous studies of clinical simulations and simulators have been able to demonstrate that students can use them to learn skills which they might otherwise have difficulty learning due to either considerations about the safety of the patient or the lack of patients upon which they can practice.  Post training levels of competence have been judged to be generally equal whether the student was training with a simulator or with a real patient.  Simulations also have the advantage of being able to repeat the same scenario over again until the skill is properly demonstrated and they can also be used for skill evaluation.

 

  • The quality of the teaching is more important than the medium by which it is delivered.  The way in which an educational experience is designed be it a large group lecture or a self-instructional, web-based module has more to do with whether or not student will learn from it than the medium by which the students receive it.  A well organized self-instructional module that takes advantage of the well-known principles of teaching is more likely to result in learning than the ill-designed, disorganized, poorly presented lecture.  Similarly a well organized lecture presented by a highly motivated instructor with an appealing speaking style will most likely be more effective than a poorly-designed web module.  To borrow from and perhaps contradict Marshall MacLuhan – the message is more important than the medium.

 

It is important to note that there have been relatively few high quality studies of the impact of TEL strategies on students and faculty members.  One of the principal recommendations of every review article that has been published is that well designed and controlled studies need to be conducted to further delineate the impact of such strategies on student learning.  At best the literature does tell us that TEL strategies can be used to effectively teach students.  The decision to employ a particular TEL approach must therefore rely upon informed judgments of the value of such approaches that may be outside the current scope and findings of the educational literature.


Recommendations

In this section we present the recommendations of the task force in the priority order identified by the group.  These recommendations are presented individually and are accompanied by explanatory text that provides the rationale and expected benefits, technologies and infrastructure that will required, and the estimated investment and operating resources that would be required. Investments are defined as the costs of the initial implementation of the recommendation.  Continuing expenses are those annual personnel and operating costs necessary to maintain the operation of the implemented recommendation.

Recommendation: Learning Spaces (Rank 1)

 

 

Create/adopt common, standard, secure, permanently available student-centered on-line learning spaces that will:

  • Promote availability and sharing of courses and course materials between the two campuses.  As part of this effort work to adopt common course materials where possible and comparable materials otherwise.
  • Assure that comprehensive, electronic references are available to each student when and where they need them.
  • Provide a multi-platform (e.g. desktop, laptop, and PDA) repository or archive of instructional modules for H&P techniques, various other patient exams, basic science reviews with exemplar video and other tools such as review of surface anatomy. Use materials that are interprofessional when appropriate.
  • Provide a virtual learning study carrel for each student that includes such things as storage for annotated notes, personnel reference collections and facilitates group studying and projects. 
  • Be organized around an on-line curriculum database that accurately portrays the medical school curriculum, how it is taught at each of the campuses, and links the educational activities to curricular objectives, clinical skills and competencies.

 

Explanation

Medical students need tools and technologies that help them to learn more effectively and efficiently.  Today the learning tools of most students consist of a backpack stuffed with textbooks and class handouts with scribbled marginal notes.  These are often supplemented with paper handouts from the Knowledge Coop or the Note Coop and printouts of the instructors PowerPoint slides.  Class handouts may point them to certain references in the Bio-Medical Library.  They can review and obtain additional copies of class materials on the Meded website and can listen to lectures again at Lectures on Line.   In some courses and clerkships they can go to the course website to find instructional materials and learning modules that cover the course concepts.  The materials that students find from course to course and clerkship to clerkship vary greatly in terms of what is available, where it can be found, and how it is connected to other parts of the curriculum.  Online materials that are available to students during a course or clerkship may or may not be available to them after it is finished.  While we espouse evidence-based medicine, we provide few tools to aggregate and organize findings from the literature that are the basis of this approach.  Finally while excellent materials are being developed and supplied to students on both the Twin Cities and Duluth campuses, there is little if any ability to share those materials so that the students can take advantage of the best of both campuses.

  

With the ever expanding body of knowledge in medicine it is no longer possible for every student or practicing physician to hold the entire relevant corpus of medicine in their head.  The dependence on reference materials, literature searches and practice guidelines is growing and will continue to grow for the foreseeable future.  Most if not all of these sources now exist in electronic form and are being designed so that they can be accessible in a variety of modes from the desktop to the point of care.  Drug references are available on the Bio-Medical library’s website from any network connected desktop or laptop and similar versions are widely available for the personal digital assistant (PDA).  Many of the major references such as Harrison’s, Harriet Lane and the Five Minute Clinical Consult have electronic versions for the same range of devices.  UptoDate, one of the most popular clinical references for practicing physicians is widely available electronically and numerous consensus treatment guidelines for a variety of diseases are now available electronically.  We needed to assure that our students have the necessary access to these references wherever they are located – in the classroom, studying at home, on the wards or in a clinic. 

 

It is important to note that the Bio-Medical Library may not be able to provide all of the reference materials needs by students.  While it does provide access to many electronic resources including full-text journals, there are a number of important and useful references for the PDA that the library does not currently license.  Examples of these are the very references mentioned above. 

 

We also need to acknowledge the reality that often students learn a concept or skill the first time it must be used to take care of a patient, not the first time it is taught.  Thus for better learning it is wise to facilitate “just in time” learning by providing the ability for students to easily locate and review the knowledge and skills necessary to take care of that patient.  They need to be able to do this in a timely manner by being able to locate the relevant information as quickly as possible.  Even if the necessary information is available electronically, an organizing structure that the student understands and can readily use is necessary to promote timely access.

 

To address these issues the task force recommends that the Medical School create a permanent electronic learning space for each student that will assist them to learn more effectively and efficiently. It should allow them to create a personal knowledge base that they can continue to use and build as they progress from medical student to resident to practicing physician.  The metaphor chosen to explain this is the permanent, personal library study carrel.  Such a carrel is space assigned exclusively for use of its occupant and which can be organized in a manner that suites the needs and learning style of the person using it.  Secondly the carrel is located within the library as a knowledge resource with ready, essentially unimpeded access to all of the materials of the library whether they be reference books, scientific publications or other materials.  Third, the knowledge resources are organized in several ways for systematic retrieval of relevant information and there are numerous tools for searching and retrieving relevant knowledge.  Once that knowledge is retrieved it can be taken into the carrel and organized according to the understanding and needs of the person using it.  The task force is proposing the electronic equivalent of this study carrel in the form of an online, network-accessible “learning space” for each student where then can collect and organize the materials that they need for learning including annotated notes, class PowerPoint presentations, reference collections, learning modules, and class projects among many possible examples. 

 

To make the learning space a truly powerful and useful tool for students, it must provide easy access to all of the materials that the students need including course objectives; instructional modules for such things as H&P exams, basic science reviews and other materials that may be of assistance in learning.  This implies that there needs to be a well-organized repository of these materials analogous to the structure of a library to which the student has complete and continuing access.  Since health care is an interprofessional activity and there may well be materials developed in other disciplines that are useful for medical students such a repository should strive to include them where appropriate. Furthermore students need access to these materials is a variety of situations.  They need to access them at home from their desktop computer while studying for an exam the next day.  They need to access them from their laptop in class as part of a group discussion or problem solving exercise.  They need to access them on the wards from the computer at the nursing station or patient bedside.  And they may need access from their PDA in the exam room as they are conducting an examination.  This implies that the repository of information has to be able to provide its information to students using multiple platforms.

 

A significant component of this recommendation is that such a repository be equally accessible and useful on both the Twin Cities and Duluth campuses.  For example there have been exemplary self-instructional materials and electronic course resources developed at each of the campuses and our students would benefit from being able to access both for the purpose of learning.  We believe that such sharing will lead to a strengthening of the education programs at both campuses and may encourage the adoption of learning materials with a greater degree of commonality.  This, in turn, will allow us to even more convincing demonstrate the comparability of programs at the Duluth and the Twin Cities.

 

Finally we propose an organizing mechanism for the repository that focuses on the common set of competencies adopted by the faculty that are reflected in the curricular structure and implemented in the courses and clerkships of the curriculum.   The basis for this organization already exists in the Curriculum Database and should be expanded and modified to provide an organizing framework for the proposed repository by adding a structure of competencies and objectives.     

Equipment and Infrastructure

In many respects the individual components and basic infrastructure that can be used to build these learning spaces are already in place.  The Medical Education websites at the Twin Cities and Duluth campuses, Lectures on Line, the Medical Education Curriculum Database, the University and AHC portals, the WebCT Vista class management system, the ePortfolio system and the Bio-Medical library’s resources are all now available to our students.  The University is making a significant investment in networked storage systems that can provide very large amounts of online “space” to support this concept.  The University’s Gigabit network combined with its expanding network of wireless access points provides quick access to online materials.  From a technical standpoint implementation of this recommendation would require at most the additional of one or two servers to provide additional functionality not supplied by the existing systems.    

 

The challenge of implementing this recommendation is to assemble these building blocks into a unified and simple-to-use, integrated system that students can readily make use of for the purpose of learning.  All of the components described above were built for different purposes and in many respects do not work well together.  Significant effort will be required to create an easy-to-use, integrated system that can bring together all of these diverse components and make that system available for student use whenever they need it and wherever they are located.

 

The success of this recommendation is critically dependent on the assistance and cooperation of the Biomedical Library.  The systems that they provide for accessing the literature are especially important to medical education.  Their experience of licensing electronic resources is invaluable and will be of great assistance as we move to expand the range of resources available to our students.  The library can provide the administrative infrastructure to administer these licensing programs and the Medical School should take advantage of the support they can provide.

 

We will also need to work with the University groups that support and administer the WebCT Vista course management system so that is adapted to support this recommendation.  Specifically it is necessary for students to maintain permanent access to certain elements of their courses during their entire student career as well as their career as a practicing physician.  This requires at a minimum that the student remain “enrolled” in a WebCT course for the four years of undergraduate education and consequently that multiple editions of each course be maintained. 

 

Finally an exceedingly important component of this infrastructure is the collection of learning modules that constitute the recommended repository.  It should be recognized that there are a number of these modules already available, some of which have been created by our own faculty in Duluth and on the Twin Cities campus. While the Medical School has the rights to use certain locally developed materials such as Dr. Downing’s Histology Time, it will be necessary to purchase or license access to others and to develop additional modules to “fill in the gaps” where there are no available materials or those materials are too expensive.

Estimated Required Resources

While elements of the infrastructure are largely in place, this recommendation will require both a significant startup investment as well as financial resources for continuing support.  In particular, the idea of a Learning Space is a cutting edge concept that does not currently exist as commercially available software and so will require investment in software development rather than a direct purchase.

 

Investment:

The resources required to implement this recommendation fall into three categories:  software development required for the learning space concept, module development to create the necessary additions to the repository materials in order to make it a complete and useful resource and purchase of existing commercially available modules where needed.  It is estimated that development of the learning space software will require 2 FTE years of software developers at $160,000.  This amount would likely be spread over a three year time span.  The investment in module development will depend on the number that need to be developed.  Current estimates indicate that the development of a single instructional module costs about $7,000.  If it is assumed that from 10 to 40 modules would be needed, this would cost from $70,000 to $280,000 including instructional developer time and production costs.  This amount would likely be spread over a two year time span.  In addition we estimate that it would be necessary to purchase up to $50,000 in additional modules.  Thus the total investment amount ranges from $230,000 to $490,000.

 

Continuing Expenses:

Continuing expenses will consist of licensing fees for some modules and personnel for ongoing maintenance and continuing development of the system. 

 

Operating Costs: We would anticipate that licensing fees would range from $0 - $150,000 based primarily on the number of different references that would be included.

 

Personnel:  It is estimated that this will require a 0.4 FTE systems developer to maintain and improve the system and its components.  


Recommendation: Multi-Site Access (Rank 2)

 

 

Provide multi-site, multi-media access to clinical education activities such as seminars, Grand Rounds, Morning Reports, and clerkship didactic presentations that are accessible from anywhere with a network connection.  Record these sessions for later review and for those who need greater flexibility.

 

Explanation 

Medical students (undergraduates and residents) are located on the Twin Cities and Duluth Campus as well as scattered over multiple locations throughout the Metro area and the state of Minnesota in programs such as RPAP.  In addition they are often not available at the time that a particular educational activity is taking place.  Yet there are numerous types of activities that they do need to attend, be able to review and would certain benefit from including clerkship didactic sessions, departmental grand rounds, morning reports, class lectures and discussion groups.  Since many of these are discussion-based with only the general topic determined in advance, they do not lend themselves to pre-defined and developed on-line modules.  Rather there needs to be a method for students and others to attend such sessions from a distance or at a time other than the scheduled time.   

 

The solution proposed is to institute a system where these sessions are made available to geographically separated sites either via high quality, interactive videoconferencing or via internet-based webcasting.  For example, a Grand Rounds session typically consists of a PowerPoint presentation on a given topic accompanied by a discussion among the participants.  Using a combination of the internet to share the PowerPoint slides and interactive videoconferencing, participants at other than the originating site can fully participate in the presentation and discussions.  Similarly but less expensively, Breeze technology can be used distribute the slides and audio from the presentation to any other site connected to the internet and would allow participants to submit written comments and questions for the presenter.  The Breeze approach is significantly less expensive and could reach a wider audience due to the lower costs of participation.  Employing this technology would facilitate recording and would make possible the storage and retrieval of such sessions for later viewing.  This would be similar to what the Medical School does now for the lectures recorded as part of the Lectures on Line program at the Twin Cities campus.  At the same time the approach can take advantage of existing technologies at the University of Minnesota and the spreading availability of broadband internet access throughout the State. 

 

The task force believes that implementation of this recommendation would lead to a wider availability of educational activities and could lead to a greater comparability of educational experiences across the spectrum of education due to the ability to share the same educational activity among different geographic locations.  In addition, recording of these sessions would provide increased flexibility for students, residents and other physicians who could shift viewing these sessions to times that fit within their own schedules.

Equipment and Infrastructure

Implementation of this recommendation would require a combination of technologies and high bandwidth internet accessibility in meeting rooms where these sessions take place.    Fortunately on the Twin Cities and Duluth campuses the University has supplied high speed accessibility to every classroom and meeting room.  FUMC has similar accessibility.  For webcasting of sessions, the University provides the Breeze Live server that can record the audio and screen presentations from a session for later access and simultaneously provide the live session to those connected to the internet.  Breeze Live which can run on a laptop with an attached microphone provides limited interaction in that remote users can submit online questions or comments and it does not normally record the video from the session.   For sessions where live, interactive video is important a more sophisticated system such as a Polycom Viewstation is required.  Fortunately these units can also use the network for communications.  PC based solutions for interactive videoconferencing are rapidly declining in price and should further reduce the costs of this method in the near future. 

 

One potentially difficult technical problem that must be faced is the problem of network firewalls that exist at other hospitals.  Such firewalls often block videoconferencing and it will be necessary to negotiate separately with each institution to allow such videoconferencing.  Fortunately there is already a precedent at the Minneapolis VA where weekly Grand Rounds are held jointly with the University.

 

The deficiency in using the Breeze Live method of recording presentations is that the recorded session cannot be downloaded to a CD for later viewing as is done now in the Lectures on Line project for Years 1 and 2. If the Breeze server cannot be adapted the Medical School will have to maintain a separate system for this purpose.  This will require the continued availability of a moderate capacity video server running streaming video services such as our current Real Networks server.

 

Estimate Required Resources

 

Investment: 

To implement this recommendation a range of investments should be considered based upon the quality desired.  For the smallest investment the Medical School could provide/support the capability to distribute and record Breeze presentations (Audio/PowerPoint) from any classroom or conference room which hosts a medical school class, seminar, grand rounds or other educational activity.  Since this can be accomplished on the machine used for the PowerPoint presentation when it is connected to the network, the only additional equipment needed is a microphone and webcam.  We estimate $700 for 10 webcams with built in microphones. At a higher level the medical school could also equip a conference room in each department that has multi-site clerkships and residencies with a Polycom Viewstations capable of four site simultaneous conferencing.  We estimate the need for seven installations of $10,000 each for a total of $70,000.  This assumes that each remote site will be able to fund the necessary equipment from local sources.  Finally a video server will be required to store and distributed recorded video which the Breeze server cannot provide.

 

Continuing Expenses:

 

Annual Operating: This amount ranges from $300 per year for equipment replacement at the low end to $14,000 for Polycom maintenance contracts and $3,000 for equipment replacement for other classroom setups at the high end.

 

Personnel:  Either of the solutions requires 1.5 FTE Breeze Live, web conferencing and videoconferencing specialists supplemented by AHC personnel.  This assumes that the utilization of the technology will be on the order of 2-5 sessions per day and that some assistance will be required for setup and recording of each session.  There is also the need for someone to manage the Medical School component of the Breeze server and the Video server.  This additional 0.5 FTE person and the AHC can provide backup as needed.  This is estimated to be an annual total of $110,000 including salaries and benefits.


Recommendation: Simulations (Rank 2)

 

Increase the use of technology-based simulations and simulators for teaching and evaluating procedural skills.

 

Explanation

Simulations provide the opportunity for students to practice and learn clinical (both cognitive and procedural) skills without risk to actual patients and in situations that can reduce the cost of instruction.  They provide the advantage of practice by being able to repeat the scenario under identical conditions until the skill has been learned to the required level of competence.  Simulations also provide the ability to practice in contexts that would occur only very rarely in normal medical care but in which the necessary skills are critical to the survival of the patient.    Simulations are of proven value in a number of fields such as airline pilot training and nuclear power plant operation and numerous studies in medicine have demonstrated their utility in learning complex endoscopic and surgical skills.

 

There are at least two types of simulations.  One is strictly software-based and uses interactive screen presentations controlled by the keyboard, mouse or some other mechanical device to teach the skill.  These simulations typically have a large cognitive component and may focus more on diagnostic and therapeutic decision making.  They typically do not have a physical skill component.  The second type is the physical simulation where there is some hardware component that provides a visual and tactile simulation of the patient.  These can range from a simple venipuncture simulator that mimics the tactile feedback of inserting a needle into the patient’s vein to full body, computer-controlled simulators with eye blinks, respiration, heartbeat, an airway, etc. that can be used to teach a variety of procedural skills. 

 

This recommendation focuses on the increased use of physical simulators to teach procedural skills to medical students and residents with the possibility of evolving their use into continuing medical education offerings.    It recognizes that simulations play a dual role in education in that they can be used either to teach skills or for evaluating the skill level of students.  Both uses are equally important.  The educational component is useful as an instructional tool that can expose the students to a wider variety of situations.  The evaluation aspect is useful in creating and administering standardized scenarios to determine the student’s level of competence in a skill. 

 

The benefits to be derived from these simulations are numerous.  There can be more standardized training in procedural skills without risk to patient.  The availability of simulators can provide more opportunities for skills training since the student would no longer be dependent on the appearance of a patient in the hospital or clinic with a particular problem in order to practice the skills.  There is also the ability to assess student skill levels in a wider variety of standardize situations at reduce costs both in dollars and time.  And finally they provide the ability to repeat those assessments across students and across time under the same conditions.

 

While the Medical School and the AHC have made some initial investments in simulators such as CathSim and SimMan, the task force recognized that a further expansion of that investment is necessary in order to systematically incorporate simulations into undergraduate medical education and resident training.  The group realized that there will need to be a systematic review of the needs and available simulations that can meet those needs.  Depending on the simulators selected the infrastructure and resource requirements will be quite different.      

Equipment and Infrastructure

In the case of software-based simulations, the necessary computer technology is already available though there may be some need to purchase special items of equipment to use the simulations such as “joysticks” and 3D goggles. For physical simulations the equipment required will be the physical simulators themselves. The infrastructure required is the space necessary to house them and their accompanying support personnel.  Fortunately the AHC has developed the IERC which can provide the necessary space for such simulators and so the principal resource issue for the Medical School is the cost of the simulators, the personnel to run them and the consumable supplies that would be required.  However this space may need to be renovated to simulate an emergency room or OR suite if the decision is made to acquire high-end simulators.

Estimated Required Resources

Estimates of the amount required for equipment are dependent on the number of students that require training and the types of simulations desired.  For software simulations, the investment required is only the purchase price of the software.  For skills expected of all undergraduate medical students, multiple units of simulations will be required.  For advance training of residents in specialized skills, one or at most two simulators should be adequate. 

 

Investment:

The prices of simulations vary widely from a few hundred dollars for simple software-based simulations to more than $250,000 for complex, full body simulators. In addition communications with colleagues at other Medical Schools indicate that $500,000 in renovations to existing facilities to house one or more full body simulators in not unreasonable.  Fully equipping a simulation laboratory could easily approach  $1 million. 

 

Continuing Costs:

 

Annual Operating:  Operating costs for simulations are dependent on the consumable supplies required.  This ranges from essentially $0 for software-based simulations to $80,000 in consumable supplies for full-body simulators.

 

Personnel:  Physical simulators require attention of and management by dedicated personnel for anything beyond the simplest levels.  While the existing IERC personnel would undoubtedly handle low level simulations, any full body simulator will require a dedicated, medically trained coordinator and technician support to fully utilize since a major activity is developing and executing detailed scenarios using the simulators.  Unfortunately these simulators do not come with an extensive library of scenarios and they must be developed locally.  Thus implementation of advanced simulations would require a 1.0 FTE simulation coordinator and a 1.0 FTE simulation technician for an estimated annual total of $170,000.


Recommendation: Blended Courses (Rank 2)

 

 

Work to evolve courses with significant lecture content to a more blended approach where in-class time can be reduced and remaining time used more for teaching applications and higher level thinking skills.  A blended approach consists of a mixture of face-to-face group meetings combined with self-study modules, case-based learning, narrated slide presentations and other on-line activities that the student can complete on their own schedule.

 

Explanation

In undergraduate medical education, a significant amount of time in both basic science and clinical coursework is devoted to lectures.  It should be recognized that lectures are an important and effective instructional tool that do provide the opportunity for an instructor to interact with a large number of students in a short period of time.  They can and do provide organization structures for students and are an important motivational tool.  Students do learn effectively from lectures and there is no doubt that in medical school they attain the required levels of knowledge in the subjects taught in this mode.  Yet because a lecture must take place at a particular place and time, they do reduce the flexibility for individual students by enforcing a single pace on the learning process.  In addition while they can provide important conceptual integration and knowledge application opportunities, a significant component of most lectures is information transmission.  Evidence from the medical education and higher education literatures tends to support the position that independent learning strategies that do not require a group meeting in a given place at a given time are equally effective in terms of student learning. 

 

Accordingly the task force recommends an approach that blends the best features of both the lecture (face-to-face interactions with the instructor, modeling, and interactive knowledge application) and independent learning approaches in order to promote greater time flexibility for students and faculty.  The group believes that greater use of independent learning accommodates students who have different styles of learning and allows students to proceed through the curriculum at a pace that best suites their own needs and abilities.  They also maintain that there will be a benefit to faculty after the initial additional time investment in the development of independent learning modules.  That benefit will be that less time will be spent in lecture and that time so spent can be devoted to higher level educational activities.

 

To achieve this blended approach to instruction will require the availability of a variety of independent learning modules that cover the full range of topics in the curriculum.  Fortunately this has been an area of active development in medical education and there are numerous products on the market that address topics from anatomy to urology.  At the same time the University is providing the tools such as Breeze Presenter to easily convert the information transmission components of lecture courses to online, independent study modules in the form of narrated PowerPoint presentations.  Live group presentations can be captured with Breeze Live and preserved for future years.  The Twin Cities campus has developed tools such as the Minnesota Virtual Clinic that are designed to be used for independent learning.  The Duluth campus has developed resources such as Histology Time and Neurotime that are widely used computer-based independent learning courses.  In addition there are free national resources such as the internet-based Computerized Learning in Pediatrics Project that provides a complete set of online learning modules for pediatrics clerkships and companies like DxR Clinician that sell clinical case simulations.   We are fortunate at the University of Minnesota to have access to the Vista course management software which can be used to control access to these modules, track student progress through the modules and administer quizzes and tests as needed.  Implementing this recommendation will require the judicious purchase of commercially available learning modules along with the local development of additional modules.

Equipment and Infrastructure

Implementing this recommendation requires a network and server to distribute the independent learning modules to students.  This already is available through University provided services including the University Gigabit network, WebCT’s Vista and the Breeze servers.  To develop additional modules requires some additional facilities that are available in the University’s Digital Media Center and the AHC’s Learning Commons.  The equipment and infrastructure to accomplish the recommendation already are in place at the University of Minnesota and the required additional investment in equipment is trivial and amounts to some high quality microphones and web cameras. 

Estimated Required Resources

 

The primary resource needed to implement this recommendation is the support of technical and instructional personnel to assist in the development of new modules that are not available otherwise.  The University provides numerous resources such as the Digital Media Center and the AHC’s Learning Commons for producing the materials.  In addition assistance will be needed to making the materials available to the students for their use.  It is important to recognize that development of any new materials will require significant faculty commitment on the order to 4-6 hours for every hour of independent learning. 

 

Investment:

 

 It is estimated that approximately $35,000 will need to be spent to buy existing modules that can be used in the classes. For module development, the extent of personnel required is a function of the number of self-instructional modules that would need to be developed internally.  The undergraduate medical curriculum currently consists of over 1000 hours of unique lectures per year counting all first and second year courses and clerkship didactic sessions.  There are different scenarios that can be pursued.    One would be to replace 25% of these lectures with independent learning modules totaling 250 hours.  Assuming that about ½ of these would be available commercially the scenario would require the development of 125 hours of instruction.  Assuming that a single TEL designer could work with individual faculty members to develop 50 modules per year means that with 1.0 FTE instructional/TEL designers the task could be completed in 2.5 years for a total cost of $175,000.  If the percentage replacement was increased to 50%, the task would require 2.0 FTEs for the same period of time and would cost $$350,000.

 

Continuing Costs:

 

Annual Operating:   It is likely that some commercially available material will require licensing rather than purchase.  Depending upon the materials required we estimate an annual cost of up to $75,000.  

 

Personnel:  There will be the need for 1.0 FTE in technical support to assist with testing and making the modules available to the students.  This is an annual total cost of $55,000.   


Recommendation: Ad Hoc Communications (Rank 3)

 

 

Develop/adopt means for faculty and students to communicate easily and frequently on an ad hoc basis with each other regardless of site to facilitate advising and student group communications. 

 

Explanation

Students and residents are located at numerous sites throughout the state.  In addition to Year 1 and 2 students on both the Twin Cities and Duluth campuses, Year 3 and 4 students are located at a variety of hospitals and clinics in Duluth and in the Minneapolis and Saint Paul metropolitan areas.  Students in the RPAP program are scattered throughout the state.  Residency sites are also scattered throughout the metropolitan area.  This recommendation addresses the need for faculty to communicate with students and for groups of students to communicate with each other on an ad hoc basis where video and other media in addition to audio are important.  Examples of this are advisors in the Twin Cities needing to meet with their advisees on the Duluth campus, RPAP faculty needing to supervising students in rural hospitals and clinics and the student councils needing to hold joint sessions between the two campuses.  The motivation is to create a greater sense of community between the two campuses and among the diverse educational sites and to facilitate a higher level of interaction with students when they are not on either campus.  We believe that this would reduce the need for travel and would promote closer collaboration between the Twin Cities and Duluth campuses.

 

The Task Force suggests a combination of technologies depending on the location of the communicating parties to implement this recommendation.  When the parties have access to existing videoconferencing facilities in the Twin Cities and Duluth, they could use those.  If that is not possible or convenient then they could use a PC-based broadband version of the Polycom software to conduct such videoconferences.  This provides the same capabilities as the Polycom videoconferencing units and in addition provides the ability to share the computer screens between sites.   

Equipment and Infrastructure

Since implementing this recommendation would make use of campus networks, existing videoconferencing facilities and internet attached computers, the principal equipment purchase would be webcams and software to be used for PC-based video.

Estimated Required Resources

Investment:  

Implementing this recommendation does provide some choices.  To provide a high quality PC-based solution would require commercial videoconferencing software such as the Polycom PVX at $120 per station and a webcam at $60 for a total of $180 per station.  A PC solution with minimally acceptable quality could use Microsoft’s Instant Messenger (free) and a Webcam at $60 for a total of $60 per station.  To equip 10 student advisors, 35 RPAP student sites, and 30 residency sites is $13,500 for the high quality solution and $4,500 for the minimally acceptable solution.  

 


Continuing Costs:

 

Annual Operating: approximately $0

 

Personnel:  Either approach would require a 0.20 FTE technical support person to assist users in setting up the software and in resolving technical issues as they arise after the original dissemination of equipment and software.


Recommendation:  Electronic Medical Records ( Rank 3)

 

 

Introduce the electronic medical record as a teaching tool from the very beginning of the curriculum and work to assure that students have appropriate access to electronic records during their clerkships.

 

Explanation

Electronic medical record (EMR) systems also known as automated medical record (AMR) systems or clinic information systems (CIS) are quickly replacing the paper medical record in patient care.  Today our students must make use of these systems in their clerkship and residency programs.  They have become the fundamental tool for storing, retrieving and reviewing patient information.  The federal government and business groups through a number of initiatives are encouraging and/or requiring EMRs for both hospitals and clinic-based medical practices. 

 

The task force takes the position that just as students benefit from early exposure to patients during Years 1 and 2 of the curriculum, they would also benefit from early exposure to the information systems that, if not now, will soon be integral to the care of those patients.  Early exposure would insure that students are trained in the use of and familiar with medical record systems by the time they start seeing patients during their clerkship years.  Implementing this recommendation would have the additional benefit of helping to provide a context for the consideration of basic science concepts in the form of simulated patients as is now done with the Minnesota Virtual Clinic.  Finally such an EMR could provide a realistic organizing framework for the numerous cases that are used to illustrate clinical concepts during Year 2 and the clinical years of the curricula.   This could well be accomplished by an expansion and adaptation of the already developed Minnesota Virtual Clinic.  Alternative approaches might be to work with the VA to install a version of their public domain EMR or negotiate with one of the Hospital Information System vendors such as Cerner or Epic for access to a version of their system.

 

The task force also took the position that just as it is important to have early training in EMRs, the ability to access and use the systems in the various clerkship sites is critical to learning during the clerkship years.  Yet there are numerous reports of students having difficulty obtaining authorization to access these EMRs or even experiencing outright denial of access.  Therefore to insure a better educational experience during clerkships it is necessary to make arrangements so that these students have the necessary access to the electronic medical records at each clerkship site.  It is recognized that the implementation of this portion of the recommendation has more to do with negotiating policies for access at each clerkship site than it does with resource expenditures.

Equipment and Infrastructure

All the necessary equipment and infrastructure are already in place to implement this recommendation.

Estimated Required Resources

The following estimate is based on using either the Minnesota Virtual Clinic (MVC) or a commercial system provided free by the vendor as a means of implementing this recommendation.  Addition staff time will be required to work with the clerkship sites to insure appropriate access for students.

 

Investment: approximately $0

 


Continuing costs:

 

Annual Operating: These costs are based on the need to support an existing server for the Virtual Clinic for an annual cost $3,000.

 

Personnel:  Implementation of the recommendation will require a 0.5 FTE technical support person for technical development and maintenance tasks. It will also explicitly require the involvement of a 0.20 FTE faculty member for oversight of the expanded MVC or the commercial system.


Recommendation: Computer-based Testing (Rank 3)

 

 

Adopt and implement computer-based testing capabilities.

 

Explanation

Computer-based testing (CBT) uses information technology to present test questions to students, collect their answers and assign grades to those tests in an automated fashion.  It requires that each student have access to a computer under controlled conditions, usually connected to a network.  Test questions reside on and are administered by a server connected to the network.  In order to take such a test, the student identifies himself or herself to the server and then is presented questions.  The approach works best for automated grading with objective types of questions where the student can select from among a number of distinct alternatives.   This includes single and multiple answer multiple choice questions as well as matching questions.  Short and long answer text questions can be used but they must be assigned a grade manually.  When automated grading is used, the ability to provide the student with results immediately upon completion of the test is available and grades can be automatically entered into a grade book for the instructor.  CBT has increased flexibility over paper-based testing in the type of questions that can be used.  It allows the easy incorporation of color images and illustrations as well as annotations for those images. It provides the capability of using video and audio clips as part of the question.  It provided the ability to present questions in a random order to each student and can enforce time limits on examinations.  If desired, tests can be made available to students over a period of time for completion within a given time limit.  Finally it allows the possibility of systematically drawing questions from a test question bank so that each student can receive a different set of questions.   CBT provides a highly flexible question asking format that can be quite secure and it provides essentially immediate results to students and instructors.  Repeated use of CBT within a given course or clerkship facilitates the construction of a bank of questions concerning that can simplify the work in test makers in future years. 

 

There are two primary issues that arise in the use of computer-based testing:  security and costs of implementation.  Security is an issue from two points of view.  First, in high stakes testing, it is necessary to positively identify the person taking the test which some maintain is more difficult to do in computer-based testing.  The second aspect of security is a concern about the student’s ability to either access material on the internet while taking a test or to pre-install information on their own computer that would allow them an unfair advantage.   Both of these concerns can be addressed by using controlled testing environments such as a computer lab or supplying computers to the students specifically for the test.

 

It is important to note that the Duluth campus has taken a lead role in using CBT for its classes.  All examinations at that campus are now administered using WebCT Vista in the computer facilities available to them.  They report acceptance among the faculty and students of this approach to testing.

 

The task force recommends that the undergraduate medical education program on the Twin Cities campus adopt computer-based testing as a means of testing for all courses during the first two years in parallel to the Duluth campus.  Furthermore this approach should be initiated using the Medical Student Computer Lab for test administration.

Equipment and Infrastructure

 

A major cost issue involves the cost of equipment to administer computer-based tests.  One approach has been to emulate as closely as possible the traditional classroom exam where all students are in the same room and take the exam at the same time.  This requires that every student have access to a computer at the same time which can be quite costly in terms of equipment.  The Duluth campus has taken this approach.  But for the Twin Cities campus this could potentially require the purchase of 165 laptops for testing a single class.  Another approach could involve using the existing computer lab and testing in shifts rather than all at once.  This would be a considerably less expensive approach. 

 

The equipment and infrastructure already exist to implement this recommendation as it is formulated by the task force.  The University’s WebCT Vista course management system has the capability of conducting secure testing of the type described above.  The Student Computer Lab can provide the access to the necessary computers to administer these tests.  There in this initial step, there need be no additional investment in equipment or infrastructure beyond that used to support the lab currently.

Estimated Required Resources

 

Investment:  

 

Approximately $0.

 

Continuing Costs:

 

Annual Operating: Using the computer lab for high stakes testing requires that all of the equipment remains operational and up-to-date.  In order to assure this, it will be necessary to replace all of the computers every three years and to keep a replacement supply available at all times.  The cost of doing this is estimated to be $25,000 every 3 yrs for the Twin Cities campus.

 

Personnel:  Personnel will be required to assist the faculty in the development of the computer-based exams during the initial phases of the project.  This will require a 0.5 FTE evaluation specialist who will be able to work with individual faculty members to develop suitably sophisticated test questions that take advantage of the full capabilities of computer-based testing.  It will also require 0.25 FTE for technical support of test administration.  This is an annual total of $48,750. 


Recommendation: Performance Tracking ( Rank 3)

 

 

Develop/adopt a unified, standard student performance tracking system that incorporates all forms of evaluation including classroom tests, faculty evaluations and procedural skills and competencies checklists and incorporates the technologies currently in use.

 

Explanation

The evaluation of student performance takes numerous forms over the course of a physician’s education.  In undergraduate medical education there are classroom tests, faculty evaluations of clerkship performance, certification of skills and competencies, as well as recommendations for residencies.  In residencies these take the form of periodic evaluations of performance, procedural documentation, presentations, recommendations, etc.  This information now resides in a variety of systems from paper-based grade books, Excel spreadsheets, Vista exam results, paper files, E*Value reports and files and the Registrar’s Peoplesoft system.  To aggregate all of this information for a given student is a challenging and time consuming task that discourages attempts to judge that student’s overall competence. The current systems do not incorporate mechanisms for judging and certifying the competencies required of graduating medical students or residents. Yet it is one of the primary responsibilities of the medical school to judge and certify the competence of its graduates.

 

To address this problem and to lay the groundwork for a more competency-based approach to medical education the task force is recommending the creation of a unified, standard performance tracking system that will be shared by the Twin Cities and Duluth campuses and that aggregates all of the evaluation information for each student and makes it available in a readily useable form while remaining in compliance with FERPA regulations.  This would have the additional advantage of providing a means of tracking student performance in order to identify those in need of additional assistance to successfully complete their education and should result in reduced staff time devoted that these activities.     

Equipment and Infrastructure

 

In order to assure security from intrusion and to protect the records from loss a separate server will be required.  The balance of the infrastructure (the network) is already in place.

Estimated Required Resources

Investment:

 

A server for this purpose can be obtained for a cost of $3,000.  The cost for software development will require 2.0 FTE developers for 0.5 years for a total of $90,000.

 

Continuing Costs:

 

Annual Operating:  The server will require $3,000 for server support and maintenance from the AHC’s AIS group.

 

Personnel: A 0.2 FTE information technology professional will be required for system maintenance and support for a total annual cost of $27,500.


Recommendation: Parity ( Rank  4)

 

 

Create parity in learning opportunities for medical students regardless of their personal financial ability to afford the equipment necessary to take advantage of e-learning technologies.

 

Explanation

For medical students to take advantage of the web-based or e-learning activities either required or made available by the implementation of the recommendations of this report they must have personal access to standard software, standard computer configurations, broadband internet, and devices such as PDAs.  Ideally these devices should provide the same level of functionality and access for all medical students.  Yet the current situation falls short of this ideal in that students possess and use a variety of equipment and software that is quite variable in its utility.  Too often those differences are dictated by financial concerns and an inability to purchase the necessary computer hardware and software.  While medical students all have access to the facilities of the Medical School Student Computer lab, this parity of access does not extend into the classroom, clinical site or home.  Students must supply their own laptop to use in class and their own PDAs for clinical rotations.

 

The thrust of this recommendation is that all medical students should have equal access to the hardware and software necessary for their education.  Inequality can arise from a number of sources but one certainly is the lack of a common configuration of hardware and software that is identical for each student.    This is an ideal situation where there is both physical and functional equality since everyone has exactly the same hardware and software.  One alternative is to approach this from a functionality perspective by working to insure that all students have access to the same functionality even though they may use different hardware to achieve this functionality.  At present the school does require that students have access to a computer when they matriculate but makes no further requirements.  This works reasonably well for educational activities that require only a web-browser and as long as high-speed access is not necessary.   Students can meet this requirement either by having any of a variety of different computers at home connected by slow dialup or by making use of the computers in the student lab.  This approach also works reasonable well for supplemental learning materials.  But this level of requirement does not facilitate student use of information technology in classroom or in clinical settings since access in these settings requires portable devices such as laptops or PDAs.   In order to take greater advantage of the educational uses of information technology it will be necessary to expand the functional requirements of the hardware and software that students must have.  For example if we decide that students should have readily available access to drug information during their clerkship rotations, then they will need to have a PDA that is capable of running the microMedix software that is available from the Bio-Medical Library.  Similarly if classroom or group discussion activities require that students be able to conduct ad hoc literature searches, then they will need to have wireless laptops or PDAs to accomplish this. 

   

To address this perceived problem the task force recommends an effort to provide parity by insuring that all students have access to the same hardware and software at all of the locations where learning takes place.  The basic functions needed by every student are the ability to access all of the online educational materials provided by the Medical School, to be able to access online resources during class sessions, and to have the ability to readily access electronic reference materials while they are clerks or residents.  The Medical School needs to provide the resources to students to make this possible.

 

Equipment and Infrastructure

To completely implement this recommendation each student should have a portable computer with broadband wired and wireless internet capabilities, equipped with basic productivity software (e.g. Microsoft Office and a web browser and the necessary add-ins such as Flash and Java Runtime), and a personal digital assistant capable of containing the reference resources needed for clerkship and residency. While the University provides much of the technology infrastructure to support this through its Gigabit network and wireless accessibility, it will be necessary to extend this broadband access to the student’s home in order to fully realize this recommendation.

Estimated Required Resources

 

Assuring equality of access as recommended here does require a significant investment of resources to acquire the necessary hardware, software and network access for each student.  The question is “Who will assume the burden?”  There are a range of solutions that can address the problem.  At one extreme the Medical School can require that students invest their own resources in this equipment and broadband access based on the argument that these are now basic learning tools akin to textbooks.  This is supported by the policy that computer expenses are consider legitimate expenses in terms of calculating the financial aid package.  This approach requires essentially no investment funds from the school other than to provide support equipment such as network storage or loaner machines for students whose own equipment requires repair. An intermediate approach would be to set up a no-interest leasing scheme where the students could spread out their expenses over the course of their education and would own the equipment when they graduate.  In this case the school would have to invest an initial amount for each student but would recover it over the four year period of their education though an interest subsidy would be born by the school.  At the other extreme is the possibility of supply this equipment and broadband internet access directly to each student as a part of their medical education just as we now supply course handouts.  This last approach would also require that the school provide the necessary additional space on the order of 8,000 square feet to house, distribute and service the equipment in addition to the costs of that equipment.  

 

Investment:

 

In terms of investment we will consider the two extreme scenarios described above.  The minimal cost to the school would be the Student Responsibility Scenario where students would be required to buy certain equipment and subscribe to a broadband service.  In this approach, the school would still need to invest in loaner laptops ($30,000) and PDAs ($5,000) to provide backup for students when they experience equipment problems.  In the School Responsibility Scenario, the Medical School would purchase all of the equipment for the students and would provide subscriptions to broadband services.  This would require the same investment in loaner equipment ($35,000).

 

Continuing Costs:

 

Annual Operating: We estimate that in the Student Responsibility Scenario, there would be an annual cost of $33,000 for replacement parts, supplies and software.  In the School Responsibility Scenario, there would be the same need for replacement parts, etc. ($33,000). In addition there would be annual subscriptions to broadband services that would total $422,400 for 880 students, and the cost of equipment for each incoming class which would be approximately $420,000 per year.

 

Personnel:  The Student Responsibility Scenario would require 2.0 FTE technical support specialists to assist students with set up of their machines and troubleshooting technical difficulties.  The total annual cost would be $110,000.  The School Responsibility Scenario would require a 1.0 FTE administrator to oversee the equipment acquisition and distribution process, a 0.25 FTE secretarial support for the administrator and 2.0 FTE technical support specialists to assist students with set up of their machines and troubleshooting technical difficulties.  This would amount to a total annual cost of $220,000. 

 

While it might be possible to replace the technical support specialists with outside contract services but the cost is likely to be about the same.  Similarly it would also possible to outsource the entire process for these scenarios but again it not reasonable to expect significant annual cost savings.


Recommendation: eLearning Preparation (Rank 4)

 

 

Prepare trainees for life-long e-learning by defining competencies in and teaching students e-learning methods.

 

Explanation

Medical students are a product of their educational background.  To be admitted to medical school they have demonstrated that they have acquired highly successful strategies for learning in the current context of higher education.  That context relies heavily on the lecture format and our students are very good at learning in traditionally structured educational situations.  If the medical school decides to implement any of these recommendations that increase the use of  approaches that make greater use of technology-based instructional strategies, there will likely be resistance from students and they may well have more difficulty learning and attaining the required competencies.  In a TEL environment students need to be skilled using and learning from asynchronous teaching modules; intelligently and effectively communicating with email, bulletin boards and online discussion groups, instant messaging, teleconferencing, and videoconferencing; effectively and efficiently using a variety of internet search engines; developing online presentations; and evaluating the quality of information on websites.  These are learned skills which we cannot now assume our students have when they enter medical school.

 

The task force recommends that the medical school identify and adopt a specific set of elearning competencies that address the skills and knowledge that medical students need to make effective use of the new information technology-based approaches to teaching and learning.  Furthermore, the medical school should include as part of its undergraduate medical curriculum the educational experiences that address these competencies.  A cost effective approach to implementing this recommendation would be to develop a set of online instructional and evaluation modules that would teach these competencies to medical students.

Equipment and Infrastructure

Implementation of this recommendation would require no additional equipment or infrastructure beyond what is currently available to students.

Estimated Required Resources

 

The required resources to implement this recommendation if it were accomplished as a series of educational modules would be the costs associated with development of the modules. 

Investment:

 

We estimate that six modules need to be developed at a cost of $7,000 per module for a total of $42,000.

 

Continuing Costs:

 

Annual Operating: Approximately $0

 

Personnel:  To support this program a 0.10 FTE instructor would be required to manage the use of the modules and to interact with the students as they complete the modules and demonstrate their competence for a total annual cost of $20,000.


Recommendation: eLearning Resource (Rank 4)

 

 

Serve as an e-learning resource for teaching faculty (e.g.  RPAP preceptors, residents, attendings), practicing physicians and health care systems in Minnesota.

 

Explanation

The Medical School has a wealth of instructional resources that are available now and implementation of some of the other recommendation will add greatly to that collection.  These range from basic science courses such as anatomy to numerous clinical case presentations and modules teaching bioethics and professionalism to name but a few examples.  However these are now primarily used by medical students and residents as part of their educational program.  Yet there are a number of additional groups of medical practitioners who might benefit from access to these materials.  Certainly the teaching faculty in the clerkships and residencies should have access to all these materials in order to understand what the students are learning in their course of study and to appropriately design educational experiences for their students.  At the same time such access could serve as a form of reward for their teaching efforts and could possibly be associated with CME credits.  These materials could also be repurposed for use by practicing physicians and other health professionals either as individuals or in conjunction with local health systems.  This could lead to additional offerings and revenue generation for the continuing medical education program and perhaps cooperative offerings with other health professions such as nursing and pharmacy.      

Equipment and Infrastructure

 

Implementation of this recommendation would require no additional equipment or infrastructure beyond what is currently available to students. However it might require negotiations with the University to allow access the resources on University and Medical School servers.

Estimated Required Resources

 

These estimates are based on the strategy of repurposing existing resources for the uses described above.  As such the only additional resources required would be those associated with any modifications required to make them available to outside professionals or groups.

 

Investment: Approximately $0

 

Continuing Costs:

 

Annual Operating:   Approximately $0

 

Personnel:  It is estimated that a 0.5 FTE coordinator and instructional designer would be required to oversee the use of these materials by the groups described in the recommendation.


Summary

 

The task force puts forward these recommendations with the confidence that their implementation will address the Dean’s charge to the group and will create a better educational environment for medical students, residents, the faculty and the physician community.  They are based on the group’s understanding of the literature with respect to technology enhanced learning and the perceived needs of our educational programs. 

 

The group realizes that there is a significant range of cost estimates given choices to be made in the implementation of the various recommendations.  In the Table 2 below we attempt to summarize those cost estimates on a per enrolled medical student basis (excluding residents) in terms of high and low cost options and in terms of possible subsets of recommendations that might be selected.

 

 

Cost Category

Investment – 3 years

Continuing

Estimates

Low

High

Low

High

Recommendations

 

 

 

 

All

$490

$2676

$658

$1713

Ranks 1 – 3

$402

$2094

$433

$887

Ranks 1 – 2

$308

$2000

$170

$710

 

 

 

 

 

Learning Space (1) 

$261

$557

$36

$207

Multi-site Access (2)

$1

$88

$144

$144

Simulations (2)

$1

$1136

$0

$284

Blended Courses (2)

$239

$402

$63

$148

Ad Hoc Communications (2)

$5

$15

$13

$13

Electronic Medical Record (3)

$0

$0

$76

$76

Computer-based Testing (3)

$0

$0

$67

$67

Performance Tracking (3)

$94

$94

$35

$35

Parity (4)

$40

$534

$163

$764

eLearning Preparation (4)

$48

$48

$23

$23

eLearning Resources (4)

$0

$0

$40

$40

Table 2.  Cost per Enrolled Medical Student of Investments and Continuing Costs for Each Recommendation and Certain Combinations.  ()’s indicate Ranking


 

 

 

 

Appendix A:

TEL Task Force White Paper


 

 

 

 

 

 

 

 

 

 

 

 

 

Technology Enhanced Learning

in Medical Education at the

University of Minnesota Medical School:

 

A White Paper

 

 

 

 

October 12, 2004

 

 

Stuart M. Speedie, Ph.D.

Edward Ratner, M.D.

 


Introduction

 

The rapid developments in computer and information technologies and tele-communications in the last decade have the potential to profoundly affect Medical Education. These new information technologies provide a set of attractive attributes that can significantly affect many different aspects of the educational environment. Some of these characteristics are:

 

·        Connectivity - The Internet provides an unprecedented ability to access resources and websites throughout the world.  The resources of the National Library of Medicine are now just as accessible as those in the Bio-Medical Library and the reference books on the office book shelf.  The barriers of access to information due to geographic distance are rapidly disappearing when such resources are available on the Internet.  As a result there is an unprecedented breadth of information available at any internet-connected computer located anywhere in the world.  We are also quickly moving toward the ideal of ubiquitous accessibility with the proliferation of network connections and wireless access points.  Now in major cities one literally can use the internet wherever one happens to be located – the street corner, the local coffee shop, a hotel room, or office.  With the advent of digital satellite network any place on earth that has a source of electric power can link to the Internet as was illustrated by a group of climbers who conducted internet-based telemedicine consultations and physiologic monitoring while ascending Mount Everest. 

 

·        Speed – We now have the ability to move information across the internet at speeds that open new communications vistas.  The contents of a scientific paper including graphs and pictures can be copied from the publisher’s computer to a local machine in a matter of seconds.  The available high speeds allows one to “page” through a reference book with relative ease, even though the electronic copy of that reference may be located in another state.  Those same speeds not only facilitate textual communication but also now permit the use of the internet for voice and even live video interactions.    Much of the information we need and use on the Internet is available at our fingertips in a matter of fractions of a second. 

 

·        Multimedia – The advent of digital technology provides a wider variety of modes for information sharing.  Anything that can appear on a printed page can now be digitized and transmitted over the Internet.  All of the visual and aural modes such as text in any of thousands of fonts and orientations, high resolution pictures and either black and white or color, sounds, animated graphics and recorded video are now available.  Rudimentary tactile information in being transmitted in the form of force (haptic) feedback.  The only remaining sense yet to be addressed is the olfactory.  By combining these modes into multimedia messages the internet can be used to transmit a great variety of experiences from simple email to real-time, total-immersion, minimally invasive surgical procedures. 

 

·        Interactivity – The internet is capable of providing a set of experiences in which the user is encouraged or required to be an active, rather than a passive participant.  This is the essence of the World Wide Web in which the person viewing a web page makes choices about what they would like to see or do next by clicking on links within the page.  Based upon the choice they are presented with new information.  This type of interaction forms the basis for many of the web-based instructional programs because it provides the means of implementing automated feedback to the participant.  Using this technology we can construct instructional sequences that give constructive feedback depending on the choice that the user makes and can guide them to materials to help them learn from and correct their mistakes.  It allows and facilitates implementing the principles of active learning into web-based instruction.  It also permits the use of new modes of testing where the next question asked depends on the last answer given by the student.   

 

Institutions of higher learning and professional schools have taken advantage of the benefits of these information technologies to create technology enhance learning or TEL.  The University of Minnesota defines TEL as follows:

Technology-enhanced learning (TEL) encompasses the broad range of experiences and environments in which technology is used to enhance teaching and learning. Technologies are relentlessly and seamlessly merging, and the lines separating the traditional classroom, the technology-enhanced classroom, and distance learning are disappearing rapidly. TEL initiatives use technology-based resources--video, audio, images, simulations, and library tools--to enrich the learning environment and to extend it from the classroom to the residence hall, the home, the workplace, and the mall.

An Overview of Technology Enhanced Learning in Medical Education

 

There are many different types of TEL applications that are designed for various educational purposes.  In the following sections with we will enumerate the various categories and relate them to medical education.

 

Classroom aids (e.g. audiovisual equipment)

At the most basic level, TEL includes the traditional audiovisual equipment found in medical school classrooms, including projectors (overhead, slide, and LCD), video recorders connected to televisions or LCD projectors. Certain classrooms are equipped with graphics tablets that allow the instructor to “write on” or annotate presentation slides.  In some U of M Schools classrooms are equipment with personal computers.  Other classroom learning aids available include infrared voting devices and connections for students and faculty to the Internet (typically wireless).   Some other medical schools have adopted handheld computers as tools to achieve wireless connections between a lecturer and students.

 

Instructional Tools (PC, Web, CD, Handheld)

To supplement and sometimes replace verbal presentations, a variety of TEL tools are widely used.  These include audiovisual instructional aids presented in classroom settings (e.g. PowerPoint slides), Web-based instruction, multi-media CDs, and software for handheld computers.  Web-based tools are sometimes formatted inside instructional software such as WebCT (or Vista).  

 

Examples of significant use of TEL instructional aids include:

  • Physician and Society  (Virtual Patient Panel, WebCT multimedia lectures, Web-based simulated health plan)
  • Physician and Patient (multimedia Web site for home visit experience, CD for physical exam, demonstration videos for history taking),
  • Medicine and Pediatric Clerkship (CD, advanced Web site)
  • Primary Care Clerkship (advanced Web site, Web-based patient log, handheld computers
  • Histology Time and Neurotime (Duluth)
  • HIPAA training offered by the AHC

 

Job Aids /  Reference Tools (e.g. Web sites, ebooks)

Tools to assist professionals in consistent or enhanced performance of specific tasks are sometimes called job aids.  They can reduce training time and the need for memorization. They have been shown to reduce errors due to forgetfulness or distraction.   The most recognizable example of a job aid is a pilot’s pre-flight check list.  This type of  just-in-time TEL in health care is commonly in the format of Web sites or handheld computer software and is sometimes integrated with electronic medical records.  Health care examples include:

  • organizational or government policies or procedures (e.g. Medicare reimbursement or coverage rules, hospital procedures for discharge against medical advice)
  • guidelines for care (e.g. ACLS protocols, ISCI guidelines, immunization recommendations)
  • disease or treatment related calculators (e.g. creatinine clearance based upon serum creatinine, pediatric dosage calculators, interpretation of blood gases)
  • how to perform clinical procedures, including even major surgery
  • adverse drug reaction tools
  • differential diagnosis tools

 

The federal government and virtually every disease and specialty-specific organization maintains Web sites with clinical information for clinicians.   The Biomedical Library supports a widely-used Web-based decision-support/reference tool called Up-To-Date (also available for Windows Mobile handheld computers).  There are a large number of free or modestly priced medical reference books, documents or calculators are available for handheld computers.  These include popular print books such as Washington Manual and comprehensive specialty-specific Palm OS-only tools such as Renal-to-Go (created by a U of M nephrologists).   The State of Minnesota has supported development of regionally focused handheld software tools for care of the elderly.

 

Simulations

Simulations and simulators allow students to practice clinical skills without endangering real patients through .  The Medical School has access to two clinical simulators located in the Interprofessional Education and Research Center.  SimMan is a programmable patient simulator that provides scenarios to challenge and test clinical and decision-making skills during realistic patient-care scenarios.  CathSim is a set of hardware and software tools for intravenous catheterization and phlebotomy training. 

 

Databases

Another common form of just-in-time health care TEL is electronic databases, searchable on Web sites or handheld computers.  The Medical School maintains the Curriculum Database that records and indexes the contents of all courses, lectures and other types of educational sessions in undergraduate medical education.  The most commonly used external databases are formularies and drug databases.  The National Library of Medicine maintains the largest health care bibliographic database, MEDLINE.  Other databases include information from the human genome project.  Even electronic telephone directories for specialty referral sources or community resources could be considered just-in-time TEL.  The Bi-Medical Library supports a bibliographic search engine, OVID that provides access to several on-line databases including Medline. In addition it supports access to a variety eJournals and electronic versions of well-know medical references.   A large percentage of physicians use a free (or low cost for advanced version) drug database for handheld computers, Epocrates.  The State of Minnesota maintains a Web and handheld versions of its database of Minnesota-based community services, MinnesotaHelp.info.

 

Communication Tools

Electronic interaction between students and teachers and from students to each other can be an important form of TEL.  Communication can be synchronous (parties on at the same time) or asynchronous, one way (e.g. group voicemail) two-way, or multi-party.  Methods include email, teleconferencing (voice or video), and Web-based instructional tools such as bulletin boards and on-line discussion groups that are provided at the University by WebCT.   Transfer of content or files from teacher to learner or from one learner to another can occur via direct connections (voice mail), local area computer networks, the Internet via WebCT or custom websites, more often now using wireless tools such as WiFi, Bluetooth, or infrared. 

 

Beyond email, the U of M Medical School’s primary communication tool, the RPAP program and other groups uses teleconferencing routinely.  Fairview University Hospital has the ability to link operating rooms to interactive teleconferencing to classrooms to allow students to remotely observe operations.  

 

Learner Documentation Tools

An increasingly recognized educational method is asking students to reflect upon and document their learning experiences.  This can be in prose format or as a log of encounters.  As healthcare related TEL, this takes the form e-portfolios and logs of patients seen or procedures completed.  The University has invested in and supports a life-long e-portfolio system, but its format has not been customized for medical student or resident use. The one current example in undergraduate medical education is the use of a Web-based patient log is used in the Primary Care Clerkship.

 

Assessment/Evaluation Tools

Evaluation of student or teacher performance is often technologically enhanced to improve efficiency of data entry, communication, and analysis.   Even scannable paper evaluation tools used in Objective Structured Clinical Examinations represent a form of TEL.    Another form of TEL assessment is the video-enabled examination room, which allows live or taped review of real or standardized clinical encounters.  The Medical School uses a Web-based software for resident and medical school course evaluation, E-value. 

 

The Duluth campus has implemented a comprehensive on-line testing program for all of their courses  using the capabilities of WebCT.  Students use laptops provided by the school to take their exams and results are immediately available for review by faculty and students.

 

Use of TEL Across the Medical Learner Continuum

 

Pre-clinical Undegraduate

In the first two years of medical school, TEL is focused on classroom aids and instructional aids.  The final exam in Physician and Patient at the end of Year 2 uses the video-enable exam rooms for an OSCE.  Email is the standard communication tool.

 

Clinical Undergraduate

During Year 3 and 4 of medical school, use of TEL by faculty is limited to selected clerkships that have course directors interested in TEL.  Almost all students carry handheld computers (purchased personally) with a variety of ebooks or other reference materials installed.  MEDLINE is used occasionally, but textbooks and Web sites are more common sources of learning about diseases.  As noted above, teleconferencing assists learning in RPAP.  Communication with preceptors is almost entirely verbal

 

Graduate Medical Education

During residency, TEL is focused on handheld computer applications, with the largest residency programs purchasing handhelds for their residents (IM and FP).  E-books and other reference tools are indispensable for most residents.   MEDLINE is used frequently by residents as their knowledge needs are more urgent and surpass textbook offerings.  Classroom education has little TEL use, as it is most often small group (e.g. on rounds).   A number of specialty or sub-specialty specific CDs are available for self-learning.  Communication with preceptors is almost entirely verbal but E*Value is used for evaluations of resident performance.

 

Continuing Medical Education

Formal continuing medical education uses classroom aids for on-site conferences.  Web-based CME is widely available, although the U of M has not pursued this approach to any significant degree.  Web-based, video, and phone based teleconferences are common for 1-hour CME sessions.  There have been some efforts to expand formal CME to handheld computers (in 5-10 minute increments as part of drug database searches). 

 

Practicing Physicians

Few practicing physicians outside academics use MEDLINE more than rarely, but reference CDs such as Up-to-Date are popular for office-based use.   Board re-certification programs often include CD-based instructional aids.  Physician decision support within EMRs remains in its infancy and is not yet significant within the UMP EMR system.   About ½ of practicing physicians carry handheld computers (declining with age), with drug databases the most popular medical database software.

 

Current Technology Infrastructure

 

The University of Minnesota has provided excellent Internet access throughout the campus.  In addition to the classroom equipment described above, every classroom has at least one network connection and all classrooms used by the Medical School have wireless capability.  The network is being upgraded to Gigabit speeds and is fully able to support a myriad of TEL applications.  First and second year students have access to computers in the Medical School Student Computer Lab (Minneapolis), (Duluth), the Bio-Medical Library and the Learning Commons on the fifth floor of Diehl Hall.  Access to computers and the Internet is much more variable one the student enters the medical workforce as a clerk, intern, resident or independent practitioner.  The Medical School and the Academic Health Center operate servers that support a variety of websites, databases, and recorded lectures and website space is provided by the University to all students and faculty members.  The Bio-Medical library provides a comprehensive collection of electronic references including citation databases, full text articles and references in addition to their collections of hardcopy books and journals.  TEL relating teaching facilities include the IERC and the Learning Commons.   

 

What does the research literature tell us about the impact of TEL?

 

Accompanying this White Paper are several references that review findings with respect to the use of TEL in medical education.  They attempt to address a number of questions including:

 

  1. Does TEL function reliably?
  2. Does TEL improve effectiveness of learning (e.g. retention, ability to use learning in practice)?
  3. Does TEL improve efficiency of learning (reduced teacher or learner time, reduced printing, reduced travel)?
  4. Does TEL improve access to learning (across distance, flexibility when learner can use it)?

 

The following summarizes the conclusions of those articles.

 

·        TEL works. It is possible to create technology enhanced learning applications like those described above that are capable of achieving their stated objectives.  They work reliably and students can use them for learning, evaluation and communication.  Students can and do make use of and learn from TEL materials and strategies.

 

·        Computer-based independent learning is at least equivalent to standard teaching methods in producing student learning.  Repeated well-designed studies have demonstrated this finding.  Some studies have found that TEL based independent learning methods produce greater learning than traditional teaching methods.

 

·        Learning can be faster with TEL materials.  Of the several studies that have examined the issue of time to achievement in TEL-based independent learning, most have demonstrated learning may take longer using traditional teaching methods.  This likely due to the fact that students in traditional teaching are controlled by the instructor’s pace, but in TEL-based independent learning they can proceed at their own pace.  If this pace is quicker than that set by the instructor, faster learning is likely to result.

 

·        Given a choice, students generally prefer face to face teaching.  While students can use and learn from TEL based independent learning, they report a continuing preference for a live teacher.  This may well be due as much to the fact that their entire educational experience is within the conventional system and they have not yet developed the skills to learn independently in a TEL environment.  There is also the very real advantage that the learning environment in which teaching is done by a live instructor can be more flexible and responsive to students than an TEL environment.

 

·        TEL methods are likely to be more work for the instructors than the conventional lecture.  Development of TEL materials is often reported to require more time and resources than the lectures even with the advent of PowerPoint presentations.  In many ways the development of TEL materials is equivalent to writing a textbook – except that the textbook is in an electronic format and is in a more complex format than the typical printed textbook.  In addition those TEL strategies that involve electronic communications with students either through email, discussion boards or chat rooms may require more instructor time than traditional office hours.

 

·        On line materials are relatively expensive to develop.  Creation of TEL-based learning materials is often beyond the expertise of the typical teacher and usually requires the time and skills of educational technology specialists.  These same specialists are also often needed to maintain and update the materials.  In addition special purpose hardware is sometimes requires such as when creating video clips or conducting videoconferences.  All these are costs over and above the normal costs of instruction.

 

While the technology does provide these new opportunities we must recognize that we are at the very beginning of the development of educational technology in Medical Education.  There is a well know cycle of technology development in which the first applications of the technology replicate existing processes and methods in a new form.  Granted that there may be significant improvements in some ways, but the underlying processes and approaches have not changed.  Only after a new technology is explored for a while are truly innovative and unexpected uses made of it.  Many of the uses of technology in Medical Education today exemplify this first stage of the cycle.  Class websites provide links to electronic versions of printed handouts, independent study modules are often lecture recordings with accompanying PowerPoint slides.   On line tests are reproductions of paper tests in an electronic environment.  PDAs provide quick access to pocket diaries.  Electronic search engines like PubMed provide the same means of searching the literature as was available through the Index Medicus.  While each of these examples does reproduce an existing educational method in a new form they, in many cases also provide significant improvements upon that form.  We need to accept the fact that the process of innovation in Medical Education, like any other area of endeavor must proceed through this development cycle.

 

The Challenge

 

The challenge we face in examining the uses of technology in Medical Education is to make the best judgments how to take advantage of its positive characteristics to improve the educational process and achieve the education mission and goals of the School.  We do not use technology for its own sake, but rather as a method of enabling more effective and efficient achievement of that mission.  The Dean has identified three areas of education priorities and has asked us to address each by considering if technology enhanced learning could facilitate them and if so which ones and at what price.

 

In this paper we have reviewed the promise of the technologies that are used in TEL, described the different categories of TEL use in medical education and summarized the findings of the educational research literature with respect to technology enhanced learning.  Our task is to make use of this information to address the issues we face and to provide the best possible answers to the following questions:

 

1.      How can we best use these technologies to ensure comparability of learning experiences between campuses and among program sites as required by LCME,  ACGME and ACCME?  

 

  • What do we mean by comparability and how do we measure it?
  • Are there examples of this being done now either at the University of Minnesota or other schools?
  • What technology strategies should we employ that would yield the greatest benefit at a reasonable cost in terms of required faculty and student time, financial outlays, and physical space?

 

2.      What is the role of these technologies in promoting scheduling flexibility and independent learning in our educational programs?

 

  • To what degree do we need to provide scheduling flexibility and independent learning within the curricula?
  • Which technology approaches have the best evidence for effectiveness in promoting independent learning in terms of student outcomes and satisfaction?
  • Are there examples of this being done now either at the University of Minnesota or other schools?
  • How do the costs of implementing these approaches compare to the benefits derived?

 

3.      How can these technologies be used to facilitate rigorous, reflective, outcomes-based evaluations of student knowledge and skills.

 

  • What kinds of evaluations do we need to conduct in order to certify that our students are achieving the required educational objectives?
  • Are there examples of this being done now either at the University of Minnesota or other schools?
  • What advantages are to be gained by using TEL approaches to evaluation?
  • How do the costs of implementing these approaches compare to the benefits derived?

 

   


 

 

 

Appendix B:

Literature Review


 Follow this link to view document

 

 

 

 

 

Appendix C:

TEL Best Practices at the University of Minnesota Medical School

 


TEL Best Practices at the University of Minnesota Medical School

 

Best Practice

Description

Contact

Anatomy

The Program in Human Anatomy Education web site is a source of information on Anatomy courses at the University of Minnesota as well as a jumping off point for course materials for students enrolled in our courses.  The Program's two largest courses are INMD 6150 for medical and dental students and INMD 3001/2 for undergraduates.  The site is the access point for course descriptions, course syllabi, links to course materials (lecture notes, additional handouts and digital anatomy/radiographic images, quizzes, lectures on line, PowerPoint presentations, etc) and course announcements via a blog page.  The goal of the web site for our enrolled students is to give the students access to the central learning materials of the course from any web access point. 

Ken Roberts

Physician and Society 

Physician and Society is a set of courses in the first and second years of the curriculum that focus on a variety of social science, epidemiologic, cultural, ethical and health system issues.   The course is supported by a website that maintains a number of resources for the students and is available at www.meded.umn.edu.  The course uses an online, anonymous survey to collect health information and medical care preferences from the students and constructs a simulated health plan around those references.  The students assume the role of plan administrators and throughout the Year 1 course consider a variety of plan-related issue and make decisions by an online voting process that feeds back into class presentations.  The Minnesota Virtual Clinic is used to present patients that raise issues about cultural and ethical considerations.  In Year 2 the course has converted a number of classes to the asynchronous presentation approach.  During certain weeks, the students are required to sign on to WebCT, view narrated PowerPoint presentations and answer questions regarding those presentations.

James Pacala

Karyn Baum

Physician and Patient

This second year medical school course is designed to provide students with their first hands-on experience performing histories and physical examinations on patients in a variety of real-life clinical settings.   Technology is used to both teach and evaluate students.  One of the 18 full-day sessions of this course includes a structured home-visit experience to a senior to teach students house call etiquette and comprehensive assessment of the older patient; the experience is called Seniors as Teachers.   It uses multi-media Web site (www.geriatrics.umn.edu) that contains detailed written instructions on why and how to perform a house call, photos of seniors' home environments, and streaming video of Dr. Ratner interacting with seniors.   This Web site effectively substitutes for one-on-one mentoring in the home setting, which would be logistically impossible with over 160 medical students per year. In the past four years, over 1800 visits to this site have been made by students and faculty from the U of M and elsewhere.  At the end of the Physician and Patient series of courses, students are assessed through a set of simulated patient encounters that are video recorded for faculty review and, if necessary, use in remediation. 

  

Sharon Allen

Ed Ratner

Histology Time

Histology Time on CD is an interactive multimedia computer program that instructs the student in the fundamentals of histology and provides a mechanism for student self-testing of those fundamentals in a non-punitive way.  It incorporates a large number of histology images (~5,000 images) from the videodisc Histology: A Photographic Atlas (Stephen Downing, author).  Histology Time on CD is divided into 19 chapters that cover the basic subject areas covered in a typical histology course.  Each chapter has four parts: (1) a MicroLab session where the student is given basic information about the specific tissue or organ system being covered; (2) a Quiz Time session where the students are quizzed on their understanding of the visual material; (3) an MC Exam session which tests the students on their understanding of material typically covered in the lecture component of a histology course; and (4) a K Exam session which also tests students on their understanding of material typically covered in the lecture component of a histology course.  Significant benefits of using this program include the following: (1) eliminates the need to use microscopes in our curriculum; (2) students learn the material in a fraction of the time that was formerly required; (3) the amount of faculty time that needs to be devoted to laboratory exercises has been greatly reduced; (4) permits rapid and extensive review of histological principles and histological images; (5) makes tutoring of students on the visual components of histology very easy; (6) makes the laboratory component of histology very portable (students can have the entire program on a single CD-ROM and can study at home, on an airplane, or wherever they have access to a computer).  There are PC- and Macintosh- compatible versions available and the program is currently in wide use throughout the United States and abroad.  Many medical schools have purchased site licenses for the use of the program in their curricula and students at many institutions have their own personal copies of this program as well.

Stephen Downing

 

Neurotime

NeuroTime® is an interactive computer-based learning tool developed to facilitate the learning of neuroanatomical structures, their relationships and terminology using high quality images of intact and dissected gross specimens plus a series of whole brain sections in the coronal, horizontal and sagittal planes.  Corresponding magnetic resonance images (MRIs) enable the student to apply this structural knowledge to the interpretation of magnetic resonance scans.  In Identification Mode the student selects the name of a structure and a corresponding transparent, colored overlay appears over the structure. In Quiz Mode the student can assess their level of knowledge.  Since this tool is delivered on CD-ROM for Macintosh or Windows systems, students are able to study all gross specimens, brain sections and MRIs at any time and place they have access to a computer.  Students find this to be a very efficient way to study neuroanatomy since 1) they do

not waste time looking for specimens in the lab, searching for the names of structures or waiting for an instructor's feedback 2) they can navigate to the areas they wish to study and receive immediate feedback and 3) they can study when they are prepared to learn. NeuroTime® can be used in conjunction with gross specimens or independently where specimens are unavailable.  Since it can replace wet specimens, its use reduces exposure of faculty and students to formalin, saves faculty time and eliminates the cost of maintaining a wet lab facility.  NeuroTime® can be adapted for use in any curriculum that requires a knowledge of neuroanatomical structures.

Donna Forbes

 

Pathology

Currently the Pathology courses website offers students 24 hour access to course information including: lecture handouts and PowerPoint presentations, laboratory preview images and laboratory answers, laboratory review images and handouts, grades, and announcements.  Each period is self-contained and password protected for safe and efficient access to materials. All presentations are in a downloadable/printable format. Currently we are working on a database system to allow faster access and to have a more interactive site. The main goal of this effort will be a comprehensive and fully searchable materials and image database to allow students to more easily reference any pathologies that they were taught during the Years 1 and 2 courses.

Dan Dykoski

Medicine and Pediatric Clerkships

The Medicine Clerkship has the website www.imclerkship.umn.edu which has general information regarding the course.  It has links to each of the clerkship sites (Abbott, HCMC, VAMC, Regions, FUMC) with site specific information that students can use to prepare for the rotation.  The site also has the coursebook as well as a number of the lectures and handouts available for students to download for review or to look at if they missed a lecture. There is a clerkship CD ROM with clinical cases that is distributed on the first day.  The CD has all of the papers that the students use to prepare for the lectures and all of the course handouts and information they can use to improve their write-ups and presentations.  It also contains all of the materials they use for their portfolio.  The students also get a Physical Diagnosis Teaching CD with video clips, heart sounds, lung sounds and papers that are meant to supplement the physical diagnosis rounds the students have. 

 

In the Pediatric Clerkship students have access to the website www.peds.umn.edu which has site specific information similar to the medicine clerkship.  Within the website is the coursebook, practice questions,  a link to interactive practice cases (www.clippcases.org) and a link to an on-line site featuring videos used to teach the pediatric physical exam. We also encourage students to use their PDAs and give them a link to the PDA version of Pediatric One Minute Clinical Consult.

James Nixon

Brad Benson

Primary Care Clerkship

The primary care clerkship provides clinical experiences for medical students at a wide variety of sites scattered throughout the Minneapolis/Saint Paul and Duluth metro areas.  To maintain contact with the students, the course employs a multi-function website with links to a variety of resources and supportive materials.  The website is also used as a tool to distribute announcements and class schedule changes.  In addition the site provides access to a web-based patient log that is used to report the kinds of patients seen by the students.  They also have access to a clinical skills checklist which students use to report their learning of those skills.  The course also provides PDA resources for the students in terms of reference materials such a the Micromedix drug reference, medical calculators and the Five Minute Clinical Consult.  If a student does not own a PDA the course provides a loaner for the duration of the clerkship.

David Power

Neurology Clerkship

The neurology clerkship website provides a number of resources for the student.  The neurology tutorial syllabus is available and allows medical students to study the syllabus anywhere, anytime, provided they can access the web. There are also links to supplementary information at appropriate locations throughout the syllabus. We have attempted to provide some vertical integration of curricula by linking to relevant course materials from the year 1 medical neuroscience and year 2 pathophysiology courses, as well as American Academy of Neurology practice guidelines, supplementary chapters on stroke and neuromuscular disease which Dr. Walk has written in order to provide more in depth clinical information, useful external websites, and a neurologic examination streaming video. Dr Tuite has added supplementary videos and powerpoint presentations. We hope to add more information over time, and have charged a radiology faculty member with adding a neuroradiology tutorial.  The clerkship also has used webCT combined with a modified version of the Minnesota Virtual Clinic for a virtual patient designed to teach fundamental principles in medical professionalism and ethics.  The site also has an internal NBME shelf-style examination for the students to take as a practice test on webCT.  One of the clerkship sites is in Duluth which until recently had too few faculty to teach the clerkship tutorial, so we are including students from Duluth in our tutorial sessions at the U via Polycom or webconferencing.

David Walk

HIPAA training offered by the AHC

The Academic Health Center’s HIPAA training and certification program is one of the most successful early applications of the WebCT course management system combined with an online, interactive training program using Macromedia Flash.  It has trained and certified over 30,000 people to date in programs that are customized for the responsibilities of different users.  When a user signs on to WebCT they are presented with the modules that they must complete according to their role in health care at the University.  The system takes each user through each unit where they are presented with information and required to answer questions to demonstrate their understanding.  When each module is completed, the results are reported to the AHC and become part of the personnel record at the University.

Janet Shanedling (AHC)

Duluth Testing Program

The Duluth campus has implemented a comprehensive on-line testing program for all of their courses using the capabilities of WebCT. All questions are developed using the Quiz Tool and reside in the question database for each course. Students use wireless laptops provided by the school to take their exams in their classrooms and results are immediately available for review by faculty and students.

Richard Hoffman

Lectures on Line

Lectures on Line is a student initiated and Medical School supported activity that attempts to capture all of the lectures during Years 1 and 2 of the curriculum.  Using a computer located in the AV booth in each classroom connected to a video camera and the room sound system, the audio and the contents of the screen in the front of the room are captured and saved as a streaming video file.  These files are uploaded to a Real Networks video server for review by students in those classes from any network connected computer.  The lectures are accessible through the published class schedule on each Year’s meded web page and can only be viewed by members of that class. 

Stuart Speedie

Curriculum Database

The curriculum database is a collection of information representing the first four years of the medical school’s curriculum.  In the first two years it is organized around class schedules and in the last two years by clerkship.  The goal of the database is to capture and represent the information, knowledge and skills covered in every educational experience that occurs during those years.  For each Year 1 and 2 lecture a topic list is obtained and used to index that lecture.  In addition, if a PowerPoint presentation was used or supplemental materials such as handouts were provided in electronic form they are also included as well as links to Lectures on Line.  Information in the database is retrievable in two ways.  A master calendar provides links to all sessions of all courses in the first two years.  There is also a search mechanism similar to Google™ that allows free text searching on any term.  These searches return all of the lectures and clerkships that include the search term.  Thus it presents a powerful tool for faculty and students to locate where and when in the curriculum a particular topic is covered. 

Stuart Speedie

E*Value

The E*Value system is a web-based system for evaluating students during their clerkships and for students to provide feedback on their experiences in those clerkships. If presents a series of questions on a web page that were designed by evaluation experts in the Medical School and to which the evaluator responds to by selecting the appropriate option or typing in a comment.  Central to the system is a master schedule that details when each student will complete each clerkship.  When the end of a clerkship approaches, the system automatically emails the clerkship director or site director with a reminder to complete the evaluation for the current students.  It tracks completion of the evaluations and will send reminders when needed.  The system also emails the students to remind them to complete their evaluations of the clerkships.  E*Value provides a series of summary reports of these evaluation activities that are used by the clerkship directors to assign grades and to make  changes in the clerkships based on student evaluations.  E*Value plays a similar role for graduate medical education and is used by a number of residency programs for the same purpose. 

Theodore Thompson

Classroom Response System

Classroom response systems are typically used during a lecture to solicit individual responses during class.  After covering a topic, the lecturer presents a multiple choice question and ask the students to select which alternative they think is correct. These systems uses individual response units much like a TV remote with a series of buttons.  Receivers in the room are capable of detecting when a button is pressed on a unit and each unit has a unique identifier so that responses are not counted multiple times.   These receivers are connected to a computer that automatically collects the responses and aggregates them into a histogram or some other figure.  Based on the responses the lecturer can adjust his or her presentation.  The Duluth campus has used this technology for the past several years in their classrooms and discussion groups and it is available for use in PWB 2-470 on the Twin Cities campus as well as for checkout use in other rooms.

Richard Hoffman

Laptop Purchase Program

Working with campus information technology partners, the Duluth campus was able to negotiate a laptop purchase program that allows students to buy a fully-equipped tablet computer with a four-year service plan and wireless capability, all within the constraints of financial aid packages. About 60% of the incoming Duluth students currently use laptop computers in the classroom.

Richard Hoffman

SimMan, CathSim

Since Fall 2003, SimMan has been used to train medical, nursing, and pharmacy students in everything from basic physical exam skills to the elements of successful team performance in acute care scenarios.  SimMan has also been used to train U.S. embassy medical staff from around the world in best practices for responding to complex medical emergencies.  CathSim has been used by medical and nursing students to teach the basic elements of IV insertion using six virtual patients, ranging from an infant to a geriatric patient.  Working with Emergency Medicine instructors, IERC staff developed a rubric for assessing medical student competency in IV insertion before they started clinical rotations.  CathSim's ability to collect data on each student's performance provides, for the first time, an objective measure of each student's abilities before they encounter actual patients in clinical settings.  With the acquisition of additional human patient simulators in the near future, planning is underway for significantly expanding the use of simulations at every level of training in each of the health sciences at the University of Minnesota.  The Interprofessional Education and Resource Center (IERC) which houses these simulators assists faculty interested in using them in training students, residents, and practicing professionals.

Jane Miller

GME Online Core Curriculum

The GME online core curriculum provides an opportunity for residents who are not able to attend the live presentations of the GME Core Curriculum to review those presentations at a later time.  The video and audio of all core curriculum presentations are recorded and converted to streaming video files.  A WebCT course was created for the core curriculum and provided links to each presentation.  When a student selects a particular presentation, they must identify themselves and indicate their residency program.  This is followed by a short, pre-presentation confidence assessment concerning the topic.  The student then goes to a page that has a short biography of the presenter along with links to the streaming video presentation and the PowerPoint slides which the student can view at the same time in different windows on the screen.  When the viewing is completed, the system records the amount of time spent viewing the video and asks a series of evaluation questions and another confidence assessment.  By completing this process the resident can provide evidence that they have completed the necessary core curriculum sessions.

Lisa Wichman

 

 

 


 

 

 

 

 

Appendix D:

Technology Enhanced Learning Methods

 


Technology Enhanced Learning Methods

 

The following table lists a variety of TEL methodology for instruction and evaluation.  They are presented as a list of alternative approaches that can be used in medical education.  The table provides a brief description of each methodology and lists some of the tools that are available to implement that methodology.

 

TEL Method

Description

Tools

Instructional Methods

Recorded presentations

Recorded presentations often combine PowerPoint slides and audio narration that is captured digitally and made available to students at their convenience.  They provide the student with the ability to study the material at a time optimal for them and to review the exact content numerous times.  Students and rewind and replay as desired.  They are typically created either by recording a live lecture or by the instructor narrating the presentation at the computer.

Breeze Presenter

Narrated PowerPoint

Lectures on Line

Live distributed  presentations

Live distributed presentations are presentations that are distributed to one or more additional locations using videoconferencing systems such as Polycom or using networks and servers such as Breeze Live.  These sessions can be interactive with the ability of students to ask questions and respond to the leader’s questions.

Videoconferencing

Breeze Live

MS Instant Messaging

Polycom PVX PC

Live Group discussions

Live group discussions can use the same technology as live distributed presentations but the instructional goal is to teach by discussion rather than lecture.  They can also be entirely web-based using online chat technologies like instant messaging.  They all facilitate discussions among geographically dispersed participants when all are available at the same time.

WebCT Vista Chat

MS/AOL/Yahoo IM

Breeze Live

Videoconferencing

Asynchronous Group discussions

The goal of asynchronous group discussions is to promote discussions among groups of geographically distributed participants when those persons cannot or do not wish to be available during the same time period.  This method allows extended interactions for the purpose of exchanging information or reacting to others ideas.

WebCT discussion groups

Blogs

myAHC Portal discussions

Online Case Presentations

Online case presentations are cases delivered to students via web pages that illustrate various clinical scenarios and will often lead the student step by step through those scenarios. They may be a simple as a static web page or may involve multimedia presentations and user interactions in order to teach about clinical skills or other topics.  There are a number of tools for creating these case presentations that incorporate templates for case descriptions. 

Web Pages

Minnesota Virtual Clinic

DxR

CLIPPs

Numerous others

Online readings

Online readings are a method to link students to the literature available on the Internet.  This is typically accomplished by embedding web links into web pages that will take the student to the desired article of particular location in a reference.  

Web pages

Full text databases

Full text references

Bio-Medical Library tools.

Online assignments

Online assignments are a means to deliver assignments to students and to collect the assignments that are submitted for review and grading.  WebCT provides the means to grade these assignments and distribute feedback to individual students.

WebCT Assignments

Active Websites

Online Group Projects

Online group projects combine online assignments with group discussion methods to assist student to work together on projects when they cannot meet face to face.  It is possible to limit the discussion groups to just the participating students and an instructor if desired.

WebCT Private discussions and assignments

myAHC discussion groups

Interactive, directed Learning

Interactive, directed learning is a method of online teaching where the information to be presented is divided up into small units that can be presented on a typical web page.  Often the students will be required to take some action such as answer a question to verify that they understand the presentation before proceeding.  Multimedia elements including video and audio clips as well as animated illustrations may be part of such units.  The student may proceed through each unit in a sequence determined by the designer or may be allowed the freedom to choose their own path..  This approach is also commonly referred to as a learning module.

Custom websites

Macromedia Director

Computer-based simulations

A computer-based simulation uses a model of some medical phenomena such as electrolyte balance to teach about a particular topic.  The model allows the student to explore the consequences of various changes that can occur or are made to occur as the result of different disease states or therapeutic decisions.  It can also be used to create various scenarios for the student to interact with much like the case presentations described above.

Custom websites

Commercial Products.

Physical simulations

Physical simulations are similar to or even incorporate computer-based simulations but realize the results of the simulation in a physical model that students can touch and feel in order to practice and learn physical clinical skills such as intubation, venipuncture or endoscopic skills.  These simulators provide tactile feedback to help students learn their skills.  They are usually quite complex interactions of physical and computer components.  To be used as a teaching tool, scenarios need to be defined that determine how the model will operate and react to the inputs provided by students.  

SimMan

CathSim

Many other commercial simulations

Virtual Reality

Virtual reality is a form of computer-based simulation which attempts to provide a 3D environment in which the student is totally immersed.  One example is derived from the Visible Human project that provides the means to “fly through” the GI system to view internal components that would be difficult to demonstrate otherwise.  This is a cutting edge technique this is just being developed for educational purposes.

Visible Human

 

Self-testing with feedback

Self testing quizzes are a useful device for students to test their knowledge in a particular area.  Asking a question and providing different choices of answers along with explanations of why each answer is either or correct or incorrect.  The WebCT quiz tool allows the instructor to make these self-testing quizzes available for students to use at their convenience.

WebCT quiz tool

Custom websites

Breeze Presenter

Evaluation

Multiple choice testing

These are questions in which the instructor poses a question or situation in the stem of the question with directions for the student to make some choice or choices.  The question may include pictures, illustrations, video and audio clips if desired. A set of alternatives is provided and the student is expected to select one or more in response to the directions.  In online testing neither the number of choices or the number of correct answers is not limited to specific number.  Answers can be automatically scored and recorded for each student.

WebCT testing tool

Short answer questions

In this type of question, the instructor poses a question and sect of directions and the student is required to respond with a one or two word answer.  If the instructor supplies a list of correct answers, these questions can be automatically scored and recorded.

WebCT testing tool

Essay questions

Essay questions can be used in online testing and the system will record the student’s answers, but they must be manually graded by the instructor or their representatives.  The system will record the scores and incorporate them into a final test score if desired.

WebCT testing tool

Matching

Matching questions are an efficient means of asking a number of multiple choice questions about a particular topic such as identifying anatomic structures.  The WebCT testing tool permits partial and weighted scoring of matches. 

WebCT testing tool

Computer-based simulations

Computer-based simulations have been described above as teaching tools.  They can also be used for evaluation of student knowledge and skill by presenting the student with a scenario to which they are required to respond.  The responses are evaluated by an expert or can be automatically scored if acceptable responses are predefined.

Custom websites

Commercial Simulations

Physical simulations

Physical simulations can be used for evaluation in the same manner as computer-based simulations with the added benefit that performance of physical skills can be observed and evaluated.

SimMan

CathSim

Many others

Expert judgments

Information technology can be used to facilitate the observation and expert evaluation of clinical skills.  It does this by providing online reminder lists of what should be observed and evaluated as well as providing a mechanism for recording and reporting the results of those evaluations.

E*Value

CoursEval

Custom websites

Performance review

Information technology can facilitate the review and evaluation of a variety of work products that students may generate to demonstrate their competence in various areas of medicine.  This is accomplished by created a repository for individual student’s digitally realized works that they can share with and have evaluated by faculty.  These might include patient education materials, educational videos, recorded clinical performance, written documents among but a few examples. Tools such as ePortfolio provide a means to store and retrieve these materials and make them available for evaluation. 

ePortfolio

E*Value

Skills and Competency checklists

Skills and competency checklists are a means of tracking student performance and achievement over time.  Information technology can make such lists available online and can provide the means for multiple individuals including students and instructors to record and verify achievement of those skills and competencies.

E*Value

Custom websites