The Instructional System Design Model: A Framework for Development of a Web-based Program
Maria Bishop, M.D., B.S.N.; Susannah Reiser, M.A.; Ann M. Taylor, M.P.H., C.H.E.S.; Judith A. Rein, Ph.D.; John Hall, Ph.D.
Quality-of-life concerns have resulted in a need for physicians to become competent in end-of-life care (EOLC). The web-based Hospice Model of Care program introduces medical students to EOLC. The Instructional Systems Design model provides the program framework, including content development, case-based video integration, and formative evaluation; Groove Virtual Office provides the collaborative working environment. Evaluation results facilitated revision of the instructional strategy including video demonstrating authentic patient-physician encounters. Developers of similar curricular programs would benefit from our experiences.
Many medical schools use technology-based teaching methods in their predoctoral programs (Gordon et al. 1999). Approaches range from interchangeable learning objects—e.g., digital images of the brain—to the delivery of complete courses online (Wiley 2000). Development of these instructional methods requires considerable time and effort from a team of experts. Murphy (2005) states that 18 hours of faculty time are required to complete one hour of web instruction. Developing a complete course may take up to one month of the subject matter expert’s time. Combine this with the effort contributed by programmers, web developers, and instructional designers and it becomes apparent: Web-based education is expensive and time-consuming. Consequently, team collaboration is vital to the conceptualization, development, and production of a professional-quality product.
Given the proliferation of multimedia authoring tools such as Adobe Flash™, interactive applications which mimic the instructor can now be constructed. An application that displays the digestive system and its functions (Boudinot and Martin 2001) is one such example. Another example is the Harvey system, a cardiology patient simulator designed to imitate common and rare cardiac conditions in a practical and safe manner (Gordon et al. 1999). These multimedia applications accommodate a wide range of users with varied schedules and offer an environment that promotes learning on a personal level.
Given the time and expense required to effectively create and evaluate these newer learning technologies, especially in the medical environment where schedule demands often requires asynchronous collaboration, the goal of the current project was twofold. The first of these is to evaluate the effectiveness of using a traditional model of instructional design applied to newer teaching technologies. The second is to evaluate the effectiveness of a computer-based collaboration tool to implement that instructional design model given the fact that collaborators could rarely meet yet needed to be included in the decision making process.
The project’s focus was the creation of a web-based program that is part of the Comprehensive Education in End-of-life Care (CEPEC) grant, the goal of which is to develop, implement, and evaluate a cancer and EOLC curriculum at the University of Arizona, College of Medicine (UA CoM). Learning objectives, instructional modules, and evaluation instruments, essential components of a cancer and EOLC curriculum, are under development for key topics (e.g., communication, symptom management, treatment/outcomes, cultural competence, financial issues, advance directives/ethics, palliative care, spirituality, and provider self-care). This case study tracked development of the first web-based program, The Hospice Model of Care.
The Hospice Model of Care (HMOC) is characterized by providing compassionate care for terminally ill patients and their families, delivering health care via an interdisciplinary team, and improving quality-of-life by symptom management and supportive care services that address physical, psychological, and spiritual issues. This web-based instructional program conveys essential HMOC principles and consists of case-based videos with audio narration and other features that deliver uniform and interactive learning to students.
The CEPEC team used the Instructional Systems Design (ISD) model as a framework to develop the HMOC web-based instructional program (Dick, Carey and Carey 2001). Fundamental characteristics of ISD include interrelated components of instruction—resulting in a systems approach. These components consist of the (1) learner, (2) instructor, (3) materials, and (4) learning environment. Each component, individually and collectively, have important roles to consider when designing instruction as they must interact effectively for the intended goal of learning.
Within the model there are distinct phases to follow, allowing design details to become the focal point (Table 1). Another advantage of using a systematic approach is that it facilitates integration of technology into an existing curriculum by providing a detailed documentation process.
The HMOC program team members consist of an instructional designer, subject matter experts (SME), video producer, videographer, graphic designer, and programmer. The development of HMOC program materials were facilitated by the multiphase character of the ISD model allowing complete coordination of team efforts.
In the Analysis Phase instructional goals are defined, instructional analysis is conducted, and entry behaviors and characteristics are identified. Based on the CEPEC project pre- and post-tests administered to 1st-year and 3rd-year students, cancer-related EOLC education and training is warranted in the UA CoM curriculum (Bishop et al. 2005). The instructional goal for the HMOC Program is to increase 2nd-year medical students’ knowledge and skills. Accordingly, issues related to hospice are the primary focus of this program. EOLC and hospice are characteristically multi-dimensional; hence the program has three learning sections: (1) Diagnosing end-of-life, (2) Hospice Model of Care, and (3) Individualized Services—including bereavement support. Because learning EOLC begins with conceptual knowledge that eventually leads to clinical application, a cognitive framework formed the basis for the instructional analysis. Student entry behaviors and characteristics are similar (e.g. education level, basic science courses completed, experiential knowledge, volunteer work) for our population.
In the Design Phase learning objectives are written and the assessment instrument and instructional strategy are designed. Learning objectives were established as part of the CEPEC research project. From these objectives, thirteen HMOC program learning objectives were selected and refined for content development. The HMOC Program learning objectives also provide the framework for test item development; each objective has a matching test item. The assessment instrument consists of ten pre- and post-test multiple-choice items, ten parallel practice test items, and one abbreviated Subjective-Objective-Assessment-Plan (SOAP) note. The program sections include practice test items intended for interactivity and retention. Each practice test item has a corresponding rationale designed as a learning aid. A review of the literature substantiates the rationale for each item.
The instructional strategy relies heavily on student interactivity and is designed to engage students with modeling of appropriate behaviors by experts and a thought-provoking practice test, rather than simply clicking on buttons alone (Mayer 2001). Although lack of student motivation is a serious problem as demonstrated by attrition rates between 50% and 70% for non-required online courses (Frankola 2001), the HMOC Program was a required assignment as part of UA CoM Social and Behavioral Sciences course for 2nd-year students. Thus, issues related to attrition rates became less of a concern for our population.
In the Development Phase the content and instructional materials are produced and revised. The instructional designer created a flowchart and storyboard which were instrumental during the materials production and revision process. The flowchart provides a concrete visualization of the program’s content points and user interactivity. The detailed storyboard contains text such as user instructions, objectives, test items, and other documentation. Team members, synchronously and asynchronously, shared and revised documents, and communicated by using Groove Virtual Office™ (Groove), a web-based and user-friendly collaboration tool. Groove has many secure features wherein a team can efficiently share files instantaneously, co-edit documents, manage project tasks, use instant mail messaging, and set up multiple project workspaces with the option to selectively invite members to collaborate. Other features include, but are not limited to, a discussion board, live chat function, and a Microsoft Office Outlook™ compatible calendar. Since some team members were geographically separated, Groove provided the “virtual office” where team members met, enabling participants to telecommute in an efficient and effective manner (Figure 1).
Production of the video segments was initiated early in program development due to the time-intensive nature and because the program consists of multiple sets of materials (e.g., video editing, narrator voice-over, programming, graphic design) produced at different times. Members of the team began developing content by creating a fictitious patient’s past medical history and audio taping a mock physician-patient encounter based on the program’s learning objectives. The audio tape was transcribed and revised, then transformed into a script format from which the video participants would read during the video taping.
The ISD model implies frequent assessment of mock-ups and drafts to create an iterative process that refines the effectiveness of instructional materials during development. After video taping and editing were complete, the team reviewed the video and unanimously concluded that the dialogue was verbose and stilted; there were unrealistic expectations that standardized patients and clinicians could effectively deliver lines using cue cards (e.g. someone with acting experience). Consequently, team members revised the video by including additional bulleted, on-screen, text to address salient points, also including voice-over narration for dialogue reduction. After the team reviewed the revised video for accuracy, the instructional designer prepared the formative evaluation instrument.
In the Evaluation Stage the evaluation instrument is designed and evaluation is conducted. A 7-item multiple-choice formative evaluation instrument was constructed to obtain feedback regarding program content (e.g. clarity, distractions, redundancy). Two 3rd-year students who had completed three clinical rotations (e.g., pediatrics, medicine, surgery) agreed to review the video component of the program. The students had already received background information about the project and were given a pen-and-paper evaluation form to complete after viewing the video. Based on their responses the team concluded: (1) the video did not hold their attention, (2) the actors did not display adequate emotion, and (3) realistic role modeling behavior was needed. In addition, recent exposure to first time, real-world clinical experiences contributed to student reviewers’ strong desire to learn about ambiguous situations regarding patient interactions. To address these weaknesses, the team discussed incorporating the content in a didactic yet appealing way. Although design changes were needed the basic content would remain unchanged.
In the Revision Stage instructions are reworked and improved according to the evaluation outcomes. By using Groove, revisions were expedited, avoiding a prolonged and cumbersome process that would have been necessary in the face of varying schedules and locations of team members. Based on the formative evaluation results the instructional strategy was revised to include a new portrayal of role modeling within the case-based videos accompanied by documentary interviews with three experts in EOLC. The cast remained the same with respect to the physician and hospice nurse. However, the patient and patient’s wife were replaced by new cast members with some acting experience. Another physician expert portraying himself addressed hospice concepts and spirituality by using a technique to establish a patient's spiritual history. Videotaping locations were also changed to include a private meeting space (e.g., without interruptions) and for background variety. A narrator voice-over was created to supplement content points not addressed. Upon completion of video-taping, the instructional designer edited eight hours of footage to a total of 30 minutes. Subsequently, four 3rd-year students evaluated the video using the original formative evaluation instrument. Results indicated: (1) good role modeling, (2) concise, clear, relevant information, and (3) remarkable improvement from the previous video. After reviewing the video, the instructional designer and programmer determined appropriate time points for voice-over narration including the introduction, instructions on using the program, and points that required bridging to meet objectives. A professional narrator was hired to perform the voice-over narration. After compressing the digital assets, various video and audio file sizes were tested over different computer locations for best usability. Once the graphic design and user interface (e.g., look, feel, style, and navigation features) were in place (Figure 2), programming began using Adobe Flash 8. After programming was complete, CEPEC project staff beta-tested the program for function and navigation. Based on these findings, minor navigational and content changes were made. After releasing the program, a few students encountered technical problems due to firewalls and difficulties when installing Flash Player™. Overall, technical problems were minimal.
Hmoc Optional Elements
In response to student reviewers’ input, optional interactive features were developed including: (1) an Ask-The-Expert question video database, (2) a Resource Library, and (3) a threaded discussion board. The Ask-the-Expert activity (Figure 3) includes a database of 37 questions related to EOLC; questions were submitted by students and developed by SMEs (see Figure 3). The interactivity allows the student to click on a question and images of experts appear, the student then selects an expert’s (thumbnail) image who answers the question in video format. Fifteen experts from various fields (e.g. oncology, EOLC, surgery, pharmacology, ethics) volunteered their time to answer questions, thus allowing students to learn about both common and unique points of perspectives.
The Resource Library (Figure 4) contains a concise downloadable student manual, a list of EOLC web resources, a glossary, and references. Finally, the threaded discussion presents case scenarios and questions intended to elicit student discussion (e.g., board posts).
During a lecture CEPEC staff administered a pencil-and-paper, 10-item multiple-choice pre-test before allowing the students to access the HMOC program. The parallel post-test was a required and an embedded component of the HMOC program. Results of the post-test indicate that after accessing and interacting with the HMOC program, more students received higher scores on the post-test compared with lower scores on the pretest (Bishop et al. 2006).
After finishing the program, students also completed an optional user-satisfaction survey. These survey results indicated that students were pleased with the web-based program. Of the 95 students who completed the program, 84 (88%) took the optional survey that consisted of 5 items using 6-point Likert scales (e.g., completely satisfied to completely dissatisfied, completely agree to completely disagree, a great deal to none), a fill-in item, and comment space. In response to how satisfied students with this program, 82% were satisfied, and 89% felt they learned information about palliative/EOLC because of using the program, 89% agreed that they wanted to learn more about palliative/EOLC.
Students commented liberally about the program. Comments attest to the overall positive appeal of the program. Students also felt the information was presented well, wished more classes were like the HMOC Program, enjoyed the patient-physician portrayal, and liked the program’s interactivity.
Students used two new features only infrequently—the Ask-the-Expert series and the threaded discussion board; time constraints (e.g. schedule conflict with midterms) and elective material, contributed to their limited use since they were optional activities. The next iteration of the HMOC program slated for release in February 2007 will require students to view relevant Ask-the-Experts videos in preparation for related and required small-group discussions in the same SBS course. The next iteration of the HMOC program will not include the threaded discussion board. Interactive discussion features already exist on class-specific medical student websites (e.g., AZMed website); integrating the HMOC feature will allow students to interact and pose both course- and HMOC-specific questions in one location. This decision is consistent with our institution’s long-term goal of incorporating use of technology in all facets of medical education.
The HMOC program is intended to teach medical students multi-dimensional elements of EOLC theory and practice. Because the video is a key learning aid, it became important to insure an accurate portrayal of realistic scenarios. The initial formative evaluation, driven by the ISD model, provided the team with valuable feedback about the use of video to teach medical students. While the original video was verbose and stilted, the new documentary-style video produced a streamlined and engaging portrayal of the patient-physician encounter delivering bad news, discussing spiritual issues at the end-of-life, and hospice and bereavement care (Table 2).
Research shows that many web-based courses have significant dropout rates implementing the program as compulsory produced complete student participation (Frankola 2001). An integrated pre-test, post-test, and satisfaction survey provided a method to collect student data to comprehensively evaluate the effectiveness of the program for future use.
The ISD model rendered a systematic way to perform learner analysis, develop content, evaluate, and revise accordingly. The ISD model also facilitated the team collaboration efforts by modularizing the tasks to be completed by the various team members. The use of Groove allowed those tasks to be shared in the form of documents for review and revision as appropriate.
The use of a network-based collaboration tool was integral to implementation of this instructional program. The team was able to learn and utilize Groove easily due to the application’s moderate learning curve and intuitive interface. While not originally intended for program development, this collaborative tool became an integral part of a successful model for the team. Groove provided a communication tool and central repository for current versions of all materials generated by applying the ISD model. It also enabled team members to work from various remote locations and at different times. Using Groove for the HMOC program promoted team efforts and efficiency while enhancing project development. This case study demonstrates the usefulness of employing a traditional instructional design model (e.g., ISD) with newer technologies; it also demonstrated that such a complex model can be successfully implemented, even when team members are geographically separated.
We encourage colleagues to accept new instructional development challenges presented by emerging technologies. Those technologies allow for a choice of time and place for interaction with an infinitely patient virtual instructor. However, we also encourage instructional designers not to abandon the traditional educational principles that have helped to create effective educational media with older technologies.
For colleagues who find themselves in an environment where it is difficult to meet, we strongly suggest investigating the use of a collaboration tool such as Groove Virtual Office for program development. Such tools can aid in the normally time-consuming production phase by providing instant communication, file sharing, online meeting, and record tracking and retention thus, increasing overall efficiency, effectiveness, and productivity. Additionally, the ISD model promotes a structure that allows an intuitive documentation and revision process. The positive outcomes associated with development of the HMOC Program have prompted team members to continue working efficiently with these tools on an ongoing basis.
Table 1: ISD phases listed in sequential order (Dick, et. al. 2001)
Table 2: Comparisons of video dialogue before and after revision
Bishop, M., Taylor A. M., Rein, J.A., Ahner, H., and Olson-Garewal, K. 2005. First year medical students' knowledge and attitudes regarding end of life. Abstract presented at the annual meeting of the American Association of Cancer Education, Cincinnati.
Bishop, M., Reiser, S., Taylor, A.M., Rein, J.A., Hall, J. R. 2006. Results of a self-paced, web-based hospice model of care program for predoctoral students. Abstract presented at the annual meeting of the American Association of Cancer Education, San Diego.
Boudinot, S.G., and B.C. Martin. 2001. Online anatomy lab (OAL): A self-regulated approach to the instruction of human anatomy. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 3, no. 1 (April). http://imej.wfu.edu/articles/2001/1/01/index.asp (accessed October 18, 2005).
Dick, W., L. Carey, and J.O. Carey. 2001. The systematic design of instruction, 5th ed. New York: Addison-Wesley Educational Publishers.
Frankola, K. (2001). Why online learners drop out. Workforce Management, (October). http://www.workforce.com/archive/feature/22/26/22/index.php
Gordon, M.S., S.B. Issenberg, J.W. Mayer, and J.M. Felner. 1999. Developments in the use of simulators and multimedia computer systems in medical education. Medical Teacher, 21 no. 1, (January): 32-36. http://taylorandfrancis.metapress.com/(q5pbn555dgsot54504pq5ka3)/app /home/contribution.asp?referrer=parent&backto=issue,6,24;journal ,54,60;linkingpublicationresults,1:101208,1 (Link accesses journal from the publisher's site) (accessed August, 10, 2005).
Mayer, Richard. E. 2001. Multimedia Learning. New York: Cambridge University Press.
Murphy, E. 2004. Moving from theory to practice in the design of web-based learning using a learning object approach. E-Journal of Instructional Science and Technology, 7 no. 1, (March). http://www.usq.edu.au/electpub/e-jist/docs/Vol7_No1/content.htm (accessed August 16, 2005).
Wiley, D. A., ed. 2000. Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version, http://www.reusability.org/read/#1 (accessed August 29, 2005).
Funding is provided by the National Cancer Institute (Grant R25 CA092517) and the University of Arizona.
The following individuals have graciously participated by giving their expertise and time. We are grateful for their efforts.
AZ Health Sciences Center Biomedical Communications Team
Maria Bishop, M.D., B.S.N. is a board-certified internist and oncologist with clinical practices at the University of Arizona (UA), Arizona Cancer Center and the Southern Arizona Veterans Affairs Health Care System (SAVAHCS) in Tucson, AZ. She is an associate professor of clinical medicine at the UA, College of Medicine and is actively involved in teaching both pre- and postdoctoral students. As Principal Investigator of the Comprehensive Education in Palliative/End-of-life Care (CEPEC) project, Dr. Bishop oversees development, implementation, and evaluation of the cancer and end-of-life care curriculum for predoctoral students.
Susannah Reiser, M.A., is the instructional designer for the CEPEC project at the Arizona Cancer Center where she specializes in curriculum development, production and technical documentation of digital learning objects using systematic methodologies, and technologies such as audio, video, and interactive evaluation instruments.
Ann M. Taylor, M.P.H., C.H.E.S. is the director for the CEPEC project at the Arizona Cancer Center where she coordinates the daily research-related activities. She has considerable experience developing curricula for diverse populations, coordinating cancer-related educational research in community- and academic-based settings, and has co-authored several articles on curriculum development/evaluation projects. She also teaches sessions for 3rd-year medical students.
Judith A. Rein, Ph.D. is the curriculum and evaluation specialist for the CEPEC project at the Arizona Cancer Center where she designs and produces evaluation and instructional materials.
She has taught various measurement courses for the Department of Educational Psychology at the University of Arizona. She also directed the Testing, Evaluation, and Assessment section of the Division of Academic Resources at the UA College of Medicine.
John Hall, Ph.D., the Associate Director of the Division of Biomedical Communications at the Arizona Health Sciences Center. His academic interest includes communication of mediated appeals related to health messages. He has participated in numerous studies concentrating on the conditions under which people learn healthy behaviors such as the recent creation and investigation of smoking cessation curricula. Additionally he has co-authored several articles on media-based message construction that persuade people to avoid unhealthy behaviors. Much of his expertise centers on using web-based instruction including producing and teaching multimedia and web-based applications.
2007, The Journal of Biocommunication, All Rights Reserved
Table of Contents for VOLUME 32, NUMBER 3