Currently, virtual classrooms and virtual laboratories expand the use of computers for individual learning at home. These new virtual learning places integrate all given technology and didactic methods and enhance their usability by net-based communication between learning students, online tutors and connected real devices. We thus labeled class 2 text. Class 3 was composed of 54 studies Interactivity and instructional design principles were nearly nonexistent.
We thus labeled class 3 Web conference. Class 4 contained 18 studies 7. We thus labeled class 4 instructional. The four right-hand columns in Figure 4 depict the composition of each class and the relative weight of each variable on class assignment. Of the articles included in this study, we identified conducting interventions on the end points of interest Figure 5 summarizes the results for this section and depicts the percentage of studies for each intervention characteristic, and the relative contribution of intervention end point variables to the intervention end point latent class described below.
Prevalence of articles per intervention feature and intervention endpoint latent class. Only variables regarding assessment of knowledge, attitudes, skills, and online activity the 4 last panels were used to determine intervention end point latent classes. CBL: computer-based learning.
A total of 81 of studies were conducted with medical students from preclinical years In addition, 32 studies were conducted with specialist medical doctors In addition, 84 studies repeatedly tested participants in a pre-post approach A total of 61 studies were randomized Of the studies, 28 studied controlled interventions between blended learning approaches and traditional lectures A total of 8 studies compared spaced repetition versus bolus learning 3.
In addition, 5 studies compared the use of 3D models versus 2D images 2. A multitude of other comparisons were performed, such as exploratory versus blocked learning approaches [ 73 - 75 ], complex versus simple user interfaces [ 73 , 76 , 77 ], immediate versus delayed completion of lectures in CBL systems [ 78 ], and multimedia versus text on CBL media [ 73 , 79 - 81 ]. Multimedia Appendix 3 lists the different comparison groups we identified for each of the articles reporting interventions. Knowledge outcomes were assessed in of articles Objective knowledge was assessed using multiple choice quizzes in 98 of studies Judgments of knowledge were collected using Likert scales in 27 studies Researchers directly assessed knowledge in 9 studies 7.
A total of 31 studies were conducted in a laboratory setting Knowledge assessment was part of a final examination in 39 articles Of the studies, 90 reported that interventions improved knowledge acquisition Of studies, assessed student attitudes A total of 29 studies were conducted in a laboratory setting In addition, studies found positive attitudes toward interventions Of studies, 31 assessed subject skills Online activity was measured in 76 of studies We considered 3 distinct classes to group the studies taking into consideration intervention end point variables.
Class 1 contained articles assessing knowledge and attitudes We labeled class 1 knowledge and attitude. Class 2 contained 25 intervention studies In addition to assessing knowledge and attitudes, articles in this class also assessed skills. We labeled class 2 knowledge, attitude, and skill. Class 3 contained 12 studies that assessed online activity, specifically through the number of posts and number of reads 5. Attitudes were always assessed, but knowledge and skill assessment were nearly absent. We labeled class 3 online activity.
Table 2 reports model statistics for the intervention end point latent classes, and Figure 5 depicts the prevalence of articles per intervention feature and intervention end point latent class. Latent class analysis model fit per number of latent classes for intervention end points. This decision was based on picking the model with the best interpretability and the lowest BIC. Of studies, 25 correlated different variables with knowledge outcomes Of these, 1 study correlated system interactivity with knowledge scores and concluded that lower levels of interactivity benefitted knowledge acquisition [ 73 ].
Correlations between knowledge gains and time spent using online platforms were also sought. These were found to be positive in 4 studies [ 49 , 87 - 89 ] and neutral in 1 study [ 76 ]. In addition, 1 study described a modest positive correlation between increased knowledge scores on the learning system and an increase in examination scores [ 90 ]. Increased learning platform usage was correlated positively with knowledge acquisition in 5 studies [ 90 - 94 ], while 4 found no association [ 46 , 95 - 97 ]. Other studies found positive relationships between knowledge and the number of posts in online forums [ 98 , 99 ] and comprehensiveness of student study materials [ ].
Regarding attitudes, 2 articles found a mild positive correlation between judgments of knowledge and knowledge score [ , ]. Other correlations were assessed, namely confidence and skill [ ], study duration and skill [ ], and study duration and learning style [ ], but did not reach statistical significance.
We obtained references and citations for of the articles included in this review The mean number of references was In total, the abstracted articles had references to other articles. The most referenced articles were from Ruiz et al [ 4 ], Cook et al [ 12 ], Chumley-Jones et al [ ], Greenhalgh [ ], Ward et al [ ], Muller [ ], and Ellaway and Masters [ ].
The mean number of article citations of the abstracted articles was More than half of the references were common to various abstracted articles, while a smaller percentage of studies had independent sets of references. Of the articles, had at least one reference or citation in common with other abstracted articles A total of 58 articles were not related to any other article, since they did not share references or citations The mean number of related studies for each article included in this review was 4. Relationships between articles included in this review indicated by nodes.
Links between nodes indicate that articles have references and citations in common. The width of the link indicates the number of studies in common, ranging from 1 to 5. About a quarter of the studies have no common references or citations. Only of the studies were included in this analysis due to missing information Studies comparing traditional versus CBL methods were cited a mean of Studies comparing different CBL methods were cited a mean of Figure 7 shows this result. Regarding educational software latent classes, articles in the multimedia class had a mean of 3.
Articles from the Web conference class had a mean of 5. Regarding the intervention end point latent classes, articles in the knowledge and attitude class had a mean of 2. Articles from the online activity class had a mean of 6. Finally, articles not citing the Cook et al work had a mean related article count of 4. Figure 8 plots the complete results for this section. Mean number of related articles per latent class and reference to the Cook et al review. Number of related articles is adjusted for publication date. P values indicate intraclass pairwise differences from the topmost element of each color-coded class.
Significant relationships are marked in bold typeface. Of the interventions, most did not use comparison groups and lasted between 1 week and 3 months. Articles comparing different CBL methods were cited more often than were studies comparing traditional versus CBL methods, independent of publication date. Articles reporting instructional design principles, articles measuring online activity, and articles citing the Cook et al CBL reviews had significantly more references and citations in common than did other articles.
The last systematic review and meta-analysis of this topic encompassed data from to and highlighted the problems of intervention variability and lack of evidence for comparative effects of CBL methods [ 12 , 13 , 28 ]. Recent reviews have also demonstrated that practice exercises, interactivity, feedback, and repetition can favorably influence learning outcomes [ 13 , 49 ].
Other reviews summarized technologies and methods used [ , ], and addressed specific topics such as the role of blogs [ ], wikis [ ], portfolios [ ], simulations in general [ ] and for surgery in particular [ ], gastroenterology [ ], catheterization [ ], and airway management [ ].
Other authors focused on specific aspects of the effects of Web-based learning on problem-based learning [ ], and the implications of recent Web capabilities, namely Web 2. Our study complements previous reviews by encompassing recent work concerning these fields over a large base of abstracted articles. Despite the considerable time overlap with similar reviews, assessments such as latent class analysis and citation network analysis had not yet been conducted during the considered time period [ 13 ].
This study has limitations. We scrutinized databases that frequently index medical education articles. However, we did not abstract the gray literature or references from other articles, and thus our article search cannot be considered exhaustive. We narrowed the study participants to medical education only. This can be considered a limitation insofar as these findings cannot be generalized to other health professions. Other reviews have performed similar searches including articles in health professions in general [ 12 ].
We performed the article abstraction step manually. While the independent reviewing method and ICC results indicate a low probability of coding error, we cannot completely exclude it. Variables regarding instructional design and assessment outcomes were often not explicitly declared and relied on reviewer judgment.
We could not retrieve references and citations for 27 of the articles This study also has strengths. We performed a broad analysis of the literature and accounted for aspects that, to our knowledge, were not previously assessed, such as specific platforms and their features, and correlations assessed between learning end points and types of comparisons.
We systematically summarized data using latent class analysis, which, to our knowledge, was for the first time performed in this setting. We described the article citation network and explored relationships between these and the article latent classes and CBL considerations, which, to our knowledge, were also for the first time performed in the field.
Finally, we have made these results available through an interactive visualization that allows researchers to deeply explore articles [ ]. Our findings show that, while CBL in medical education varies significantly, most published articles are from medical schools in a small set of countries. Medical education has geographical specificities, which makes contributions from different geographical areas particularly enriching and should incite more schools to conduct research in this field. Over 25 platforms and software projects were built specifically for medical education, despite having significant overlap in goals and features.
While a few provided means to interact with learning materials, such as microscopy images [ 72 ], in ways not before possible, it would be worthwhile for researchers to try to develop open and generalizable systems addressing specific learning contexts that can be reused by researchers from other medical schools. Initiatives to design pluggable modules for mainstream learning management systems and reusable learning materials, such as learning objects [ ], aimed at specific medical contexts should be preferred over building closed systems from scratch.
The diversity of methods encompassed by CBL in terms of delivery medium, context, learner, and purpose, without reports of instructional design considerations, obfuscates the effect of different intervention aspects, for which instructional design—or the lack of it—is partly accountable [ 8 , 9 , 13 , ]. The value of reporting interactive tools, such as quizzes with feedback, would also increase. Determining which principles best apply to different medical settings and medical knowledge is an issue of interest for future research [ 8 ].
Studies generally report positive outcomes on knowledge, attitudes, and skills. Interestingly, studies that found no positive effect in any of the learning outcomes were often randomized controlled trials [ 66 , 83 - 86 ], some of them running in multiple institutions [ , ]. Studies with little or no description of the learning and teaching methodology had neutral findings [ 82 , ].
Once again, the lack of comparable arms, such as CBL versus traditional instruction, makes it difficult to assess intervention outcomes. Furthermore, data showing that objective knowledge assessment and skills increase with interventions can be used in deeper ways. Real-time collection of student activity, together with objective performance assessment through multiple choice quizzes, may have predictive value. Judgments of knowledge together with other student activity metrics may provide data for a next generation of intelligent tutoring systems able to track, manage, and predict student performance [ ].
An increase in studies reporting online activity measurements and correlations with other learning outcomes using reproducible tools, as described before, would generate useful evidence on the effectiveness of CBL methods in enhancing learning [ ].
On-Board Training and Operational Support Tools Applying Web Technologies
Metrics could include, for example, student communication style and sentiment [ , ] and time spent on different types of materials [ ]. Indeed, we found that, while traditional versus CBL articles were more numerous, articles comparing different CBL methods were cited more often than articles comparing CBL versus traditional settings. We take this as a sign that recommendations put forward by previous authors are being taken into consideration [ 8 , 9 , 11 ]. Articles in the instructional and online activity latent classes, as well as those citing the Cook et al meta-analysis [ 12 ], had more references and citations in common with other articles, demonstrating greater awareness of research in this field and possibly indicating future research directions.
The shift to student-centered models needs to continue. However, only a few reports put students at the center of the education process, focusing usually on aspects related to teaching [ ]. Part of the success of CBL features comes from empowering students to conduct study sessions at their own pace, providing them with richer interactions with learning materials, and facilitating communication, which were not otherwise feasible. Promoting student self-directedness through social media and reward-based systems may lead to increased engagement and improved learning outcomes [ ].
Active learning through engagement in collaborative user-generated content, facilitated communication, and feedback in which instructors act as moderators may further promote this change [ ]. Engaging students in the creation of content can be a good way to help faculty cope with the increasing demand for learning material [ ]. Social media tools such as wikis have been used in the medical context for various purposes [ ], but in medical education they still are limited in their format, management, and collaborative features [ ].
Other approaches using 3D virtual worlds may offer great potential to learners through immersive exploratory worlds and a rich feedback environment that may be used to engage learners and simulate real-world scenarios of medical doctors [ ]. We have come a long way in CBL in medical education.
While the field is highly variable and some studies seemed to be unaware of advances in the field, recommendations on comparing different CBL methods seem to have been taken into consideration. Incorporating instructional design principles in the design of learning materials and developing further educational software in ways that can be shared between researchers are paths for further improvement.
A focus on measuring online activity and correlating it with other outcomes may provide insights into ways to keep promoting student-centered approaches tailored to specific learning settings. Authors' Contributions: TTG designed the study, abstracted the articles, conducted statistical analysis, wrote the manuscript, and developed the Web app.
PF created the initial data extraction sheet, designed the study, abstracted the articles, and wrote the manuscript.
- A Womans Sundance.
- Belly Fat Weight Loss For Beginners.
- Bears (Animals We Love Book 1).
- ISBN 13: 9783639856309.
- Before He Became a Monster: A Story Charles Mansons Time at Father Flannigans Boystown.
- 30 Desserts and Smoothies for a Vegetarian Pregnancy!
ITG designed the study, abstracted the articles, and wrote the manuscript. MS designed the study and conducted the statistical analysis. MAF designed the study, wrote the manuscript, and gave overall approval and direction. Conflicts of Interest: Conflicts of Interest: None declared. National Center for Biotechnology Information , U. J Med Internet Res. Published online Aug 1. Author information Article notes Copyright and License information Disclaimer. Corresponding author. Corresponding Author: Tiago Taveira-Gomes moc. This article has been cited by other articles in PMC.
Abstract Background Computer-based learning CBL has been widely used in medical education, and reports regarding its usage and effectiveness have ranged broadly. Objective In this systematic review, we aimed to characterize recent studies of the development of software platforms and interventions in medical education, search for common points among studies, and assess whether recommendations for CBL research are being taken into consideration.
Methods We conducted a systematic review of the literature published from through Results We analyzed articles. Conclusions Studies in this field vary highly, and a high number of software systems are being developed. Keywords: medical education, internet-based learning, computer-based learning, e-learning, b-learning, systematic review.
Introduction Medical education is a field that reflects the constant revision of medical knowledge, educational technology, and teaching strategies. Methods Study Eligibility We included medical education studies written in English regarding the development of educational software, interventions using educational software, or both. Study Selection Working independently and in duplicate, reviewers PF, ITG screened all article titles and abstracts, and in full text all potentially eligible abstracts, abstracts with disagreement, or abstracts with insufficient information. Data Analysis We manipulated and prepared data for statistical analysis using NumPy [ 20 ] and pandas [ 21 ] libraries for the Python language.
Results Study Eligibility, Identification, and Selection The search strategy yielded citations, of which we identified potentially eligible articles based on their abstract. Open in a separate window. Figure 1. Study Analysis The number of publications rose over the years, from 13 of the publications in — 5. Figure 2. Figure 3. Web-Based Learning Software Of the studies assessed, reported blended learning environments Figure 4.
Platform Type A total of studies used websites Media Support Of the studies, provided content in text format Interacting With Content A total of studies reported unspecified features Sharing Content Of the studies, 47 reported communication and content sharing through discussion forums Instructional Design Principles The media principle was apparent in 74 studies Latent Classes We considered 4 distinct classes for educational software, according to the model statistics in Table 1.
Table 1 Latent class analysis model fit per number of classes for educational software. Interventions Of the articles included in this study, we identified conducting interventions on the end points of interest Figure 5. Study Design and Study Sample A total of 81 of studies were conducted with medical students from preclinical years Conducted Comparisons Between Groups Of the studies, 28 studied controlled interventions between blended learning approaches and traditional lectures Knowledge End Point Knowledge outcomes were assessed in of articles Attitude End Point Of studies, assessed student attitudes Skill End Point Of studies, 31 assessed subject skills Online Activity End Point Online activity was measured in 76 of studies Intervention End-Point Latent Classes We considered 3 distinct classes to group the studies taking into consideration intervention end point variables.
Table 2 Latent class analysis model fit per number of latent classes for intervention end points. Reported Correlations Between Assessment Outcomes Of studies, 25 correlated different variables with knowledge outcomes Reference and Citation Network Analysis Reference and Citation Analysis We obtained references and citations for of the articles included in this review Related Article Analysis Of the articles, had at least one reference or citation in common with other abstracted articles Figure 6.
Figure 7. Associations With Latent Classes and the Cook et al Review Regarding educational software latent classes, articles in the multimedia class had a mean of 3.
Figure 8. Comparison With Previous Reviews The last systematic review and meta-analysis of this topic encompassed data from to and highlighted the problems of intervention variability and lack of evidence for comparative effects of CBL methods [ 12 , 13 , 28 ]. Limitations and Strengths This study has limitations. Platform Development Should Avoid Reinventing the Wheel Over 25 platforms and software projects were built specifically for medical education, despite having significant overlap in goals and features. Instructional Design Considerations Should be Reported The diversity of methods encompassed by CBL in terms of delivery medium, context, learner, and purpose, without reports of instructional design considerations, obfuscates the effect of different intervention aspects, for which instructional design—or the lack of it—is partly accountable [ 8 , 9 , 13 , ].
Interventions Should Focus on Assessing Unexplored Outcomes Studies generally report positive outcomes on knowledge, attitudes, and skills. Conclusions We have come a long way in CBL in medical education. Multimedia Appendix 1 Search queries. Click here to view. Multimedia Appendix 2 Educational software of abstracted papers. Multimedia Appendix 3 Complete reference of abstracted papers. Multimedia Appendix 4 Interactive article explorer. Footnotes Contributed by Authors' Contributions: TTG designed the study, abstracted the articles, conducted statistical analysis, wrote the manuscript, and developed the Web app.
References 1. Dewey J. The School and Society and the Child and the Curriculum. How we use social media to supplement a novel curriculum in medical education. Med Teach. Eysenbach G. Medicine 2. The impact of e-learning in medical education. Acad Med.
On-Board Training and Operational Support Tools Applying Web Technologies
Computer-supported collaborative learning in the medical workplace: students' experiences on formative peer feedback of a critical appraisal of a topic paper. Ozuah PO. Undergraduate medical education: thoughts on future challenges. BMC Med Educ. Reforming medical education to enhance the management of chronic disease. Med J Aust. Friedman CP. The research we should be doing. Adler M, Johnson K. Quantifying the literature of computer-aided instruction in medical education. Hagler P, Knowlton J. Invalid implicit assumption in CBI comparison research.
J Comput-Based Instr. Cook D. The research we still are not doing: an agenda for the study of computer-based learning. Internet-based learning in the health professions: a meta-analysis. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis.
Revelle W. Evanston, IL: Northwestern University; Ann Intern Med. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventionsxplanation and elaboration. Mayer R. Applying the science of learning to medical education. Med Educ. Association between funding and quality of published medical education research. Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue. J Gen Intern Med. The NumPy array: a structure for efficient numerical computation.
Comput Sci Eng. McKinney W. Data structures for statistical computing in Python. Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study.
Struct Equation Modeling. Linzer D, Lewis J. J Stat Softw. Wickham H. Selenium WebDriver. Selenium HQ; Cohen A Fuzzy Wuzzy.
German 1 - Beginners
Adaptive name matching in information integration. Where are we with Web-based learning in medical education? Peixoto T. The Graph-Tool Python Library. Acute medicine teaching in an undergraduate medical curriculum: a blended learning approach. Emerg Med J. Scaling-up undergraduate medical education: enabling virtual mobility by online elective courses. Croat Med J. Puljak L, Sapunar D. Web-based elective courses for medical students: an example in pain. Pain Med. Initial experiences in radiology e-learning 1. Virtual patient simulation for learning and assessment: superior results in comparison with regular course exams.
Integrating an open-source course management system Moodle into the teaching of a first-year medical physiology course: a case study. Adv Physiol Educ. Worm B, Jensen K.
- Inquiry-Based Learning - Undergraduate Research.
- Learn German Easily!
- Login using?
- ÖWR - CBT: Österreichische Wasserrettung - Computer Based Training (German Edition).
Does peer learning or higher levels of e-learning improve learning abilities? A randomized controlled trial. Med Educ Online. The development and evaluation of a distance learning system in ophthalmology.