Phone-based Audience Response System as an Adjunct in Orthodontic Teaching of Undergraduate Dental Students

Design:

The trial was designed as a cross-over clustered randomized control trial (each group was a cluster), so that each group acted as their own control for their knowledge's retention and perception.

Setting and consent: College of Dentistry, Prince Sattam Bin Abdulaziz University/Alkharj/ Saudi Arabia. Written consents were obtained from students before commencing the trail.

Participants : The cohort of the trial included the whole fourth-year undergraduate class (34 undergraduate dental students) in order to eliminate selection bias. Participants in this study had no prior orthodontic teaching, hence, no bias associated with increased knowledge in any orthodontic field was anticipated.

Randomization: Students were allocated to one of two even groups (G1 or G2) using computer generated randomization. each group (G1 and G2) consisted of 17 male students with a mean age of (23.27 years ± 0.86).

Intervention :

Lectures : Simultaneously, G1 and G2 attended two lectures (L1\&L2): L1 was titled "Management of Class III malocclusion", L2 was titled "Management of open bite and cross-bite". Both lectures were presented at the main campus of Prince Sattam Bin Abdulaziz College of Dentistry and they were conducted in an identical manner in all aspects including:

• Presentation in PowerPoint (Microsoft Corp, Redmond, WA)
• Both lectures were delivered by the same registered specialist orthodontist (FA),
• Learning outcomes of the delivered lectures (T1 and T2) were based on teaching outcomes specified by the National Commission for Academic Accreditation and Assessment in Saudi Arabia, and
• Both lectures delivered over the same period of time. Prior to lectures, students were instructed to register with the audience response system (ARS) and download its application (Poll Everywhere, San Francisco, California, USA, https://www.polleverywhere.com). Additional smart phones were available to students who do not have smart phones at the time of the lecture. Students were unaware of their allocation until the beginning of the first lecture.

In the lecture hall, G1 was separated from G2, and both groups completed an assessment of written multiple-choice questions (MCQs) scored out of 20, before the lecture. During L1, phone based (ARS) questions (PB-ARS) secretly displayed on the smart phones of the students in G1, and they were allowed to read the question and answer it, while those in parallel arm of the trial (G2) were blinded to those questions and answers. At the end of L1, both groups completed the same written MCQs assessment . A similar protocol was undertaken during L2 a week later, except that the groups were crossed-over, hence, the group which has had ARS integrated lecture in the previous week, has received in a non-ARS integrated lecture, and vice versa.

Formative exam:

MCQs formative tests consisted of 20 questions related to the taught topics during L1 and L2 was undertaken. In order to reduce the carry over effect, the PB-ARS questions during the lectures were not similar to the MCQs sheet. The used MCQs were piloted between authors to ensure its content validity and reliability. Contents validity was tested using test matrix and expert judgment. The test reliability was estimated using inter-rater reliability, a correlation of more than 0.7 was considered acceptable.

Summative exam:

Both groups attended their final written summative exams. L1 was undertaken 11 weeks prior to final exam while L2 was undertaken 10 weeks before the exam. The final exam was in MCQs format and included questions from all subjects taught in the second semester of the year including five questions relevant to the subjects taught in L1 and L2. The exam questions were identical for all students and were delivered at the same time under controlled conditions. Students who failed to attend the final summative exam were excluded to reduce the effect of time factor as confounding factors. The final exam scores of all student to questions related to TL1 and TL2 were anonymously collated using an excel sheet by independent tutor to reduce reporting bias. The maximum achievable score for the five questions relevant to the subjects taught in L1 and L2 was 5.

Students' perception:

At the end of L1, both groups were asked to fill a questionnaire regarding their experience with the lecture and intervention group participants were asked specific questions about their experience with PA-ARS . The used questionnaire was modified version from a previously used questionnaire to assess students experience with ARS in a recent trial . Students' responses were saved in an excel sheet after each lecture (L1, L2).

Analysis of the results:

Students' scores were saved in an excel sheet after each test. A specialized statistical analysis software was used (IBM SPSS statistics 22, version 22) The statistician was blinded to the allocation group. Scores were analyzed and compared to assess knowledge retention using cross-over analysis with Mann-Whitney U test for pre and post lecture quiz and t-test for summative test.