Research: Collaborative document editing + revision order 00172124

Methodology and Planning Criteria for Participant Selection In this experimental study, there were three classes of pupils from a Belgian school in their 5th year of TSO secondary education. The first class comprised 25 students, while the second had 7 students. In addition, another class, which comprised 16 students in the same educational stage, served as a control group. One of the primary criteria used was the learner’s language proficiency or level of English. Standardized language proficiency tests measured language proficiency levels depending on how long a student studied the English language and their fluency in reading, writing, listening, and understanding. This approach resulted in a broader range of linguistic issues being addressed in the feedback process, benefiting all participants. Data Collection The data collection process for the intervention incorporated a range of methods, including surveys and moderator observations. Quantitative data were analysed using Microsoft Excel with measures such as mean, standard deviation, and the t-test as measures of possible relationships between study variables. A combination of methods was utilized for qualitative data, comprising surveys, direct observations, and narrative analysis. The survey questionnaire was a combination of open-ended (1 question) and closed-ended questions (11 questions). The 12 questions were divided into five sections, each with different responses from participants. The first part sought responses on students' overall evaluation of the impact of peer feedback on their writing skills, namely content, organization, language use, and mechanics. To quantify this evaluation, a 3-point Likert scale was used. The measurements and descriptors used were: slight improvement, moderate improvement, and considerable improvement; slightly, moderate, and significantly and; very comfortable, somewhat comfortable, and neither comfortable nor uncomfortable. Learners involved in the study were expected to comment on how they believed their writing skills had improved as a result of peer feedback by answering the three provided options – slight improvement, moderate improvement, and considerable improvement. The first five questions were generally answered with a larger response scale. The second part was still closed-ended questions and related to the specific evaluation of peer feedback, including openness to feedback, comfortability of giving feedback, comparison of peer feedback to teacher feedback, ability to encourage others to participate in peer sessions, and experiences on specific moments during peer feedback sessions. The expected responses in this category varied with the expected responses being “Moderately, Significant, Slightly, and Not at all” in one category and “Very comfortable, Somewhat comfortable, and Neither comfortable nor uncomfortable” in another. Other responses included “Better, About the same, or Worse” particularly in the question where the researcher asked learners how peer feedback compares to teacher feedback in terms of helping them improve their writing. The last two sections of the questionnaires recorded a smaller response scale as compared to the first section. The last part was an open-ended question where learners had the freedom to answer the posed question without a predetermined limit. Learners were asked how they experienced the whole peer feedback writing process and their responses were then analysed thematically. In addition, the direct observation method was used, where the teacher took notes and recorded learner interactions in the EFL classroom. The specific behaviors observed included learner’s participation level, collaboration among groups and peers, engagement with feedback, communication skills, peer interaction dynamics, and teacher interventions. To ensure the effectiveness of these observations, a well-established protocol was used, which outlined clear criteria and standards. This protocol was designed to assess the quality of discussions and verify that the peer feedback was consistent with the proposed guidance. For enhanced sessions, a dedicated website was developed to purposefully provide comprehensive content on feedback principles, thesis statement construction, content analysis, compliance with writing standards, guided feedback reflection on how to give and receive feedback, collaborative document editing techniques, effective topic sentence formulation, and other vital writing skills. Participants also learned to provide detailed, constructive, and research-aligned feedback, considering critical aspects such as content, organization, language proficiency, and adherence to writing conventions. This approach ensured systematic feedback focusing on essential writing aspects. Additionally, in-depth narrative analysis was used to increase the level of understanding of qualitative aspects of collected data, especially students’ reflections on working with their peers during reviewing, classification of the feedback they received, and outcomes for their writing process and competence. These data collection methods allowed for a comprehensive examination of peer feedback's effectiveness in improving EFL learners' writing skills. These methods helped gather insights on peer assessment experiences from educators and students, focusing on student engagement, feedback quality, and session involvement. Feedback training The research intervention began with a thorough pre-assessment section, where selected students individually wrote an essay on specified topics provided in the Appendix. This crucial step of the research study served as a baseline for evaluating language proficiency at an individual level while identifying areas for improvement, thus guiding subsequent peer groupings and feedback training sessions. Following the pre-assessment, students were arranged into groups of three based on their initial performance, marking the onset of 20-minute feedback training sessions conducted twice a week for three weeks. The small groups allowed students to actively engage, articulate their ideas, and cooperate efficiently without individual shyness getting in the way. Furthermore, each feedback session led to small tasks, including thesis development, checking and understanding writing conventions, and developing transitional body paragraphs. Collaborative document editing (CDE) was used within the groups to enable collaboration and togetherness in the learning process. To maintain the independence of the impact of peer feedback, the pre-test results were kept from pupils during the study, preventing teacher feedback from influencing their writing. The effect of teacher feedback would be separately assessed in later evaluations, ensuring the research's integrity. Feedback-guided reflections were critical in helping participants incorporate feedback into EFL writing for improved results. Other types of guided reflection included feedback integration reflection, where earlier feedbacks were used; comparison reflection, in which students compared comments from peers; and language skill emphasis reflection, which emphasized language proficiency. Upon completion of training, the intervention progressed to a post-test phase, where each student was expected to write an essay and then share it with their group members through CDE. Group members had the opportunity to apply what they had learned in feedback sessions by reviewing each other's essays. They offered suggestions on various aspects, such as subject-verb agreement, word count, spelling, and noun usage. Additionally, they provided positive or negative feedback on the essay's topic, structure, and any inconsistencies. Peer editing also enabled students to inquire about specific details, arguments, and essay language use. The individual post-test was designed to assess and evaluate each student's grasp of EFL writing skills, thus identifying individual learning needs and progress. The post-test also allowed students to reflect on how the peer feedback and collaborative writing process influenced their learning journey individually. By incorporating these individual reflections and assessments, the intervention sought to ensure a well-rounded evaluation of each student's capabilities in the context of peer feedback through CDE intervention. Ethical considerations Ethical considerations are of great importance in research. This research study's main ethical considerations included fair participant treatment, consent and privacy, and equity. To adhere to ethical considerations, all study participants were treated equally and fairly. Fair treatment refers to the lack of bias or discrimination based on gender, age, sex, or language. Participants were informed earlier about set study goals, their expected contribution, and their roles, which allowed them to give informed consent. Other ethical considerations that were given attention include confidentiality and privacy, where participants' data was collected and stored with adherence to data protection principles. The researchers adopted relevant measures to ensure prejudice was avoided entirely and also ensured that formal research approvals were obtained from relevant ethical authorities before the research began. Reliability and Validity The research methodology adopted different approaches to attain the desired validity and reliability. First, sample selection was done regarding key language proficiency aspects to ensure the research was relevant. Secondly, research methods used in data collection, namely observations, surveys, and narrative analysis, gave room for data triangulation. Thirdly, the participant's selection criteria were designed to encourage measures such as the learner’s educational ability and scores, thus enhancing research consistency. Fourthly, the project's experimental nature and strategically adopted control measures helped minimize any possible impact of confounding variables. This careful structuring of participants played a key role in enhancing research reliability and validity. Fifthly, research instruments used at both stages of research (pre-test and post-test) underwent extensive refinement. This refinement ensured that the research tools were correctly calibrated to measure the intended outcomes effectively, thus increasing reliability and validity. Finally, the standardized writing assessment pre-test and post-test reinforced the methodological rigor. Results This research findings follow a mixed-method approach to assess the effectiveness and impact of peer feedback on 5th-year TSO EFL students in Belgium. The results section is subdivided into two sections: a quantitative section (mean, STDEV, Sample-Paired T-test) and a qualitative section (surveys, direct observations, and narrative analysis). Because of observed differentiation during the procedure of the intervention, the experimental groups’ data were not combined. The disparities were attributed to varied approaches to intervention and the extent of compliance by groups with the given protocol. Quantitative analysis Each criterion in the pre-test essays was analyzed based on a scale of 1 to 4 with each criterion scored on a scale of 1 to 4. The criteria comprised content, structure, language use, and mechanics, which helped to generate a full-range assessment of the participant's writing proficiency. In Table 1, the mean scores and paired t-tests for pre-tests and post-tests for Experimental Groups 1 (Class 5F), Experimental Group 2 (Class 5H), and the Control Group (Class 5G) are shown. Table 1: Pre-Test Mean Scores and STDEV Criterion Group Pre-test Mean Pre-test STDEV Total score 4 Content Experimental 2.45* 0.69 Control 1.92 0.67 Organization Experimental 1.69 0.54 Control 1.58 0.51 Language Use Experimental 1.66 0.55 Control 1.67 0.75 Mechanics Experimental 1.62 0.56 Control 1.67 0.65 ∗ p < 0.05 ∗∗ p < 0.01 ∗∗∗ p < 0.001 The use of inferential statistics, particularly the paired t-tests, will aid in the exploration of any possible differences in the pre-test scores within the two study groups. The analysis was conducted to establish which of their performances was statistically different for all participants before the start of the intervention. Pre-test analysis indicates that there was no significant difference between the experimental group and control group except for content in the pretest. For the Organization, Language Use, and Mechanics criteria, the mean scores between the experimental group are highly comparable with the control group's mean pre-test and indicate no statistical significance. Based on these numbers, we can infer that both groups started on somewhat equal footing regarding their pre-test performances across all the assessed criteria except for content. This equivalence in the baseline measurements is essential for the subsequent evaluation of any interventions or treatments applied to the experimental group. Table 2: Mean Scores and STDEV Criterion Group Pre-test Mean Post-test Mean Pre-test STDEV Post-test STDEV Total score 4 4 Content Group 1 2.50 2.91** 0.67 0.29 Group 2 2.29 2.86** 0.76 0.38 Control 1.92 2.00* 0.67 0.85 Organization Group 1 1.68 3.05*** 0.57 0.58 Group 2 1.71 3.00** 0.49 0.58 Control 1.58 1.75* 0.51 0.62 Language Use Group 1 1.73 2.59** 0.55 0.50 Group 2 1.43 2.29** 0.49 0.49 Control 1.67 1.75* 0.75 0.75 Mechanics Group 1 1.68 2.41*** 0.57 0.59 Group 2 1.43 2.43** 0.53 0.53 Control 1.67 1.83* 0.65 0.83 ∗ p < 0.05 ∗∗ p < 0.01 ∗∗∗ p < 0.001 Post-test, there were notable differences between the experimental and control groups, as seen through mean changes across the four measures in Table 2. However, the degree of improvements post-test was significantly higher for the experimental groups across all four measured criteria. With the content score, strong evidence (p < 0.01) suggests improvement in post-tests among learners in experimental groups. Additionally, organization and mechanics emerged as the two categories with the most noticeable gains (p < 0.001), with learners in experimental groups showing significant improvement in writing structure and the ability to present their ideas more coherently, spelling, and punctuation, which are crucial components of written texts. The advancements in language use were another area of improvement in the experimental group (p < 0.01), with this group showing an enhanced grasp of grammar, vocabulary, and expression and an indication of improvement in the clarity and effectiveness of their communication. For the paired t-test, all values show a statistically significant improvement in the four categories for the experimental groups. All these improvements played a role in the overall improvement of their respective writing performances. In contrast, the mean scores post-test for the control group indicate a slight improvement across all four categories. Content creation and language use recorded the smallest mean growth post-test (p < 0.05), indicating a low to moderate improvement in the student's ability to articulate and organize ideas. Similarly, organization and mechanics recorded a small growth post-test (p < 0.05). The technical aspects of writing, such as mechanics, also saw minimal improvement (p < 0.05). Overall, groups 1 and 2 recorded higher mean post-tests. With the mean difference in the two treatment groups being higher, it is inferred that after the intervention in experimental groups, the students in the treatment group (EG 1 and EG2) achieved higher writing skills than students in the control group. Finally, the standard deviations for pre-test and post-test scores across the three categories are relatively low. This suggests consistency within each group, indicating that almost all participants' responses were closer to the group mean. There were lower standard deviations, as seen with experimental group content, indicating some degree of homogeneity in content-related assignments. For the control group, the STDEV for the pre-test and post-test scores were slightly higher than the experimental groups, indicating slightly higher variability in performance within the control group. Overall, the STDEV indicates that the experimental groups exhibit consistent levels of variability in performance, with an indication of improvement after the intervention. The control group records slightly higher performance variability than the experimental groups but indicates higher stability in performance levels over time. Qualitative analysis Written surveys were administered to the experimental groups (1 & 2). Of 25 participants in experimental group 1, 23 completed the survey forms, with the two unsubmitted forms attributed to absenteeism. In experimental group 2, there was a 100% response rate, with all 7 participants fully responding to the required questions. Table 3: Survey Impact of Peer Feedback on Writing Skills in Experimental Groups 1 & 2 Measurement Overall writing skills Content Organization Language use Mechanics Slight Improvement Group 1 (N=25) 5 (22%) 9 (39%) 3 (13%) 10 (44%) 7 (30%) Group 2 (N=7) 1 (14%) 2 (29%) 1 (14%) 4 (57%) 2 (29%) Moderate Improvement Group 1 (N=25) 12 (52%) 8 (35%) 8 (35%) 10 (44%) 12 (52%) Group 2 (N=7) 4 (57%) 5 (71%) 3 (43%) 3 (43%) 3 (43%) Considerable Improvement Group 1(N=25) 6 (27%) 6 (26%) 12 (52%) 3 (13%) 4 (17%) Group 2 (N=7) 2 (29%) 0 3 (43%) 0 2 (29%) No Improvement Group 1(N=25) 0 0 0 0 0 Group 2 (N=7) 0 0 0 0 0 From Table 3, students in experimental group 1 hold varying attitudes towards the peer feedback process and its impact on their writing skills. With language use, 43.47% agree that peer slightly improved their language use, and another 43.47% also reported moderate improvement after peer feedback, with only 13.04% reporting considerable improvements. The area most learners acknowledge to have changed due to peer feedback was organization (52.17%). For experimental group 2, the figures were slightly similar to experimental group 1 except for a higher, largely moderate improvement in content (71%). Overall, writing skills were perceived by the majority to have moderate improvements (52.17%), and content cited by the majority (39.13%) as having significant improvement. Learners admit moderate improvements across the four measurement criteria after peer feedback sessions. No participant in both groups and across the four categories reported a single case of “No Improvement”. Table 4: Perceived Impact of Peer Feedback in Experimental Groups 1 and 2 Question Holistic Score Experimental Group 1 (N=25) Experimental Group 2 (N=7) Peer Feedback Influence on Essay Quality Significantly 6 (26%) 4 (57%) Moderately 13(57%) 3 (43%) Not at all 4 (17%) 0 Openness to Implementing Peer Feedback Very comfortable 8(35%) 4 (57%) Somewhat uncomfortable 10(44%) 1 (14%) Neither 5(22%) 2 (28%) Comfort Level in Giving Feedback Post-Sessions Very comfortable 6(26%) 3 (43%) Somewhat comfortable 13(57%) 1 (14%) Neither 4 (17%) 3 (43%) Satisfaction with Peer Feedback in Collaborative Editing Very satisfied 0% 1 (14%) Somewhat satisfied 16(70%) 4 (57%) Neither 7 (30%) 2 (29%) Peer vs. Teacher Feedback Effectiveness Better 14 (61%) 0 About the same 7 (30%) 7 (100%) Worse 2 (8%) 0 Likelihood to Recommend Peer Feedback Sessions Probably would 11(44%) 2 (30%) Might or Might not 8(35%) 5 (70%) Probably not 4 (17%) 0 As it appears in Table 4, responses for both Experimental Groups 1 and 2 slightly varied. The effect of peer feedback was moderate for Group 1 (57%) and considerable for Group 2 (57%). Openness to feedback was reported as high in Group 2 (57%) and moderate in Group 1 (44%). Moreover, learners in Group 1 were somewhat comfortable with giving feedback, whereas the majority in Group 2 (43%) were very comfortable. The satisfaction with peer feedback varied as well, with the majority in Group 1 (70%) reporting somewhat satisfied and 57% in Group 2 reporting moderate. Regarding comparing peer feedback to teacher feedback, 64% of Group 1 participants found peer feedback to be better, whereas all participants in Group 2 (100%) reported it as "about the same." Concerning the likelihood of recommending peer feedback to others, 70% of Group 2 participants were undecided ("might or might not"), while the majority of Group 1 (44%) indicated they would probably recommend it. This nuanced feedback underscores the varied perceptions and experiences with peer feedback across the two experimental groups. After completing all peer feedback sessions, in Experimental Group 1, which consisted of 23 students, 26% felt very comfortable giving feedback to their group members, translating to 6 students. In comparison, 57% (13 students) felt somewhat comfortable, and the remaining 17% (4 students) were neither comfortable nor uncomfortable. In Experimental Group 2, with 7 students, 43% (3 students) rep...