Master Quest: An online writing game to improve EFL proficiency at school
Bárbara Echard (b.echard@rseduca.cl)
Cristian Bravo Lillo (cristian.bravo@rseduca.cl)
RSEduca, Santiago, Chile, September 2016
Abstract and summary
This paper describes how an EFL online writing contest carried out in 2015 to 3,333 students can foster proficiency in the English language of students from 5th to 6th grade in Chilean schools.
We implemented an EFL online writing contest for 3,333 students of 5th to 8th grades from 19 public and subsidized schools in Chile during the 2015 school year. We measured English proficiency on the skills of Listening, Reading and Writing (spelling) with a test administered before and after the implementation. The results show a significant improvement in test scores of those students from 5th and 6th grades who participated in the contest at least once versus those who did not. Students’ topic preferences for writing in English are also described and further discussed.
This research was presented at TESOL Chile 2016, Temuco, Chile.
Introduction
There exists the common idea that proficient writing in a second language is the product of having mastered the target language after a long learning process mostly consisting of strengthening receptive skills, i.e. listening and reading in that language. However, producing language is by itself a learning activity and promotes the overall interlanguage development (Swain, 1985). Based on the notion that writing may help students to learn English as a second language, we designed an online English writing contest (“Master Quest”, or MQ) in which Spanish-speaking students from 5th to 8th grades are both exposed to different topics and encouraged - not forced - to write about these topics in English. This raises the question of whether writing in a foreign language is indeed helpful to young students who are learning the language.
In this paper, we describe the design of the contest, and the experience we have had at carrying out the contest during 2015 with 3,333 students in 19 different schools, 14 of them located in the Antofagasta and Coquimbo region of Chile, and 5 schools located in the Metropolitan Area of Santiago, also in Chile.
In the rest of this section, we explore the use of writing as a tool to enhance language learning and review previous research related to it. In the next section, we describe our experimental design, beginning with our participants, our research questions and our hypotheses. Finally, in the results section, we describe the demographics of our participants, the results of our hypotheses testing, some discussion about our results, as well as some conclusions.
Writing as an aid to learning
The relationship between writing in English as a second language and learning at school has been studied from different perspectives, most commonly focusing on the role of corrective feedback and accuracy on the quality of writing (Chukharev-Hudilainen et al, 2016; Hartshorn et al, 2010; Sheen et al, 2009; Bitchener et al, 2005) or on the impact of ESL writing instruction on writing skills (Clúa Artero, 2013; Reynolds, 2004). However, there seems to be little experience regarding the impact of writing in English as a second or foreign language on learners’ interlanguage overall development.
Writing in English as a foreign language can help students improve their knowledge of the language. Swain’s output hypothesis (Swain, 1985) claims that any type of language production helps learners notice gaps in their knowledge, test the structures they already know, and reflect about how they are learning it. Therefore, providing opportunities to write in the target language with a communicative purpose may trigger and even consolidate acquisition processes in language learners. This hypothesis has been tested mostly in adults (Izumi, 2002), but remains relatively untested in Young Learners.
The advantages of output in learning a second language in adults have been shown in studies such as Izumi’s (Izumi, 2002), where activities that included output and enhanced input were more effective in the improvement of acquisition of relative forms than enhanced input alone. In this study, output took the form of writing notes when being exposed to language input, to complete a text reconstruction task. Participants who used this note-taking method performed significantly better in learning the specific grammar structure than those who were instead only exposed to input when completing the task. This shows that writing activities involve mental processes that facilitate the intake of new forms, which in turn lead to language acquisition.
A common issue in the schools included in this study is the scarce classroom time exclusively dedicated to teaching or even promoting the skill of writing in English as a second language. In order to contribute with opportunities to write and encourage the writing activity itself without demanding any additional action on the teacher’s part, RSEduca Munk developed Master Quest (MQ), an online writing contest, which design was based on the methodology of Guided Writing (Hyland, 2003), in which students write from a given model. This way, students are able to write autonomously, without necessarily taking up extra time from the teacher.
The Munk program
RSEduca is a small software company that offers educational services to English teachers and students from 5th to 8th grades. The main goal of these services is twofold: to complement teachers’ classes and to allow students to practice English as a Foreign Language skills. Most of these services are provided through an online gaming platform: the Munk platform (http://www.munk.cl). Students connect weekly to this website by entering their username and password, and they play short educational curriculum-based games to develop communicative skills in English, during 45 or 90 minute-long sessions. These sessions are managed by mediators, teachers who are part of our team and who are experts in facilitating the delivery of digital contents. The Munk program covers almost the whole school year, usually from May to November. Students are measured twice a year in their level of proficiency of the English language, first prior to any other activity in the program (a pre-test), and then as the very last activity before closing the school year (a post-test). These tests are meant to measure language proficiency in different dimensions: listening comprehension, reading comprehension and spelling of vocabulary items in 5th and 6th grade; and listening comprehension of short dialogues, reading comprehension of short texts and use of English in aspects such as grammar and vocabulary in sentences in 7th and 8th grades.
Since 2015 we have been running the Master Quest (MQ) English writing contest for over 3,000 students from 5th to 8th grade, as one of the learning activities that are provided through the Munk platform.
For the study that we describe ahead, we were interested in understanding how participation in the MQ contest may improve students’ overall proficiency in the English language. Thus, we used their scores in both the pre- and post-tests to understand how voluntary participation in the Master Quest contest might have influenced students’ performance in the tests.
The Master Quest contest
Master Quest (MQ) is an English writing contest held every two or three weeks as one of the activities within the Munk program. The purpose of the contest is to encourage students to write regularly about specific topics. Participation in the contest is voluntary but encouraged by our mediators.
Between 12 and 16 topics are offered as different MQ contests. Each bi-weekly contest tackles one topic in turn. Topics for the contest are carefully selected based primarily on the curricular goals suggested by the Ministry of Education curriculum (MINEDUC, 2012), but also on our field experience with students. Some topics are related to their everyday lives, such as describing themselves, their family and their school community. Other topics deal with more abstract issues, such as travel, vacations, professions, or even popular events such as Copa América soccer cup or National Holidays.
After logging into the website, (http://www.munk.cl), the student sees a welcoming page (see Appendix A) where he or she finds a link that takes it to a special environment on the web where instructions and tools for the contest are given (see Appendix B). In this space, students can read the instructions carefully in their native language, Spanish. They may consult a glossary of related vocabulary they can use, and they may also access some popular online dictionaries. Most importantly, in the Master Quest space students find model texts so they are able to understand what is requested of them. They can also write and submit their answer if they decide to participate.
After a contest is closed, two evaluators assess and classify all the texts based on a rubric that contains four categories and six sub-categories, rating aspects such as the length of the text, accuracy, and originality (see Appendix C). According to the category in which the text is placed, the student receives a predetermined feedback on the aspects that should improve overall (see Appendix D). After all of the submissions have been assessed, students receive their feedback through the same web page (see Appendix F).
One winner is selected in each grade for each county for every contest. The winner is recognized in the community of schools and published in another space on the website called "Hall of Heroes" (Salón de los héroes, see Appendix E).
Students have a fixed amount of time during which they can participate in the contest and submit their texts. The allotted time for each contest is about 15 days, depending on the school schedule. The basic rules of the game are: producing an original text in English, trying to write as much as possible (about 50 words for 5th and 6th grades, and about 100 for 7th and 8th grades) and maintain a formal and respectful language. Once the student writes and submits a text, he or she can no longer edit it.
As stated above, participation in the Master Quest contest is completely optional. Students can participate only once per contest, but they may choose to participate in any (or all) of the contests. School teachers do not have access to students’ texts; therefore, no school marking is involved. The rationale for this is to provide a private, safe and neutral space where students feel intrinsically motivated to participate.
Experimental design
Participants
Our target population is children from 5th to 8th grade studying at Chilean public and subsidized schools, who are around 10 to 15 years old. In Chile, teaching English as a foreign language is mandatory only from 5th grade on; thus, when 5th-graders reach our platform, in most cases they are exposed to the English language for the very first time.
The Munk program is a complement to the regular 2 to 3 hours per week of English instruction in their curriculum, and as such it reaches around 20 schools across the Antofagasta region of the country and the Metropolitan area. For this particular study, participants are a group of students that belonged to the Munk program during 2015, who took either the pre-test, the post-test, or both. Although not common, and mainly due to school logistical constraints, some students do not take any tests. As we defined test scores as one of our dependent variables, we excluded from our dataset those students who took no tests.
Research questions
In order to understand the effect of writing regularly in a second language on the overall target learning and motivation of students towards the target language, we developed the following set of questions and hypotheses:
- Does the writing English contest (MQ) help students improve their skill of writing in English? Does the contest help students improve their overall knowledge of English?
We wanted to know whether participating in a writing contest in a second language helps to improve students’ overall knowledge of the English language by triggering one of the functions of output described by Swain (1993). Students would be forced to reflect on the structures needed to convey a message which may, in turn, cause students to either search their knowledge and find the correct way to do it, or to create an original way to communicate their message and corroborate whether it is understood, i.e. hypothesis testing.
2. What factors influence students participation in the Master Quest writing contest? Since participation in the MQ contest is voluntary, we wanted to know which students become interested in participating, and understand if there is a pattern that can be replicated in other scenarios.
3. Is there any relationship between the frequency of participation and the improvement in their overall learning of the target language, if any?
We wanted to test whether the frequency of participation may cause a greater improvement, since the regular practice of writing in a second language may trigger recursive cognitive processes that may ultimately lead to better learning of the target language.
4. What kind of students get the more learning benefits from the writing contest?
We also wanted to know if the nature of this activity, i.e., a gamified writing activity, may result more beneficial for certain students than others, regarding (for example) their grade, geographic area, gender, etc.
5. Which topics are the most interesting to students?
Finally we intended to find those topics that draw more interest from students, in order to better understand how to encourage students to write in a foreign language.
Metrics
In order to formulate our research hypotheses, we used the following metrics:
- Pre-test score: Score that students get in the English test which is administered before any other activity has taken place.
- Post-test score: Score that students get in the English test which is administered as the very last activity in the school year.
- Test improvement: It is the score a student had in the post-test, minus the score the student had in the pre-test. This quantity is measured in percent points. If positive, it means the student improved between the pre-test and the post-test.
It is important to note that 5th and 6th grade belong to the elementary school cycle according to Chilean curriculum, whereas 7th and 8th grades mark the beginning of the secondary cycle. Because of this, teaching approaches and methods are different and a linear comparison may result in misleading and inadequate. Moreover, due to the scarce target language input that students receive in their school and home contexts, and therefore the incipient proficiency level that students in 5th and 6th grades may show in English as a foreign language, we measured 5th and 6th grades in a different way from 7th and 8th, grouping the former two grades in one category and the two latter in another.
Research hypotheses
Not all of our research questions were transformed into hypotheses as some of our questions were exploratory. We reformulated two of the previous questions into the following research hypotheses:
H1.a: Students who decided to participate at least once in MQ have the same average test improvement than students who decided not to participate in MQ at all.
H1.b: Students who decided to participate at least once in MQ have the same average pre-test score than students who decided not to participate in MQ at all.
Since the test that is taken in 5th and 6th grades is different from the one used in 7th and 8th grades, we tested H1 hypotheses separately for these two groups. Hypothesis H1.b covers the possibility that there is self-selection bias; i.e., that students who decided to participate in the contest are precisely those who have higher language proficiency.
H2: The number of participations in MQ does not influence test improvement.
Similarly to above, this hypothesis should be tested separately for the same two groups. We do not expect necessarily that there is a linear relation between the number of participations and test improvement; because of this, we might need to explore the data before proposing a more specific hypothesis.
Results
Demographics
The 2015 MQ contest was carried out between April and November in 19 public and subsidized schools located in the Antofagasta, Coquimbo and Metropolitan regions in Chile. We collected 3,069 writing samples from 1,403 students (that is about 2.2 writing samples per student in average), out of 3,333 students (N=3,333) who took either the pre-test, the post-test, or both. We complemented this data set with the gender and grade of each student, along with data about the type of sessions they had.
48.9% of students were male and 45.7% of students were female. We had no gender information for 5.4% of students, as we had no records for their gender and their names did not convey a gender either. 35.3% of students attended schools in the Metropolitan region (“Center”), and the rest attended schools located in the Antofagasta and Coquimbo region (“North”). Table 1 shows a summary of students’ gender and geographical zone. Students for whom we knew their age were 13.02 years old on average; however, in a large proportion of cases (46%) we had no age information available. Thus, we report no age information in this paper.
Male | Female | No Information | Total | |
Center | 515 (15.5%) | 489 (14.7%) | 174 (5.2%) | 1,178 (35.3%) |
North | 1,115 (33.5%) | 1,035 (31.1%) | 5 (0.2%) | 2,155 (64.7%) |
Total | 1,630 (48.9%) | 1,524 (45.7%) | 179 (5.4%) | 3,333 (100%) |
Table 1: Students’ gender and school’s geographical zone.
Participation in the contest
There were 14 contests for each grade, except for 8th grade in which there were 13. Table 2 shows the number of students who participated in at least one MQ contest, by grade. Overall, 42.3% of students participated in at least one contest; the proportion of students who participated decreased steadily between 5th grade (12.3% of total) and 8th grade (7.5% of total). Table 3 shows student participation by gender.
Did not participate | Participated in at least one contest | Total | |
5th | 575 (17.3%) | 411 (12.3%) | 986 (29.6%) |
6th | 587 (17.6%) | 346 (10.4%) | 933 (28%) |
7th | 382 (11.5%) | 395 (11.9%) | 777 (23.3%) |
8th | 386 (11.6%) | 251 (7.5%) | 637 (19.1%) |
Total | 1,930 (57.9%) | 1,403 (42.1%) | 3,333 (100%) |
Table 2: Number of students who participated in at least one MQ contest, per grade.
Male | Female | No Information | Total | |
Did not participate | 890 (26.7%) | 921 (27.6%) | 119 (3.6%) | 1930 (57.9%) |
Participated in at least one contest | 740 (22.2%) | 603 (18.1%) | 60 (1.8%) | 1403 (42.1%) |
Total | 1630 (48.9%) | 1524 (45.7%) | 179 (5.4%) | 3333 (100%) |
Table 3: Student participation by gender.
Figure 1 shows the number of students who participated zero, one, or more times, per grade. All four curves show a sharp decrease in the rate of participation, which is expected from a voluntary activity. 85 students (6% of those who participated at least once) engaged in half of the contests or more (that is, 7 or more contests).
Figure 1: Number of participations vs. number of students per grade.
Figures 2 and 3 show the number of writing samples we collected for each contest. Contests are in decreasing order of participation, from top to bottom. Since each student is allowed to participate only once per contest, the number of collected writing samples within each bar is the same as the number of students who participated. The length of each bar thus gives an idea of how interesting the corresponding contest was to students. Figure 2 shows the topics which elicited more participation from students in 5th and 6th grades, while Figure 3 shows the same for students in 7th and 8th grades. Not all contests are presented here, as some of them elicited just a few answers.
Figures 2 and 3 also show how writing samples were rated according to the rubric shown in Appendix C. Writing samples rated with a 1 or a 2 (that is, those texts considered deficient) were excluded from these graphs, as they were either written entirely in Spanish, their native language, or considered as garbage text (e.g., “skjh skjd hfkjsf...”).
Figure 2: Number of collected writing samples per MQ contest, for students in 5th and 6th grades, in zones Center (left) and North (right).
Figure 3: Number of collected writing samples per MQ contest, for students in 7th and 8th grades, in zones Center (left) and North (right).
In general, contests in the North zone (rightmost graphs in Figures 2 and 3) elicited between 1.3 and 3 times more answers than in the Center zone.
Participation in 7th and 8th grades in the Center zone (leftmost graph in Figure 3) shows that the topic “Who are you?” (which chronologically was the first one) attracted more than 120 answers; after that, participation in the contest quickly decayed (as the rest of the topics elicited between 30 and 40 answers, less than a third of the initial number of writing samples). Similarly, 182 students from 7th and 8th grades participated from the “Sports” contest in the North zone (top bar in the rightmost graph in Figure 3); however, this topic was not the first one in the timeline of the MQ contest.
Participation in 5th and 6th grades shows that the topic “A new hero” attracted the most participations in these grades, in both the center zone (leftmost graph in Figure 2) and the north zone (rightmost graph in Figure 2). The “A new hero” contest asked participants to create and describe an imaginary epic character. In all other contests the winner would be publicly displayed in the Hall of Heroes (see Appendix E), but in this one, the winners would have their hero illustrated by the design team of RSEduca and offered as one of the publicly available avatars in the Munk platform. This reward was announced before the opening of the contest, thus, we expected a large participation in all levels. However, only 5th and 6th graders had a major interest in epic characters, whereas older students show a preference for other topics.
Hypotheses testing
To test the H1.a hypotheses, we used a Welch (two-samples) t-test to compare the mean of test improvements of students who both decided to participate and decided not to. Table 4 shows the results of two such tests, one for the 5th/6th grades, and another for the 7th/8th grades. While in both cases there was an increase of the test-scores improvement among students who did not participate in any MQ contest and students who participated at least once, this difference was slightly significant only for the 5th/6th grades (t=-1.9166, p=0.0556).
Students who did not participate at all | Students who participated in at least one MQ contest | Welch t-test | |
5th/6th grades | M=13.92, SD=14.54 | M=16.03, SD=18.78 | t=-1.9166, p=0.0556 |
7th/8th grades | M=6.59, SD=15.48 | M=8.48, SD=17.37 | t=-1.606, p=0.1087 |
Table 4: Results of t-tests for test improvement among students who participated in at least one MQ contest and students who did not participate at all.
Students who did not participate at all | Students who participated in at least one MQ contest | Welch t-test | |
5th/6th grades | M=52.90, SD=17.88 | M=58.89, SD=18.56 | t=-6.9118, p<7*10-12 |
7th/8th grades | M=36.44, SD=17.42 | M=37.20, SD=19.41 | t=-0.6982, p=0.4852 |
Table 5: Results of t-tests for pre-test scores between students who participated in at least one MQ contest and students who did not participate at all.
In order to test H1.b hypotheses, we used another Welch t-test to compare the mean of pre-test scores. Table 5 shows the results of these tests. Similarly, only the 5th/6th grades showed a significant difference between those students who later would decide to participate and those who did not participate in any contest.
Results from Tables 4 and 5 suggest that in 5th and 6th grades, the MQ contest makes a difference in terms of students’ improvement between pre- and post-tests. However, this effect might be partially explained by the fact that those students with better pre-test scores are deciding to participate. This, in turn, suggests that one of the factors that lead students to participate is their confidence in their ability to write in English (a factor that is known in the psychology literature as self-efficacy; see for example Committee on Communication (2002)). A similar alternative is also possible: that the lack of self-efficacy inhibits students from participating.
In order to further understand what factors influence participation in the MQ contest, we applied a logistic regression with participation as the binary dependent variable. We used gender and zone (categorical variables), grade (an ordinal variable), and the score in the pretest (a continuous variable) as independent variables. As before, we used one regression for the 5th/6th group (Table 6), and another for the 7th/8th group (Table 7).
Variable | Estimate | Std. Error | Z value | P value |
Gender (Female, Male) | -0.242244 | 0.096691 | 2.505 | 0.01223 |
Grade (5,6) | -0.286775 | 0.097492 | -2.942 | 0.00327 |
Pretest score | 0.020544 | 0.002783 | 7.381 | 1.57 * 10-13 |
Zone (Centro, Norte) | 0.290691 | 0.110221 | 2.637 | 0.00836 |
Table 6: Results of logistic regression applied to the participation decision for the 5th/6th group.
Variable | Estimate | Std. Error | Z value | P value |
Gender (Female, Male) | -0.235061 | 0.123000 | -1.911 | 0.05600 |
Grade (7,8) | -0.390092 | 0.124975 | -3.121 | 0.00180 |
Pretest score | 0.001316 | 0.003392 | 0.388 | 0.69793 |
Zone (Centro, Norte) | -0.093150 | 0.129994 | -0.717 | 0.47364 |
Table 7: Results of logistic regression applied to the participation decision for the 7th/8th group.
The results above show that for both groups, gender is a predictor of the decision to participate in the contest: in both cases, girls are less prone to participate (by a probability of about 44%). Similarly, the grade was a predictor for the participation decision: in the first group, 5th graders were about 42.9% more likely to participate than 6th graders; while in the second group 7th graders were about 40.4% more likely to participate than 8th graders. Finally, in the case of the first group both the pretest score (p<2*10-13) and the zone (p<0.009) were significant predictors of the decision to participate. For each additional point in the pre-test score, a student was about 50.5% more likely to participate; while students in the north zone were about 57.2% more likely to participate than their mates in the center zone.
In order to understand if there is a relation between the number of participations and the degree to which a student increases his or her test difference, we first graphed the number of participations vs. the test improvement.
Figure 4: Number of participations in MQ contests vs. test improvement. We show in blue the least squares fitting lines, and in red we show the LOWESS smoother, which uses locally-weighted polynomial regression.
In the graph in Figure 4, the blue line shows a very slight positive slope, while the red, non-linear regression line shows a small “hump” between 1 and 3 participations. This suggests that if there is any influence of the number of participations in test improvement, it reaches up to 2 (maybe 3) participations. In order to test the previous hypothesis, we applied a linear regression with test improvement as the dependent variable, and the truncated number of participations as a predictor. This last variable has the values 0, 1, 2, and 2+ (meaning “more than twice”). We found a significant relation between the variables (e=0.6587, p=0.0396). However, this finding should be taken with caution since, as it can be seen from Figure 4, data is sparse and heteroscedastic.
Finally, we wanted to explore who benefitted the most among our students who participated in at least one contest. We applied a linear regression with gender, grade, zone and the pre-test score as predictors, and the test improvement as the dependent variable, restricting our data to only those students who participated in at least one contest. Tables 6 and 7 show the results of linear regressions for the 5th/6th and 7th/8th groups.
As it can be seen in Tables 8 and 9, in both groups the pre-test score was a significant negative predictor (in both cases p<2*10-16) of the test improvement. Similarly, in both cases, students in the north zone have significantly lower test improvement (p=0.0124 for the 5th/6th group, and p=0.00326 for the 7th/8th grades) than their counterparts in the center zone. In other words, controlling for all other factors, students with lower pre-test scores get the more benefit from participating in MQ, as well as students in the center zone. Interestingly, 6th graders have significantly lower test improvements than 5th graders (p=2.63*10-5); however, this does not happen in the 7th/8th group. This suggests that more research regarding these learning trajectories is needed.
Variable | Estimate | Std. Error | Z value | P value |
Gender (Female, Male) | -0.40869 | 1.51810 | -0.269 | 0.7879 |
Grade (5,6) | -6.62629 | 1.55965 | -4.249 | 2.63*10-5 |
Pretest score | -0.52940 | 0.04461 | -11.866 | <2*10-16 |
Zone (Centro, Norte) | -4.18168 | 1.66452 | -2.512 | 0.0124 |
Table 8: Results of linear regression applied to the test difference as dependent variable for the 5th/6th group, for those students who participated in at least one contest.
Variable | Estimate | Std. Error | Z value | P value |
Gender (Female, Male) | 0.49499 | 1.54814 | 0.320 | 0.74935 |
Grade (7,8) | 0.32855 | 1.58269 | 0.208 | 0.83566 |
Pretest score | -0.48570 | 0.04095 | -11.861 | <2*10-16 |
Zone (Centro, Norte) | -4.69853 | 1.58672 | -2.961 | 0.00326 |
Table 9: Results of linear regression applied to the test difference as dependent variable for the 7th/8th group, for those students who participated in at least one contest.
Discussion and Conclusions
Improvement
We found that students in the 5th and 6th grades who decided to participate had significantly better performance in the tests than students who did not participate at all. These results resonate with the roles of output in learning a second language. MQ contest activates cognitive processes which draw learner’s attention to formal aspects of the language, it gives them time to think about the language (Harmer, 2007). Moreover, due to the guided writing methodology, it is likely that students not only engage in reflecting on their own linguistic knowledge, but they may also search the available input (in this case, the model and the glossary, dictionaries) for the necessary information. All this cognitive process contributes to consolidate existing knowledge and acquire new information. However, this improvement may be attributed partially to a self-bias: students who decided to participate had higher pre-test scores than those who decided not to participate
The improvement we observed seems to dim in older grades. Although we found a similar improvement in the 7th and 8th grades, such improvement was not statistically significant. This difference in improvement for younger grades (5th and 6th) versus older grades (7th and 8th), may be due to the fact that for most of the young students, their level of proficiency is reduced and any intervention might cause great learning in contrast with their initial level. However, this might also be explained by the differences in our measurement instruments: we use qualitatively different tests for the 5th/6th group and the 7th/8th group.
Improvement in language proficiency should not be expected to be linear in time. Although we did not find evidence of generalizable improvement, this should not be understood as lack of any effects (as usual, absence of evidence is not evidence of absence). In our setup, we had several confound variables that are hardly separable one from each other, given the nature of the service our company offers. For example, we included a zone variable in several tests; however, this categorical variable was conflated with several other variables:
- Schools in the north zone had almost no mediators attending sessions, while schools in the center zone had mediators in every held session.
- Schools in the north zone (Mejillones, Tocopilla and La Serena counties) have in average a very different level of cultural and economic development than schools in the center zone; thus, differences in performance for both groups of schools are expected.
- Schools in the north zone lack a good Internet connection, a critical factor in the success of the experience, which does not happen in the center zone.
- During the 2015 school year, schools in the north zone suffered from several natural catastrophes (an earthquake, a subsequent tsunami and even a flood) that prevented students from attending schools for several months. In comparison, students attending schools in the center zone did not experience such interruptions.
Participation
Participation in the MQ contests is probably due to different reasons in the 5th/6th group than in the 7th/8th group. Topics that are most popular seem to be those that are either familiar to students, or those that are appealing to their age. A case for this can be found in the most popular topic in 5th and 6th grades: the contest “A new hero”. In this contest, students are encouraged to create an imaginary epic hero and to describe it in English. The great interest it attracted might be due to two main factors. On the one hand, the novelty of the prize to the winners: the best description would be drawn by the designing team at RSEduca and published on the Munk website; on the other hand, this might be due to the nature of the topic itself. We believe this latter explanation to be more plausible, considering that exactly the same topic and prize was offered to all grades, but it only elicited a large participation in the 5th/6th group. This is consistent with the fact that 5th and 6th graders are still going through the process of building their identity, which, according to Kagan (1958, 1971) is strengthened by role models. In this case, heroes take the place of models, and children describe the kind of person they want to become.
In general, we found that girls are less likely to participate than boys. This is slightly surprising, considering the general expectation that girls are more prone to engage in linguistic activities (Papalia and Olds, 1992). However, our results are consistent with research on games: boys have a higher preference for games played on a computer (Ito, 2008), and they seem to feel more comfortable, engaged and motivated with computer games’ reward system than girls (Hoeft, 2008). Master Quest is in and of itself a game played online, and as such, it might be perceived as a competition which may inhibit girls from participating.
Older students are also less willing to participate than younger students. Younger students (5th/6th grades) who had higher pre-test scores were more likely to participate; this was not observed in older students (that is, pre-test score was not a predictor of the decision to participate). This might be due to the fact that in Chile the 5th grade is the first level with a mandatory English curriculum. Thus, improvements in the first year are relatively larger than what can be achieved in later years.
Conclusions
During 2015, we ran an online writing contest - Master Quest - for 3,333 students of 5th to 8th grades from 19 public and subsidized schools located in Chile. We designed this contest in the belief that writing in a foreign language would enhance students’ knowledge of the language itself, following Swain’s output hypothesis (1985). We measured improvement in language proficiency through English language pre- and a post-test at the very beginning and the very end of a school year. In between, students could decide whether to participate in any number of about 14 different contests - or not to participate at all. We tested whether this voluntary activity led to improvements that could be attributed, at least partially, to participating in the contests. To the best of our knowledge, we are the first to report about such a learning experience.
We found that students in the 5th and 6th grades who decided to participate had significantly better performance in the tests than students who did not participate at all. However, this may be attributed partially to a self-bias: students who decided to participate had higher pre-test scores than those who decided not to participate. Although we hypothesize that this might be due to poor self-efficacy, this was not tested and it requires more research.
Although we found a similar improvement in the 7th and 8th grades, such improvement was not statistically significant.
Girls are less prone to participate than boys, and older students are also apparently less willing to participate than younger students. Younger students (5th/6th grades) who had higher pre-test scores were more likely to participate; this was not observed in older students (that is, pre-test score was not a predictor of the decision to participate).
We observed a slight but significant improvement in test scores with the increase in contest participations; however, this improvement reaches a plateau after 2 participations.
Finally, we observed that students with lower pre-test scores who attended schools in the center zone obtained the most benefit from participating in the Master Quest contest.
Acknowledgements
The authors would like to thank the whole team at RSEduca for their support and help with the activities that led to this study. This research was partially funded by the PAI program at CONICYT (Concurso Nacional Inserción en el Sector Productivo, Convocatoria 2014, Proyecto 781401008).
References
Bitchener, J., Young, S. Cameron, D. (2005). The effect of different types of corrective feedback on ESL student writing, Journal of Second Language Writing, 14 (3), 191 - 205.
Chukharev-Hudilainen, E. & Saricaoglu, A. (2016). Causal discourse analyzer: improving automated feedback on academic ESL writing, Computer Assisted Language Learning, 29 (3), 494-23.
Clúa Artero, J. (2013). Writing in an EFL context: an insight into young English learners’ writing skills. Bachelor’s thesis. Retrieved from http://ddd.uab.cat/record/112441 on Oct/25/2016.
Committee on Communication for Behavior Change in the 21st Century: Improving the Health of Diverse Populations, Board on Neuroscience and Behavioral Health (Ed.). (2002). Theory. Speaking of health: Assessing Health Communication Strategies for Diverse Populations (pp. 28-75). The National Academies Press. Retrieved from http://www.nap.edu/catalog.php?record_id=10018 on Nov/3/2016.
Fidaoui, D. Bahous, R. & Bacha, N. (2010). CALL in Lebanese elementary ESL writing classrooms, Computer Assisted Language Learning, 23 (2) 151-168.
Harmer, Jeremy (2007) How to teach English. Essex: Pearson Longman.
Hartshorn, K. J., Evans, N. W., Merrill, P. F., Sudweeks, R. R. Strong-Krause, D. & Anderson, Neil J. (2010). Effects of Dynamic Corrective Feedback on ESL Writing Accuracy, TESOL Quarterly, 44(1), 84 - 109.
Hoeft, F., Watson, C. L., Kesler, S. R., Bettinger, K. E., & Reiss, A. L. (2008). Gender differences in the mesocorticolimbic system during computer game-play. Journal of Psychiatric Research, 42(4), 253-8.
Hyland, K. (2003). Second Language Writing. New York, NY: Cambridge University Press.
Ito, M. (2008). Gender Dynamics of the Japanese Media Mix. In Beyond Barbie and Mortal Kombat: New Perspectives on Gender, Gaming, and Computing, edited by Yasmin Kafai, Carrie Heeter, Jill Denner, Jen Sun, MIT Press.
Izumi, S. (2002). Output, input enhancement and the noticing hypothesis. Studies in second language acquisition. 24(4), 541-577.
MINEDUC (2012). Bases curriculares Idioma Extranjero Inglés, Ministerio de Educación, Chile. Retrieved from http://www.curriculumenlineamineduc.cl/605/w3-propertyvalue-52066.html on Oct/25/2016.
Papalia, D. E., Olds, S. W., & Feldman, R. D. (1992). Human Development. McGraw Hill.
Reynolds, D. (2004). Linguistic correlates of second language literacy development: Evidence from middle-grade learner essays. Journal of Second Language Writing. 14, 19-45.
Sheen, Y., Wright, D. & Moldawa, A. (2009). Differential effects of focused and unfocused written correction on the accurate use of grammatical forms by adult ESL learners. System, 37(4), 556-569.
Swain, M. (1985). Communicative competence: some roles of comprehensible input and output in its development. In S. Gass &C. Madden (Eds.). Input in second language acquisition (pp. 235-253). Rowley, MA: Newbury house.
Swain, M. (1993). The Output hypothesis: Just speaking and writing aren’t enough. The Canadian Modern Language Review, 50(1), 158-164.
Swain, M. and Lapkin, S. (1995). Problems in Output and the Cognitive Processes They Generate: A Step Towards Second Language Learning. Applied Linguistics, 16, 371-391.
Appendix A: Munk Welcoming page
Appendix B: Master Quest contest page
Appendix C: Texts assessment rubric
Level | Category | Description |
7 | Original texts in English | The text complies with the topic and length requested. The student writes an original text (does not copy the model). The text is completely written in English. The text has no grammatical or spelling errors. |
6 | The text complies with the topic and length requested. The student writes an original text (does not copy the model). The text is completely written in English. The text has some grammatical and spelling errors. | |
5 | Text based on the provided model | The text complies with the topic and length requested. The student writes a text based on the model available on the MQ website. The text is completely written in English. The text has no grammatical and spelling errors. |
4 | The text partially complies with either the topic or length requested. The student writes a text based on the model available on the MQ website. The text is completely written in English. The text has some grammatical and spelling errors. | |
3 | Partial use of English
| The text complies poorly with either the topic or length requested. The student submits a written text in English and Spanish in between or evident abuse of an online translator can be detected. The text has grammatical and / or orthographic errors that interfere with its comprehension. |
2 | Insufficient | The text does not comply with the topic or length requested or the student copies the model completely. Or the student submits too short a text (sentence) and / or has serious grammatical and / or orthographic errors that interfere with the comprehension of the text. |
1 | The student delivers a text written entirely in Spanish, isolated words in English, or writes incoherent characters. |
Appendix D: Feedback for students
Nivel | Categoría | Descripción |
7 | ¡Excelente! | Tu texto es original y está muy bien escrito. Se nota que trabajaste cuidadosamente. Sigue así! |
6 | Tu texto es original. Has usado bien el inglés, pero necesitas poner atención en la ortografía y la gramática. Recuerda leer varias veces un texto, antes de enviarlo. | |
5 | ¡Buen trabajo! | Has producido un buen texto usando el modelo que el Munk Master te ofrece. La próxima vez, intenta crear algo más original. Ánimo! |
4 | Has producido un texto usando el modelo que el Munk Master te ofrece, pero necesitas poner atención en la ortografía y la gramática de las palabras que usas. La próxima vez, intenta crear algo más original. Ánimo! | |
3 | ¡Sigue intentando! | El Munk Master sabe que el inglés es difícil, pero te recomienda que intentes escribir más. Lee atentamente las instrucciones del Master Quest y al escribir tu texto, apóyate más de diccionarios y no mucho de traductores. Esto ayudará a que tu texto no presente errores gramaticales o de ortografía. |
2 | ¡Esfuérzate! | El Munk Master te recomienda intentarlo otra vez. Lee atentamente las instrucciones del Master Quest, escribe más y asegurarte de que lo que escribiste pueda entenderlo otra persona. |
1 | El Munk Master necesita más esfuerzo de tu parte. Lee atentamente las instrucciones, sigue las reglas del juego. Recuerda que debes escribir un párrafo en Inglés. |
Appendix D: Feedback for students (translation)
Level | Category | Description |
7 | Excellent! | Your text is original and very well written. It shows that you worked carefully. Keep it up! |
6 | Your text is original. You have used the English language well, but you need to pay attention to spelling and grammar. Remember to read a text several times before submitting it. | |
5 | Good job!
| You have produced a good text using the model that the Munk Master offers. Next time try to create something more original. Cheer up! |
4 | You have produced a text using the model that the Munk Master offers, but you need to pay attention to the spelling and grammar of the words you use. Next time try to create something more original. Cheer up! | |
3 | Keep trying!
| The Munk Master knows that English is difficult, but he recommends that you try to write more. Read the instructions of the Master Quest carefully and when writing your text, rely on more dictionaries and not much on online translators. This will help you avoid grammatical or spelling mistakes. |
2 | Work harder! | The Munk Master encourages you to try again. Read the instructions of the Master Quest carefully, write more and make sure that what you wrote can be understood by someone else. |
1 | The Munk Master needs more effort from you. Read the instructions carefully, follow the rules of the game. Remember to write a paragraph in English. |
Appendix E: Master Quest Hall of Heroes
Appendix F: Feedback for student
Comentarios
Publicar un comentario