DUOLINGO EFFECTIVENESS STUDY
EXECUTIVE SUMMARY
The research study of Duolingo effectiveness was independently conducted in September – November of 2012.
The study lasted for approximately eight weeks. A random representative sample was selected from Duolingo users who studied Spanish.
The participants were at least 18 years of age, native speakers of English, not from Hispanic origin and not advanced users of Spanish, and all of the participants resided in the United States.
The participants took one college placement Spanish language test in the beginning of the study and one test at the end of the study. The test results were measured in points (the higher the better).
The improvement of language abilities was measured as the difference between the final and the initial language test results.
The effectiveness of Duolingo was measured as language improvement per one hour of study.
MAIN RESULTS
Overall the average improvement in language abilities was 91.4 points and the improvement was statistically significant.
The effectiveness measure showed that on average participants gained 8.1 points per one hour of study with Duolingo.
The study estimated that a person with no knowledge of Spanish would need between 26 and 49 hours (or 34 hours on average) to cover the material for the first college semester of Spanish. This result is based on the language test’s cut-off point for the second college semester and the 95% Confidence Interval of the effectiveness measure.
The main factor for higher effectiveness was the motivation of the participants, with people studying for travel gaining the most and people studying for personal interest gaining the least.
Another factor for higher effectiveness was the initial level of knowledge of Spanish with beginners gaining the most and more advanced learners gaining the least.
Effectiveness
Usually in these kind of studies with two measures (initial and final test) a simple measure of the progress is computed and analyzed. This is the difference score which is the difference between the final and initial score.
The problem with the progress measured as difference score is that it does not take into account how much time each participant actually studied.
Factors for Effectiveness
From an analytical point of view it is interesting to know what factors if any, determine improvements in language skills.
We started with demographic factors.
Gender results were unexpected; men had better progress than women.
Under further investigation we found that 11 out of 44 women decreased slightly their results.
There were some differences in progress by age groups with the 31– 40 year old group having the highest results but the differences were not statistically significant.
There were expected differences between the education groups with the MA/PhD group having the highest progress report but these differences were not statistically significant.
There was no effect on progress of the presence or absence of relatives or friends who spoke Spanish. The same lack of influence was true for the type of employment.
The only significant factor (p=.006) was the reason for studying Spanish. The best
progress was achieved by people studying for travel with an average progress of 17.6 points improvement per one hour of study.
The majority of the people studied for personal interest and school (60 personal interest and 2 school) but this group had the least progress with 5.7 points improvement per one hour of study.
Only two factors were shown to matter for the progress of the participants per one hour of study.
First, their motivation; with people who studied Spanish to travel having the biggest improvement. People who studied mainly for personal interest and school had more modest improvement.
Second, the initial level of knowledge of Spanish was another contributing factor.
People who were beginners (Semester 1) had the biggest improvement and more advanced people (Semester 2 and 3) had the smallest
improvement.
No hay comentarios:
Publicar un comentario