Duolingo Launches Its Certification Program To Take On TOEFL

"Standardized English exams like the Test of English as a Foreign Language (TOEFL) that almost every foreign student who has ever wanted to study in the U.S. has to take are a big business and essentially operate like monopolies. As far as I am aware, Duolingo is the first startup to take these standardized tests on directly."

TechCrunch: Duolingo Launches Its Certification Program To Take On TOEFL

I seem to remember reading on HTLAL that duolingo does not take one far, as in only 2000 words, or a year or two of high school /college. Is that correct? Because if it is, unless English is an exception, then how can they rate more than a minimal level of English competence? Also I have read complaints about inaccuracies, to be expected in anything crowd sourced, but not really a strong point for a site that seeks to run a certification program.

The statement above is interesting as well that it uses "monopolies" as in plural of "monopoly". Do different companies actually offer TOEFL or just one? If more than one, that isn't a monopoly unless there is implicit/explicit price collusion (and really then it is an oligarchy I guess).

I know nothing of the TOEFL or how it is run, so the questions above are a product of that ignorance. The other thing that comes to mind, is why doesn't duolingo take on CEFR testing? That's a pretty big and expensive business too. Don't the Goethe and Cervantes institutes deserve some competition as well?

Diploma mills make money with very low overhead I am told .

I am sure that the TOEFL certifying companies will lobby to have Duolingo certifications not be recognized by educational institutions. It will come down to whether Duolingo can actually test to the same standards, or will dumb them down, and whether some colleges would knowingly accept dumbed-down certifications (pretty sure some would since all high school diplomas are not equally indicative, i.e. some are debased). It would be interesting for an impartial third-party agency to vet the certifications by having a group of test subjects of varying levels of ability, take both tests in rapid succession, and then compare the corresponding certifications.

Duolingo has a great idea, as such competition is what the internet is built for. But whether they can execute without dumbing down standards or attempting to redefine terms is the question.

Monopolies is in plural, because they're referring to various countries. TOEFL isn't as big in the UK, where Cambridge and Trinity exams are the normal standard. For academic entry, it's typically the IELTS exam that's required.

The article talks about having to travel to test centres for the TOEFL, but at's no longer true, as they've moved it online (TOEFL iBT).

The big issue is one of getting institutional acceptance. My last employer was an accredited testing centre for City & Guilds, who have a pretty solid academic reputation, and yet very few places accept the certificate. If C&G can't do it, what chance does an upstart like Duolingo have?

Maybe I'm cynical, but technology startups make everything sound like a noble crusade to change the world. In Silicon Valley, you can't just say, "I've created product X, because I think it will be very useful for my customers, and they will happily give me a big pile of money, which is the whole point." No, you need to talk about "disruption" and "revolution" and the evil established interests that you're going to overthrow. Sure, this is fun, and sometimes the established interests really are awful, but it gets to be a cliché after a while.

I can't speak about CEFR exams in general, but my DELF B2 exam was expensive, it took half a day, and it required driving two and a half hours each way. But it was also very well run, especially the oral exam:

• They used two trained examiners: one to interact with me, and another to observe and take notes. I'm told by other examiners that this makes a big difference, because it's hard to notice subtle language details while actually speaking with someone.
• They were very good at forcing me out of my comfort zone, and they knew all the tricks. They weren't going to be fooled by the diversionary tactics in Shekhtman's How to Improve Your Foreign Language Immediately, or by changing the subject of conversation.
• The topics they chose were actually quite challenging for B2, at least from what I've heard about about other CEFRL exams. I think that this is part of a general French strategy of pushing even well-prepared students until they struggle, the better to measure what they can really do.

Now, I'm sure that lots of English and CEFRL exams are poorly run, and I've heard some horror stories about DELF examiners, too. But even so, language exams are long, strenuous and expensive partly because assessing language skills is genuinely difficult, and partly because students prepare extensively beforehand.

But Duolingo is claiming they can do a good job with a 20-minute, online exam. Of course, since they're charging for this exam, they face the usual temptation: Do they dumb down their exam in order to earn as much money as possible, or do they keep it rigorous and try to uphold standards? The fact that the exam is 20 minutes long suggests that they're willing to compromise on rigor a bit, at least compared to the better CEFR exams. Of course, language schools face a second perverse incentive: They want a large fraction of their paying students to pass the exam, and so they risk adjusting the curriculum and the exam until most of their students can pass.

I don't think that many universities want to choose which students to admit based on a 20-minute, online exam run by somebody who sells language courses. Well, unless those students are really rich and the university has a lot of remedial ESL courses. But in that case, the university will probably run its own exams, charge high prices, and offer little financial aid to ESL students.

In any case, the real customer for language exams are universities and very large employers. Duolingo would be much better served by finding out what universities and employers hate about the TOEFL, and designing an exam which addresses those needs. After all, the consumer has exactly zero decision-making power in this market. My guess is that this is just random flailing around on Duolingo's part, and they don't really know what they're doing. The most likely outcome, in my opinion, is that they wind up with a cute exam that they can sell to their own students, but which isn't useful for anything in the real world.

Marking is still a notable cost in exams. Automated marking of the comprehension questions cuts down costs considerably, but the biggest time sink is marking productive skills.

As for the 20 minute thing, I wouldn't be too worried about that. In this instsnce, they're talking to universities on their own terms. The article talks about test results as a predictor of ability, not a measure. Universities have been talking like this for over a decade, trying to find ways to reduce the amount of marking required without compromising the accuracy of the marks. They're very heavily invested in this idea, and blind to the fact that marked assignments are as much an opportunity for students to learn from their own mistakes as a means of assessment. They think they're just reducing the marking load, but they're killing their courses. Now that they're invested, that have to be unblinking in their belief, which works in Duolingo's favour.

I don't get why they've taken the open approach, rather than piloting it as an entrance test with specific universities. 20 minutes extra in the application process wouldn't be an onerous requirement, even for those who already have a TOEFL certificate, and they could recoup their costs direct from the university, rather than relying on the test-sitter to pay, which they can't do until it gains some kind of recognition anyway.

I also wonder if another language might not provide an easier first target than English. How about Spanish? The Instituto Cervantes has a much stronger monopoly than TOEFL, and while most English-speaking countries have roughly comparable economies, there's a massive difference between Spain and South America. If you want to go to Peru, the cost of the DELE is going to be ridiculously high relative to your expected salary or the cost of your university study.

Do the universities that require/accept these certificates have such a demand for their services, that they have no inclination to just let anyone be admitted and sink or swim their first semester? I had a a friend, a fellow American, who was also fluent in French, and took courses in Italy in Italian. She told me she could not follow the course of instruction well in class, but that she read and wrote Italian much better and always managed to get good grades on the exams since they had no aural component. And this was probably at a masters equivalent level. Obviously her study time of the written materials was likely far more time-consuming than for students who had a better grasp of all the 4 skills in Italian.

I'm sure that's true. As an aside, fooling people is not Shekhtman's main thing, and that is not his most thorough book.

One of the problems with breaking into the language exam market is that the student pays all the costs—the money, the time, the travel—but the universities and employers decide which exams to buy. Markets like this are hard to change, because the decision makers don't feel the pain, and thus have no incentive to change. Cainntear is absolutely right that Duolingo should have piloted this with real universities.

Oh, don't get me wrong, I'm a huge fan of Shekhtman's books, and he's co-authored several very good texts about advanced language learners. But his tricks really do work quite well, both in polite conversation, and for bamboozling inexperienced language examiners. It's usually pretty easy to fake an extra CEFR level if you can control a one-on-one conversation. And so high-quality oral exams really ought to try to take this into account. As sfuqua wrote:

During my time as a Peace Corps volunteer, I set my self up to be a test subject for new FSI testers, so I did a lot of FSI tests. In a conversational test, there is a lot more to it than just raw grammar and vocabulary. One strategy I used was, "control the conversation," or "get talking about something I know how/want to talk about," or "I don't know much about outrigger canoes; but I do think that the Mt. Vaea club is a great place to go after a day of working on a canoe, or a day of teaching junior high school. During happy hour..." These strategies are similar to those used my US politicians (all politicians?) during debates or press conferences.

Language testing interviews are a battlefield.

(More…)​

This kind of "change the subject" strategy won't work on the DELF B2, unless you're very subtle. For example, if the examiner asks you whether Paris should have London-style congestion charges, but you're much better at talking about the environment, then you can argue "toll charges would be good for Paris because they're good for the environment." If you'd rather talk about the economy, then argue, "Congestion charges are bad for the economy, because of X, Y and Z."

Hahaha, my thought exactly. SV startups the past five years or so have really been soaked in this '60s spirit reshaped into pumped-up business seminar rhetoric. Pretty annoying really.

Quite possibly there's an issue with visa requirements. If the immigration department delegates responsibility to the universities, the universities will be forced to maintain some kind of standards.

Sounds like you need my new web service, positiviT.wow . It will change the world by training that nasty ol' cynicism right out of you. Everyone who has completed the course agrees it works!!