Known and unknown in mLearning


It is the question that will not die and doesn’t evolve as it should. Can technology improve learning outcomes?  We have known for decades that certain things can help in the learning process: computer aided instruction, Sesame Street, reminders, feedback since way before the proliferation of mobile phones. The right questions are how?, how can we measure that?,  and most important: how much does that cost?

Known knowns

space-learningBefore lamenting the state of evidence on the role of technology in improving education; I wish to lament the state of evidence on improving education in developing countries more generally.  That is not without reason; these are difficult to reach places. In some countries in remote areas we don’t even know if the school building physically exists to the extent we need GPS enabled smartphones, drones or satellite imagery just to be sure that the school is actually physically there.  Knowing or measuring what students are learning, assuming they actually physically come to the building, is a new challenge.  So: A known known is that we often don’t know what students and teachers know or don’t know there if we don’t so much as know that classes are running there.

What does mobile learning really include? Well using apps is a form of computer aided instructions.  There are serious games, so game based learning research is hardly out the window.  Reminding learners about something via SMS is a form of reminder nevertheless.  Even we have research from Matthew Kam’s MILLEE project that shows this research still holds true when applied to using games for learning on mobile phones: there were statistically significant improvements in learning outcomes.

Patrick McEwan just published meta analysis of 76 RCT’s locatable on education improvements in developing countries (that is up from a total of 9 being available in 2003) – even the dramatic rise in RCTs means there is a maximum peak rate of 6 or 7 RCT experiments in developing countries published per year.  The number one program type that achieved statistically significant improvements in student learning: “computers or technology”.  Just ahead of teacher training. RCTs are not cheap exercises; insisting on RCTs before more basic experimenting creates a no innovation chicken and egg loop.


We know we know that students can know more faster with technology enhanced learning and we know this is applicable to small portable computers (also known as mobile phones).

Known unknowns

Key recommendation of the report: report cost data. 56% of these interventions had no data on cost; that clearly is a problem. We still don’t have sufficient comparable cost data on what works to use for decision making on opportunity costs. Right now it looks like computers or technology are relatively expensive: so let’s rework the question:

The question is not “does it work?” The question is: how much does any of this cost and then how can we do that cheaper and sustainably?

We don’t know how to build capacity to create mobile learning applications. That’s what we’re working on with Ustad Mobile. I know if you look in the app store you won’t find any way to make your own materials for low cost devices without learning how to code BREW or Java 2 Micro Edition (and re-invent the wheels of audio video slides, quizzes and mini games). Given that we know cost is a problem spending time and therefor money re-inventing this is a problem.

We know that we don’t know enough about the cost of technology for people to learn to know stuff.

Mike Trucano of the World Bank has a very good answer that addresses this: The best technology is the one you already have, know how to use, and can afford. I’d say we really have to do better with the devices we have: there’s a whole lot that needs more than 160 characters in an SMS message to teach.  But these $30+ Java enabled feature phone of which there are 3 billion out there now can do audio, video and small apps. Ustad Mobile will enable all of those to provide a full computer aided learning experience.

Unknown unkowns…

Ehhh… hmm…  Where’s my hoverboard already?

How SciDev.Net article fails in mobile learning analysis

In this article on titled “How teachers in Africa are being failed by Mobile Learning” Niall Winters asserts that:

  1. Teachers are being excluded en masse from mobile learning projects with a whole sub-heading called “excluding teachers” (though the examples cited are irrelevant to mobile learning such as a project that fixes computers inside brick walls)
  2. “We know that many mobile learning projects are funded by sizeable donations made under corporate social responsibility budget” (without examples)
  3. “The idea that techno-centrism or even solely content-based solutions can address important educational challenges by themselves must be dropped. Research shows they can’t” (by referencing a blog post the author himself wrote)

I agree that these are mistakes that have been seen; and no one is arguing what is being cited is good practice; but this article is extremely flawed and unbalanced.  I’ve been working with education technology in Afghanistan for 6 years. The idea that you need to involve teachers (and maybe even students and parents to some extent) in an education solution is no more novel than the idea that you will need to speak with astronauts when designing a space suit or company managers when designing a business analytics software.

hiwel_050-300x225The first example cited in the article:

“A good example of this is how the technology community has openly welcomed 2013 TED Prize winner Sugata Mitra’s work on learning through self-instruction and peer-shared knowledge, even though his approach to achieving this is highly contested among educational researchers and practitioners.”

The author’s first and apparently prime example when discussing a failure of mobile learning for teachers doesn’t seem … um… very mobile. It is a system that involves a computer physically locked inside a fixed brick wall. That someone won a TED prize does not mean the “technology community” has overwhelmingly endorsed it. A computer lab in brick walls to attempt to bring about peer based learning is about as relevant to assessing if mobile technology can help students and teachers as it is mobile.

The author fails to consider relevant examples like Eneza Education in Kenya where a teacher-led company makes a system that helps around 100,000 pupils to do test preparation by SMS, or our own Ustad Mobile in Afghanistan where we built a system that lets curriculum designers put together audio, video, quizzes and games for learners to use outside the classroom precisely because the teachers don’t have enough time in class. We can then also rapidly help teachers understand which pupils need further help. The models where developed after consultation with education stakeholders (not excluding teachers).  Matthew Kam’s research with MILLEE show children with a game on a low cost feature phone could statistically significantly improve test scores.

The author also fails to consider the informal learning environment outside the classroom – like the demand for learning English and the success of programs like BBC Janala where users call in on a subsidized phone number to cheaply access english language lessons (working together with television and radio programs).

The author asserts:

“Third, learn from the One Laptop per Child programme. Its uptake in Sub-Saharan Africa was generally judged to have failed because of a lack of integration with education ministries.”

This is asserted interestingly without any reference. People more familiar with it might wonder if a lack of uptake had something to do with the cost being around $400 per laptop per child over it’s life… The argument that every child should have a laptop is incomparable to the value proposition of using mobile phones people already have in the family home for supplementary learning.

And finally:

“The idea that techno-centrism or even solely content-based solutions can address important educational challenges by themselves must be dropped. Research shows they can’t [5]”

This is quite a definitive statement – let’s look at the source of the certainty:

[5] Winters, N. Why mobile learning on its own won’t solve the access problem (LIDC blog, 13 November 2012)

If one were to reference Wikipedia there would be only the suspicion that you yourself actually controlled the content; here it references a blog post the author himself wrote so we have certainty that is the case.

I would agree with the author that teachers in Africa (and throughout the developing world) have a difficult life and there is no technological magic bullet; the article however ignores how teachers and students can and already do in many examples around the world both benefit from well designed mobile technology.

In summary the article does not present any coherent argument or evidence that teachers are being failed by mobile learning. In mobile learning as with any new technology there will be instances of failure and there are instances of success. Articles should consider relevant examples, and real alternatives (technological or not) to those relevant examples. They should examine what’s behind success or failure in order for it’s recommendations or opinion to be of any use.