The flaw of averages

By John McDermott

When Todd Rose was at high school, he asked a lot of questions, quizzing teachers about why they taught the way they did. On more than one occasion he was sent to detention for using a different method to the one his maths teacher recommended. He was lonely and bored at school, and won few prizes. His grade-point average (GPA) was 0.9, or a D-minus. It was not good enough to graduate, so, when he was in his final year the head teacher told him to leave.

A few weeks later Rose’s girlfriend told him she was pregnant. “You could probably imagine, in Utah, that’s not a typical thing,” he recalls. Both sets of parents (Mormons, like 60% of Utahans) were shocked; hers feared that she was tethered to a layabout. Over the next three years Rose had about a dozen minimum-wage jobs, each one demanding grinding conformity. A factory had him stamp the same aluminium blocks in the same way every day. A call-centre forced him to use the same script on every customer or risk being fired. So he quit.

In 1995, married with two sons and fearful for his family’s future, he enrolled in night classes at Weber State University, a local college that did not ask for his GPA. His wife sold blood plasma to help pay for tuition. He discovered statistics and psychology. He learned what interested him, at his own pace. He graduated with a 3.9 GPA (near enough straight As) and an offer to study psychology at Harvard, where today he is director of the Mind, Brain and Education programme at the Graduate School of Education.

His research is in the field known as “the science of the individual”. He argues that the myth of an “average” person, around which today’s educational systems are built, stunts people’s intellectual growth and damages their lives. A class of pupils has an average height and an average score in a test but when you look closer at individuals, the elements are “all over the place”. Very few pupils are average across most dimensions: they learn in different ways, at different speeds and along different paths. He expounded his ideas in “The End of Average” in 2016.

Had Rose merely been an academic with an interesting theory, his ideas might not have got far beyond Harvard’s lecture halls. But – perhaps because so many of them are drop-outs themselves, perhaps because technology is central to implementing the model – Silicon Valley’s biggest names have bought into his theory. The form of education Rose advocates is generously supported by both the Gates Foundation and the Chan Zuckerberg Initiative, an investment fund set up by Mark Zuckerberg, Face­book’s founder, and his wife Priscilla Chan. Zuckerberg has also lent Facebook’s software engineers to Summit Public Schools, the leading exponent of the model, which Rose also advises. Zuckerberg says he wants 1bn children around the world to benefit from personalised learning.

The model is spreading. Over the past few years dozens of schools using such an approach have opened, and many more are on the way. More than 3,000 of America’s 13,000 school district leaders have signed a pledge to make their schools “future ready” by moving towards “personalised, digital learning”.

When explaining what he means by the “tyranny of the average” Rose starts with the United States Air Force. In 1926 the USAF measured pilots’ heights, leg lengths, arm spans and so on. For each the air force calculated a simple mean and used those data to build a standardised cockpit. By the late 1940s, there had been a spate of plane crashes. Gilbert Daniels, a researcher, took another 140 measurements from 4,000 pilots. He assumed that the vast majority of pilots would be within a narrow range across most dimensions. None were. Some were tall with short legs; others had small hips and a wide neck. The cockpit would have been slightly wrong for all of them. In response to Daniels’s findings, the air force brought in an adjustable cockpit. Crashes became less common.

For Rose, schools and universities are like those 1940s planes. Since they are designed with a mythical “average” user in mind, they are designed for nobody.

Rose notes similar issues in other fields. When neuroscientists publish research about which parts of the brain are involved in an activity (say, storing memories), they often publish scans with the relevant areas highlighted. The images are “average brains”: composites of several scans. But when some researchers compare the images of individual brains with the “average brain”, they find that few match.

The study of child development is full of cases in which the use of group averages leads to poor predictions about individuals, says Rose. Charts that depict the stages children “should” reach at particular times are based on averages of thousands of pathways taken by other kids as they learn to walk, talk or read. Yet these fixed stages obscure that children can take different paths to the same milestone.

Responses to Rose’s work range from “so what?” to a belief that he is attacking the foundation of all research. Studying groups inevitably means you miss individual differences, but by identifying common patterns, statistical analysis allows researchers to theorise about cause and effect. A comparison of two groups – one that has been treated in some way (an anti-cancer drug, a lower tax rate, an unemployment scheme) and another that has not – helps establish what works.

Rose concedes that averaging out results allows statisticians to draw distinctions between groups, and that if you want to make predictions about an individual it is useful to know something about the group to which they belong. Most pupils with a 0.9 GPA will earn less in later life than those with straight-As, so if you want to understand a pupil’s potential, knowing their GPA is helpful.

But, says Rose, it’s not that helpful. He argues that extrapolating about an individual from a group can lead to huge errors. Imagine a group of typists, he suggests, who type at different speeds, with different levels of accuracy. On average faster typists are more accurate. But if a slow typist were suddenly to type faster the number of errors they made would probably increase. Knowing something about the overall correlation between speed and accuracy would probably result in a false conclusion.

One problem, according to Rose, is that “talent is always jagged”. Standardised tests try to reduce ability to a single number. Results on tests such as the sat are correlated with, for example, completing the first year of college. But the link is weak. One study suggests that sat scores explain just 16% of the variation in first-year completion rates. And once you disaggregate abilities, there is a huge amount of variation, says Rose. Two people with the same sat score may have wildly different verbal and numerical reasoning scores, for example.

“Average-arian” thinking gives rise to another problem, says Rose. Edward Thorndike, one of the most influential psychologists of the 20th century, thought that, “the quick learners…are the good retainers.” To this day exams are time-limited; pupils are placed in age-specific grades; timetables feature specific times for each subject. All of which reflect the belief that there is a straightforward relationship between learning ability and learning speed. But it turns out that whether you can master a subject is not related to how long it takes to do so, says Rose.

Rose contests that there is another way of looking at the world, one that places the individual at the centre of analysis. Traditionally researchers take one observation at one time from multiple people. By contrast the likes of Rose, Kurt Fischer – Rose’s mentor at Harvard – and Peter Molenaar, at Penn State University, take multiple observations of a single individual. They then look for common patterns in the way that those individuals develop. This way is “analyse, then aggregate”, rather than “aggregate, then analyse”.

When you do that, argues Rose, you get a more dynamic picture of how children learn. “It’s not that we’re all snowflakes,” he says; but it shows that different brains develop along different paths. They are not infinite, but they are multiple.

Though newly fashionable, these ideas have a long history. In his novel “Emile”, Jean-Jacques Rousseau argued that children should “learn by experience alone”. In 1916, John Dewey, a philosopher and psychologist, published “Democracy and Education”, arguing that the pupil, not a government-mandated curriculum, should be at the centre of a school. In ordinary schools, he said, the child is not allowed to “follow the law of his nature”, and is therefore “thrown into a passive, receptive or absorbing attitude”.

Technology has given these ideas a new momentum. Providing children with bespoke attention typically means hiring a tutor or raising the teacher-pupil ratio – too expensive for most parents or schools. But while a blackboard can show only one set of sums, new software claims to display whatever sums are appropriate to a child’s level and should free up teachers’ time to spend less time marking and preparing lessons, and more with individual pupils. In theory, then, such technology should put personalised education within the reach of every school.

On a mezzanine level at Summit Public School in Redwood City, just south of San Francisco, teenagers slouch and tap at their laptops. They look as though they could be doing work experience at one of the many tech companies nearby. Inside the classrooms, however, few are staring at screens. Instead they are working on projects. In one room students take it in turns to practice the first line of a speech they have written. The openings hint at the backgrounds of pupils’ families in this poor, mostly Latino community. “Imagine crossing the border without food or water,” begins a 16-year-old girl.

The blend of technology and self-directed projects at each of Summit’s 11 schools is deliberate. Pupils spend about 16 hours per week (roughly half of that at home and half at school) learning on their laptops. They use the Summit Learning Platform, designed pro bono by engineers from Facebook.

Textbooks are replaced by “playlists” which offer materials on which students can draw to help them learn. Pupils take tests and submit essays on the platform. Marking is often automated. The assessment data are used to create a profile of every student across the 36 cognitive skills and five “habits of success” that Summit believes they should be able to master before leaving.

The platform allows students to learn at their own pace, argues Andrew Goldin, Summit’s chief of schools. At conventional schools, “there are many students waiting for others to catch up.” Rather than rely on high-stakes tests, the platform tracks individuals as they go along. It prompts pupils, Goldin says, to reflect on why and how they learn.

Pupils in a senior year English class seem to agree. One 17-year-old girl says that, unlike her friends at other schools, “I actually like my school”, since it offers lots of chances to try out new things. Her deskmate, also 17, adds that, “you really get to know the teachers, and they to know you.” It helps that teachers at Summit spend less time than their peers marking lessons. They can use that time mentoring pupils, coaching them in “habits of success” such as stress management and curiosity and helping with pupils’ projects.

Not everybody is as delighted as these pupils are by the spread of such techniques. Benjamin Riley, executive director of Deans for Impact, a charity, believes that giving children too much control over the pace and content of what they learn can be dangerous. “Effortful thinking is what our minds are built to avoid.” While some may thrive in such a system, many others will take the easy way out and not think too much.

Cognitive scientists such as Daniel Willingham of the University of Virginia worry that autonomy can be taken too far. If children can opt out of learning important facts, he says, they will find it harder to understand more complex ideas at a later stage.

Groups representing minorities have also expressed scepticism. They point out that it took African-Americans until 1954 to earn the legal right to be taught in the same school as white people, and almost another half-century before a president vowed to ensure that “no child [be] left behind”. The average-arian school may not be perfect – but at least it has minimum standards, for which they have fought long and hard.

Rose acknowledges many of these concerns. Personalisation should not be an excuse for dodging the basics, he insists. Teachers need to challenge and mentor pupils, and in a personalised system they have more time to do so. Goldin points out that most of the students at the Summit School in Redwood City do not have university-educated parents, yet the school’s results in nationwide maths and reading tests are higher than its demographics would suggest. 93% of Summit students graduate, compared with 83% across the state. And 99% of Summit graduates go to university.

Worries about such heavy reliance on technology do not relate only to its impact on the nature of education. Platforms like Summit’s generate vast quantities of data about the intellectual and social skills of the children using them. Pupils may benefit from this – but they may not be the only beneficiaries. Data are a resource, so these deep, detailed profiles could become exceedingly valuable to the companies that are supplying the technology. That’s why some critics suspect that the tech barons who are promoting personalised education may not be doing so purely out of altruism.

The question of who owns these data – the child, the school or the company providing the platform – therefore becomes crucial. Rose argues that ownerships should lie with the child, for if pupils do not own their data, they will not be able to take them to another platform or school, or will have to pay to do so. He argues that the platforms used by schools and the data they generate should be regarded as a public utility.

America is in the very early stages of a big pedagogical experiment based on old ideas given new life by digital technology and the techies’ money. There isn’t enough evidence yet to conclude that this blend of technology and personalised learning serves pupils better than the status quo, but the revolution is gathering pace.

It could, Rose acknowledges, “go horribly, horribly wrong”. If it does, a lot of children’s lives will have been damaged; but then it is hardly as though the existing system is releasing the full potential of America’s young people. For Rose, giving children more control over what they learn and how they learn it is central to achieving that. Ultimately, he says, “you should have a right to know who you are.”

PHOTOGRAPH CHARLIE MAHONEY