AI and Learning: There’s a Fraction too Little Friction

Leave a reply

I’ve been thinking quite a lot recently about what happens to learning when we introduce generative AI.

My concern is that if AI makes it easier to complete a task, are we also making it easier to avoid the thinking that gets us there?

That’s why I was interested in a recent paper looking at human-AI collaboration at scale.

AI, Claudius

The researchers analysed almost 250,000 real-world conversations with Claude. Rather than looking at what AI could do in a controlled experiment, they looked at what people were actually doing with it.

There’s quite a lot of learning going on.

In 67% of learning and upskilling conversations, Claude engaged in some form of teaching. This wasn’t just school or university work. Software development, health and lifestyle, and business were among the most common areas.

The most common approach was combining conceptual explanation with step-by-step guidance.

So far, so good. But teaching and learning are not the same thing.

I can ask AI to explain how something works. I can also ask it to do the work for me. The two interactions might look superficially similar, but the consequences for learning are very different.

Then I read that almost half of the conversations involved some form of friction.

The AI misunderstood the question, made a mistake, couldn’t do what the user wanted, or perhaps the user’s instructions weren’t clear enough.

Normally, we think of this as something technology should get better at eliminating.

However, in 78.7% of these cases, users tried to recover from the problem, and when people challenged the AI’s reasoning, those attempts were successful 81.5% of the time.

This is much more interesting than the headline that AI can teach.

Resistance is not futile

There is a version of AI-supported learning that looks like this:

Ask a question. Get an answer. Move on.

There is a more productive version:

Ask a question. Get an answer. Notice something doesn’t make sense. Challenge it. Work through the disagreement. Correct the answer.

The second interaction looks rather like learning.

There is effort, uncertainty, feedback. There is the possibility of being wrong. The learner still has to think.

This brings me back to a question I have been exploring in some of my recent writing about AI and education:

Are we using AI to make learning more effective, or to make tasks more efficient?

A student who uses AI to produce a better piece of homework may well get a better piece of homework. But that doesn’t tell us whether the student has become better at the underlying task.

On the other hand, an AI system that helps a student formulate a question, tests their understanding, gives them feedback and challenges their reasoning could potentially be a very powerful learning tool.

The difference is what the human is doing.

The answer isn’t to keep AI away from learning. Quite the opposite. The potential is enormous.

But we should be careful about celebrating every reduction in effort.

Effort is the point. Productive effort drives learning.

If AI removes all of it, we may discover that we have superficially made learning “easier” but also thrown the baby out with the bathwater.

The interesting question isn’t whether AI can teach: it can.

The question is whether we can use AI to make learning easier without taking the learning out of it.

AI in Education: Are We Automating Learning or Augmenting Learners?

1 Reply

I was struck by a recent article in the Economist about whether AI is stopping children from learning.

The headline finding is extraordinary.

Recent research, based on 30 months of data from 26,811 Chinese secondary-school students, found that after adopting generative AI, students’ homework scores increased (on average by 18%), while the time they spent doing their homework fell (by 30%).

So far, so good. Except that their exam scores subsequently fell by 20%.

In other words, students were producing better homework, faster, but learning less.

The authors found that the problem was concentrated among students whose behaviour was consistent with outsourcing their homework to AI. Those who used AI but continued to spend roughly the same amount of time on their work experienced only small learning losses.

The Power of Productive Struggle

This made me think about productive struggle.

Learning is not simply about getting the right answer. It involves building and retrieving knowledge, making connections, trying something, getting it wrong, receiving feedback, revising your thinking and trying again.

That process requires effort and involves some friction or struggle. In the case of learning, in contrast to that of work, that effort is not a bug in the process but rather a key part of it.

This is consistent with the idea of “desirable difficulties” developed by Robert and Elizabeth Bjork: conditions that make learning feel harder or less efficient in the short term can, under the right circumstances, produce more durable learning.

The Tutoring Paradox

I keep coming back to Benjamin Bloom’s famous “2 sigma” research. Bloom found remarkably large gains from one-to-one tutoring compared with conventional classroom instruction. Shouldn’t using AI (tutors) result in similar learning gains?  

That would be the Holy Grail, but maybe we’re not there yet.  Tutoring isn’t simply a matter of giving each student the right answer. The tutor continually diagnoses where the learner is, asks questions, provides feedback, adjusts the level of challenge and keeps the learner engaged in the task. There is a human relationship involved too.

This is the question for AI in education.  How can AI provide the benefits of a tutor without removing the struggle that makes the learning happen?

Augmentation Works

Interestingly, a recent randomised experiment by Zara Contractor and Germán Reyes found that students using AI to learn an unfamiliar subject performed better on subsequent unaided knowledge tests, with the gains persisting a week later. But the strongest delayed gains came from students who used AI to augment their learning. For example, by asking it to explain concepts rather than using it to automate the production of their work. The authors found that AI users shifted time away from drafting and towards reading and searching for information.

To Be or Not To Be

The real distinction we should be looking at isn’t between “AI” and “no AI”.

It is between AI that does the thinking for you and AI that makes you think better.

If we optimise education for productivity (higher homework scores in less time) we may accidentally optimise away something much more important: the learning itself.

🎯 Seeking three executive coaching clients – pro bono

2 Replies

Are you navigating a leadership transition, stepping into a new role, or facing a significant challenge as a leader?

I’d like to offer three people an executive coaching engagement consisting of three one-to-one coaching sessions.

What this involves:

🔹 Three one-to-one coaching sessions focused entirely on your goals and agenda

🔹 Sessions recorded solely to meet ICF accreditation requirements

🔹 Full confidentiality throughout

🔹 Provided at no cost

These sessions will form part of the final stage of my Professional Certificate in Executive Coaching at Henley Business School and also count towards my ICF accreditation.

You’ll receive the same coaching approach, commitment and professionalism that I bring to my paid clients. The only additional element is that the sessions will be recorded to support my qualification process.

If this sounds valuable—or someone comes to mind who would benefit—please contact me at john@executive-mentor.com.

I’d be delighted to have a short conversation to explore whether it’s a good fit.

I’d really appreciate your support, and please feel free to share this with anyone who might benefit.

Three Meters Below Sea Level (And Finally Equipped to Handle It)

Leave a reply

Today, after six months of lessons, I passed my swimming diploma (C). 

I can now swim a few hundred metres fully clothed (including coat and shoes), dive into the deep end and swim 15 metres underwater through a hole in a screen, and tread water in the deep end for as long as I like.

Living in the Netherlands, this feels less like a hobby and more like sensible risk management. We are surrounded by canals, crossed by rivers, and bordered by the North Sea. To add a little existential spice, my own house sits three metres below sea level. Until now, my relationship with the local geography was based largely on faith and  sturdy dikes. Now, at least, I have a fighting chance if the water ever wins. 

I still remember those first few lessons, struggling to master the breaststroke. Who invented that technique—with the arms, the frog legs, and the sequencing? It doesn’t feel like a naturally human movement. It took a surprising amount of effort to persuade my adult body to cooperate. But eventually it did. When I stopped thinking.

Looking back on the past six months, the most surprising part of the experience was how much I enjoyed learning.  In fact, I enjoyed learning to swim more than actually swimming.

The Joy of Being a Beginner

It’s funny which learning experiences stay with you. The two standout experiences in my life so far have been:

  • Learning to play the double bass as a child.
  • Learning to swim as an adult.

The two have more in common than you might think. In both cases, your ego has to leave the room. You can’t think your way through it or compensate with experience from somewhere else. You simply turn up, practise, look slightly ridiculous, improve by small increments, and one day realise you’re doing something that seemed impossible a few months earlier.

There’s something deeply satisfying about that. 

People Made the Difference

What I’ll probably remember most from these lessons are the people.

The instructors were endlessly patient, encouraging and quietly demanding in exactly the right proportions. I remember saying “I don’t think I can do it” when my instructor told me to swim through a hole in a screen after diving into the deep end.  Her reply – “don’t think, just do it” – was exactly the feedback I needed at that moment.


My fellow students were just as generous. There is a surprising amount of camaraderie that develops among people who are all trying to learn to swim as an adult. We all had our own reasons for not learning to swim earlier. Mine was a lack of talent, fear of water and physical clumsiness. (I’m great with books). I expected to feel embarrassed. Instead, I felt supported—and proud that I was doing it.

When your confidence wobbles—as it inevitably does when learning something completely new—the people around you matter enormously. They gave me the encouragement to keep going until I could do it.

So yes, at 56, I finally have my swimming diploma. I also have a newfound energy to tackle the things I’d love to do, even if I don’t naturally have the talent for them.

Anyone for tennis?

Time to Think

Leave a reply

There is a rhythm that takes over when you spend days in the saddle. The world narrows to the width of the road, the steady turning of the pedals, and the sound of the English countryside—rain included.

The purpose of my cycling journey across England was simple: to visit my newborn grand-niece, Florence (born on the same day as Florence Nightingale). I also wanted the journey to carry a second thread—reflection on thinking, learning, and attention. I took Nancy Kline’s Time to Think with me.

Newborn Florence. What a beautiful, pure and curious soul she is!

A mobile laboratory

Kline’s premise is simple: the quality of everything we do depends on the quality of the thinking we do first—and good thinking requires space and attention.

A bicycle tour turns out to be an unusually effective laboratory for this. No meetings. No notifications. No agenda beyond the next village. Just repetition, motion, and a mind disconnecting from noise.

Well, mostly. There was rain, there were more punctures than expected, and the traffic was occasionally overconfident in its interpretation of physics. But even that becomes part of the rhythm, eventually.

An unpaved section of National Cycle Route #1!

Places that hold thought

As I moved through England, I began to notice how often thought seems to gather in certain places.

Oxford and Stratford-upon-Avon are saturated with it. You feel it in the density of books, weight of stone, and arguments made over the centuries.

Oxford

But deep thought doesn’t only happen in institutions. Further north, at Woolsthorpe Manor, Newton’s birthplace, the story goes that much of his transformative thinking on gravity and calculus took shape not in Cambridge, but in his family’s orchard during the plague years. Whether exact or embellished, the image stays with you: revolutionary ideas generated not in a frantic hub of activity but in stillness.

That contrast stayed with me as I rode on.

Newton’s Orchard and Home

Lincoln

Standing beneath the vaulted ceilings of Lincoln Cathedral, I was struck by the scale of what people can build across generations. Stone laid upon stone, intention upon intention, each generation adding something lasting to something far larger than itself.

It made me wonder: building things that outlast us requires ultra-long-form attention that seems incompatible with today’s ‘notification economy’.

Lincoln Cathedral

The real distance

The kilometres on the bike were only the visible measure of the journey. They were scaffolding.

The real distance was mental: the slow clearing that comes from sustained movement and an open uncluttered mind.

Whether in Newton’s orchard, beneath the dreaming spires of Oxford, inside the stone vastness of Lincoln Cathedral, or in the quiet presence of Florence, the same idea kept returning in different forms:

We think best—and perhaps live best—when we step out of the noise and give ourselves time and space.