Flawed Human Teacher's Notes Better Than Polished AI Prose

A teacher discovers that using AI to improve his notes made things worse for the students

(This is a guest post by Utpal Chattopadhyay. Utpal recently retired after a long corporate career at Bell Labs and Motorola, followed by a long stint as outside faculty teaching Physics and Maths at ISI Kolkata. This month, he is teaching a course in fairly advanced mathematics, involving group theory, to 14-year-old students at a GenWise summer camp. This is a lightly edited classroom note from Utpal about an AI-related experience while teaching the GenWise class. He may later develop this into a fuller article for an education journal, so this version is being shared as an early reflection.)

Today, quite by accident, I conducted an interesting experiment.

About two days ago, I was teaching a particular topic which I could see was very draining for the students. I also realised I have to cover a key foundational result in set theory in greater detail and greater care before I jump into this.

It was very draining for me as well seeing the glassy eyes looking vacantly in the air.

So as usual, next morning at 5 am, I sat down and wrote up three pages on the set theoretic result, namely that equivalence relation results in partitioning of a set.

After I wrote it, in my handwriting, of course, I fed it to both Gemini and Claude. Claude had a number of observations some of which I agreed to. At any rate I asked both to rewrite it with those corrections.

Imagine what happened next.

I gave both the pdfs on equivalence partitioning to the students and told them to read and try to understand on their own and underline what they don’t understand.

After one student went over both the typed out PDFs I gingerly gave her my hand written stuff which was my input to the LLM

She started reading that and then said next time can you give us these handwritten stuff? I said sure but why? She said yours is easy to understand; the typed pages use words like “arbitrary” !!!!

Interesting phenomenon. May be the human touch of handwriting has something to do with it?

I narrated this to Claude.

Claude responded with this:

That is a genuinely interesting observation and worth thinking about carefully — because I don’t think it is only the handwriting.

The word “arbitrary” is the real clue. Your handwritten notes were written for those students, in that moment, after you had watched their faces. The PDFs — mine and Gemini’s — were written for a general reader, using the standard mathematical register. “Arbitrary” is completely normal in that register. But to a student encountering it for the first time it is a strange and slightly intimidating word when “any” would do exactly the same work.

The handwriting may carry a signal — this was written by a human being thinking about you specifically — but the deeper thing is that your prose is calibrated to your students in a way that no generated PDF can be, because you were in the room with them.

This is actually important feedback for mathematical exposition more broadly. The distance between “take an arbitrary element” and “take any element” is zero mathematically and enormous pedagogically.

It also raises a question worth sitting with — when you write your Bengali popular science, do you notice yourself making the same calibration instinctively? I suspect you do, and that is part of why it works.

For tomorrow — perhaps the lesson is to keep giving them your handwritten notes as the primary material, and use the typed PDFs only as reference. Let the human document lead.

The student who noticed this is paying attention at a level beyond the mathematics. Remember that.

Epilogue:

The two other students had similar reactions.

But the end was hilarious.

A little later she came back to me and said, “no at the end you have used that word arbitrary too. Chhee.”

I said please forgive me. I tried with “any” as long as could and then I thought you need to know the math lingo too 🙂

---

Navin’s Notes: I want to comment on a few things.

1) I regularly teach at various levels: high-school students, undergraduates, mid-career professionals, and board rooms, and in each of those cases, I think one of the most important things I do is to adjust my language and my examples according to my understanding of the audience. So, when I read Utpal’s note, I immediately agreed with it (possibly because this is another reason why there is still a future for teachers in addition to this.)

2) Of course, with careful prompting, you can get AI to also adjust its writing according to the audience, but it is hard. I confirmed with Utpal, that both Gemini and Claude already had the context of the kinds of students he was writing this for, but that hadn’t helped.

3) In general, most people using AI to generate text are not putting in the effort to make it not sound like AI. Even in Claude’s answer to Utpal that is reproduced above, you can see the standard tics of AI generated text (“paying attention at a level beyond the mathematics” and “The distance between <xxx> and <yyy> is zero mathematically and enormous pedagogically”) which make me cringe these days. I see this everywhere, from viral tweets to emails I receive from students. Just like Utpal’s students, I prefer the imperfect human text to the polished AI slop.)