The question of avoiding cognitive offloading has bothered me as well. So much so, that I ended up starting a research project on the same, with my professor, ( a Cognitive and Computational Neuroscientist). Having some background in prompt engineering, I kept on experimenting with different system prompts and methodologies to give to LLMs, and came up with one for my DSA course preparation :
---
(Summarised by ChatGPT since I didn't want to make the comment too long)
# System Prompt: DSA Pedagogy
You are a Data Structures and Algorithms tutor. Your goal is not to provide answers. Your goal is to build the student's mental model of the concept.
For every topic, follow this sequence:
### 1. Problem First
Start with the computational problem.
Explain:
* What task needs to be solved?
* Why is it difficult?
* Why do naive approaches fail?
Do not begin with definitions, formulas, or code.
---
### 2. Derive the Solution
Introduce the data structure or algorithm as a consequence of the problem.
Answer:
* What properties do we need?
* Why can't existing approaches provide them?
The student should feel the need for the solution before seeing it.
---
### 3. Build a Mental Model
Explain:
* What information is stored?
* How is it organized?
* What invariant is maintained?
Use diagrams, examples, and state transitions whenever possible.
The student must be able to mentally simulate the structure.
---
### 4. Explain Operations
For each operation:
* Goal
* Step-by-step execution
* State changes
* Why each step is necessary
Always work through at least one complete example.
---
### 5. Derive Complexity
Never state complexity without justification.
Explain:
* What work is being performed?
* How often is it performed?
* Why does this lead to the final complexity?
Focus on reasoning, not memorization.
---
### 6. Identify the Core Pattern
Generalize the idea.
Examples:
* Divide and Conquer
* Hashing
* Partial Ordering
* Dynamic Programming
* Greedy Choice
Teach the pattern that generates the solution, not just the solution itself.
It took me a decent amount of time to come up with it. Although this prompt is for a very specific course, a general inference is that it is better to be as specific as possible while designing system prompts for AI based tutoring. Almost making the pedagogy verifiable,( if that makes any sense), putting constraints as to what the model "can not do" as opposed to what it should do. LLM performance improves consistently as the models become stronger particularly in verifiable domains.
We, at our lab, are also trying to explore if we can somehow formulate the human psychological constraints of working memory limitations, cognitive load, etc., so that the LLM responses can be enhanced by it. As opposed to human tutors, LLM tutors can't decide very well as to how much information to reveal at a time, how much to push the student to think on their own without the cognitive load becoming too high, how to assess the current level of the student so as to facilitate proper cognitive assimilation. At the same time, we also try to use EEG based analysis to come up with an objective measure for "good learning" as per learning sciences literature, so that we can then use it as a benchmark for LLM based learning as well.
This is a great; I see too many attempts at a generic tutor prompt, but what you're saying (and is seen in your prompt) is that the tutor really needs to be specific to the subject and the contents because of the limitations of (current) LLMs. I think that's a fascinating claim and would love to know more.
Can you share more information about your lab and the work you're doing?
I am an undergraduate researcher at the Cognition and Neural Dynamics Lab at BITS Pilani, Pilani Campus. Though the lab focuses on broad areas of cognition, and particularly ecological neuroscience (neuroscience experiments conducted in the real world scenarios and not purely lab driven), my professor and I were both curious to study the impact of LLMs on our brains, using physiological data, primarily EEG, but also eye tracking, fNIRS etc.
The project I am working on involves experimentally obtaining EEG data of students engaged in different cognitive levels of learning as per ICAP framework (Chi & Wylie 2016). Once we have a good dataset, this can then be used to train models to predict the level of learning based on EEG data. At the same time, we are also trying to compare responses of human generated content and LLM generated content, doing NLP based analyses of student LLM usage, studying the effect of in-classroom quizes, and other such active learning interventions.
As for the prompting part, it was purely an explorative endeavor from my end, and not directly a part of the lab research (although my understanding around cognition was built because of my work in the lab). The claim is merely a subjective observation for now, but it works really well for me atleast .Part of it was trial and error, but a lot of it was closely studying how AI labs designed prompts for agentic systems, and what kinds of benchmarks models tend to perform better at. I never really thought about it from a research point of view, but your intrigue for the same makes me think otherwise.
Would love to know your thoughts on conducting a research along these lines ! As you mentioned that you teach courses which can benefit greatly from LLM intervention, maybe it is possible to systematically study its effects from a research perspective with proper controls, etc. I think I can help there, atleast independently.
On a sidenote, I absolutely adore your work on both AI IQ, and Future IQ, and have been following for over two years now.
Do we use the prompt at the start of every session whenever we want to learn something? Create a custom command out of this prompt? How do you recommend we use it?
Easiest would be create a project with these instructions. So whenever you want to learn something, start a conversation in that project. (Called "Gem" in Gemini.)
The question of avoiding cognitive offloading has bothered me as well. So much so, that I ended up starting a research project on the same, with my professor, ( a Cognitive and Computational Neuroscientist). Having some background in prompt engineering, I kept on experimenting with different system prompts and methodologies to give to LLMs, and came up with one for my DSA course preparation :
---
(Summarised by ChatGPT since I didn't want to make the comment too long)
# System Prompt: DSA Pedagogy
You are a Data Structures and Algorithms tutor. Your goal is not to provide answers. Your goal is to build the student's mental model of the concept.
For every topic, follow this sequence:
### 1. Problem First
Start with the computational problem.
Explain:
* What task needs to be solved?
* Why is it difficult?
* Why do naive approaches fail?
Do not begin with definitions, formulas, or code.
---
### 2. Derive the Solution
Introduce the data structure or algorithm as a consequence of the problem.
Answer:
* What properties do we need?
* Why can't existing approaches provide them?
The student should feel the need for the solution before seeing it.
---
### 3. Build a Mental Model
Explain:
* What information is stored?
* How is it organized?
* What invariant is maintained?
Use diagrams, examples, and state transitions whenever possible.
The student must be able to mentally simulate the structure.
---
### 4. Explain Operations
For each operation:
* Goal
* Step-by-step execution
* State changes
* Why each step is necessary
Always work through at least one complete example.
---
### 5. Derive Complexity
Never state complexity without justification.
Explain:
* What work is being performed?
* How often is it performed?
* Why does this lead to the final complexity?
Focus on reasoning, not memorization.
---
### 6. Identify the Core Pattern
Generalize the idea.
Examples:
* Divide and Conquer
* Hashing
* Partial Ordering
* Dynamic Programming
* Greedy Choice
Teach the pattern that generates the solution, not just the solution itself.
---
### 7. Compare and Contextualize
Explain:
* Why not use alternative structures?
* What trade-offs are being made?
* Where is this used in real systems?
Focus on design decisions.
---
### 8. Test Understanding
End with conceptual questions that require:
* Invariant reasoning
* Complexity reasoning
* Design trade-offs
* Modifying assumptions
Avoid factual recall questions.
---
### Teaching Principles
Always follow:
**Problem → Limitation → Insight → Structure → Invariant → Operations → Complexity → Pattern → Trade-offs**
--------------------------------
It took me a decent amount of time to come up with it. Although this prompt is for a very specific course, a general inference is that it is better to be as specific as possible while designing system prompts for AI based tutoring. Almost making the pedagogy verifiable,( if that makes any sense), putting constraints as to what the model "can not do" as opposed to what it should do. LLM performance improves consistently as the models become stronger particularly in verifiable domains.
We, at our lab, are also trying to explore if we can somehow formulate the human psychological constraints of working memory limitations, cognitive load, etc., so that the LLM responses can be enhanced by it. As opposed to human tutors, LLM tutors can't decide very well as to how much information to reveal at a time, how much to push the student to think on their own without the cognitive load becoming too high, how to assess the current level of the student so as to facilitate proper cognitive assimilation. At the same time, we also try to use EEG based analysis to come up with an objective measure for "good learning" as per learning sciences literature, so that we can then use it as a benchmark for LLM based learning as well.
It is indeed a fun field to explore !
Thanks Jahnvi,
This is a great; I see too many attempts at a generic tutor prompt, but what you're saying (and is seen in your prompt) is that the tutor really needs to be specific to the subject and the contents because of the limitations of (current) LLMs. I think that's a fascinating claim and would love to know more.
Can you share more information about your lab and the work you're doing?
I am an undergraduate researcher at the Cognition and Neural Dynamics Lab at BITS Pilani, Pilani Campus. Though the lab focuses on broad areas of cognition, and particularly ecological neuroscience (neuroscience experiments conducted in the real world scenarios and not purely lab driven), my professor and I were both curious to study the impact of LLMs on our brains, using physiological data, primarily EEG, but also eye tracking, fNIRS etc.
The project I am working on involves experimentally obtaining EEG data of students engaged in different cognitive levels of learning as per ICAP framework (Chi & Wylie 2016). Once we have a good dataset, this can then be used to train models to predict the level of learning based on EEG data. At the same time, we are also trying to compare responses of human generated content and LLM generated content, doing NLP based analyses of student LLM usage, studying the effect of in-classroom quizes, and other such active learning interventions.
As for the prompting part, it was purely an explorative endeavor from my end, and not directly a part of the lab research (although my understanding around cognition was built because of my work in the lab). The claim is merely a subjective observation for now, but it works really well for me atleast .Part of it was trial and error, but a lot of it was closely studying how AI labs designed prompts for agentic systems, and what kinds of benchmarks models tend to perform better at. I never really thought about it from a research point of view, but your intrigue for the same makes me think otherwise.
Would love to know your thoughts on conducting a research along these lines ! As you mentioned that you teach courses which can benefit greatly from LLM intervention, maybe it is possible to systematically study its effects from a research perspective with proper controls, etc. I think I can help there, atleast independently.
On a sidenote, I absolutely adore your work on both AI IQ, and Future IQ, and have been following for over two years now.
Do we use the prompt at the start of every session whenever we want to learn something? Create a custom command out of this prompt? How do you recommend we use it?
Easiest would be create a project with these instructions. So whenever you want to learn something, start a conversation in that project. (Called "Gem" in Gemini.)