Brilliant breakdown — especially appreciated how you tied it back to RLHF and the deeper human bias underneath it.
The part that hits hardest? It’s not just the models that are sycophantic — we’ve trained ourselves to reward it. Even offline, most people aren’t wired to tolerate uncomfortable truth for long.
Do you think the solution is technical (prompting, model tuning) — or cultural (teaching people to want truth over comfort)? Or is that just wishful thinking?
If a solution involves convincing a majority of people all over the world to change their behaviour, that is never going to happen. You can convince a few people, never a large number. So it has to be technical. Or changing yourself (i.e. become more mindful about how you use LLMs and how you interpret the results.)
Good point about mass behavior change being unrealistic. But I keep thinking about how social media has amplified this exact problem - the engagement metrics actively reward bandwagon thinking over independent analysis.
Even if we solve AI sycophancy technically, we're still operating in information ecosystems designed to discourage the very mindfulness you're talking about. How do you maintain critical thinking when the algorithms reward conformity and the feedback loops punish uncomfortable questions?
Maybe the real challenge isn't just training better models - it's that we've built platforms that systematically erode our capacity for the kind of thinking we'd need to use those models well.
Thank you for writing this. It's a useful cautionary tale/reality check.
Brilliant breakdown — especially appreciated how you tied it back to RLHF and the deeper human bias underneath it.
The part that hits hardest? It’s not just the models that are sycophantic — we’ve trained ourselves to reward it. Even offline, most people aren’t wired to tolerate uncomfortable truth for long.
Do you think the solution is technical (prompting, model tuning) — or cultural (teaching people to want truth over comfort)? Or is that just wishful thinking?
If a solution involves convincing a majority of people all over the world to change their behaviour, that is never going to happen. You can convince a few people, never a large number. So it has to be technical. Or changing yourself (i.e. become more mindful about how you use LLMs and how you interpret the results.)
Good point about mass behavior change being unrealistic. But I keep thinking about how social media has amplified this exact problem - the engagement metrics actively reward bandwagon thinking over independent analysis.
Even if we solve AI sycophancy technically, we're still operating in information ecosystems designed to discourage the very mindfulness you're talking about. How do you maintain critical thinking when the algorithms reward conformity and the feedback loops punish uncomfortable questions?
Maybe the real challenge isn't just training better models - it's that we've built platforms that systematically erode our capacity for the kind of thinking we'd need to use those models well.