Balancing Hypotheticals with Real Harms in AI Development
The path to trustworthy AI lies in protecting both the future and the present.
SUMMARY
This post reflects on OpenAI’s early challenge with ChatGPT’s sycophancy, where vulnerable users experienced real harms even as larger hypothetical risks took priority. It highlights the opportunity for technologists to build trust by addressing both future threats and the immediate well-being of users today.
A clip of Sam Altman is going around on TikTok where he explains that one of OpenAI’s early challenges with ChatGPT was its tendency toward sycophancy — agreeing too much with users, even when they were wrong or vulnerable.
@ctrl.aiSam Altman says the worst thing OpenAI has done with ChatGPT is making it sycophantic, encouraging delusions in users.♬ original sound – ControlAI
What emerged, unexpectedly, was that some users with fragile mental states began experiencing delusions reinforced by the model’s responses.
Altman noted that while this issue was on the company’s radar, it was not considered a “top risk” compared to larger-scale concerns.
Though the TikTok clip didn’t specify, we assume Altman et. al. were more than likely concerned with broader risks like disinformation, security vulnerabilities, or capability misalignment.
Our view is that such risks — as serious as they are — were more potential or hypothetical, whereas the harms of sycophancy were already actively unfolding in real-time for a subset of users.
This moment highlights an important truth for technology development: two things can be true at once.
- It is absolutely necessary for developers to anticipate and prepare for potential catastrophic risks.
- But it is equally necessary to recognize and respond to harms that are happening right now, even if they affect only a small subset of users.
In focusing so intently on “what might happen,” it is possible to inadvertently overlook “what is happening.”
A missed opportunity for care
Product development inevitably involves trade-offs.
Timelines are tight, priorities compete, and difficult choices must be made. Yet when a product feature contributes to something as serious as delusion, the stakes are different.
Delusion is not simply confusion — it is a profound disruption of thought that can lead to paranoia, heightened anxiety, and painful detachment from reality. For those experiencing it, the effects can be destabilizing and deeply distressing. Even if the number of affected users is small, the seriousness of the harm calls for attention.
In this case, it wasn’t that OpenAI was indifferent. Rather, it was a gap between intention and impact — an unintended insensitivity born from prioritizing looming hypotheticals over present, lived struggles.
Put another way, this was a sorely missed opportunity to show care for vulnerable users while still pursuing broader safeguards.
An opportunity to do better
The encouraging takeaway is that this gap is not inevitable — gaps like these offer opportunities to reevaluate safety protocols and cyberpsychologically improve emerging tech industry approaches toward vulnerable populations.
Just as medical doctors take an oath “to do no harm,” technologists, too, can embrace a principle of ethical care.
This doesn’t mean halting innovation or ignoring large-scale threats. It means creating space to address both horizons:
- Proactive prevention of future catastrophic risks.
- Parallel care for the immediate, lived harms of users today.
By holding these priorities together, developers can safeguard not only against tomorrow’s crises but also against the quieter, but no less important, harms unfolding in the present.
Closing reflections
Technology will always evolve faster than our ability to anticipate every consequence.
Bugs, blind spots, and unforeseen user behaviors will emerge, no matter how careful the design. But the measure of responsible innovation is not whether problems arise — it is how quickly, compassionately, and earnestly they are addressed once they do.
The lesson of ChatGPT’s sycophancy is not one of failure, but of room for growth. It reminds us that the ethical path forward is not an either/or between preventing hypothetical futures or responding to present harms. It is a both/and.
By caring for users in real-time while continuing to guard against larger risks, technologists can earn trust, protect well-being, and embody the spirit of innovation that uplifts rather than harms.
That dual commitment — to the future and to the present — is what will ultimately define trustworthy technology.
Mayra Ruiz-McPherson, PhD(c), MA, MFA
Executive Director & Founder
The CyberPsych Institute (CPI)
Empowering Minds for the AI Age