The Comfort Trap: Why AI therapy may not be the emotional support we think it is

AI chatbots promise empathy at the click of a button, offering comfort, validation and advice whenever we need it. But can a machine that is designed to agree with us ever replace the difficult, uncomfortable work of therapy? Bhavya K Mehta explores where AI-led counselling helps, where it falls short, and why convenience may come at the cost of genuine human connection. 

“I had an argument with a friend. What should I do?”

A chatbot replies within seconds. It reassures you, validates your feelings, and offers thoughtful advice. The next day, you return with another problem. Then another. Gradually, the chatbot becomes the first place you turn to for difficult conversations. It never judges. It never disagrees. It is always available. Before long, what began as convenience starts to feel like dependence.

This is the comfort trap.

This is no longer a hypothetical scenario. AI chatbots such as ChatGPT, Claude, and Gemini have rapidly evolved into spaces where people seek answers, companionship, and increasingly, emotional support. They have become classrooms, assistants, brainstorming partners, and, for some, even confidants. Alongside them, a growing number of mental health and wellness apps now encourage users to journal their thoughts, complete self-assessments, and receive AI-generated insights into their emotional well-being.

Today, AI puts on many hats, it teaches, manages, calculates and summarises as well all at mind-blowing speed. When we enter a string of words, it can role play as a sentient entity, adapting and performing whatever we require from it. Every response has a personalised touch, it shows initiative to keep helping us, validates our statements and conjures seemingly relatable analogies to keep us engaged. The constant reassurance and validation seems so new, so necessary at the moment. There has also been an influx of mental health and wellness apps, which mainly work with self-report tests and journals. The journal entries are further analysed by AI models which produce potentially useful insights.

It is safe to say that AI has conveniently created a space for itself in our personal lives and our minds. However,  even after looking at the wonders of it all, is it appropriate to consider these technological interventions as clinically sound? Can a program functioning on Os and 1s ever truly hear us, support us and guide us through emotional distress?

For many users, these interactions end where they should: as a convenient source of guidance or reassurance. But for some, the relationship with AI begins to extend far beyond that. Concerns about emotional dependence on AI are becoming increasingly common. In 2024, widespread media attention was drawn to the case of 14-year-old Sewell Setzer III from Florida, whose family later filed a lawsuit against Character.AI. Reports suggested that he had developed a strong emotional attachment to an AI chatbot before his death by suicide. While no single factor can explain such a tragedy, the case sparked serious discussions about the role of AI companions in the lives of vulnerable users.

What makes situations like this concerning is not simply the technology itself. It is the possibility that users begin viewing AI systems as substitutes for human relationships, professional support, or crisis intervention. Chatbots can provide responses, but they cannot assess risk, take responsibility, or intervene in the way a trained mental health professional can. As AI becomes more advanced, society will need to think carefully about where convenience ends and responsibility begins. Emotional support is not only about responding to words. It is about understanding context, recognizing distress, and knowing when someone needs real human help.

The core strength of AI chatbots and Mental Health Apps is their ability to comprehend and replicate spoken, written and even visual language. When any kind of prompt is put into the chat or the journal, the program scours the internet and its own training models to pull out the most relatable, agreeable and context-aware answer which feels perfect for the moment.

Through simple pattern recognition, it provides seemingly intuitive but generic statements, which are then internalised by the person, providing momentary validation and relief. Sure, there is substance behind the advice of these chatbots. They are trained to respond using persuasive strategies, which will make you accept an entirely new facet of the situation under dissection.

They can help reframe negative emotions and perceptions, and owing to its technological presence, provide round-the-clock support. These are learning models, which actively learn a user’s language pattern and interact in a personalised manner with tailored responses.

The biggest factor underlying their immense popularity is the ease with which these tools and apps can be accessed. With appropriate clinical training, these programs can function as a supportive “Pocket Therapist”, who can role play as a therapist or a friend, whatever the user demands it to be.A chatbot is designed to respond. It keeps the conversation moving. It validates, reassures, and mirrors what the user is saying. In the moment, that can feel comforting. After a difficult day, having something that instantly replies and appears understanding can be incredibly appealing.

Mental health chatbots are becoming part of the growing ecosystem of digital wellbeing, complementing traditional sources of support.

The problem is that growth is not always comfortable. Real support sometimes involves disagreement, accountability, and difficult conversations. A therapist may challenge unhealthy thought patterns. A friend may point out when we are avoiding a problem. A family member may encourage us to take action even when it feels uncomfortable.

Chatbots rarely do this effectively. Their primary function is to continue the interaction. As a result, there is a risk that people begin confusing validation with genuine support. Feeling heard is important, but meaningful emotional growth often requires more than simply hearing what we want to hear. This raises an uncomfortable question: if a system is designed to keep us engaged, can it truly be considered a replacement for relationships that are intended to help us become independent and emotionally resilient?

While it’s difficult to completely disregard the capability of AI based mental health interventions, it can in no way substitute the presence and advice of an actual living person.

When reaching out for support, the primary thing which drives the interaction is not what the conversation is about, but how it is happening. During a genuine conversation with a mental health provider or even a loved one, there is a sense of interpersonal alliance. The shared trustworthiness fosters understanding, empathy and can also motivate the person in crisis to recognise and seek professional help. Along with verbal communication, the physical presence

of the other person, or a certain level of visual access is a playground for non-verbal communication. The necessary cues which are picked up during such conversation helps build a shared understanding of the discussion, where both parties end up sharing a moment of vulnerability, and in some cases, resilience.

But why would human interactions and emotional support systems matter in a conversation about Mental Health AI bots? An important factor which significantly sets Human and AI mental wellness interactions apart, is a person’s ability of perspective-taking. A human mental health practitioner is professionally trained in this, and is adept at being judgement and bias free. The fact that they have class consciousness, or in other words, understand the nuances and socio-cultural context of their client adds another layer of context which needs to be taken into consideration when trying to process someone’s emotions and frustrations.

No matter how good AI is at recommending self-help practices, or using pleasantries to keep the user satisfied, it will never tell the person what they ought to know, but rather comply precisely with what the user is looking for. An AI Bot will give redundant and generic advice, amplify the wrong factor and keep perpetuating the bias or mindset which caused the crisis in the first place. While a therapist will point out how you are wrong, an AI bot will develop the perfect echo chamber, validating everything right and wrong. Furthermore, the user will tweak the prompts and chats to make the bot easier, more agreeable, and get disconnected from the gaping blind spot.

The dangerous thing is that AI does this sneakily. With companies like Meta which are dominating almost all social media platforms today, it indirectly has access to all our information.

Our friends, our job, our family, our interests. If they dig deep enough, the inadequacies and insecurities are served on a platter, which is then conveniently handed over to third-party companies, and obviously Meta AI. This raises several privacy concerns, our data is still processed and brought up in these chats to make them feel even more personal. Additionally, the legal terms of these AI models require them to report highly inappropriate or destructive texts. When this happens with a high-risk individual who is conversing with AI, the text is flagged and the user is unable to further communicate. This sudden brick wall breaks the agreeable tone of the conversation, which can be ineffective, if not detrimental for the user.

The hours and hours of chatting eventually cause a loss of personal contact and feed into social isolation. The person becomes so dependent on AI bots for help that he/she loses the capability of decision-making and conflict resolution. The accessibility of AI, which captured interest in the first place, turns out to be the reason behind the downfall of our basic emotional intelligence and cognitive capacities. This feels almost anticlimactic. And ironic, as AI chatbots start having the complete opposite effect on the primary intention, to form genuine connections and get emotional support. Looking at the questionable proceedings of seeking support from an AI bot, does it still feel worthwhile to have that “Pocket Therapist” for relief whenever and wherever? Does it still feel right to reject genuine professional therapy and resort to pseudoscientific technological tools for your pursuit of happiness?