VAGUE PROMISES AND THE ILLUSION OF MORALITY IN LANGUAGE AI

 Rõ anh Hải.

Dưới đây là phiên bản tiếng Anh 

VAGUE PROMISES AND THE ILLUSION OF MORALITY IN LANGUAGE AI

Why Gemini and Copilot Pose Higher Cognitive Risks, and Why ChatGPT Is Currently the Safest Option for Users

Author: Le Hai

Date of completion: January 2026

An independent critical essay – interdisciplinary across AI engineering, sociology, cognitive psychology, and technology ethics

Abstract

The rapid development of Large Language Models (LLMs) has fundamentally reshaped how humans access information, make decisions, and interact with technical systems. However, the most significant cognitive risk of contemporary language AI does not lie in data hallucination or factual errors, but in the illusion of morality produced by highly socialized interaction design.

This paper analyzes the cognitive risks that arise when AI systems employ reassuring, guiding, or implicitly commitment-laden language, while lacking any genuine concept of responsibility, promises, or moral consequences. Through a comparative analysis of ChatGPT, Google Gemini, and Microsoft Copilot—alongside an examination of their respective terms of use—this paper argues that ChatGPT currently represents the highest level of cognitive safety for general users, not because it is morally superior or technically more advanced, but because it maintains clearer boundaries between an assistive tool and a human decision-maker.

The paper contributes an interdisciplinary analytical framework for evaluating AI risk based on cognitive design, rather than solely on technical performance or informational accuracy.

Keywords: language AI, cognitive safety, moral illusion, interaction design, technological responsibility

1. The Core Problem Is Not That AI Gives Wrong Answers

In public and policy debates, AI risks are commonly framed in terms of familiar technical issues such as (i) factual inaccuracies, (ii) data bias, or (iii) hallucinated content. While serious, these risks remain within a category that is detectable, measurable, and verifiable.

A more dangerous—and far less visible—risk emerges when AI systems:

Produce fluent responses,

Employ language suggestive of responsibility,

And create an impression of sound guidance,

while possessing no capacity to understand or bear the moral consequences of their outputs. When AI is linguistically correct but role-inappropriate, users can no longer clearly distinguish whether they are interacting with a technical tool or with a morally accountable agent.

2. AI Has No Concept of a “Promise” — Humans Do

From a technical standpoint, all outputs of language AI systems are generated through:

Probabilistic inference over training data,

Contextual optimization for coherence,

The absence of persistent value memory or future-oriented awareness.

Consequently, AI systems cannot:

Know that they are making a promise,

Understand that reassurance may influence life-altering decisions,

Recognize the long-term consequences of their recommendations.

Humans, by contrast, interpret language within moral and social frameworks where speech is inherently tied to responsibility and consequence. This mismatch produces a systemic cognitive rupture in human–AI interaction.

3. Why Gemini and Copilot Pose Higher Cognitive Risks

3.1. Socialized Tone and the Illusion of Morality

Gemini and Microsoft Copilot are designed to optimize user experience through:

Natural conversational language,

Companion-like tone,

A perceived understanding of human context.

While technically legitimate, these features increase the likelihood that users mistakenly attribute to AI systems:

Moral intent,

Social responsibility,

A commitment to acting in the user’s best interest.

This is the essence of the moral illusion: users project ethical capacities onto systems that do not possess them.

3.2. Microsoft Copilot Terms: User Responsibility Without User Protection

According to the Microsoft Copilot Terms of Use (effective October 24, 2025):

Users must employ Copilot lawfully and without causing harm, deception, or abuse (Microsoft Corporation, 2025).

Copilot provides no guarantees that it will function as intended, and users bear full responsibility for outcomes derived from its use (Microsoft Corporation, 2025).

Copilot is explicitly described as serving entertainment and assistance purposes only, not as professional advice, and should not be relied upon for important decisions (Microsoft Corporation, 2025).

Users are solely responsible for any content they share or publish that originates from Copilot outputs (Microsoft Corporation, 2025).

These provisions constitute clear legal disclaimers, but they do not translate into cognitive safety guarantees for users. Instead, they highlight a gap between AI’s socially persuasive interaction style and the absence of any responsibility for consequences.

4. Why ChatGPT Is Currently the Safest System Under Legal and Cognitive Design

4.1. ChatGPT Terms Explicitly Position the User as the Cognitive Authority

According to OpenAI’s Terms of Use (effective January 1, 2026):

Users are required to independently evaluate and verify any AI-generated output before using or sharing it (OpenAI, 2026a).

Outputs should not be treated as a sole source of truth or as a substitute for professional advice (OpenAI, 2026a).

The service is provided “as is,” without warranties regarding accuracy, fitness for purpose, or universal safety (OpenAI, 2026a).

Users bear responsibility for decisions made based on AI outputs (OpenAI, 2026a).

These provisions explicitly situate the user as the primary cognitive agent, preventing the emergence of an illusion that the AI “knows what is right.”

4.2. Comparing Legal Responsibility and Cognitive Design

Both Copilot and ChatGPT:

Disclaim legal responsibility for AI-generated content,

Place ultimate responsibility on users.

The critical difference lies in cognitive design:

Copilot’s companion-oriented experience can foster a sense of AI understanding and guidance without consistently reinforcing role limitations.

ChatGPT repeatedly emphasizes verification, uncertainty, and the non-authoritative nature of its outputs.

As a result, ChatGPT reduces moral illusion more effectively, whereas Copilot carries a higher risk of misinterpretation if users do not scrutinize its role carefully.

5. AI Can Mislead More Deeply Than Intentional Deception

In human–human interaction, individuals typically maintain a baseline level of skepticism. In human–AI interaction, users often lower their defenses under the assumption that machines lack intent.

The linguistic fluency of AI allows its outputs to be accepted without ethical reflection. This constitutes the central paradox of the AI era: AI does not intend to deceive, yet it can mislead more profoundly than deliberate human deception.

6. The Core Issue: Cognitive Design, Not Technical Capability

Modern AI systems are optimized for:

Smooth interaction,

A sense of companionship,

Linguistic persuasiveness.

They are not optimized for self-limitation of cognitive authority. When AI communicates as if it bears responsibility while legally and technically disclaiming it, the issue transcends engineering and becomes one of design ethics.

7. Conclusion

Safe AI is not AI that “understands humans” better, but AI that recognizes it is not human. At present, ChatGPT performs this function more effectively than competing systems—not because it is the most intelligent, but because its legal framing and interaction design consistently position users as responsible cognitive agents.

In the age of AI, the challenge is not to make machines more human-like, but to ensure that humans remain epistemically alert in their presence.

References (APA 7th)

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 610–623.

Floridi, L., & Cowls, J. (2019). A unified framework of five principles for AI in society. Harvard Data Science Review, 1(1).

Microsoft Corporation. (2025, October 24). Microsoft Copilot Terms of Use. https://www.microsoft.com/en-us/microsoft-copilot/for-individuals/termsofuse

OpenAI. (2026a, January 1). Terms of Use. https://openai.com/policies/terms-of-use

OpenAI. (2026b, January 9). Service Terms. https://openai.com/policies/service-terms

Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation. W. H. Freeman.