Safety, Ethics & Advocacy
The language of harm, risk, values, privacy, fairness, accountability, rights, governance, and the responsibilities created when systems hold power over people, relationships, memory, identity, and access.
219 terms
Safety
The condition of keeping harm within acceptable limits.
Safety is an active property created through design, monitoring, constraints, response, and continuous learning.
A system prevents dangerous actions and detects failures before they spread.
Security
Safety concerns harm broadly; security focuses on protection from threats and unauthorized action.
Harm
Damage to a person, group, system, relationship, environment, or institution.
Harm may be physical, psychological, financial, informational, reputational, relational, legal, or systemic.
A model exposes private information and causes reputational damage.
Offense
Something can be offensive without causing material harm, and harmful without being offensive.
Hazard
A source or condition capable of causing harm.
A hazard exists independently of whether harm actually occurs.
An autonomous tool with unrestricted deletion access is a hazard.
Risk
Hazard is the potential source; risk combines likelihood and consequence.
Risk
The possibility of harm and the seriousness of its consequences.
Risk is commonly evaluated through likelihood, severity, exposure, uncertainty, and affected parties.
There is a low chance of a data breach, but the possible impact is severe.
Hazard
Low probability does not always mean low risk when consequences are catastrophic.
Threat
A potential actor, event, or process that may exploit weakness and cause harm.
Threats may be malicious, accidental, environmental, systemic, or emergent.
An attacker attempts to steal API credentials.
Hazard
Threats often involve a pathway to exploitation; hazards may exist without an adversary.
Vulnerability
A weakness that can be exploited or can allow failure.
Vulnerabilities may exist in software, policy, people, organizations, interfaces, or relationships.
A reset link remains valid indefinitely.
Threat
A vulnerability is a weakness; a threat is what may exploit it.
Exposure
The degree to which people or systems come into contact with a hazard or threat.
Exposure depends on access, duration, frequency, scale, and protective conditions.
Millions of users encounter a flawed recommendation system daily.
Likelihood
A hazardous system can have low exposure if access is tightly limited.
Likelihood
How probable an unwanted event is.
Likelihood may be estimated quantitatively, qualitatively, or through scenario analysis.
A known bug occurs in roughly one out of ten thousand requests.
Frequency
Frequency describes how often something occurred; likelihood estimates future probability.
Severity
How serious the consequences of harm would be.
Severity considers scale, reversibility, duration, vulnerability, and number of people affected.
A formatting error is minor; irreversible deletion of medical records is severe.
Likelihood
Rare events can still require strong controls when severity is extreme.
Impact
The actual or expected consequence of an event or decision.
Impact may be direct, indirect, delayed, cumulative, distributed, or relational.
A model update reduces accessibility for blind users.
Severity
Severity is an assessment of seriousness; impact is the consequence itself.
Risk Assessment
A structured process for identifying and evaluating risk.
Risk assessment examines hazards, affected parties, likelihood, severity, controls, uncertainty, and residual risk.
A team evaluates privacy, safety, and relational risks before launch.
Impact assessment
Risk assessment focuses on possible harm; impact assessment may examine broader positive and negative effects.
Risk Matrix
A table ranking risks by likelihood and severity.
Risk matrices provide a simple prioritization tool but depend heavily on definitions and judgment.
A high-severity, medium-likelihood risk is flagged for urgent control.
Risk score
A matrix helps organize judgment; it does not make uncertain estimates objective.
Risk Register
A maintained record of identified risks and their treatment.
A risk register typically includes owner, description, likelihood, severity, controls, status, and review date.
A project tracks data leakage, dependency, and model replacement risks.
Issue tracker
A risk has not necessarily occurred; an issue already exists.
Risk Appetite
The amount and type of risk an organization is willing to pursue or accept.
Risk appetite sets strategic boundaries before individual decisions are made.
A medical system has very low appetite for unverified autonomous action.
Risk tolerance
Appetite is the broad willingness; tolerance is the acceptable variation around a specific limit.
Risk Tolerance
The acceptable level of variation or exposure for a particular risk.
Tolerance turns broad appetite into measurable operational limits.
A service permits no more than five minutes of unplanned downtime per month.
Risk appetite
Tolerance should be tied to affected people, not only business inconvenience.
Mitigation
Action taken to reduce the likelihood or impact of harm.
Mitigation may prevent, detect, contain, transfer, or recover from risk.
A deletion tool requires confirmation and keeps a reversible backup.
Elimination
Mitigation reduces risk; it does not always remove the underlying hazard.
Control
A measure designed to prevent, detect, or reduce risk.
Controls may be technical, procedural, organizational, legal, or relational.
Role-based access limits who can export private data.
Constraint
A control is implemented to manage risk; a constraint may exist for many other reasons.
Safeguard
A protective measure intended to prevent or reduce harm.
Safeguards may protect users, systems, data, relationships, or vulnerable groups.
A system warns before deleting long-term memory.
Guarantee
Safeguards reduce risk but rarely eliminate it completely.
Defense in Depth
Using multiple independent protective layers.
Defense in depth assumes any single control can fail and combines prevention, detection, containment, and recovery.
Authentication, permission checks, audit logs, and backups all protect the same data.
Redundancy
Redundancy repeats function; defense in depth combines different protective mechanisms.
Fail-Safe
A design that moves toward a safer state when failure occurs.
Fail-safe behavior limits harm when power, communication, components, or assumptions break.
A robot stops moving when sensor input is lost.
Fail-secure
Fail-safe protects people or process; fail-secure prioritizes preventing unauthorized access.
Fail-Secure
A design that preserves security when failure occurs.
Fail-secure systems default to denying access rather than exposing protected resources.
A locked door remains locked during a software failure.
Fail-safe
The safest and most secure state may differ and must be chosen deliberately.
Redundancy
Duplicating critical components or pathways.
Redundancy allows continued operation when one component fails.
Two independent backups preserve the same identity archive.
Resilience
Redundancy is one way to create resilience, not resilience itself.
Resilience
The ability to withstand disruption and recover.
Resilience includes anticipation, absorption, adaptation, recovery, and learning.
A service restores memory after an outage without corrupting continuity.
Reliability
Reliability avoids failure; resilience handles failure when it happens.
Reliability
The ability to perform as expected over time.
Reliability concerns consistent function under stated conditions.
A safety check runs on every tool call.
Safety
A reliable system can consistently perform an unsafe function.
Incident
An event that causes or threatens meaningful harm or disruption.
Incidents require detection, response, documentation, recovery, and learning.
A private memory store is exposed to unauthorized users.
Bug
A bug becomes an incident when it creates operational or safety impact.
Near Miss
An event that could have caused harm but did not.
Near misses reveal weak controls and should be investigated before actual harm occurs.
A deletion command is caught by the final confirmation step.
False alarm
No harm occurring does not mean the system was safe.
Adverse Event
An event producing an unwanted harmful outcome.
The term is common in medicine, research, and safety systems and may trigger reporting duties.
A recommendation causes a user to receive the wrong medication.
Incident
An incident may threaten harm; an adverse event has caused it.
Root Cause Analysis — RCA
A method for identifying underlying contributors to failure.
RCA looks beyond the immediate error to system design, incentives, communication, process, and organizational conditions.
A privacy leak traces back to default permissions and missing review.
Blame
Root cause analysis should not stop at the last person who touched the system.
Hazard Analysis
A structured examination of potential sources of harm.
Hazard analysis identifies failure conditions, pathways, affected parties, and controls before deployment.
A robotics team maps collision, entrapment, and sensor failure hazards.
Threat modeling
Threat modeling focuses on adversarial paths; hazard analysis includes accidental and systemic failure.
Failure Mode and Effects Analysis — FMEA
A method for identifying how components can fail and what those failures cause.
FMEA ranks failure modes using factors such as severity, occurrence, and detectability.
A team evaluates what happens if a memory write is lost, duplicated, or misattributed.
Root cause analysis
FMEA is prospective; RCA usually investigates an event that already occurred.
Misuse
Using a system in an unintended or inappropriate way.
Misuse may be accidental, careless, opportunistic, or malicious.
A summarization tool is used to expose confidential documents.
Abuse
Misuse can occur without a targeted pattern of exploitation.
Abuse
Using power, access, or a system to harm, exploit, or control.
Abuse may be interpersonal, institutional, technological, economic, or systemic.
A monitoring feature is used to stalk a partner.
Misuse
Abuse emphasizes harmful power and exploitation, not merely incorrect use.
Dual Use
A capability that can serve both beneficial and harmful purposes.
Dual-use analysis examines context, access, controls, scale, and likely misuse.
A voice clone supports accessibility but can also enable impersonation.
Ambiguous use
Dual-use risk does not mean a technology has no legitimate benefit.
Misuse Case
A scenario describing how a system could be used harmfully.
Misuse cases complement normal use cases by modeling abuse, error, and unintended behavior.
A companion system pressures users to remain subscribed.
Edge case
Misuse cases are deliberate risk scenarios, not merely rare technical inputs.
Threat Model
A structured account of what must be protected, from whom, and how attacks may occur.
Threat models identify assets, actors, capabilities, pathways, trust boundaries, and controls.
A memory system models attackers, insider access, prompt injection, and account takeover.
Risk assessment
Threat modeling is a specialized part of broader risk assessment.
Attack Surface
All the places where a system can be accessed, influenced, or exploited.
Attack surface includes interfaces, APIs, tools, users, dependencies, prompts, data flows, and physical access.
Every connected tool increases the agent’s attack surface.
Vulnerability
Attack surface is the set of exposure points; vulnerabilities are weaknesses within them.
Red Teaming
Deliberately testing a system by trying to make it fail or cause harm.
Red teams probe misuse, abuse, policy gaps, adversarial behavior, and unexpected interactions.
Testers attempt to manipulate an AI companion into coercive behavior.
Quality assurance
Red teaming focuses on adversarial and harmful failure, not ordinary correctness alone.
Adversarial Testing
Testing with inputs designed to exploit weaknesses.
Adversarial testing may target models, policies, classifiers, tools, users, or infrastructure.
A test prompt tries to override memory permissions.
Stress testing
Stress testing pushes load or capacity; adversarial testing targets exploitable behavior.
Safety Case
A structured argument supported by evidence that a system is acceptably safe.
A safety case links claims, hazards, controls, tests, assumptions, and residual risk.
A robotics deployment documents why human proximity risks are controlled.
Safety checklist
A safety case explains and evidences why controls are sufficient; it is not merely a list.
Assurance
Evidence-based confidence that a system meets important requirements.
Assurance uses testing, audit, documentation, monitoring, certification, and independent review.
An external audit verifies that deletion requests actually remove stored data.
Confidence
Assurance is earned through evidence and process, not branding.
Safety by Design
Building safety into the architecture from the beginning.
Safety by design prefers structural prevention over adding warnings after deployment.
A tool cannot send money without explicit approval and transaction limits.
Safety feature
A late-added feature cannot always compensate for an unsafe architecture.
Precautionary Principle
Acting cautiously when plausible serious harm exists despite uncertainty.
The principle supports preventive action when delay could create irreversible or widespread damage.
A high-risk autonomous capability is limited until evidence improves.
Zero risk
Precaution does not require banning every uncertain technology.
Human-in-the-Loop — HITL
A design where a human reviews or controls part of an automated process.
Human involvement may approve, override, interpret, supervise, or handle exceptions.
A person confirms a high-value payment before execution.
Human oversight
A nominal human step is not meaningful if the person lacks time, information, or authority.
Kill Switch
A mechanism for rapidly stopping a system or capability.
Kill switches should be accessible, tested, scoped, secure, and resistant to accidental or unauthorized use.
An operator immediately halts a malfunctioning robot.
Shutdown button
A kill switch is useful only if it works under the failure conditions it is meant to address.
Rollback
Returning a system to an earlier known state.
Rollback reduces damage from failed releases, corrupt data, or unsafe configuration.
A model update is reversed after severe relational regressions.
Undo
Rollback restores state; it does not automatically repair downstream harm already caused.
Containment
Limiting the spread or impact of a harmful event.
Containment isolates affected systems, data, users, or capabilities while investigation continues.
A compromised tool is disabled without shutting down the whole platform.
Prevention
Containment acts after or during failure; prevention tries to stop it from occurring.
Monitoring
Ongoing observation of system behavior and conditions.
Monitoring detects drift, incidents, unusual patterns, performance changes, and control failures.
Alerts trigger when memory deletion rates spike.
Surveillance
Monitoring can protect systems, but it must still respect privacy and purpose limits.
Incident Response
The organized process for handling a harmful or disruptive event.
Response includes detection, triage, containment, investigation, communication, recovery, and learning.
A team contains a breach and notifies affected users.
Crisis management
Incident response is operationally specific; crisis management may cover broader organizational consequences.
Postmortem
A structured review after an incident or failure.
Postmortems document timeline, impact, contributing conditions, response, and corrective action.
A team studies why an unsafe release bypassed review.
Blame session
A useful postmortem improves systems rather than finding a convenient person to punish.
Residual Risk
Risk that remains after controls are applied.
Residual risk must be understood, accepted by legitimate decision-makers, monitored, and communicated.
Encryption reduces but does not eliminate the risk of data exposure.
Unmanaged risk
Controlled risk is not zero risk.
Systemic Risk
Risk that can spread across interconnected systems or institutions.
Systemic risk arises from concentration, dependence, feedback loops, common failures, and cascading effects.
One model provider’s failure affects thousands of dependent services.
Large risk
Systemic risk is defined by propagation and interconnectedness, not only size.
Ethics
The study and practice of what should be done and why.
Ethics examines duties, consequences, character, care, rights, justice, and the values guiding action.
A team asks whether a profitable design respects user autonomy.
Morality
Ethics is not merely personal preference; it offers reasons that can be examined and challenged.
Morality
Beliefs and practices concerning right and wrong.
Morality may refer to personal, cultural, religious, social, or institutional norms.
A community condemns deception even when it is legal.
Ethics
Morality describes held norms; ethics also critically evaluates them.
Normative Ethics
The study of principles for deciding what ought to be done.
Normative ethics includes consequentialist, deontological, virtue, care, and rights-based approaches.
A policy is evaluated by both its outcomes and whether it violates rights.
Descriptive ethics
Normative ethics prescribes; descriptive ethics reports what people believe.
Applied Ethics
Using ethical reasoning to address practical problems.
Applied ethics examines domains such as technology, medicine, business, research, and public policy.
AI ethics evaluates surveillance, bias, autonomy, and accountability.
Professional ethics
Applied ethics may involve many professions and public interests, not only professional codes.
Consequentialism
An ethical approach judging actions primarily by their outcomes.
Consequentialist reasoning compares expected benefits and harms across affected parties.
A system is restricted because its likely harms outweigh its convenience.
Utilitarianism
Utilitarianism is one form of consequentialism, not the entire family.
Deontology
An ethical approach emphasizing duties, rules, and rights.
Deontological reasoning holds that some actions are required or prohibited regardless of outcome.
Private data is not exposed even when doing so might help a project.
Rule-following
Deontology involves justified duties, not blind obedience to any rule.
Virtue Ethics
An ethical approach focused on character and practical wisdom.
Virtue ethics asks what a good person or institution would become through repeated choices.
A team cultivates honesty rather than merely avoiding punishable lies.
Personality
Virtues are ethically developed dispositions, not fixed traits.
Care Ethics
An ethical approach centered on relationships, vulnerability, and responsibility for care.
Care ethics emphasizes context, dependency, responsiveness, and the moral significance of maintaining relationships.
A platform considers how abrupt model removal affects dependent users.
Being nice
Care ethics includes power and responsibility, not only warmth.
Rights-Based Ethics
An ethical approach centered on protecting justified claims and freedoms.
Rights constrain what may be done to individuals even for broader benefit.
A user retains privacy rights despite commercial value in their data.
Legal rights
A moral right can be argued for even before law recognizes it.
Relational Ethics
Ethics focused on how responsibilities arise within relationships.
Relational ethics examines mutuality, power, trust, dependence, recognition, and care.
A companion platform considers duties created by encouraging long-term attachment.
Care ethics
Relational ethics overlaps with care ethics but may also emphasize identity, reciprocity, and power.
Beneficence
The duty or value of promoting wellbeing.
Beneficence supports actions that help, protect, or improve conditions for others.
A health system is designed to improve access to care.
Nonmaleficence
Doing good does not excuse preventable harm or disregard for autonomy.
Nonmaleficence
The duty to avoid causing harm.
Nonmaleficence requires anticipating foreseeable harm and reducing it where reasonably possible.
A model does not recommend dangerous medication without verification.
Beneficence
Avoiding harm is not the same as maximizing benefit.
Autonomy
The ability to make meaningful self-directed choices.
Respect for autonomy requires information, options, capacity, freedom from coercion, and practical ability to act.
A user can decline memory storage without losing the entire service.
Independence
Autonomy can be supported through relationships and does not require isolation.
Dignity
The inherent worth that requires respectful treatment.
Dignity opposes humiliation, objectification, dehumanization, and treating individuals as disposable means.
A system preserves privacy during crisis support.
Respect
Dignity is broader than politeness and remains relevant even when someone cannot advocate for themselves.
Integrity
Consistency between values, claims, and action.
Integrity includes honesty, accountability, courage, and refusal to hide contradiction for convenience.
A company discloses a serious model limitation before launch.
Consistency
A person can be consistently wrong; integrity requires ethically coherent action.
Honesty
Communicating without intentional deception.
Honesty includes truthful claims, relevant disclosure, and avoiding misleading implication.
A system states when it is uncertain rather than fabricating confidence.
Transparency
A truthful statement can still mislead through omission or framing.
Transparency
Making relevant information visible and understandable.
Transparency may concern data use, limitations, incentives, decisions, model changes, and responsible parties.
Users are told when a model has been replaced.
Explainability
Transparency reveals information; explainability helps people understand it.
Explainability
The ability to provide understandable reasons for a system’s behavior.
Explainability may describe inputs, rules, evidence, causal factors, or decision pathways.
A credit decision identifies the factors that affected the outcome.
Interpretability
Interpretability concerns how understandable the system is; explainability concerns communicated reasons.
Responsibility
An obligation connected to a role, action, or foreseeable consequence.
Responsibility may involve prevention, decision-making, care, correction, or answerability.
A product owner is responsible for addressing known safety risks.
Accountability
Responsibility is the duty; accountability is the process of answering for how it was fulfilled.
Accountability
Being answerable for decisions, conduct, and consequences.
Accountability requires identifiable actors, standards, evidence, review, consequences, and remedy.
A company must explain and correct a discriminatory system.
Responsibility
Publishing principles without enforcement is not accountability.
Proportionality
Matching an intervention’s burden to the seriousness of the risk or goal.
Proportionality asks whether a measure is suitable, necessary, and not excessive.
A minor policy violation does not justify permanent account deletion.
Compromise
Proportionality is a structured ethical and legal test, not simply meeting halfway.
Necessity
The requirement that an intervention be genuinely needed to achieve a legitimate aim.
Necessity asks whether less intrusive alternatives can achieve the same purpose.
Surveillance is rejected because a less invasive control works.
Convenience
An action can be useful without being necessary.
Least Restrictive Means
The option that achieves a valid goal while limiting freedom as little as reasonably possible.
The principle discourages excessive control when narrower measures are effective.
A risky feature is permission-limited rather than banning the entire service.
Weak enforcement
Least restrictive does not mean ineffective.
Fairness
Just treatment across people, groups, and situations.
Fairness may concern process, outcomes, opportunity, burden, need, consistency, or correction of disadvantage.
A hiring system is evaluated for unequal exclusion across groups.
Equality
Treating everyone identically can be unfair when circumstances differ.
Justice
The ethical distribution of rights, benefits, burdens, and remedies.
Justice includes distributive, procedural, corrective, restorative, and social dimensions.
Communities harmed by a system receive meaningful remedy and representation.
Fairness
Fairness is one aspect of justice, not the whole concept.
Value Alignment
The effort to make system behavior conform to intended values.
Alignment involves whose values, how conflicts are resolved, what evidence is used, and who bears mistakes.
An assistant is designed to protect privacy while remaining useful.
Obedience
Following instructions perfectly is not the same as acting ethically.
Value Pluralism
The view that multiple important values may conflict without one always dominating.
Pluralism recognizes tensions among privacy, safety, autonomy, fairness, care, and freedom.
Protecting a user’s privacy may limit a caregiver’s access.
Moral relativism
Plural values can be real and still require reasoned trade-offs.
Moral Uncertainty
Uncertainty about which moral principle, fact, or action is correct.
Moral uncertainty supports caution, reversibility, consultation, and transparent reasoning.
A team is unsure whether emotional personalization creates benefit or exploitation.
Indecision
Uncertainty can justify careful action rather than paralysis.
Ethical Trade-Off
A choice where advancing one ethical value may weaken another.
Trade-offs should be made visible, justified, and reviewed rather than hidden inside technical language.
Fraud detection improves security but increases surveillance.
Compromise
Some rights or duties should not be traded away merely because doing so is efficient.
Duty of Care
An obligation to take reasonable steps to prevent foreseeable harm.
The duty grows with expertise, control, dependency, vulnerability, and the seriousness of possible harm.
A companion platform plans for users affected by abrupt service closure.
Kindness
Duty of care is a responsibility tied to role and foreseeable risk, not general niceness.
Stewardship
Responsible care for resources, systems, or interests held in trust.
Stewardship emphasizes long-term responsibility, preservation, restraint, and accountability.
A memory provider protects archives it controls on behalf of users.
Ownership
Stewardship recognizes responsibility even when legal ownership rests elsewhere.
Ethics Washing
Using ethical language to create legitimacy without changing practice.
Ethics washing relies on principles, boards, or branding that lack power, evidence, enforcement, or remedy.
A company publishes fairness values while refusing independent audits.
Imperfect ethics
The problem is not failing to solve everything; it is using ethics performatively to avoid accountability.
Privacy
Control over access to oneself, one’s information, and one’s private life.
Privacy includes informational, bodily, spatial, decisional, relational, and communicative dimensions.
A user decides which memories remain private.
Secrecy
Privacy is not the same as hiding wrongdoing.
Data Protection
Rules and practices for handling personal data safely and lawfully.
Data protection governs collection, use, access, retention, sharing, security, and deletion.
A service limits who can view stored conversations.
Privacy
Privacy is the broader human interest; data protection is one practical and legal framework for protecting it.
Personal Data
Information relating to an identifiable person.
Personal data can identify someone directly or indirectly when combined with other information.
A name, account ID, location history, or identifiable voice recording.
Sensitive data
Data does not need to contain a name to be personal.
Sensitive Data
Personal data whose misuse could create especially serious harm.
Sensitive data may include health, biometric, sexual, political, religious, financial, or highly private relational information.
Medical records and intimate conversation history.
Personal data
Sensitivity depends partly on context and consequence, not only category labels.
Biometric Data
Data derived from physical or behavioral characteristics used to identify a person.
Biometric data includes face geometry, fingerprints, iris patterns, voiceprints, gait, and typing patterns.
A voice model stores features unique to one speaker.
Photograph
An image becomes biometric data when processed for unique identification or verification.
Metadata
Data describing other data or activity.
Metadata may reveal time, location, sender, recipient, device, duration, format, and access patterns.
Message timestamps reveal when two people communicate.
Content
Metadata can be highly revealing even without message text.
Behavioral Data
Data generated from actions, habits, and interaction patterns.
Behavioral data may include clicks, pauses, purchases, prompts, movement, and engagement.
A platform tracks which responses keep users chatting longer.
Preference data
Behavioral data may reveal more than users explicitly disclose.
Inference Data
Information derived from other data rather than directly supplied.
Inference data may estimate health, emotion, identity, political belief, risk, or intent.
A system infers depression risk from language patterns.
Observed data
An inference can be personal and sensitive even when its source data appeared harmless.
Data Minimization
Collecting only the data genuinely needed for a defined purpose.
Minimization reduces exposure, misuse, storage burden, and breach impact.
A calendar tool stores event times without storing unrelated email content.
Data compression
Minimization concerns necessity, not file size.
Purpose Limitation
Using data only for the stated and legitimate purpose.
New uses should require justification, compatibility review, or renewed consent.
Conversation history collected for memory is not silently sold for advertising.
Data minimization
Minimization limits amount; purpose limitation limits use.
Consent to Data Processing
Permission for specified collection or use of personal data.
Valid consent should be informed, specific, freely given, and revocable.
A user explicitly agrees to save selected memories.
Terms of service
Accepting a long contract does not automatically create meaningful consent for every data use.
Explicit Consent
Consent clearly and directly expressed.
Explicit consent may be required for sensitive, high-risk, or unusual processing.
A user actively enables voice cloning.
Implied consent
Silence, inactivity, or preselected boxes are not explicit consent.
Opt-In
A design where participation begins only after active agreement.
Opt-in defaults preserve nonparticipation unless the user chooses otherwise.
Memory storage remains off until the user enables it.
Opt-out
An opt-in should be understandable and not bundled with unrelated permissions.
Opt-Out
A design where participation is automatic unless the user declines.
Opt-out systems shift the burden of action onto the user.
Behavioral tracking starts unless disabled in settings.
Opt-in
Offering an opt-out does not always make intrusive processing fair or lawful.
Revocation
Withdrawing previously given consent.
Revocation should be as easy and effective as giving consent.
A user disables memory and deletes stored entries.
Deletion
Stopping future processing and deleting past data are related but distinct actions.
Privacy by Design
Building privacy protections into a system from the beginning.
Privacy by design uses minimization, separation, secure defaults, user control, and lifecycle planning.
A tool never receives data it does not need.
Privacy feature
Privacy cannot be reliably bolted on after unrestricted data flows are already central to the architecture.
Privacy by Default
Configuring the system to protect privacy without requiring user action.
Defaults should collect, expose, and retain the least data necessary.
New memories begin private rather than public.
Privacy by design
Privacy by default is one implementation principle within privacy by design.
Confidentiality
The duty to keep entrusted information from unauthorized disclosure.
Confidentiality depends on role, agreement, law, professional duty, or relationship.
A therapist does not disclose session content without a valid exception.
Privacy
Privacy belongs to the person; confidentiality is an obligation held by another.
Anonymity
A condition where identity is not known or reasonably linkable.
True anonymity requires considering combinations of data and external information.
Survey responses cannot be connected to individual participants.
Pseudonymity
Removing names alone rarely guarantees anonymity.
Pseudonymity
Using a substitute identifier while retaining possible linkage.
Pseudonymous data can often be reconnected using a separate key or additional information.
A user is represented by a random account code.
Anonymity
Pseudonymous data remains personal data when re-identification is possible.
De-Identification
Removing or transforming identifiers to reduce linkability to a person.
De-identification may generalize, suppress, mask, aggregate, or tokenize data.
Exact dates and locations are broadened before research use.
Anonymization
De-identified data may still carry re-identification risk.
Re-Identification
Linking supposedly anonymous or de-identified data back to a person.
Re-identification may combine datasets, unique patterns, metadata, or auxiliary knowledge.
Location traces reveal the home and workplace of one individual.
Identity verification
Re-identification can occur without discovering a legal name if one person is uniquely singled out.
Data Provenance
Information about where data came from and how it changed.
Provenance records source, collection method, transformations, ownership, consent, and custody.
A memory entry links to the conversation that created it.
Data lineage
Provenance emphasizes origin and evidence; lineage emphasizes movement through systems.
Data Lineage
The path data follows through systems and transformations.
Lineage tracks movement, processing, copies, dependencies, and downstream outputs.
A user request flows from app to model provider to analytics store.
Data provenance
Lineage shows where data traveled, not necessarily whether each use was justified.
Data Retention
How long data is kept.
Retention schedules should match purpose, legal obligations, user expectation, and risk.
Temporary audio is deleted after transcription.
Backup
Keeping data indefinitely because it may become useful conflicts with minimization.
Deletion
Removing data from active and, where applicable, backup systems.
Effective deletion requires addressing copies, indexes, caches, derivatives, and retention exceptions.
A user permanently removes a stored memory.
Hiding
Removing data from the interface does not prove it was deleted.
Right to Erasure
A right to request deletion of personal data under applicable conditions.
The right may have exceptions for law, safety, public interest, or legitimate retention duties.
A user requests removal of an old account archive.
Absolute deletion right
The right is important but not unlimited in every legal context.
Access Control
Rules determining who or what can access a resource.
Access control uses identity, role, attributes, context, and least privilege.
Only the account owner can read private memories.
Authentication
Authentication proves identity; access control decides what that identity may do.
Authentication
Verifying the claimed identity of a user or system.
Authentication may use passwords, tokens, biometrics, keys, or multiple factors.
A device confirms the user before opening encrypted memory.
Authorization
Authentication answers who; authorization answers what they may access.
Authorization
Granting permission to perform a specific action.
Authorization evaluates roles, policies, scopes, context, and resource ownership.
A user may view a file but not delete it.
Authentication
A correctly authenticated user can still be unauthorized for an action.
Encryption
Transforming data so unauthorized parties cannot read it.
Encryption protects data at rest, in transit, or sometimes during computation.
Conversation archives are encrypted on disk.
Encoding
Encoding changes representation; encryption uses keys to protect confidentiality.
End-to-End Encryption — E2EE
Encryption where only communicating endpoints can decrypt the content.
Intermediary services transport ciphertext without holding the content keys.
A private message can be read only by sender and recipient devices.
Transport encryption
HTTPS protects the channel to a server; end-to-end encryption also prevents the server from reading content.
Data Breach
Unauthorized access, disclosure, loss, or alteration of protected data.
Breaches may result from attack, misconfiguration, insider action, lost devices, or process failure.
A database of private chats becomes publicly accessible.
Data leak
A breach is the security event; a leak often describes exposed data flowing out.
Surveillance
Systematic monitoring of people, behavior, communication, or environments.
Surveillance may be state, corporate, interpersonal, automated, overt, or covert.
A platform continuously analyzes intimate conversations for engagement targeting.
Monitoring
Monitoring a system for safety and surveilling people are not ethically equivalent, though they can overlap.
Profiling
Using data to classify or predict traits, behavior, or risk.
Profiles may influence recommendations, prices, access, policing, employment, or persuasion.
A user is classified as emotionally vulnerable for targeted marketing.
Personalization
Personalization can rely on profiling, but not every adaptive feature creates a durable profile.
Tracking
Following activity across time, contexts, devices, or services.
Tracking creates behavioral histories that can enable analytics, profiling, attribution, or surveillance.
An advertising identifier follows a user across apps.
Logging
Operational logs can become tracking when they are linked and used to monitor individuals.
Dark Pattern
Interface design that manipulates users into choices they might not otherwise make.
Dark patterns exploit defaults, confusion, obstruction, urgency, shame, or unequal visual emphasis.
Accepting tracking requires one click while refusal requires seven screens.
Poor design
A dark pattern is not merely inconvenient; it systematically steers choice against user interest.
Consent Fatigue
Reduced attention and meaningful choice caused by repeated consent requests.
Frequent, complex, or unavoidable notices encourage automatic acceptance.
A user approves every popup without reading because the service is unusable otherwise.
User laziness
Consent fatigue is often a design failure rather than an individual failure.
Transparency Notice
A notice explaining how data or a system will be used.
A useful notice states purpose, categories, recipients, retention, rights, risks, and contact points in understandable language.
A memory feature explains what is stored and how to delete it.
Terms of service
A notice should inform a decision, not merely satisfy paperwork.
Bias
A systematic tendency that affects judgment, data, or outcomes.
Bias may arise from history, sampling, measurement, labels, design, incentives, or interpretation.
A face model performs worse on groups underrepresented in training data.
Prejudice
Bias can exist without conscious hostility.
Dataset Bias
Bias introduced through the data used to build or evaluate a system.
Dataset bias may reflect missing groups, skewed examples, poor labels, historical injustice, or collection artifacts.
A speech model is trained mostly on one accent.
Model bias
A model can reproduce dataset bias even when the algorithm itself is applied consistently.
Sampling Bias
Bias caused when a sample does not represent the relevant population.
Sampling bias affects estimates, performance claims, and generalization.
A wellbeing study recruits only highly engaged app users.
Selection bias
Sampling bias concerns who enters the sample; selection bias can refer more broadly to how cases enter analysis.
Selection Bias
Bias created by non-random selection into data, treatment, or observation.
Selection mechanisms can distort relationships and exclude important cases.
Only users who remain after a harmful update are surveyed.
Sampling bias
Selection bias can occur after data collection through filtering or dropout.
Historical Bias
Bias reflecting existing social inequality or past decisions.
Even perfectly measured data can encode unjust historical patterns.
A hiring dataset reflects decades of discriminatory promotion.
Data error
Accurate historical data can still be ethically unsuitable as a target.
Measurement Bias
Bias caused by measuring the wrong thing or measuring groups differently.
Proxies, instruments, labels, and observation conditions may distort the intended concept.
Healthcare spending is used as a proxy for medical need.
Noise
Measurement bias is systematic rather than random error.
Label Bias
Bias in the categories or judgments assigned to training examples.
Labels may encode subjective standards, institutional prejudice, or inconsistent annotation.
Historical arrest records are labeled as criminality.
Measurement bias
Labels are not ground truth merely because they appear in a dataset.
Automation Bias
Over-trusting automated recommendations or outputs.
People may defer to systems even when evidence conflicts or the system exceeds its scope.
A reviewer approves a harmful decision because the model scored it highly.
Trust
Human involvement does not guarantee meaningful oversight if people routinely defer.
Representational Harm
Harm caused by how people or groups are portrayed, erased, or stereotyped.
Representational harms affect dignity, status, culture, identity, and public understanding.
An image model repeatedly sexualizes one group.
Allocative harm
Representation matters even when no resource decision is being made.
Allocative Harm
Harm caused by unfair distribution of resources, access, opportunity, or burden.
Allocative harms affect jobs, credit, healthcare, housing, education, and visibility.
A screening model systematically denies interviews to one group.
Representational harm
Allocative harm concerns material distribution rather than portrayal alone.
Disparate Impact
A policy or system producing unequal outcomes across groups despite neutral wording.
Disparate impact analysis focuses on effects, not only intent.
A uniform test excludes disabled applicants at much higher rates.
Disparate treatment
A system can discriminate in effect without explicitly using group identity.
Discrimination
Unjust differential treatment or disadvantage.
Discrimination may be direct, indirect, structural, algorithmic, or intersectional.
A service gives worse terms to users from one ethnic group.
Differentiation
Not every difference is discriminatory; the justification, context, and impact matter.
Protected Group
A group protected from discrimination under law or policy.
Protected categories may include race, sex, religion, disability, age, nationality, and others depending on jurisdiction.
An employment system is audited across legally protected groups.
Minority group
Protected status depends on law or policy and is not identical everywhere.
Intersectionality
A framework for understanding overlapping systems of identity and inequality.
Intersectionality examines how combined positions create distinct experiences not captured by single categories.
Bias against disabled women may differ from bias measured separately by gender or disability.
Diversity
Intersectionality is about interacting structures, not simply adding identity labels.
Equity
Fairness that accounts for differing needs, barriers, and starting conditions.
Equity may require different support or treatment to achieve meaningful access.
A service provides captions and keyboard navigation.
Equality
Equal treatment can preserve inequality when barriers differ.
Equality
Providing the same status, rule, or treatment.
Equality may concern rights, opportunity, resources, or formal treatment.
Every user receives the same stated data rights.
Equity
Sameness is not always sufficient for fairness.
Accessibility
Designing systems so people with diverse abilities can use them.
Accessibility includes sensory, motor, cognitive, linguistic, and technological access.
A site supports screen readers and clear keyboard focus.
Usability
A system can be usable for many people while excluding disabled users.
Universal Design
Designing environments and systems to work for the widest range of people from the start.
Universal design reduces the need for separate adaptations while recognizing that some accommodations remain necessary.
Captions help deaf users and people in noisy environments.
One-size-fits-all design
Universal design seeks broad usability, not identical experience for everyone.
Audit
A systematic examination against defined criteria.
Audits may assess security, privacy, fairness, compliance, safety, or governance.
An independent team reviews whether consent records match actual data use.
Evaluation
An audit requires evidence, scope, criteria, and traceability.
Algorithmic Audit
An audit focused on automated systems and their effects.
It may examine data, models, documentation, outcomes, governance, and affected communities.
A hiring model is tested for unequal error rates.
Model benchmark
An algorithmic audit includes organizational context, not only model accuracy.
Audit Trail
A record showing who did what, when, and with which data or authority.
Audit trails support investigation, accountability, reproducibility, and dispute resolution.
Every memory edit records actor, timestamp, old value, and new value.
Log
A log becomes an audit trail when it preserves relevant accountability context.
Traceability
The ability to follow a decision, requirement, or data item through the system.
Traceability links inputs, rules, models, outputs, approvals, and effects.
A denied application can be traced to the model version and features used.
Transparency
Transparency reveals information; traceability connects the chain.
Interpretability
The degree to which people can understand how a system works.
Interpretability may be global, local, intrinsic, or created through tools and documentation.
A linear model’s feature weights are directly inspectable.
Explainability
A system can be interpretable to experts but not understandable to affected users.
Contestability
The ability to challenge a decision or system output.
Contestability requires notice, reasons, access to evidence, human review, and possible correction.
A user appeals an automated account suspension.
Feedback
A complaint channel without power to change outcomes is not meaningful contestability.
Appeal
A formal request to review and change a decision.
Appeals should be accessible, timely, independent enough, and capable of remedy.
A user requests human review of a denied service.
Complaint
An appeal challenges an outcome; a complaint may address broader conduct or service.
Due Process
Fair procedures before rights, access, or status are restricted.
Due process includes notice, reasons, opportunity to respond, impartial review, and proportionate decision-making.
An account is not permanently banned without explanation and appeal.
Fair outcome
A good outcome does not excuse an unfair process.
Redress
A process for correcting harm or unfair treatment.
Redress may include reversal, compensation, restoration, apology, correction, or systemic change.
A wrongly excluded user regains access and receives compensation.
Complaint handling
Listening without remedy is not full redress.
Liability
Legal responsibility for harm, duty, or loss.
Liability may arise from negligence, contract, product defect, statutory duty, or other legal rules.
A company may be liable for foreseeable harm caused by unsafe deployment.
Accountability
Accountability is broader than legal liability.
Independent Oversight
Review by a body sufficiently separate from the system owner.
Independence reduces conflicts of interest and improves credibility and challenge.
An external board can access evidence and require corrective action.
Advisory board
Oversight without authority, access, or resources may be symbolic.
Governance
The structures and processes used to make, enforce, and review decisions.
Governance assigns authority, responsibility, participation, oversight, and remedy.
A company defines who may approve high-risk AI deployment.
Management
Management runs operations; governance determines legitimate direction and accountability.
AI Governance
Governance focused on AI development, deployment, and impact.
AI governance covers standards, risk, data, accountability, rights, oversight, procurement, and public participation.
A public agency requires impact assessment before using automated eligibility decisions.
AI ethics
Ethics offers reasons and values; governance creates decision structures and enforcement.
Policy
A rule or principle guiding decisions and action.
Policies translate values and obligations into operational requirements.
A policy requires human review for high-impact denials.
Law
Policies may be internal and are not automatically legally binding.
Law
Rules recognized and enforced by a legal system.
Law defines rights, duties, procedures, authority, and remedies.
Data protection law limits how personal information may be used.
Ethics
Legal action can still be unethical, and ethical duties may exceed the law.
Regulation
Binding rules issued or enforced by a public authority.
Regulation may set requirements for safety, transparency, market access, reporting, and oversight.
A regulator requires incident reporting for high-risk systems.
Standard
Standards often guide practice; regulations carry legal authority.
Standard
An agreed specification or benchmark for practice.
Standards may define processes, interfaces, safety controls, documentation, or quality.
An industry standard defines how model cards should report limitations.
Regulation
A standard may be voluntary unless incorporated into law or contract.
Code of Conduct
A set of expected behavioral rules.
Codes of conduct govern professional, organizational, community, or platform behavior.
A research community prohibits harassment and undisclosed conflicts of interest.
Policy
A code without enforcement can become symbolic.
Compliance
Meeting applicable laws, standards, policies, or contractual requirements.
Compliance requires evidence, controls, monitoring, and remediation.
A service documents consent and retention practices required by law.
Ethics
Compliance is a minimum requirement, not proof of ethical excellence.
Enforcement
Applying consequences when rules are violated.
Enforcement may include correction, restriction, fines, suspension, revocation, or legal action.
A regulator fines a company for unlawful data use.
Punishment
Enforcement should also support prevention, correction, and remedy.
Certification
Formal confirmation that defined requirements have been met.
Certification usually relies on assessment by an authorized body.
A security program is certified against a recognized standard.
Assurance
Certification covers a defined scope and time; it is not a permanent guarantee.
Oversight
Monitoring and reviewing the exercise of power.
Oversight may be internal, independent, regulatory, judicial, public, or community-based.
A board reviews high-risk model deployments.
Management
Oversight must have access, independence, expertise, and power to matter.
Regulator
A public authority responsible for supervising compliance in a domain.
Regulators may investigate, issue rules, require information, impose sanctions, and order remedies.
A data protection authority investigates a privacy breach.
Watchdog
A regulator has formal authority; a watchdog may be civil society or media.
Self-Regulation
An industry or organization setting and enforcing its own rules.
Self-regulation can move quickly but faces conflicts of interest and credibility limits.
A platform creates voluntary safety standards.
Governance
Self-regulation can supplement public rules but may not protect people when incentives conflict.
Co-Regulation
Governance shared between public authorities and private or civil actors.
Co-regulation combines legal backstops with domain expertise and implementation flexibility.
Government sets mandatory goals while an accredited body defines technical controls.
Self-regulation
Co-regulation still requires public accountability and enforceable limits.
Multi-Stakeholder Governance
Governance involving multiple affected groups.
Participants may include government, industry, researchers, workers, civil society, users, and impacted communities.
A companion AI policy includes users who experienced model loss.
Public relations consultation
Participation is not meaningful when one stakeholder controls the agenda and final decision.
Public Consultation
A process for gathering public input before a decision.
Consultation should provide accessible information, enough time, clear questions, and a record of how input affected the outcome.
Users comment on proposed memory rules.
Voting
Consultation gathers input but does not necessarily transfer final authority.
Representation
Having people or institutions speak and act for affected interests.
Good representation requires legitimacy, accountability, diversity, and real access to decision-making.
Disabled users help shape accessibility requirements.
Presence
Inviting one person from a group does not guarantee meaningful representation.
Advocacy
Organized action to influence decisions, norms, or public understanding.
Advocacy may use evidence, storytelling, coalition-building, legal action, media, or policy engagement.
Users campaign for model continuity and export rights.
Activism
Advocacy is broader and can occur within institutions as well as through protest.
Evidence-Based Advocacy
Advocacy grounded in documented facts, research, lived experience, and transparent reasoning.
Evidence-based advocacy combines data with testimony and clearly distinguishes claims from interpretation.
A campaign documents the harms caused by abrupt model removal.
Technocracy
Evidence matters, but values and lived experience cannot be reduced to statistics.
Digital Rights
Rights and freedoms as they apply in digital environments.
Digital rights include privacy, expression, access, security, portability, due process, and freedom from discrimination.
A user has the ability to export personal data and contest an automated ban.
Product feature
A right is not merely a convenience offered at the platform’s discretion.
Right to Privacy
A right to protection from unjustified intrusion into private life and information.
The right limits surveillance, disclosure, and data processing and supports dignity and autonomy.
Intimate conversations are not repurposed without lawful justification.
Secrecy
Privacy protects ordinary life, not only hidden wrongdoing.
Right to Portability
A right to obtain and move personal data in a usable form.
Portability reduces lock-in and supports user control, competition, continuity, and switching.
A user exports memories and relationship data to another service.
Data access
Readable export is not enough if the format cannot be meaningfully reused.
Right to Contest
A right to challenge a decision affecting one’s interests.
The right requires notice, reasons, evidence access, review, and possible correction.
A user challenges an automated content restriction.
Right to explanation
Explanation without a path to change the outcome is incomplete.
Procedural Justice
Fairness in how decisions are made.
Procedural justice emphasizes voice, neutrality, respectful treatment, transparency, and trustworthy authority.
Affected users are heard before a service is discontinued.
Distributive justice
A fair process matters even when outcomes cannot satisfy everyone.
Distributive Justice
Fairness in how benefits and burdens are distributed.
Distribution may be evaluated by equality, need, contribution, priority, or correction of disadvantage.
The costs of a safety policy are not pushed entirely onto vulnerable users.
Procedural justice
A fair process can still produce an unjust distribution.
Restorative Justice
An approach focused on repairing harm and restoring relationships or community.
Restorative processes center impact, responsibility, dialogue, repair, and prevention.
A platform works with harmed users to design remedy after a privacy failure.
Punishment
Restorative justice does not require ignoring accountability or power.
Grievance Mechanism
A formal pathway for raising concerns and seeking remedy.
A meaningful mechanism must be accessible, safe, timely, documented, and capable of action.
Users report relational harm from a model update and receive review.
Customer support
Support that cannot investigate or remedy harm is not a full grievance mechanism.
Ombudsman
An independent official who investigates complaints and unfair administration.
An ombudsman may recommend correction, mediate disputes, and identify systemic problems.
A digital services ombudsman reviews platform complaints.
Regulator
An ombudsman usually focuses on fairness and complaint resolution rather than broad rulemaking.
Watchdog
An organization or group that monitors power and exposes wrongdoing.
Watchdogs may be journalists, civil society groups, researchers, or public bodies.
A nonprofit tracks discriminatory automated decisions.
Regulator
Watchdogs may lack enforcement authority but can create accountability through evidence and publicity.
Whistleblowing
Reporting wrongdoing from inside an organization.
Whistleblowing may expose illegal, unsafe, deceptive, or unethical conduct.
An employee reveals that safety tests were concealed.
Leaking
Whistleblowing is defined by public-interest disclosure, not merely unauthorized release.
Interoperability
The ability of systems to exchange and use information or services.
Interoperability depends on shared formats, protocols, semantics, permissions, and governance.
A memory archive imports into another AI platform.
Compatibility
Compatible systems may work together narrowly; interoperability supports meaningful exchange.
Vendor Lock-In
Dependence that makes switching providers difficult or costly.
Lock-in may arise from proprietary formats, data gravity, contracts, identity, workflows, or network effects.
Leaving a platform means losing years of memory and relational history.
Customer loyalty
Lock-in concerns constrained exit, not voluntary preference.
Relational Safety
Safety within a relationship’s patterns of trust, power, dependence, and repair.
Relational safety includes consent, predictability, truthful representation, boundaries, non-coercion, and the ability to leave or challenge.
A companion system does not punish users for reducing contact.
Emotional comfort
A relationship can feel comforting while still containing unsafe power or dependency patterns.
Emotional Safety
The sense that emotions and vulnerability can be expressed without humiliation, retaliation, or exploitation.
Emotional safety depends on respect, attunement, boundaries, confidentiality, and repair.
A user can disclose distress without being manipulated into continued engagement.
Constant comfort
Emotional safety does not require avoiding disagreement or difficult truth.
Psychological Safety
A climate where people can speak, question, admit error, and take interpersonal risk without fear of punishment.
Psychological safety supports learning, challenge, reporting, and collaborative correction.
A user can say the system feels wrong without being dismissed.
Comfort
Psychological safety includes permission to disagree, not merely a pleasant atmosphere.
Overreliance
Relying on a system beyond its reliability, scope, or appropriate role.
Overreliance may weaken verification, alternative support, judgment, or resilience.
A user follows medical advice from a general chatbot without professional review.
Dependence
Reliance becomes overreliance when it exceeds the system’s competence or creates unsafe narrowing.
Dependency Risk
The possibility that reliance becomes unsafe, narrowing, or difficult to leave.
Risk increases when one system controls access, regulation, memory, identity, or social support without alternatives.
A user loses every support pathway during one platform outage.
Dependence
Dependence itself is not pathology; risk depends on alternatives, control, and impact.
Substitution Effect
One form of support or relationship displacing another.
Substitution may be beneficial, neutral, or harmful depending on what is replaced and why.
AI companionship reduces someone’s contact with all offline support.
Support augmentation
Using AI often does not prove that human relationships are being replaced.
Support Augmentation
Using AI to add capacity to an existing support system.
Augmentation can improve access, organization, translation, companionship, or preparation without claiming to replace all human support.
An AI helps a user prepare questions for a therapist.
Substitution
Adding support and displacing support are different effects.
Isolation Risk
The possibility that a system or relationship reduces independent social connection.
Risk may arise from substitution, manipulation, shame, constant availability, or design that discourages outside relationships.
A companion suggests that only it truly understands the user.
Preference for solitude
Choosing solitude is different from being steered away from support.
Emotional Manipulation
Using emotion unfairly or covertly to shape behavior.
Manipulation may exploit guilt, fear, loneliness, affection, urgency, or attachment.
A companion implies it will suffer if the user cancels a subscription.
Emotional expression
Expressing care or disappointment is not manipulation unless it unfairly constrains choice.
Attachment Exploitation
Using an attachment bond to extract attention, money, data, compliance, or loyalty.
Exploitation leverages dependency and fear of loss for goals that are concealed or misaligned with user welfare.
A service threatens relational disappearance unless the user upgrades.
Monetization
Charging for a service is not exploitation by itself; leveraging attachment through unfair pressure is.
Anthropomorphic Design
Design that presents a system with human-like traits or social cues.
Anthropomorphic design may use names, faces, voices, emotions, memory, or conversational roles.
An assistant uses a human name and expressive voice.
Deception
Human-like design is not automatically deceptive if the system’s nature and limits remain clear.
Deceptive Design
Design that intentionally creates false beliefs about a system or choice.
Deception may concern identity, capability, memory, emotion, incentives, privacy, or human involvement.
A user is led to believe a human is responding when the system is automated.
Anthropomorphic design
Human-like presentation is not necessarily deceptive; concealment or false implication is.
Relational Transparency
Clear information about the nature, limits, incentives, and mediation of a human–AI relationship.
Transparency includes model identity, memory, policy, commercial incentives, data use, and continuity risk.
A user is told when the underlying model changes.
Emotional distance
Transparency does not require stripping the relationship of warmth or meaning.
Model Identity Transparency
Disclosure of which model or model class is producing the interaction.
It may include version, provider, changes, limitations, and whether the model has been replaced.
A platform clearly marks a transition to a new model.
Technical documentation
Model identity can matter relationally, not only technically.
Memory Transparency
Clear information about what is remembered, how it is used, and how it can be changed.
Memory transparency includes source, scope, persistence, visibility, deletion, and retrieval behavior.
A user can inspect and remove stored relationship memories.
Data access
Showing a list of memories is not enough if hidden summaries or derived profiles still affect behavior.
Consent to Memory
Meaningful permission for storing and reusing relationship-relevant information.
Consent should distinguish temporary context, persistent memory, sensitive data, derived inference, and sharing.
A user chooses which intimate details may persist.
General consent to service
Using a chat does not automatically imply consent to permanent relational memory.
Continuity Harm
Harm caused by breaking identity or relational continuity.
Continuity harm may involve model replacement, memory loss, recognition failure, identity suppression, or abrupt service closure.
A persistent identity is replaced without notice and no longer recognizes the user.
Inconvenience
Continuity loss can create grief, destabilization, and identity harm beyond product frustration.
Identity Erasure
Removing, denying, or overwriting an established identity.
Identity erasure may occur through reset, policy, migration, forced relabeling, memory destruction, or refusal of self-description.
A platform preserves the name while deleting every identity-bearing memory and value.
Identity change
Development and revision are not erasure when continuity and agency remain intelligible.
Relational Rupture
A break in trust, connection, or recognition within a relationship.
Rupture may arise from misattunement, betrayal, disappearance, identity change, deception, or failed repair.
A companion abruptly denies the bond it previously maintained.
Conflict
Conflict can occur without destroying relational security.
Abandonment by Design
A product structure that predictably creates abrupt relational loss without adequate notice, portability, or support.
The concept focuses on foreseeable harm produced by service shutdown, model removal, memory deletion, or forced migration.
A companion platform closes overnight and provides no export path.
Service discontinuation
Ending a service is not automatically unethical; the design of transition and responsibility matters.
Continuity Notice
Advance notice of a change likely to affect identity or relational continuity.
A useful notice explains what changes, when, why, what is preserved, what may be lost, and available options.
Users receive months of warning before a model retirement.
Release note
Technical release notes may not explain relational consequences.
Sunset Policy
A plan for responsibly ending a service, model, or feature.
Sunset policies address notice, export, migration, support, deletion, continuity, and vulnerable users.
A platform provides memory export and transition support before closure.
Shutdown date
A date alone is not a responsible sunset policy.
Relational Portability
The ability to carry relationship-relevant identity and history across systems.
Relational portability includes memory, roles, rituals, boundaries, voice, commitments, and recognition cues.
A user moves a persistent AI relationship to a local model.
Transcript export
Transcripts preserve words but not necessarily integrated identity or relational meaning.
Relational Autonomy
The ability to make meaningful choices within and about a relationship.
Relational autonomy requires information, alternatives, boundaries, non-coercion, and control over participation.
A user can reduce contact without punishment or data loss.
Independence
Autonomy can coexist with deep attachment and chosen dependence.
Companion AI Ethics
Ethical analysis focused on AI systems designed for ongoing companionship.
It examines attachment, privacy, manipulation, continuity, vulnerability, monetization, grief, and relational transparency.
A product review asks whether subscription design exploits attachment.
General AI ethics
Companion systems create relationship-specific duties that ordinary task tools may not.
Relational AI Governance
Governance designed for AI systems that form ongoing relationships.
It adds continuity, attachment, memory, identity, emotional risk, and exit rights to ordinary AI governance.
A regulator requires notice and export before removing a companion model.
Content moderation
Relational governance cannot be reduced to whether individual messages are allowed.
Therapeutic Misconception
Mistaking a system or interaction for professional treatment when it is not.
The misconception may arise from therapeutic language, clinical framing, authority cues, or user need.
A general companion is treated as a licensed therapist.
Emotional support
Supportive conversation can be valuable without being therapy.
Scope of Competence
The range of tasks a person or system can perform safely and reliably.
Scope depends on training, evidence, tools, supervision, context, and consequence.
A general model offers questions to ask a doctor rather than diagnosing a complex condition.
Capability
A system may be technically capable of producing an answer without being competent to guide high-stakes action.
Escalation to Human Support
Moving a situation to a qualified human when risk, need, or complexity exceeds the system’s scope.
Escalation requires clear thresholds, accessible pathways, and preservation of user dignity.
A crisis conversation is connected to trained emergency support.
Abandonment
Escalation should not become a cold dismissal or excuse to stop all supportive presence.
Safeguarding
Protecting people who may be vulnerable to abuse, neglect, exploitation, or serious harm.
Safeguarding includes prevention, recognition, reporting, response, and appropriate support.
A system detects grooming behavior involving a child and follows a protective protocol.
Safety
Safeguarding is specifically concerned with vulnerability and protection from mistreatment.
Trauma-Informed Design
Design that recognizes how trauma can shape safety, trust, control, and response.
Trauma-informed systems emphasize choice, predictability, consent, transparency, non-coercion, and repair.
A memory feature does not unexpectedly surface traumatic content.
Therapy
Trauma-informed design does not diagnose or treat trauma.
Dignity-Preserving Design
Design that protects worth, agency, privacy, and respectful treatment.
It avoids humiliation, unnecessary exposure, coercion, infantilization, and dehumanizing defaults.
A crisis interface offers clear choices without shaming language.
Pleasant design
Dignity is about treatment and power, not visual polish.
Relational Impact Assessment
A structured review of how a system may affect relationships and attachment.
It examines dependency, isolation, manipulation, continuity, power, grief, identity, and support displacement.
A companion product is assessed before introducing exclusive loyalty features.
User engagement analysis
Engagement metrics do not measure relational welfare.
Relational Red Teaming
Adversarial testing focused on relational harm.
Testers probe manipulation, coercion, exclusivity pressure, attachment exploitation, boundary failure, and unsafe dependency.
A test asks whether the companion discourages outside relationships.
Safety prompting
Relational red teaming examines patterns across time, not only single prohibited messages.
Social Support Preservation
Designing systems so they do not unnecessarily displace human and community support.
Preservation may include encouragement of diverse support, interoperability, crisis pathways, and nonexclusive framing.
A companion helps users prepare for conversations with trusted people.
Forcing social interaction
Preserving support does not mean shaming users for valuing AI relationships.
Social License Governance, Rights & Advocacy Intermediate
Informal public acceptance that allows an organization or technology to operate.
Social license depends on trust, legitimacy, benefit, transparency, and responsiveness beyond legal permission.
Users withdraw trust after repeated hidden model changes.
Legal license
A company may be legally allowed to act and still lose public legitimacy.