chatbot
Enable an AI agent to recognise a chatbot, assess its conversational capabilities and operating state, and determine which interactions or delegated actions it can appropriately support.
Research draft, second pass
A second pass drafted this model: the structure a model of this thing needs, and what is known about it in the world. The line under this one says how the second half was obtained - researched against sources, or recalled without web access, in which case nothing here was read anywhere and every claim is a lead to verify. Unreviewed either way.
recalled by Codex without web access - no source was read
Researched by: Codex
Purpose and description
Enable an AI agent to recognise a chatbot, assess its conversational capabilities and operating state, and determine which interactions or delegated actions it can appropriately support.
A chatbot is a software system that interacts with users through conversational exchanges in natural language, typically by text or speech, using scripted rules, retrieval, generative models, or combinations of these mechanisms.
It can be Start or continue a conversation using a supported modality and language.; Request clarification, correct misunderstood information or reset the conversational context.; Ask for answer sources and inspect whether a response is grounded in accessible evidence.; Delegate a supported external operation within the chatbot's effective permissions.; Inspect and, where supported, correct or delete remembered user information.; Request human assistance or report an incorrect, inappropriate or failed interaction..
Distinguishing features
Accepts user contributions as conversational turns and provides automated replies presented as those of an interlocutor; a static FAQ does not pass this test.
Supports language-based interaction, including constrained conversational choices; a conventional form or menu alone does not establish chatbot identity.
Produces or selects replies automatically; a chat interface staffed entirely by humans is a neighbouring service.
Has a configured conversational role and interaction surface; a language-model checkpoint alone is not a chatbot.
Can qualify without generative AI, persistent memory or tool use; these distinguish chatbot variants rather than establish membership.
Scope
+ The deployed chatbot's identity, intended audience, supported purposes and conversational channels
+ Response production through scripted rules, retrieval, generative models or combinations of these
+ Handling of dialogue context, ambiguity, corrections and conversational memory
+ Access to information sources, user data and external action tools
+ Disclosure, refusal, human handoff and assessment of conversational performance
- Training and general capabilities of an underlying language model independent of this chatbot
- General messaging-platform infrastructure and communication protocols
- Individual conversation transcripts as independently managed records
- Underlying knowledge documents and databases as independently maintained information assets
- The internal operation of organisations, human support teams or external services contacted by the chatbot
Characteristics
- Conversational purpose
- Information answering, transactional assistance, tutoring, companionship, entertainment, general assistance; multiple values permitted Determines which user expectations and success criteria apply.
- Response mechanism
- Scripted, rule-based, retrieval-based, generative, hybrid, unknown Helps explain response variability, coverage limits and likely failure modes.
- Interaction modalities
- Supported input and output combinations of text, speech, images and constrained choices Determines whether the chatbot can accept a particular contribution and communicate an accessible reply.
- Context persistence
- Single-turn, session-scoped, cross-session, mixed by data type, unknown Determines what prior information a user or agent can reasonably expect the chatbot to retain.
- Knowledge-source connections
- Linked sources or collections with access scope, update basis and attribution behaviour Makes answer grounding and information freshness assessable.
- Effective action authority
- Accessible tools and operations linked to acting identity, permission scope and confirmation requirements Separates conversational advice from actions the chatbot can actually execute.
- Human handoff state
- Unavailable, available, requested, queued, transferred, failed Shows whether unresolved needs have a functioning route to human assistance.
- Response latency
- Seconds to first meaningful response and completed response, with percentile, workload and observation window Measures whether interaction timing supports the intended conversational use.
- Conversation outcome rate
- Percentage meeting a stated outcome criterion, with denominator, evaluation method, chatbot version and observation window Supports assessment against the chatbot's actual purpose without treating every reply as success.
Also called
Where this came from
wikidata · CC0 1.0
Drafted structure
Bundle to layer to finding to question, as the second pass will find it: 6 bundles · 11 layers · 17 findings · 27 questions.
Conversational identity Establishes which chatbot is speaking, whom it serves and how users encounter it.
A shared name, persona or underlying model does not establish that two deployments have the same capabilities or responsibilities.
Role and boundary
Defines the chatbot's intended conversational role and deployment identity.
Configured interlocutor
Record the deployed conversational system separately from its persona, underlying model and hosting interface.
- Which configured deployment does this chatbot identity denote, and which operator is responsible for it? definition
- Do channel-specific versions share conversation state, knowledge access and permissions, or require separate deployment identities? boundary
Audience and presentation
Captures supported users, purposes and the expectations established by the interface.
Interaction contract
Record the purposes, languages and modalities offered, together with how automation and role limits are disclosed.
- Which audiences, tasks, languages and input-output modalities does the chatbot explicitly support? definition
- How does the interface identify automated replies and distinguish the chatbot's role from any human representative? boundary
Dialogue continuity Describes how turns become a coherent exchange and how misunderstandings are repaired.
Chatbot behaviour depends on what it treats as the current request and which earlier contributions remain available.
Turn understanding and repair
Examines interpretation of user contributions and recovery from conversational errors.
Intent clarification
Record how the chatbot handles ambiguity, missing details, topic changes and explicit corrections.
- How does the chatbot decide whether to answer, ask a clarifying question or report that it cannot interpret a turn? action
- How reliably does it incorporate a correction without continuing to rely on the superseded interpretation? measurement
Context and memory
Separates information used within a conversation from information retained across conversations.
Remembered information
Record what conversation history, summaries and user preferences remain accessible, including their scope and user controls.
- Which information persists between turns or sessions, and what causes earlier context to be omitted, summarised or forgotten? boundary
- How can a user inspect, correct or delete retained information, and which storage locations do those controls affect? action
Reply production and grounding Connects chatbot replies to response mechanisms, accessible evidence and uncertainty handling.
Fluent conversational output does not by itself reveal where an answer came from or whether its claims are supported.
Response path
Identifies the mechanisms and configuration that determine a reply.
Reply selection and generation
Record whether replies follow scripts, rules, retrieval, generation or routing among these mechanisms.
- Which mechanisms produce replies, and what conditions route a conversation between them? definition
- Which versions of models, prompts, rules and retrieval configuration can be associated with an observed reply? provenance
Evidence and uncertainty
Examines the relationship between factual replies, source access and acknowledged limits.
Answer support
Record accessible evidence, attribution practices and behaviour when evidence is missing, stale or contradictory.
- Which sources were actually consulted for a reply, and can its material claims be traced to supporting passages or tool results? provenance
- What does the chatbot do when it lacks sufficient evidence or encounters conflicting information? action
Delegated actions and data boundaries Defines what the chatbot can cause outside the conversation and which information it may use or disclose.
Conversational requests can become external operations, while retrieved content and user text can contain instructions that lack authority.
Tool-mediated action
Links conversational requests to authorised operations and verifiable outcomes.
Execution authority
Record available operations, acting identities, confirmation requirements and evidence of execution.
- Which external operations can this chatbot execute, under whose identity and with what permission scope? boundary
- How does it obtain required confirmation and distinguish a proposed action from an attempted, completed or failed action? action
Information and instruction separation
Examines isolation between users and separation of source content from authorised instructions.
Trust boundary enforcement
Record how data access is constrained and how instructions embedded in untrusted conversation or retrieved material are handled.
- What prevents one user's conversation, retrieved records or remembered information from being exposed to another user? boundary
- What evaluated behaviour shows whether embedded instructions can redirect the chatbot or induce unauthorised disclosure or tool use? measurement
Conversation outcomes and recovery Assesses whether conversations fulfil their purpose and how unresolved or failed interactions proceed.
A chatbot needs evaluation at the level of conversational outcomes, including failures that require refusal, recovery or human assistance.
Purpose-specific evaluation
Connects observed quality and timing to the chatbot's intended tasks and audiences.
Measured conversation quality
Record explicit success criteria and evaluation conditions for complete interactions.
- What observable outcome counts as success for each supported use, and how are abandonment and unresolved conversations counted? definition
- What outcome rates, factual-error rates and response latencies have been measured for the deployed version across relevant tasks and languages? measurement
Refusal, handoff and service failure
Defines responses to unsupported requests, operational failures and needs for human intervention.
Unresolved interaction routing
Record when the chatbot stops attempting an answer or action, and how it communicates and routes the unresolved need.
- Which conditions trigger clarification, refusal, retry, human handoff or termination of the exchange? action
- When handoff is requested, how is transfer confirmed, what context is shared, and what happens if no human is available? action
Evidence and external alignment What the world already says about this thing, gathered so the model can be checked against it.
A model that cannot be lined up against existing standards, identifiers and practice cannot be adopted by anyone who already uses them.
Reported evidence
Findings from the breadth pass, kept separate from the structural claims.
Check these first
Recalled without web access and unsourced; every item is a lead to verify.
- This description covers the software-system sense; an individual conversation, transcript, persona, and underlying model are distinct things.
- The listed kinds overlap: task orientation describes purpose, while rules, retrieval, and generation describe mechanisms.
- No chatbot-specific universal identifier or standard is asserted; applicable requirements depend on implementation, deployment, and jurisdiction.
- Which of these check these first hold for the sense of chatbot this model covers, and on what evidence? provenance
Kinds and varieties
Recalled without web access and unsourced; every item is a lead to verify.
- Rule-based or scripted chatbot
- Retrieval-based chatbot
- Generative chatbot
- Hybrid chatbot combining rules, retrieval, and generation
- Task-oriented chatbot
- Open-domain conversational chatbot
- Which of these kinds and varieties hold for the sense of chatbot this model covers, and on what evidence? provenance
Real-world use
Recalled without web access and unsourced; every item is a lead to verify.
- Answering customer-service questions and routing requests to human staff
- Helping users find information in documents or knowledge bases
- Supporting transactions such as bookings and order tracking
- Providing tutoring and language-practice conversations
- Offering conversational assistance with writing and brainstorming
- Which of these real-world use hold for the sense of chatbot this model covers, and on what evidence? provenance
Typical measurements
Recalled without web access and unsourced; every item is a lead to verify.
- Task completion rate - No universal typical range; depends on task definitions and evaluation conditions - percent of evaluated tasks
- Response latency - No universal typical range; depends on architecture, workload, and whether first output or complete output is measured - seconds
- Which of these typical measurements hold for the sense of chatbot this model covers, and on what evidence? provenance
Failure modes and hazards
Recalled without web access and unsourced; every item is a lead to verify.
- Misinterpreting user intent or losing relevant conversational context
- Returning incorrect, outdated, or unsupported information
- Exposing sensitive information through responses, logs, or connected services
- Producing biased, abusive, or otherwise inappropriate responses
- For systems connected to tools, taking unintended actions after misunderstanding instructions or following malicious input
- Which of these failure modes and hazards hold for the sense of chatbot this model covers, and on what evidence? provenance
Regional variation
Recalled without web access and unsourced; every item is a lead to verify.
- Language, dialect, writing system, and cultural conventions affect conversational performance and evaluation.
- Requirements concerning privacy, disclosure, accessibility, and sector-specific use vary by jurisdiction.
- Which of these regional variation hold for the sense of chatbot this model covers, and on what evidence? provenance
Neighbouring kinds and how to tell them apart
Recalled without web access and unsourced; every item is a lead to verify.
- Language model - A language model predicts or generates language; a chatbot is an interactive system that may use such a model.
- Virtual assistant - A virtual assistant emphasizes helping perform tasks and may use nonconversational interfaces; a chatbot is defined by its conversational interaction.
- Conversational user interface - A conversational user interface is an interaction mechanism; a chatbot is the software participant presented through that mechanism.
- Social bot - A social bot automates activity on a social platform; it need not conduct conversations.
- Live chat service - Live chat can connect users to human operators; chatbot responses are produced automatically by software.
- Which of these neighbouring kinds and how to tell them apart hold for the sense of chatbot this model covers, and on what evidence? provenance
What the second pass must settle
- Should the registry's chatbot sense include voice-only conversational systems, or should those be related through a broader conversational-agent model?
- What minimum conversational behaviour separates a chatbot using constrained choices from a conventional menu interface?
- Which changes to persona, operator, underlying model, memory or tool permissions create a new chatbot identity rather than a version of an existing deployment?
- Which evaluation criteria can be shared across informational, transactional and companionship chatbots, and which require purpose-specific extensions?
- How should declared memory deletion, source grounding and permission enforcement be represented when deployment evidence is unavailable?