There is a practical challenge across research, public programs, and evidence-based decision-making: organizations have more data and documentation than ever, but users still need help finding the right source, interpreting its limitations, and applying it responsibly. Conversational artificial intelligence (AI) can reduce that burden when it is designed not as a general-purpose chatbot, but as a controlled interface to approved evidence, data, and workflows.

Large language model (LLM)-powered conversational AI is becoming a new interface for complex evidence and data systems. Instead of requiring users to know where information is stored, which fields to query, which filters to select, or how policy documents are organized, these tools allow users to describe their needs in ordinary language. The system can then retrieve relevant evidence, generate or execute structured queries, reason across documents or datasets, and return outputs that are understandable, traceable, and fit for purpose.

At NORC, we are advancing trustworthy conversational AI systems built on LLMs, retrieval-augmented generation (RAG)—a technique that connects AI models to specific approved sources—and agentic RAG patterns, which incorporate AI agents capable of independently planning and executing multi-step tasks. These systems translate user intent into grounded summaries, structured queries, evidence tables, decision-support outputs, and auditable responses.

The technical foundation for trustworthy conversational AI combines LLMs with retrieval-grounded workflows that connect user questions to approved sources, structured data, and validation steps.

• LLMs provide the natural language reasoning and generation layer.
• RAG connects those models to curated knowledge bases, approved documents, structured datasets, metadata, and workflow tools.
• Agentic RAG adds planning, tool use, iterative retrieval, source comparison, structured data access, and output validation.

NORC senior fellow Robert M. Goerge recently highlighted how generative AI can help state agencies navigate policy change by synthesizing guidance, surfacing relevant requirements, and helping staff identify the information needed to respond consistently across programs.

For example, a user might ask whether a policy threshold changed between two program manuals and how many records would be affected. The system can retrieve the relevant sections, compare the language, translate the rule into query logic, apply access controls, retrieve the data, and return a cited summary with uncertainty markers, areas where the tool lacks confidence in its generated answer.

This capability extends beyond document-based question answering. NORC is applying conversational AI to data retrieval workflows where users often face complex query portals, technical filters, or database-specific interfaces. A natural language interface can allow users to describe the information they need, translate that request into structured query logic, retrieve relevant data, and present results as tables, summaries, downloadable extracts, or follow-up prompts. This lowers barriers for nontechnical users while preserving the precision, access controls, and auditability required for authoritative data systems.

NORC’s Trusted Health Information Assistant (THIA) is one example of how this architecture can be applied in a research setting. Rather than treating a chatbot as a stand-alone communication tool, THIA uses a controlled knowledge base and retrieval-augmented generation to create a study environment where researchers can examine how people seek, interpret, and respond to AI-mediated health information. Its value is not only in answering participant questions, but in making the interaction itself measurable, configurable, and suitable for research on comprehension, engagement, and trust.

The California CalWORKs Outcomes and Accountability Review (Cal-OAR) Assistant applies the same grounded conversational AI approach to a different evidence environment. The assistant is designed to help users navigate a knowledge base built from service system assessment and improvement documentation from each of California’s 58 counties, covering a range of program and service topics. By retrieving responses from curated source material rather than relying only on general model knowledge, the system can provide more traceable, context-specific responses while supporting responsible information access.

Together with our structured data retrieval work, THIA and the Cal-OAR Assistant demonstrate NORC’s cross-domain capability to build LLM-powered, source-grounded conversational systems adaptable to different evidence bases, data systems, workflows, user groups, and oversight requirements.

Making this capability work across domains requires more than connecting an LLM to a knowledge base. We emphasize approved-source grounding, transparent citations, privacy safeguards, accessibility, role-based controls, audit logs, abstention behavior, and human review for high-stakes uses. Evaluation should measure not only whether outputs are fluent, but whether they are accurate, complete, reproducible, well-cited, and appropriate for the decision context.

The goal of conversational AI systems is not to replace researchers, analysts, program experts, or decision-makers, but to give them better infrastructure for moving from information search to evidence-guided action while preserving trust and accountability.