AI Council 2026: How Model Verification Shapes AI Quality

AI Council 2026 showed that AI model verification affects what information can shape a model throughout its lifecycle.

Zhi Xiong, Head of Client Solutions, attended three days of sessions in San Francisco from May 12 to 14, 2026. Speakers examined verification across financial training data, reinforcement learning, synthetic data, and production agents.

AI Council 2026 ran from May 12 to 14, 2026, in San Francisco.

• Model evaluation: Test models on the intended task and consider how the training process shaped the result.
• Reinforcement learning: Open-ended tasks require expert-defined criteria to determine what constitutes a successful response.
• Synthetic data: Check generated examples against requirements for their output type before using them for training.
• Production agents: Keep runtime controls active when agents process external content or take high-impact actions.

Verification serves a different role as an AI system moves through development. During training, teams review data before it can influence the model.

Reinforcement learning introduces evaluation scores. These scores guide what the model learns, which makes the scoring criteria part of the training process. 

Synthetic data adds generated examples to the training pool. Each example needs a check tailored to its output before it is added to the dataset.

After deployment, agents can process retrieved documents or web content. They may also use tools to take action. Verification now needs to check external inputs and control agent actions.

The lifecycle creates 4 distinct verification questions:

• Training data: Is this example reliable enough to use?
• Reinforcement learning: Does the score reflect successful performance?
• Synthetic data: Does this generated example meet its required standard?
• Production agents: Can the system trust this input or approve this action?

Task Evidence Should Guide Model Selection

Model size describes parameter count. Task-level evaluation measures how a model performs on the intended workflow.

Snorkel AI's finance result reflected the model's task-specific training alongside its underlying capabilities.

Teams should compare models on representative tasks and review the training setup behind each result. Parameter count alone does not show which model fits the workflow.

Reward Design Shapes Reinforcement Learning

A reward signal tells a model which responses training should reinforce.

Objective tasks can use measurable outcomes. Open-ended tasks require domain-expert-defined criteria before the environment can assign a useful score.

Those criteria become part of the training process because they determine which behavior receives positive feedback.

Synthetic Data Needs Output-Specific Checks

Synthetic data can contain different types of errors depending on the output. Verification needs to match the failure being checked.

This synthetic data filtering determines which generated examples are suitable for training.

Production Agents Need Clear Trust Boundaries

External content introduces a trust problem once an agent begins operating in production.

Retrieved documents and web content can contain hidden instructions. Without clear trust boundaries, these instructions can influence the agent through prompt injection.

Runtime controls should separate trusted instructions from external content and restrict high-impact actions that require additional oversight.

How a 4B Model Outperformed a 235B Model on Finance

Charles Dickens, Senior Research Scientist at Snorkel AI, presented a 4B-parameter model that outperformed a 235B-parameter model from the same family on the finance task.

Training took about 21 hours using 8 NVIDIA H100 GPUs. Snorkel reported a total cost of under US$500, including the GPU resources used by a second model that graded responses.

Snorkel built the training data from financial tables in SEC filings. A separate model generated question-answer pairs from those tables before each pair passed through 3 review stages:

• Programmatic check: The answer was checked against its source table.
• Independent model review: Two language models independently reviewed the calculations and extracted values.
• Expert audit: Financial experts reviewed difficult cases before approving the data.

The model was first trained on questions using a single table. Snorkel later tested it on more complex questions that required information from multiple tables.

The result reflects the complete task setup, including the verified finance data and the training process designed for the task. The available evidence does not isolate verification as the sole cause of the performance difference.

Figure 1. Snorkel AI's financial data verification pipeline

AI-generated question-answer pairs moved from SEC financial tables through automated checks and expert review before entering model training.

Reinforcement Learning for Open-Ended Work

The AI Council session, The World Is Not Enough: RL's Environment Problem, featured Nic Ouporov of Fleet, Thais Castello Branco of Taste, and Vincent Weisser of Prime Intellect.

The panel examined how reinforcement learning works when a task has no single correct answer.

In reinforcement learning, a model attempts a task and receives a reward signal that guides later responses.

Objective tasks are easier to score because the result can be checked directly. Generated code can run against expected outputs, while numerical answers can be compared with verified values.

Open-ended work may have several acceptable outcomes. Domain experts need to define what success looks like before the environment can turn that judgment into a useful score.

The panel identified the design of the RL environment as a major bottleneck. Poorly defined criteria can reward responses that receive a high score without meeting the intended objective.

How NVIDIA Filtered Synthetic Training Data

In RLVR in Practice: From Synthetic Data to GRPO, NVIDIA's Chris Alexiuk explained how verification supports synthetic training data.

NVIDIA described how Nemotron 3 Super used about 7 million examples during supervised fine-tuning. Most of this data was synthetically generated by other models.

Before entering the training dataset, each generated example passed automated quality checks:

• Math answers: Tested for correctness.
• Generated code: Run and checked against expected results.
• Structured outputs: Validated against the required schema.
• Conversation quality: Graded by a second model.
• Multiple responses: The system retained the response with the strongest agreement.
• Benchmark overlap: Samples that matched evaluation benchmarks were removed to keep the training data separate from the testing set.

These checks filtered the generated examples before training. Each one targeted a specific type of output.

Runtime Controls Protect Agents From Untrusted Inputs

In Agentic AI: From Risk Awareness to Practical Control, Noma Security CISO Diana Kelley examined how agents handle external information during live workflows.

Kelley described an agent context that can combine system instructions, user requests, retrieved documents, and web content. Once those sources enter the same workflow, the system needs controls that preserve the boundary between trusted instructions and external material.

She presented 4 controls:

• Instruction separation: Keep trusted system instructions separate from retrieved content.
• External-input checks: Inspect external material before it reaches the model.
• Action logging: Record tool calls and agent actions for review.
• Human approval: Require approval before high-impact actions proceed.

These controls align with the OWASP GenAI Security Project and its work on agentic AI security.

Verification should target the failure that could make a result unusable or unsafe.

Teams can assess each verification point through 4 questions:

• Failure: What could go wrong at this stage?
• Evidence: What information can confirm the result?
• Decision: Can a system reliably verify the result?
• Escalation: When does the result need human review?

The answers determine the appropriate check. The verification method should align with the risk posed by the workflow.

Verification Adds Work Beyond Model Training

Model-based evaluation adds to the inference workload because another model may need to score or review the output.

Some tasks also require specialists to define evaluation criteria or investigate difficult cases. Teams need to account for this work when estimating project cost and timelines.

Verification Needs Clear Ownership

Verification responsibility can change as a system moves through development.

Teams should define who sets the evaluation standard and who is responsible for enforcing it once the system reaches production.

1. Define What Each Check Must Prove

Document the evidence required for a result to pass verification.

2. Test Verification Before Production

Run representative failure cases through the workflow and confirm that the intended checks catch them.

3. Record Where Human Review Is Required

Identify the decisions that cannot be verified reliably through automated checks.

The examples from AI Council 2026 point to a practical constraint: AI quality depends on the decisions used to evaluate it.

A system can pass its evaluation and still fail the requirements of its intended workflow. Verification only works when the check reflects the result that actually matters.

For enterprise teams, this makes evaluation design part of the decision about whether an AI system is ready for use.

AI Council founder Pete Soderling closed the conference with an original AI-themed song at the piano.

Interested in exploring the talks in more detail? Recordings from AI Council 2026 are available on the AI Council YouTube channel.