Final Year Design Project · The Trust Boundary

The Boundaries of Evidence-Grounded
Trust in Multi-Agent Scientific Debate

with a Controlled Benchmark for Sycophantic Consensus · Agentic AI · TACL / EMNLP 2028

Prepared ByMd. Atikur Rahaman
GitHubatik203
TimelineJul 2026 – Dec 2027
Key DatesThesis Jun 2027 · Defence Jun/Jul 2027 · Paper Aug–Dec 2027

Executive Summary

Multi-agent debate (MAD) improves LLM reasoning but is susceptible to inter-agent sycophancy: a confident hallucinating majority systematically suppresses correct minority agents. No existing framework provides an in-session, evidence-grounded mechanism to prevent this.

We propose a trust-calibrated deliberation system that re-weights each agent's influence dynamically based on the verifiability of its claims against retrieved scientific literature. The core contribution is a formally specified trust update: log-prior evidence accumulation, one softmax, and a floor/ceiling simplex projection with proven bounds.

Secondary contribution is an open evaluation harness (CCR/MPR/ECR metrics; 4 executable arms and a 6-arm target) for future sycophancy mitigation research.

We target ≥20% sycophancy reduction (CCR metric) vs. vanilla MAD on GPQA-Main, MMLU-Pro STEM and SciClaimBench primary sets; HLE is the abstention control, and BrokenMath/BrokenArXiv are appendix-only. The Core run (24 runs) is pre-registered; no efficacy claim is made yet.

514
P0 Tests Passing
13/13
P0 Gate Passed
24
Core Runs
300
Benchmark Target (180 floor)
3
Agent Roles in Pipeline
P0 build status
Implementation evidence, not efficacy results
514
Tests passing
13/13
P0 gate checks
24
Core runs · 1,280 GPU-h
300
Benchmark target · 180 floor
~53.3 GB
4-bit main stack · 80 GB A100

Current report evidence: 70-pp report, 19-pp journal, 0 undefined references; tags p0-gate and p0-release. Next: N1 96 GB smoke + FP8 precision replication → WP-18 placebo/oracle → 24-item calibration → verifier audit → G2 pilot (Dec 2026) → G3 Core (Jan 2027) → G4/G5 Full (Feb 2027).

The Problem

Confident wrong majorities override correct minorities in AI debate — no existing fix.

Our Solution

Evidence-grounded trust weighting: agents whose claims check out against real papers earn more influence.

The Goal

Target ≥20% sycophancy reduction on GPQA-Main, MMLU-Pro STEM and SciClaimBench; 4 executable arms and a 6-arm target, with B1–B10 as comparison definitions. Efficacy is not yet measured.

Problem & Research Gap

The Problem

In multi-agent LLM debate, a confidently wrong majority can cause a correct minority agent to abandon its position — even without any new evidence being presented. Standard majority-vote aggregation then locks in the wrong answer. This is called inter-agent sycophancy.

The Gap

Diagnosis exists (Yao et al. 2025; He et al. 2026). Efficiency optimization exists (iMAD, AAAI 2026). Static aggregation exists (MoA, ICLR 2025 Poster). Nothing existing dynamically re-weights agent trust during a debate based on external evidence. That is the specific, verified gap.

FYDP Research

The Boundaries of Evidence-Grounded Trust in Multi-Agent Scientific Debate, with a Controlled Benchmark for Sycophantic Consensus.

Jul 2026 – Dec 2027 · Thesis Jun 2027 · Defence Jun/Jul 2027 · Paper Aug–Dec 2027

Single 80 GB A100 · Open-weight Models

© 2026 Md. Atikur Rahaman · Final Year Design Project