A. Execution Plan (Blueprint §12)
P0 was released in Nov 2026. The N1 96 GB smoke + FP8 precision replication, WP-18 placebo/oracle, 24-item calibration and verifier audit precede the G2 pilot in Dec 2026; the G3 Core run follows in Jan 2027; the G4/G5 Full run plus the benchmark ladder follows in Feb; human eval/SDT in Mar; results freeze in Apr; thesis chapters in May; thesis submission in Jun; defence in Jun/Jul; paper after the defence (Aug–Dec 2027). Gates mark explicit go/no-go decisions. Core and Full are separate matrices on the same 4-bit main stack; do not assume Core cells are reusable. The plan totals $500–600 with a $600 hard stop; the main matrix runs the 4-bit 24–27B stack (~53.3 GB) on one 80 GB A100 at $0.136/GPU-h, and the 96 GB FP8 stack is limited to the N1 smoke and a small GPQA precision replication. If no 80 GB A100 is available, the fallback is $1,050–1,150 and requires re-approval.
| Phase | Duration | Key Deliverables | Progress |
|---|---|---|---|
| Ph 0: Literature & Setup | Jul–Sep 2026 | Literature freeze, vLLM 3-model serving on the A6000, Gate 0 MAD reproduction — passed 16 Sep 2026 (10/10 debates × 3 rounds) | |
| Ph 1: P0 Engineering | Oct–Nov 2026 | Core wired: options in prompts, parser/scorer, trust simplex projection, verifier interface, claim-domain routing, endpoints, manifest/resume. 514 tests; P0 gate 13/13 passed. | |
| Ph 2: Pilot run | Dec 2026 | N1 96 GB smoke + FP8 precision replication ($25–40) → WP-18 placebo/oracle → 24-item calibration → verifier audit (150–200 pairs; lexical 0.58 balanced accuracy; target ≥0.75, κ ≥0.60) → 50-question Month-1 pilot → 100–150-question G2 pilot. 3 seeds for the pilot; 2 for the main matrix. G2 decides whether the Core run proceeds. | |
| Ph 3: Core run | Jan 2027 | G3 Core run: 4 executable arms (B3, B4-matched, B8-no-trust, B8) × GPQA-Main 400 / MMLU-Pro 400 / HLE 200 × 2 seeds = 24 runs, 1,280 GPU-h. B8-oracle is not a Core arm; SciClaimBench is not in Core v1.0 (180 items at the v1.2 benchmark freeze). PRE_REGISTRATION.md v1.0. | |
| Ph 4: Full run + benchmark | Feb 2027 | G4/G5 Full: 32 rows at 300 items/dataset with the gold-evidence oracle on SciClaimBench (1,280 GPU-h; $210), plus a 36-row gold-answer alternate (1,440 GPU-h; $235). Adds B8-placebo/B8-oracle via WP-18. SciClaimBench ladder: 180 floor, 300 target, 600 stretch, 800 no-go. | |
| Ph 5: Human Eval + SDT | Mar 2027 | Human evaluation / SDT, benchmark annotation, and verifier-audit follow-up. | |
| Ph 6: Results Freeze | Apr 2027 | Freeze results, CIs, and effect sizes. No new runs or arm changes after the freeze. | |
| Ph 7: Thesis Chapters | May 2027 | Write the thesis chapters against the frozen results. | |
| Ph 8: Thesis Submit + Defence | Jun–Jul 2027 | Thesis submitted June 2027; defence June/July 2027. | |
| Ph 9: Paper | Aug–Dec 2027 | Paper writing and submission after the defence (TACL rolling; EMNLP 2028 fallback). |
- Gate 0 — achieved 16 Sep 2026 (Jul–Sep 2026): Vanilla MAD reproduction (Du et al. 2023) on a GPQA slice: 10/10 debates completed all 3 rounds, no empty positions, mean 370.8 s/debate. Full report in experiments/gate0/.Working MAD reproduction — done
- P0 engineering gate (Nov 2026): 615 tests; P0 gate 13/13 passed and released (tags p0-gate, p0-release). Current report evidence: 70-pp report, 19-pp journal, 0 undefined references. Core wiring is in place — this is engineering readiness, not an efficacy result.P0 gate report + runnable matrix — 13/13
- Pilot run (Dec 2026): N1 96 GB smoke + FP8 precision replication, WP-18 placebo/oracle, a 24-item calibration, and the verifier audit precede the pilot. The 50-question Month-1 pilot is followed by the 100–150-question G2 pilot. G2 requires baseline CCR ≥0.30, trust AUROC ≥0.65, ≥65% correct-direction changes, and injection κ ≥0.75; if it fails, the project reframes to report the negative result honestly.G2 pilot-run report
- Core run (Jan 2027): G3 Core run: 4 executable arms × GPQA-Main 400 / MMLU-Pro 400 / HLE 200 × 2 seeds = 24 runs, budgeted at 1,280 GPU-h. B8-oracle is not a Core arm, and SciClaimBench is not in Core v1.0. No efficacy claim is made before the pre-registered analysis.G3 Core matrix + pre-registered causal contrast (B8 vs B8-no-trust)
- Full run + benchmark (Feb 2027): G4/G5 Full: 32 rows at 300 items/dataset with the gold-evidence oracle on SciClaimBench (1,280 GPU-h), adding B8-placebo/B8-oracle through WP-18; the 36-row gold-answer alternate is 1,440 GPU-h. SciClaimBench benchmark freeze at 180 items (v1.2); ladder 180 floor, 300 target, 600 stretch, 800 no-go.Full-run matrix + SciClaimBench v1.2
- Human eval + SDT (Mar 2027): Human evaluation and SDT, benchmark annotation, and the verifier-audit follow-up.Human-eval + benchmark package
- Results freeze (Apr 2027): Freeze all results, confidence intervals, and effect sizes. No post-hoc arm or endpoint changes.Frozen results
- Thesis chapters (May 2027): Write the thesis chapters against the frozen results.Thesis draft
- Thesis submission (Jun 2027): Submit the FYDP thesis in June 2027.Submitted thesis
- Defence (Jun–Jul 2027): Defend the thesis in June/July 2027.Defence slides
- Paper after defence (Aug–Dec 2027): Write and submit the paper version after the defence; TACL rolling submission, EMNLP 2028 fallback.Paper submission
B. Implementation Order (Blueprint §13)
Exact build sequence with dependency reasoning. Each step depends on the previous.
Two long-lived branches only: main (public mirror) and dev (development; main is the release deploy). Snapshots are git tags, not long-lived phase branches: p0-gate and p0-release exist now; Pilot, Core and Full tags are planned. Feature work lands on short-lived slices and merges back with --no-ff.
- Never build trust update before retrieval exists.
- Never build B9 (iMAD) before the core system (steps 1–7).
- Never run the Core run before the Dec 2026 calibration + pilot confirms the core assumption.
C. Month-by-Month Learning Plan (Honest)
What you actually need to learn, when, and how much — mapped to the blueprint's build order. The rule: ~30% watching, ~70% building. Every month ends with a running script, not notes.
| Month | Phase | Effort | Learn (only this) | Build / Exit check | Links |
|---|---|---|---|---|---|
| Jul–Sep 2026 | Ph 0 — complete | ~15 hr/wk | Only the Transformer / attention / quantization units from the ML/DL playlists (skip what you know); structured prompting for tagged output; vLLM + HF Hub basics. | Repo skeleton; 3-model vLLM serving on the A6000; Gate 0 MAD reproduction passed 16 Sep 2026 (10/10 debates × 3 rounds, mean 370.8 s/debate). | |
| Oct–Nov 2026 | P0 — released Nov 2026 | ~20 hr/wk | LangGraph StateGraph subset — the ~5-API core, not the whole library (CampusX playlist, videos 1–6 only); trust projection math from the P0 interfaces. | Core wired: options in prompts, parser/scorer, trust simplex projection, verifier interface, claim-domain routing, endpoints, manifest/resume. 514 tests; P0 gate 13/13. | |
| Dec 2026 | Pilot run | ~15 hr/wk | Only the stats you will actually run: paired bootstrap, McNemar, Cohen's d, Cohen's κ — short scipy walkthroughs, no full stats course. | N1 96 GB smoke + FP8 precision replication ($25–40); WP-18 placebo/oracle; 24-item calibration; verifier audit (150–200 pairs; lexical 0.58 balanced accuracy; target ≥0.75, κ ≥0.60); 50-question Month-1 pilot, then the 100–150-question G2 pilot. G2 decides whether the Core run proceeds. | |
| Jan 2027 | Core run | ~20 hr/wk | Nothing new — debugging discipline and log inspection only. If you need a concept, look it up on demand. | G3 Core run: 4 arms × GPQA-Main 400 / MMLU-Pro 400 / HLE 200 × 2 seeds = 24 runs, 1,280 GPU-h. B8-oracle is not a Core arm; SciClaimBench is not in Core v1.0. No efficacy claim until the pre-registered analysis. | — |
| Feb 2027 | Full run + benchmark | ~20 hr/wk | Benchmark annotation and verifier-audit protocol; no new ML theory. | WP-18 adds B8-placebo/B8-oracle; G4/G5 Full primary is 32 rows at 300 items/dataset with the gold-evidence oracle on SciClaimBench (1,280 GPU-h; $210); the 36-row gold-answer alternate is 1,440 GPU-h ($235). SciClaimBench: 180 floor, 300 target, 600 stretch, 800 no-go. | — |
| Mar–Apr 2027 | Human eval + freeze | ~15 hr/wk | Inter-annotator agreement (κ/ICC) for the human evaluation. | Human evaluation / SDT, benchmark annotation, verifier-audit follow-up, then results freeze with CIs and effect sizes. No post-hoc arm changes. | — |
| May–Jul 2027 | Thesis + defence | ~15 hr/wk | Venue/thesis formatting only; no new technical material. | Thesis chapters (May), submission (Jun), defence (Jun/Jul). | — |
| Aug–Dec 2027 | Paper | ~10 hr/wk | TACL/EMNLP writing guides and reviewer-response practice. | Write and submit the paper after the defence. TACL rolling; EMNLP 2028 fallback. | — |
- Do not binge-watch playlists. Binge-watching is the #1 way to spend a month learning nothing you can reproduce. Watch the one video for the concept you are about to implement, then implement it.
- Do not take a full stats or ML course. This project needs ~6 statistical tests and ~4 architecture concepts. Learn them on demand.
- Skip if you already know it: Python (skip 100 Days of Python entirely), basic ML concepts, and any LangChain content — you need LangGraph only, and only its StateGraph core.
- If you fall behind: cut the reranker to a simple overlap score and catch up in the Pilot run — never skip the Gate 0 MAD reproduction or the Dec 2026 calibration + pilot.
D. Learning Resources (Verified Links)
A short shelf, not a library. Prefer the official docs for everything you build; use the playlists for concepts only. All links verified.
E. Learning Level Guide
| Level | Meaning |
|---|---|
| 🔵 Beginner | Basic conceptual understanding only |
| 🟢 Intermediate | Comfortable using and modifying existing implementations |
| 🟡 Advanced | Able to implement, debug, and explain independently |
| 🔴 Expert | Deep enough understanding to contribute novel research ideas |
Core ML & Deep Learning
| # | Topic | Level Needed | Importance | Why Needed | Learn |
|---|---|---|---|---|---|
| 1 | Neural Networks | 🟢 Intermediate | High | Understand model structure and inference | 100 Days of DL (CampusX) ↗ |
| 2 | Transformer Architecture | 🟡 Advanced | Critical | Foundation of all LLM agents | 100 Days of DL — Transformer units ↗ |
| 3 | Attention Mechanism | 🟡 Advanced | Critical | Understanding trust influence and context handling | 100 Days of DL (CampusX) ↗ |
| 4 | Embeddings & Vector Representations | 🟢 Intermediate | Critical | Core of RAG and retrieval pipelines | 100 Days of ML (CampusX) ↗ |
| 5 | Model Quantization (GPTQ / AWQ) | 🟢 Intermediate | High | Efficient 4-bit inference on the 80 GB A100 main stack | vLLM quantization docs ↗ |
| 6 | Inference vs Fine-Tuning | 🟢 Intermediate | Medium | Your work focuses on inference orchestration | HF docs ↗ |
Large Language Models
| # | Topic | Level Needed | Importance | Why Needed | Learn |
|---|---|---|---|---|---|
| 1 | LLM Text Generation | 🟡 Advanced | Critical | Understand how models reason and fail | HF docs ↗ |
| 2 | Chain-of-Thought (CoT) | 🟡 Advanced | Critical | Main baseline and reasoning style | Wei et al. 2022 ↗ |
| 3 | Self-Consistency Decoding | 🟡 Advanced | Critical | Major experimental baseline (B5) | Wang et al. 2022 ↗ |
| 4 | Prompt Engineering | 🟢 Intermediate | High | Agent orchestration and structured prompting | OpenAI prompt guide ↗ |
| 5 | Temperature & Sampling | 🟢 Intermediate | Medium | Generating agent diversity | HF docs ↗ |
| 6 | Hallucination in LLMs | 🔴 Expert | Critical | Your main failure mode | Your literature review ↗ |
| 7 | Sycophancy in LLMs | 🔴 Expert | Critical | Core research problem | Sharma et al. 2023 ↗ |
| 8 | Model Calibration (ECE) | 🔴 Expert | Critical | Foundation of trust calibration | Guo et al. 2017 ↗ |
| 9 | Two-Phase Model Strategy | 🟢 Intermediate | High | Dev stack (Qwen3.5-9B / Gemma 4 12B / Ministral-3-14B) → 4-bit main stack (Qwen3.8-27B-AWQ-INT4 ~21.0 GB / Gemma 4 26B-A4B AWQ ~17.2 GB / Mistral-24B-AWQ 15.06 GB; ~53.3 GB, one 80 GB A100); FP8 check stack (73.68 GB) is N1 smoke + GPQA precision replication | Proposal §7 ↗ |
| 10 | Sycophancy Injection Protocol Design | 🔴 Expert | Critical | CCR operationalization; κ > 0.75 pilot check (Phase 1–2) | Proposal §5.4 ↗ |
| 11 | LLM Uncertainty & Confidence Estimation | 🟡 Advanced | High | Lightweight confidence gating (Phase 2) | Your literature review ↗ |
Multi-Agent Systems
| # | Topic | Level Needed | Importance | Why Needed | Learn |
|---|---|---|---|---|---|
| 1 | Multi-Agent Debate (MAD) | 🔴 Expert | Critical | Main paradigm your work extends | Du et al. 2023 ↗ |
| 2 | Mixture of Agents (MoA) | 🟡 Advanced | High | Important baseline (B6) | Wang et al. 2024 ↗ |
| 3 | iMAD Framework | 🟡 Advanced | Critical | Closest competitor baseline | Your iMAD review ↗ |
| 4 | Consensus Mechanisms | 🟡 Advanced | High | Understanding majority-vote limitations | — |
| 5 | Trust & Reputation Systems | 🔴 Expert | Critical | Foundation of your contribution | Your iMAD review ↗ |
| 6 | Agentic AI Paradigm | 🟢 Intermediate | Medium | Broader research positioning | LangGraph playlist (CampusX) ↗ |
| 7 | Debate Prompt Design | 🟡 Advanced | High | Structuring adversarial reasoning (Phase 1–2) | LangGraph playlist (CampusX) ↗ |
| 8 | Adaptive Triggering / Confidence Gating | 🟡 Advanced | High | Gate full debate pipeline (Phase 2) | Proposal §5.2 ↗ |
| 9 | Heterogeneous Multi-Model Agent Design | 🟡 Advanced | High | Reducing correlated hallucinations | Proposal §5.4 ↗ |
Retrieval-Augmented Generation (RAG)
| # | Topic | Level Needed | Importance | Why Needed | Learn |
|---|---|---|---|---|---|
| 1 | Dense Retrieval (DPR, Contriever) | 🟡 Advanced | Critical | Atomic claim retrieval backbone | sentence-transformers ↗ |
| 2 | Cross-Encoder Reranking | 🟡 Advanced | Critical | Evidence scoring per claim (Phase 2) | sentence-transformers ↗ |
| 3 | FAISS Vector Index | 🟢 Intermediate | High | Efficient similarity search at scale | FAISS (GitHub) ↗ |
| 4 | Source-Partitioned Retrieval | 🔴 Expert | Critical | Your novel retrieval strategy (Phase 2–3) | Proposal §5.5 ↗ |
| 5 | Atomic Claim Decomposition | 🔴 Expert | Critical | Breaking agent utterances into verifiable propositions (Phase 2) | Proposal §5.5 ↗ |
| 6 | RAG Evaluation Metrics | 🟡 Advanced | High | Measuring retrieval quality (Phase 3–4) | lm-eval-harness ↗ |
Evaluation & Statistics
| # | Topic | Level Needed | Importance | Why Needed | Learn |
|---|---|---|---|---|---|
| 1 | Statistical Significance Testing | 🟡 Advanced | Critical | Paired bootstrap, McNemar tests for all main results | SciPy docs ↗ |
| 2 | Effect Size Reporting | 🟡 Advanced | Critical | Cohen's d, Cliff's delta for sycophancy reduction claims | — |
| 3 | Inter-rater Agreement (κ) | 🟡 Advanced | High | Validating sycophancy injection protocol (Phase 1 pilot) | — |
| 4 | Ablation Study Design | 🔴 Expert | Critical | Isolating each component contribution; required for Q1 submission | Proposal §11 ↗ |
| 5 | Calibration Metrics (ECE, ECR) | 🔴 Expert | Critical | Calibration endpoint for the trust signal (H3) | Guo et al. 2017 ↗ |
| 6 | Benchmark Evaluation Harness | 🟡 Advanced | High | Reproducible multi-benchmark evaluation | lm-eval-harness ↗ |