Industry Pivot Away from Prompt Engineering

According to the 2026 State of Context Management Report, 82 percent of IT and data leaders say prompt engineering alone is no longer sufficient to power AI at scale. The shift is reflected in investment patterns: 95 percent of data teams plan to invest in context engineering training in 2026, signalling a fundamental reorientation across the industry.

McMillan’s Landmark Study on File-Native Context

Independent researcher Damon McMillan published a peer-reviewed study in February 2026 titled “Structured Context Engineering for File-Native Agentic Systems,” running 9,649 experiments across 11 models and four context formats.

McMillan’s study tested 11 models against four context formats — YAML, Markdown, JSON, and a compact custom format called TOON — on schemas ranging from 10 to 10,000 tables.

Performance Across Model Tiers

Results split sharply by model class:

  • Frontier-tier models (Claude, GPT, and Gemini) saw file-based context retrieval improve accuracy by 2.7 percentage points.
  • Open source models experienced an aggregate 7.7 percentage point accuracy deficit with file-based context retrieval.

The gap between model tiers proved decisive. A 21 percentage point accuracy gap separated frontier and open source model tiers — the largest variable tested, exceeding any format or retrieval architecture effect.

Format Choice: A Non-Issue

McMillan’s study found that context format (YAML vs JSON vs Markdown vs TOON) had no statistically significant effect on aggregate accuracy; a chi-squared test returned p=0.484. Notably, compact context formats designed to minimise token count (such as TOON) sometimes caused models to spend more reasoning tokens on an unfamiliar format than they saved in raw size — a pattern researchers call the ‘grep tax’.

Schema Scale and Navigation

Well-structured, domain-partitioned schemas allowed file-native agents to scale up to 10,000 tables while maintaining high navigation accuracy, demonstrating that context engineering can support large-scale deployments when properly architected.

Advanced Prompting Techniques

Chain-of-Symbol (CoS) prompting outperforms Chain-of-Thought (CoT) for spatial and planning tasks by using symbols (e.g. ↑ ↓ [x]) to token-optimise the reasoning buffer.

The Metaprompt strategy involves using a reasoning model (e.g. GPT-5.2) to write the system prompt for a smaller production model (e.g. GPT-4.1-mini), achieving higher adherence at 1/20th the inference cost.

Source: Prompt AI Learning