Building Recce's AI Data Review meant working around three hard limits in Claude: 200k context window, ~90k single prompt, 25k per MCP tool response.
A lineage diff for 5 changed models blew past the 25k MCP limit. Fixes: dataframes over key-value, numeric indices over full node IDs, filter to changed + downstream nodes only.
The agent also produced wrong lineage graphs. Switched from dbt's parent_map format to explicit edge lists matching Mermaid's native format. Accurate ever since.
Our own Kent Chen wrote up the architecture decisions and fixes the team landed on.
Read the full blog here: https://blog.reccehq.com/designing-reliable-ai-agents-for-dbt-data-reviews
Recce - Trust, Verify, Ship