From Prompt to Production: Building Reliable LLM Agents with Execution Gates
16th August, 2026
From Prompt to Production: Building Reliable LLM Agents with Execution Gates
I remember a late-night PagerDuty page: a prototype voice assistant had drafted an approval call that would have cost us money if placed automatically. The model could generate fluent, plausible plans — but it wasn't trustworthy enough to hit external APIs without human-in-the-loop controls. That gap — impressive language capability versus safe, auditable execution — is the engineering problem I keep returning to. In this post I give a compact, practical pipeline: a five-role mental model, a concrete voice-agent data flow, code-grounded patterns I've implemented, common failure modes, and a hands-on checklist you can use before letting an agent act.
In this post I give a compact mental model for how agents turn a prompt into action, a concrete architecture/data-flow example, what I've implemented around these ideas in my projects, where this breaks, and a practical checklist you can use before letting an agent act.
A simple mental model
Think of an agent as a pipeline with five roles:
- Ingest: accept user intent and normalize it. - Retrieve: fetch context (knowledge bases, memory, documents). - Plan: use the model to convert intent+context into a stepwise plan (tool calls, messages). - Execute: perform those steps against external tools/providers, subject to gates. - Verify & Audit: confirm outcomes, record evidence, and escalate if needed.
This mental model highlights where failures happen: bad retrieval leads to stale context, unsafe plans become dangerous actions, execution can fail for provider reasons, and verification is sometimes missing entirely.
Minimal data-flow example (ASCII)
User Prompt -> Preprocessor -> Retriever (KB, embeddings) -> Planner (LLM) -> Action Plan -> Policy Gates (approval, consent) -> Executor (provider API / tool) -> Verifier -> Audit Log
Component responsibilities in one table:
| Component | Responsibility | |---|---| | Retriever | Surface relevant facts, citations, and recent turns | | Planner | Produce deterministic steps and tool calls, include verification prompts | | Policy Gates | Consent checks, approval UI, rate & idempotency guards | | Executor | Convert abstract calls to provider-specific APIs, handle retries | | Verifier | Reconcile provider response with expected result; record evidence |
Architecture example: a voice agent that can book a demo
Here's a concrete architecture I use in prototypes like the Amazon Voice Agent (provider-agnostic runtime):
1) Ingest: speech-to-text + language detection -> normalized intent. 2) Retriever: tenant-scoped KB + recent-turn window (local embeddings & pgvector). 3) Planner: LLM constructs a step plan: "(1) ask for date, (2) check availability tool, (3) propose slots". 4) Policy Gates: consented memory check, approval required for outbound calls, encrypted provider keys. 5) Executor: mock-first voice runtime that can be switched to real provider adapters (Twilio, Alexa, etc.). 6) Verifier + Audit: store events, transcripts, and approval artifacts for operator review.
Diagram (simplified):
User Speech -> STT -> Intent -> Retriever -> LLM Plan -> Approval UI (if needed) -> Voice Runtime -> Provider API -> Call Events -> Audit/Logs
This flow separates planning from execution. The planner reasons in an abstract tool-space; executors map those abstract actions to provider SDKs. That separation enables dry-runs, unit tests, and health probes before live execution.
What I built around this idea
I’ve built multiple systems that embody parts of this pipeline. Below I list implemented capabilities and explicit gates that remain before live/automated operation:
- Amazon Voice Agent (implemented): provider-agnostic mock-first voice runtime, multilingual detection, consented memory, audit events, approval-gated calls, encrypted provider keys, health probes, autonomous test reports, and a full operator UI. (Gates: live streaming, barge-in, transfer, and real provider acceptance are still environment gates.)
- Google WhatsApp Growth Engine (implemented): lead classification, service matching, consent/review/approved-send gates, adversarial safety suite, Growth Engine UI for discovery, qualification, tailored drafts, one-page pitch outlines, review/export, durable local review-pack storage. (Gate: message sending remains review-only; no outbound messages were sent.)
- Universal Scraper (implemented): three-stage extraction pipeline, fixture tests, and live-safe runs for Product Hunt / Finsmes / billing leads. (Gates: robots/rate limits/provider blocking and selector drift still constrain live scraping.)
- ARIL local monorepo (implemented): tenant-scoped KBs, document chunking, embeddings, pgvector retrieval, citations, evaluation metrics, and protected routes. (Gate: production Postgres migration is an environment gate.)
I link the code and project pages in References so you can inspect the implementations directly [4].
Where this breaks (failure modes and mitigations)
1) Hallucinated actions: the LLM invents a tool or parameter that doesn’t exist. - Mitigation: planner must emit structured, schema-validated action calls; executors reject unknown actions and return a clear error for human review.
2) Stale or incorrect retrieval: the agent acts on outdated facts (pricing, availability). - Mitigation: freshness checks, source-level timestamps, and conservative defaults (ask user to confirm if KB age > threshold).
3) Provider-side failures and rate limits: API throttles or partial failures cause inconsistent state. - Mitigation: idempotency keys, retries with exponential backoff, circuit breakers, and compensating rollbacks logged in the audit trail.
4) Privacy or consent violations: agent leaks or uses protected information. - Mitigation: consent gates, redaction policies, encrypted provider keys, and fail-closed access controls.
5) Selector drift / scraping brittleness (for scrapers): CSS/DOM changes break extractors. - Mitigation: multi-source redundancy, synthetic tests, selector heuristics, and automated fixture tests.
6) Adversarial inputs / prompt injection: malicious text causes unsafe behavior. - Mitigation: adversarial safety suite, prompt sanitization, and a denylist for actions that touch billing/payment.
A practical checklist (before letting agents act automatically)
1. Structured actions: require schema-validated plans rather than free-form text directions. 2. Approval gating: human-in-the-loop for any monetary, legal, or high-risk action. 3. Idempotency: generate and persist unique operation keys for retries/compensation. 4. Retrieval verification: attach source citations and freshness metadata to each plan. 5. Dry-run defaults: systems default to simulation mode until explicit environment gates are flipped. 6. Audit trail & observability: persist events, LLM prompts/responses, provider results, and approval records. 7. Secrets & keys: encrypt provider keys, use health probes, and rotate keys regularly. 8. Safety tests: include adversarial test cases and fixture tests in CI.
Conclusion
Turning prompts into reliable action requires engineered constraints: structured plans, retrieval hygiene, execution guards, and strong observability. Models are excellent at producing fluent plans; the engineering task is making those plans safe, testable, and reversible. My work on voice runtimes, local KBs, scrapers, and growth engines embodies these principles: flexible planners combined with strict execution gates and auditability.
References
[1] Anthropic — Model Context Protocol: https://www.anthropic.com/news/model-context-protocol
[2] Model Context Protocol introduction: https://modelcontextprotocol.io/introduction
[3] Tool use and self-supervision paper (Toolformer): https://arxiv.org/abs/2307.03172
[4] My portfolio and project repos: https://github.com/deepanshuvermaa/my-portfolio
[5] Example research on retrieval and grounding (selected): https://arxiv.org/abs/2005.11401
Project evidence mentioned above is available in my public repositories linked in [4].