Scoping an Agentic Application

Where to draw the line between agentic and traditional software.

Tyson Gern

Scoping an Agentic Application

Key Takeaways

• Not every application that uses an LLM needs an agentic workflow; deterministic flows are better suited to traditional software with targeted LLM calls.
• MCP is helpful when building tools for agents you don't control, otherwise tool calling is usually a better choice.
• Communication patterns between agents carry the same challenges as in any system. Make sure you understand the tradeoffs before choosing an implementation.
• Agent frameworks help teams move quickly at first at the cost of less control over agentic workflows.
• Spec driven development is showing early promise for bounded development tasks.

Introduction

We have had an influx of project scopings to kick off the new year, possibly the busiest January and February in recentover the last few years. The excitement around ChatGPT, Claude, and open-source models has led to customers pushing to build agentic systems, and we're noticing a common challenge: Teams want to build with agents but aren't sure where to draw the line between agentic and traditional software.

Over the course of these scopings we've developed a set of questions that help teams land on an effective approach to planning and architecting agentic work. These questions surface tradeoffs early before a team commits to an architecture that doesn't fit the problem.

Is the Workflow Deterministic or Non-Deterministic?

This is the first, and most important, question we ask. A deterministic flow is one where, given the same input, you expect the same output every time. Think of a data pipeline that extracts fields from a document, validates them against a schema, and writes them to a database. The steps are known ahead of time, and the order doesn't change.

A non-deterministic flow is one where the path through the system depends on reasoning that can't be fully specified in advance. A customer support agent that needs to read a ticket, decide whether to escalate or respond, look up account history, then draft a reply is making decisions at each step. The next action depends on what the model understands about the current situation.

If your workflow is deterministic, you likely don't need an agent to orchestrate it. Traditional software that makes LLM calls at specific points (when summarizing text, classifying input, or extracting entities) is simpler to build, test, and operate. You get the benefits of language model capabilities without the complexity of an autonomous loop.

If your workflow is non-deterministic, agentic orchestration can be a powerful tool. The agent evaluates the system's output, decides what tool to call next, and iterates toward a goal. Don't jump to this solution too soon. We've seen teams reach for agents when a well-structured pipeline with a few LLM calls would have been sufficient and far easier to maintain.

Is Your Main Interface a Chat Agent?

We ask this question to determine the shape of the system. If your application is primarily accessed through a conversational interface like ChatGPT or Claude, consider writing an Model Context Protocol (MCP) server that integrates well with these interfaces. MCP gives the chat interface structured access to your tools and data sources, adding new capabilities to the conversational interface, and allowing agents to reach out to your systems behind the scenes.

If your application's main interface is something other than chat (like a web application, a background service, or an internal tool with a traditional UI) then you'll likely have more success with function calling and tool use integrated into your existing architecture. In this case, the LLM interactions are a component within a larger system rather than as the primary interface. You make API calls to a model, receive structured responses, and use them to drive application logic.

The distinction matters because it shapes how you think about state, context, and user experience. A chat agent maintains a running conversation and manages context windows carefully. With the tool-calling approach the application is responsible for managing its own state. Choosing the wrong pattern leads to unnecessary complexity in one direction or a poor user experience in the other.

Is Communication Between Agents Lossy?

When building a system with multiple agents or services, think carefully about how they communicate. This is the same question distributed systems engineers have been asking for decades. It takes on new importance when agents are making decisions based on the messages they receive.

The key question is whether your system requires once-and-only-once message delivery or whether it can tolerate some loss. Agents processing financial transactions, generating compliance reports, or coordinating steps in a workflow where skipping a message means skipping work, need guaranteed delivery. A message broker like RabbitMQ provides acknowledgments, retries, and dead-letter queues. Messages don't disappear.

If your system is more event-driven, with agents reacting to streams of data where an occasional missed event is acceptable or where events might be replayed, then something like Kafka is a better fit. Kafka gives you high throughput, replay capability, and the ability to add consumers without disrupting producers. The tradeoff is that delivery semantics are at-least-once by default, and achieving exactly-once requires additional coordination.

Don't overthink this decision if your system is simple. If you have two agents passing messages in a single process, you don't need a message broker at all. But if you're building a system where agents operate independently and communicate asynchronously, get the messaging semantics right early. Fixing them later is painful.

Should I Use an Agent Framework?

This depends on the team. Agent frameworks like Google's Agent Development Kit (ADK), LangChain, and CrewAI provide scaffolding that helps teams get started quickly. They handle common patterns like tool registration, conversation memory, and agent orchestration so developers can focus on the problem domain rather than the plumbing.

For teams new to building with LLMs, a framework like ADK can be a great choice. It provides sensible defaults and enough structure to keep a team moving quickly without requiring deep familiarity with the underlying APIs. The documentation tends to be example-heavy, which helps developers learn in context.

The alternative is direct API integration using an SDK like OpenAI's or Anthropic's. This approach gives you full control over prompts, tool definitions, conversation management, and error handling. You write more code up front, but have more fine-grained control. When something goes wrong (and in agentic systems, things do go wrong) you're not debugging through layers of framework abstraction.

In our experience, experienced teams that have a firm understanding of LLMs and their HTTP APIs tend to prefer direct integration. Teams that are earlier in their LLM journey or want to prototype quickly get more value from frameworks. Neither choice is permanent. We've seen teams start with a framework, learn the patterns, and then migrate to direct integration once they outgrow the framework's constraints.

Should I Build a Distributed System or a Single Deployable Artifact?

Nothing fundamentally changes with agents here. The same principles that guide traditional architecture decisions apply. If your application runs in the cloud and needs to scale individual components independently, a distributed architecture makes sense. If your workload is modest, a single deployable artifact is simpler to operate.

Robust agentic systems can absolutely run entirely in a single process. An orchestrator agent that calls tool functions, manages state in memory, and coordinates sub-agents through direct function calls is a great starting point. A single process is easier to debug, easier to deploy, and eliminates an entire class of distributed systems problems.

On the other hand, if your agents need to scale independently, run on different hardware, or be developed by different teams, distributing them can make sense. Just be deliberate about it. The overhead of network calls, serialization, service discovery, and distributed tracing is significant. Don't distribute your system because it feels like the modern thing to do. Distribute it because the problem demands it.

Should Developers Write and Review Code?

It depends on the application. We've been having fun with spec-driven development, where developers write detailed specifications and let an AI coding assistant generate the implementation. The developer then reviews the generated code, tests it, and iterates. Spec-driven development is rapidly evolving, but because we have deep knowledge of our tools, the approach tends to work well for smaller applications using well-understood patterns. For a CRUD application or a straightforward integration, spec-driven development accelerates delivery. In some cases, close review of every line of generated code isn't necessary.

For applications that are larger or more complex, such as building a distributed file system, implementing a new consensus protocol, or working in a domain where correctness is critical, developers likely should be writing and closely reviewing code. A non-agentic approach tends to be safer here, or at least requires much more careful review of what gets generated. LLMs are remarkably capable at producing code that looks correct, but it may not be correct. Be especially cautious in domains where subtle bugs have serious consequences.

The practical advice is to match the level of involvement to the size and complexity of the application. Small, well-understood applications with familiar patterns can lean more heavily on generation. Novel or large-scale systems may require more hands-on development and thorough review regardless of whether a human or an AI wrote the initial code.

Conclusion

Scoping an agentic application differs in subtle ways from scoping a traditional software project. Many of the same questions about architecture, communication, deployment, and development practices apply. However, agents introduce non-determinism and autonomy into systems that were previously fully controlled by developers. These different characteristics demand more deliberate upfront decisions.

Start with the simplest approach that solves the problem. Use an agent when the flow is genuinely non-deterministic. Choose your communication patterns based on delivery requirements. Pick a framework if it accelerates your team, but be prepared to implement parts by hand.

These questions won't give you a perfect architecture on day one. They will help you avoid the most common mistakes we've seen: over-engineering with agents where a simple pipeline would suffice, picking an ill-suited communication protocol, and putting too much trust in generated code without appropriate review.