Agent framework vs custom harness
A framework gets the first agent running. Your own harness is what keeps it running when the model changes.
The short answer
An agent framework is a library of parts for building agents: calling models, routing steps, managing state. A custom harness is the layer you write yourself to run the agent, log its steps and enforce its rules. Frameworks are faster to start. A harness you own is easier to control and to explain. Two open-source projects are worth reading before you choose: LangGraph and the OpenAI Agents SDK. Check each repository's current documentation first.
Decision table
| Question | Agent framework | Custom harness |
|---|---|---|
| What it is | A library of agent parts, maintained by its project | Your own code for running, logging and governing the agent |
| Who owns the outcome | Your team, using the framework's patterns | Your team, fully |
| Typical duration | Days to a first working agent | Longer to a first agent, but no outside dependency to wait on |
| What you get at the end | An agent built on a framework's structure | An agent and a harness shaped to your rules |
| Main risk | Upgrades and abstractions you did not choose | Maintenance, testing and security fall to your team |
| Best for | Standard agent patterns and a fast first version | Strict logging, permissions and a long-lived system |
Use cases for agent framework
- A first agent to learn from. The team needs a working agent in a week to test a use case before committing to a design.
- A standard multi-step workflow. The steps fit a framework's pattern for routing and state, with little custom logic.
- A team that already knows the framework. Engineers have used it before, so the learning cost is low.
- A prototype that may be thrown away. Speed matters more than long-term structure, and the prototype is not yet a system.
Use cases for custom harness
- A system with strict audit and permission rules. Every tool call must be logged and checked against a policy the framework does not express.
- A long-lived agent with frequent model changes. The harness owns the model boundary, so a model change does not reach the rest of the system.
- An unusual control flow. The steps do not fit a framework's graph or routing model, and forcing them in would hide the logic.
- A team that wants no external dependency for core control. The agent's control path must be understood entirely by the team that runs it.
Trade-offs
Agent framework
Advantages. Fast to a first working agent. Common patterns are already built and tested by a community. Less code for the team to maintain at the start.
Disadvantages. The framework's abstractions shape the design. Upgrades can change behaviour you depend on. Hard logging or permission rules may be awkward to add.
Custom harness
Advantages. Control of every step, log and permission. No upgrade of a dependency can change core behaviour without a decision. Designed around your rules, not someone else's defaults.
Disadvantages. Slower to a first answer. Your team must test, secure and maintain the harness. Easy to build something clever that no one else can read.
How each option runs
Framework, step by step:
- Step 1. Pick the framework by its documented patterns, and check its current repository.
- Step 2. Build the first agent on its structure, with the team's own evaluation set.
- Step 3. Pin the version, and read the release notes before every upgrade.
Custom harness, step by step:
- Step 1. Define the harness's responsibilities: model calls, tool calls, logging, permissions.
- Step 2. Build those parts and test each on its own.
- Step 3. Keep the harness small enough that a new engineer can read it in a day.
What drives the effort
- How much the team already knows about agent frameworks and how to test them.
- How strict the logging, permission and audit rules are.
- How often the underlying models will change.
- Who will maintain the code after the first release.
This site does not publish prices. The engagements page describes what each shape includes.
A checklist before you choose
- Does a standard framework pattern fit the workflow without workarounds?
- Do audit and permission rules need to be enforced at every step?
- Can the team test and upgrade the chosen option on a schedule?
- Will the system live longer than the first project?
If the first is yes and the rest are no, use a framework. If the second and fourth are yes, the harness is worth its cost.
What people get wrong
- They pick a framework for its popularity, then fight its abstractions for months.
- They write a harness for a prototype that a framework could have run.
- They upgrade a framework without tests, and the agent changes behaviour silently.
A hybrid that often works
Use a framework for the agent's internal steps while the team learns the workflow. Put a thin harness of your own around it for logging, permissions and the model boundary. The harness is what you keep when you change the framework.
Common questions
Which framework should I use?
Choose by the patterns your workflow needs, not by popularity. Read each project's current documentation and check its release activity before you commit.
Is a custom harness reinventing the wheel?
Only if the parts already fit your needs. Logging, permission checks and the model boundary are often the parts teams most need to own.
How do I stop a framework upgrade breaking the agent?
Pin the version, keep an evaluation set, and run it before any upgrade. Treat a passing evaluation as the condition for upgrading.
Can I move from a framework to a harness later?
Yes, if the tools and the evaluation set are written independently of the framework. Keep them in your own repository.