Why we built n00dles from scratch instead of patching LangChain

I built my first LLM wrapper in 2022. I built my second in early 2023. I threw both away, and for a long time I assumed that meant the problem was my code. It took a 2am page, a production pipeline silently losing $40 of API spend on calls it had already made, and about four hours of staring at a stack trace at my kitchen table to realize the problem wasn't my code at all. It was the shape of every framework I'd tried to build on top of.

The setup

The pipeline in question was simple on paper: scrape a handful of sources, run three analysis agents over the results, merge their output, and write a summary. We'd built it on top of a popular chaining framework, the way most teams do — not because we loved its abstractions, but because hand-rolling retry logic and prompt templating for the tenth time felt like reinventing a wheel that surely somebody else had already built correctly.

Six months in, our chain definition file was 4,000 lines long. Not because the pipeline had gotten more complex — it hadn't, really — but because every edge case we hit got patched the same way: a callback here, a monkey-patch there, a custom subclass to work around a method that didn't quite do what we needed. We had retry logic in three different places, two of which disagreed with each other about what counted as a retryable error. We had a state-tracking dictionary that one engineer affectionately (and accurately) called "the haunted house," because nobody wanted to go in there alone.

The incident

Here's what happened. A pipeline run hit a rate limit on the fourth of five agent calls. Our retry wrapper caught it, waited, and retried — correctly. But the framework's internal state object didn't persist anywhere durable between steps; it lived in process memory, scoped to the request. When the retry triggered a redeploy race (unrelated, just bad timing — a routine deploy happened to land mid-run), the process restarted. The first three agent calls, the ones that had already succeeded and already cost real money, were gone. No checkpoint. No record. The retry-on-restart logic, not knowing any better, started the whole pipeline over from agent one.

⚠

The actual dollar cost was small. The actual lesson was not: a framework that can't tell you what already happened isn't safe to retry anything in. Retrying blindly is just a more expensive way of being wrong.

I got paged at 2am because the pipeline that fed our morning report had silently doubled its token spend and still hadn't produced output by the time someone needed it. I fixed the immediate bug in about twenty minutes. I spent the rest of the night unable to stop thinking about how many other places in that 4,000-line file had the exact same shape of bug, just waiting for the right race condition to find them.

The actual diagnosis

It would have been easy to conclude "we wrote bad code" and move on. We didn't, because the bug wasn't really in our code — it was in the gap between what the framework assumed and what production actually requires. Three assumptions, in particular, kept causing us trouble:

State was optional, not structural. Checkpointing was something you could bolt on if you remembered to, not something the framework did for you by default. We remembered to in some places and not others — predictably.
Retry was a wrapper, not a contract. Every agent call needed its own retry logic added by hand, which meant every agent call could silently have slightly different (or absent) retry semantics depending on who wrote it and when.
The I/O contract lived in nobody's head but the original author's. What shape of data did agent three expect? You found out by reading three other files and one Slack thread, or by running it and seeing what broke.

None of these are exotic problems. They're the same three problems every team building multi-agent systems eventually runs into, which is exactly why we decided the fix needed to be structural, not another patch.

Starting over, on purpose

We didn't rewrite from scratch out of pride. We rewrote because every attempt to fix the underlying issues inside the existing framework meant fighting its abstractions, not using them. So we wrote down three rules before writing a single line of the new core, and we've tried to break none of them since:

State is checkpointed after every step, by default, with no configuration required.
Retry, timeout, and fallback are properties of the agent definition, not something glued on afterward.
The I/O contract is the function signature. If you can read the type hints, you know what the agent expects and returns — full stop.

That third rule turned out to be the one that shaped everything else. We landed on a single decorator, @agent, where the docstring becomes the system prompt and the type hints become the validated contract:

python

from n00dles import agent, pipeline, run

@agent(model="claude-haiku-4-5")
def researcher(topic: str) -> str:
    """Research the topic. Return 3 key facts."""

@agent(model="claude-sonnet-4-6")
def writer(research: str) -> str:
    """Write a short article from the research."""

content_pipeline = pipeline(researcher >> writer, retry=3)
result = run(content_pipeline, topic="multi-agent orchestration")

No callback registry. No subclassing. No chain-of-handlers you have to trace through to find out what actually runs. researcher and writerare just functions with a contract — and the executor that runs them checkpoints state after each one finishes, with no extra configuration, because that's not an advanced feature. It's the floor.

What that actually changed

The same class of incident that paged me at 2am simply can't happen the same way in n00dles. If a node fails after exhausting its retry budget, the pipeline raises a PipelineFailure that tells you exactly which node failed and why — and every node before it is already checkpointed, so re-running the same run_id resumes from where it actually stopped, instead of replaying calls you already paid for.

✓

This isn't a hypothetical improvement — it's the literal fix for the bug that started this whole rewrite. Checkpoint-and-resume was the first thing we built, before we built parallel execution, before we built provider abstraction, before anything else.

We also made one structural bet that's easy to miss: n00dles doesn't maintain five parallel provider SDKs internally. It wraps litellm, so switching from Claude to GPT-4o to a local Ollama model is a string change, not a rewrite. We didn't want to be in the business of re-implementing every provider's SDK quirks — we wanted to be in the business of making the orchestration layer boring and predictable, which meant outsourcing the part that wasn't our problem to solve.

Why open source it

We used n00dles internally for two production deployments before we open-sourced it, and in both cases the bug class that paged me that night simply stopped showing up. Not because we got better at writing careful code — because the framework made the careless version of the code impossible to write by accident.

That's the part that convinced us to ship this publicly instead of keeping it as an internal tool. The problem we hit wasn't specific to our pipeline, our company, or our bad luck with timing. It's the default failure mode of building multi-agent systems on frameworks that treat reliability as an extension, not a foundation. If you've had your own version of that 2am page, we'd like n00dles to be the reason you don't get a second one.

If you want to see exactly what that looks like in practice, the five-minute quickstart walks through the same @agent → pipeline() → run() flow shown above, end to end, against a real model.