State management in production LLM pipelines

I own everything that has to keep running when nobody's watching it: the state store, the worker pool, the deploy tooling, the monitoring stack. If you want to find the part of a system that's lying about how reliable it is, look at how it handles state across a crash. Almost everything else in a pipeline is forgiving of small bugs. State is not — it's the one place where "mostly correct" is functionally identical to "wrong," because the failure only shows up under exactly the conditions you didn't test for.

Why in-memory state always eventually fails

Every in-memory state store works perfectly in development. That's the trap. A dictionary that tracks "which steps have completed and what they returned" is simple, fast, and has zero infrastructure dependencies — which means it's also the thing every tutorial uses, and the thing every team initially ships to production without thinking too hard about it.

It works fine until one of three things happens: the process restarts mid-run (a deploy, a crash, an autoscaler killing an idle worker), the pipeline genuinely needs more wall time than a single request-response cycle allows, or you scale past one worker process and discover that two requests for the "same" pipeline run aren't actually looking at the same memory. None of these are exotic. They're Tuesday.

What checkpointing actually means here

In n00dles, a pipeline run is backed by a PipelineContext — the original inputs, plus a growing record of every completed node's output, keyed by node name. After every single node finishes, not just at the end of the run, that context gets serialized and written to the configured state store under the run's ID. This isn't an opt-in feature you enable for "important" pipelines. It happens by default, for every run, because the moment checkpointing becomes optional is the moment someone forgets to opt in on the pipeline that turns out to matter most.

from n00dles import agent, pipeline, run

content_pipeline = pipeline(researcher >> writer >> editor)
result = run(content_pipeline, topic="agentic search")

# every result carries the run_id it was checkpointed under
print(result.run_id)

If that process dies after researcher and writer have already succeeded but before editor finishes, resuming with that same run_iddoesn't replay the whole pipeline. The executor checks each node against the saved context first — researcher and writer's outputs are already there, so only editoractually calls an LLM. You don't pay twice, and you don't wait twice, for work that already finished.

The unglamorous part: serialization

Most of the actual engineering effort in a state layer isn't the happy path — it's getting serialization right for every shape of output an agent can return. n00dles agents can return plain strings, or they can return Pydantic models when you want validated, structured output. A checkpoint has to handle both, and it has to handle the second case without quietly losing data or — worse — silently coercing a typed object into something that looks fine in a log line but breaks the moment a downstream node tries to read a field off it.

We serialize through a single explicit path: Pydantic models get model_dump(mode="json"), everything else falls through to standard JSON encoding. One honest caveat worth stating plainly because we'd rather you hear it from us than discover it the hard way: a restored checkpoint comes back as a plain dict, not the original model class. There's no per-node type registry that reconstructs the exact Pydantic type on resume. If a downstream node's contract expects the original model and not a dict, that's worth testing explicitly in your resume path today.

Picking a backend

The default backend is SQLite, and we mean that as a real recommendation, not a placeholder until you "graduate" to something else. For local development and single-node deployments — which covers a large share of real production multi-agent pipelines, not just toy projects — a file-based store with zero configuration and zero external dependencies is exactly the right amount of infrastructure. You don't need to stand up a database cluster to get crash-safe checkpointing.

from n00dles import configure

configure(state_store="sqlite:///n00dles.db") # default path: ./n00dles_state.db

There's also an InMemoryStateStore, which exists specifically for tests — fast, no file I/O, wiped clean between test runs. Both implement the same minimal StateStore interface: load_or_create, save, and delete. That interface is intentionally small, because the smaller the contract, the easier it is to implement correctly against a new backend later.

Where we are, honestly

I'd rather tell you what's actually true than let the roadmap slide read like a feature list. Today, the shipped backends are SQLite and the in-memory test store. A shared backend for horizontally-scaled, multi-worker deployments — the kind backed by Redis or Postgres, where any worker can pick up any run — is on the roadmap and described in the docs, but it isn't in your hands yet.

We'd rather ship that backend once, correctly, with the same checkpoint-every-node guarantee the SQLite store already gives you, than ship something fast that quietly weakens the one promise this whole layer exists to keep. See the state management docsfor the current backend table and what's coming next.