Why Streaming LLMs Break Temporal (And How to Fix It)

In the world of AI engineering, we are currently caught in a massive tug-of-war between frontend user experience and backend infrastructure reliability.

On the frontend, streaming tokens directly from an LLM is a non-negotiable UX requirement. Users expect to see the model’s thought process generated character-by-character.

On the backend, building reliable agents requires durable execution engines. When a process crashes, hits a rate limit, or loses network connectivity, the system must recover and resume exactly where it left off.

The problem? These two requirements are fundamentally at war. Durable execution demands strict determinism. Streaming is inherently non-deterministic, volatile, and stateful. Standard durable engines like Temporal break down when confronted with streaming LLMs, forcing developers into dangerous architectural compromises.

Here is why streaming breaks durable execution, and how Cellaflow’s Non-Persistable Zones (NPZ) finally solve the dilemma.

The Temporal Problem: Determinism vs. Volatility

Temporal is widely considered the gold standard for durable execution. It achieves durability through event sourcing: it records the history of execution and replays it to recover state.

This architecture imposes an absolute rule on developers: the core workflow code must be one hundred percent deterministic. You cannot perform random API calls, read the system clock, or iterate over unpredictable network streams inside the main workflow. To do anything non-deterministic, you must wrap the code in an isolated Activity.

If you try to stream an LLM response natively inside a Temporal workflow, you violate determinism and break the system.

The Side-Channel Nightmare and State Corruption

To get around this, engineers wrap the LLM stream inside a Temporal Activity. But here is the catch: Temporal Activities are designed to run to completion and return a single, final result to the workflow. They are not designed to yield continuous streams of data to an external client.

To make streaming work, developers must build "side-channels." As the Activity receives tokens from the LLM, it pushes those chunks to an external message broker (like a Redis pub/sub topic or a WebSocket gateway) connected to the user's frontend.

This creates a catastrophic vulnerability regarding atomicity and state corruption.

Imagine a Kubernetes pod crashes, or a network hiccup occurs halfway through the stream. The Temporal Activity fails. Temporal does exactly what it is designed to do: it automatically retries the Activity from the beginning.

However, Temporal is completely blind to the Redis side-channel. When the Activity restarts, it prompts the LLM again and begins streaming a new set of tokens into the same Redis queue. The frontend client, which already received half of the first response, suddenly receives the new chunks appended to the old ones. The user is left staring at duplicated, corrupted, and garbled text.

Even worse, if you try to build a custom orchestrator that saves incremental checkpoints midway through a stream, a crash might permanently serialize corrupted state—like an unclosed JSON string—bricking the entire execution graph permanently.

Cellaflow’s Solution: Non-Persistable Zones (NPZ)

At Cellaflow, we believe you shouldn't have to choose between a modern, streaming UX and backend durability. Furthermore, you shouldn't have to manage complex Redis side-channels just to send text to a user.

Cellaflow introduces a novel primitive explicitly designed for LLM streaming and native socket management: the Non-Persistable Zone (NPZ).

Cellaflow's engine recognizes that certain blocks of code are highly volatile. A live stream is a transient transaction—it is unsafe for mid-execution state serialization. Developers simply wrap their streaming logic in a context manager block:

# The Cellaflow Way: Native, safe streaming
@cellaflow.durable
def agent_response_workflow(state):
    # Safe checkpoint before the volatile action

    with cellaflow.npz():
        # Journal commits are suspended here
        stream = llm.stream("Explain durable execution")
        for chunk in stream:
            cellaflow.emit(chunk) # Native emit, no side-channels needed

    # Stream completed successfully. Final state is committed.
    return {"status": "complete"}

When the execution pointer enters the NPZ, Cellaflow intentionally suspends all journal commits. The engine explicitly acknowledges that it is entering a volatile state where progress cannot be safely bookmarked.

The Clean Rollback

If the system successfully exits the NPZ, Cellaflow commits the final, aggregated state to its embedded RocksDB journal.

But what happens if the pod crashes or receives a SIGTERM mid-stream?

Cellaflow's recovery mechanism detects that the crash occurred inside an active NPZ. Instead of attempting to blindly retry and appending garbage data to a side-channel, Cellaflow intelligently rolls the execution pointer backward to the exact safe state before the stream started.

Because Cellaflow manages the client connection natively, it signals the client that an interruption occurred. When the client automatically reconnects via its unique session ID, Cellaflow cleanly restarts the stream from the beginning. The frontend drops the incomplete UI state and cleanly renders the new stream.

Unifying UX and Durability

You can't build next-generation AI agents if your infrastructure forces you to treat streaming as a hacky afterthought.

By introducing Non-Persistable Zones, Cellaflow eliminates the need for fragile side-channels and incremental state corruption. We allow developers to write native, imperative streaming loops directly in the primary execution context, guaranteeing that if a crash happens, your agent rolls back safely, and your user never sees a corrupted token.