Streaming responses

Some work has an answer that arrives in pieces. An AI model writes a sentence a token at a time; an import reports the row it just processed; a build prints the step it just finished. @Sse is for those: the handler writes as it goes, the client reads as it arrives, and neither waits for the other to finish.

The half everyone gets wrong is the ending. When the person closes the tab, your producer has to stop — otherwise a model keeps generating into a stream nobody is reading, and you pay for every token of it. @Signal() is that half, and it costs you one parameter.

Quick example

// modules/ai/ai.controller.ts
import { Body, Controller, Signal, Sse, SseOut, z } from "@palbase/backend";
import type { SseWriter } from "@palbase/backend";
import OpenAI from "openai";

const openai = new OpenAI();

// ONE frame — what a single write() carries. Required: see "Why frame is
// required" below.
const Chunk = z.object({ text: z.string() });

@Controller("/ai")
export class AiController {
  @Sse("/chat", { frame: Chunk })
  async chat(
    @Body(z.object({ prompt: z.string() })) body: { prompt: string },
    @SseOut() out: SseWriter,
    @Signal() signal: AbortSignal,
  ): Promise<void> {
    const stream = await openai.chat.completions.create(
      { model: "gpt-4o-mini", messages: [{ role: "user", content: body.prompt }], stream: true },
      { signal },                       // the client leaves → OpenAI is aborted
    );

    for await (const chunk of stream) {
      out.write({ text: chunk.choices[0]?.delta?.content ?? "" });
    }
  }
}

On the client, the generated method hands you the sequence:

// web — `pb` and `pb.ai.chat` are GENERATED from the controller above
declare const pb: {
  ai: { chat(input: { prompt: string }): Promise<AsyncIterable<{ text: string }>> };
};
declare function render(text: string): void;

for await (const part of await pb.ai.chat({ prompt: "…" })) {
  render(part.text);
}

// iOS
for try await part in try await pb.ai.chat(.init(prompt: "…")) {
    render(part.text)
}

Breaking out of that loop cancels the request, which fires the server's signal, which stops your provider. You do not have to write anything to make that work — doing nothing special is already correct.

Coming from NestJS?

The pieces map one to one, and nothing about the streaming path is library-specific.

The signal is a plain AbortSignal, so it goes wherever your provider takes one: { signal } for the OpenAI and Anthropic SDKs, abortSignal: for the Vercel AI SDK, or straight into fetch.

Why frame is required

An @Sse handler returns void, so nothing can infer what one frame looks like. Without a declaration the contract would describe an untyped response, and every generated client from it would be opaque — it would compile, it would run, and part would be unknown on both platforms.

That is not a hypothetical: the deploy gate refuses a contract with untyped responses, and it refused this one before frame existed. Declaring it is what makes pb.ai.chat hand you a typed part.

Do your database work before the first write

The first out.write() settles the request's transaction, and the database is refused by name afterwards:

sse_db_after_write: POST /ai/chat touched the database after its first write().

This is deliberate. A stream can live for minutes, and your handler runs inside the request's transaction — holding one open for that long exhausts the connection pool. The failure would not show up in a test; it would show up under load, looking like "the database is slow".

So: load what you need, then start writing.

// modules/ai/ai-order.controller.ts
import { Controller, Database, Signal, Sse, SseOut, z } from "@palbase/backend";
import type { SseWriter } from "@palbase/backend";

const Chunk = z.object({ text: z.string() });

declare const provider: {
  start(history: { body: string }[], opts: { signal: AbortSignal }): Promise<AsyncIterable<{ text: string }>>;
};

@Controller("/ai")
export class AiOrderController {
  @Sse("/chat", { frame: Chunk })
  async chat(@SseOut() out: SseWriter, @Signal() signal: AbortSignal): Promise<void> {
    // ✅ before the first write — the request's transaction is still open
    const history = await Database.public.notes.findMany({ limit: 50 });

    const stream = await provider.start(history, { signal });
    for await (const chunk of stream) out.write(chunk);

    // ❌ after the first write — refused by name, not silently slow
    // await Database.public.notes.insert({ user_id: "…", body: "…" });
  }
}

If you need to persist the finished result, do it from your own accumulated copy, or hand the work to a job.

Errors

Where the failure happens decides how it is reported, because after the first frame the status line is already spent:

• Before the first write — the ordinary error envelope and status, exactly like any other route. Throw NotFound, Forbidden, whatever fits.
• After the first write — a terminal event: error frame carrying the request id, then the stream closes. The generated clients surface this by throwing out of the for await loop.

A frame that fails to decode also ends the stream with an error rather than being skipped: a silently dropped frame in a token stream is a hole in the middle of the user's text.

Backgrounding an app is not a disconnect

A client that LEAVES ends the stream: the socket closes, your handler's signal fires, and the provider stops. That is the ordinary case and it is measured — a killed client stopped a real OpenAI stream at 212 chunks where an uninterrupted one pulled 1125.

Backgrounding is the exception, because iOS suspends the app WITHOUT closing the socket. Measured (iPhone 17 Pro, 2026-08-30): URLSession ended the client's task with -1001 timed out while the server carried on. Nobody was reading.

The size of that is worth stating plainly, because it is smaller than it sounds: your handler ends when your provider's stream ends, so what you pay for is the TAIL OF ONE ANSWER, not an open tap. It is not a leak. It is the rest of a reply the user walked away from.

So do not treat "the user switched apps" as a stop. If a background stint should end the work, end it explicitly:

@Environment(\.scenePhase) private var scenePhase
// …
.onChange(of: scenePhase) { _, phase in
    if phase == .background { streamTask?.cancel() }   // this DOES stop the server
}

Cancelling the task is the same event as breaking out of the loop: the request ends, the handler's signal fires, and the provider stops. That path is measured too — breaking the loop on the device stopped the producer in under a second.

Nothing is cancelled on your behalf, deliberately: a two-second glance at a notification should not throw away a half-written answer. Where the line falls is your product's call, not the SDK's.

Limits worth knowing

• A stream stays open as long as it keeps emitting. One that emits nothing for five minutes is cut, on the assumption that it is stuck.
• A route cannot be both @Sse and @Upload. One method cannot be a streaming response and an upload completion handler, and the build says so by name rather than letting one silently win.
• The client leaving is a normal ending, not an error. Your handler's signal fires, your loop ends, and nothing is logged as a failure.

See also

• Controllers — the decorators an @Sse route shares with every other route
• Direct uploads — the other route kind whose body does not travel the ordinary way
• Jobs — for work that should outlive the request that started it