Cache

The Cache service is a JSON-typed key-value store held in the memory of the backend process that is serving the request — not a shared cache, not a database, and not something that survives a deploy. It is one Map with a 50 000-entry ceiling, and its point is speed: a value already computed costs a map lookup instead of a query or a round trip to a third party. Anything that has to be correct across processes or present after a push belongs in the Database instead.

Quick example

A controller does not reach a platform service — the caching, like every other decision, lives in the service the controller names:

// modules/stats/dto/todo-stats.ts
import { z } from "@palbase/backend";

// A route's return type must be a NAMED zod schema that exists as a VALUE in
// scope — the deploy stager reads it, and an interface alone is rejected.
export const TodoStats = z.object({
  total: z.number(),
  done: z.number(),
});
export type TodoStats = z.infer<typeof TodoStats>;

// modules/stats/stats.service.ts
import { Cache, Database, Injectable } from "@palbase/backend";

import type { TodoStats } from "./dto/todo-stats";

@Injectable()
export class StatsService {
  forUser(userId: string): Promise<TodoStats> {
    // A miss runs the fill once; concurrent misses in this process wait on it.
    return Cache.getOrSet<TodoStats>(`stats:${userId}`, 300, async () => {
      // `where` is a NAMED field: the same object also carries `orderBy`,
      // `limit` and the operators, so the filter has to say which it is.
      const todos = await Database.public.todos.findMany({ where: { user_id: userId } });
      return {
        total: todos.length,
        done: todos.filter((t) => t.done).length,
      };
    });
  }
}

// modules/stats/stats.controller.ts
import { Controller, Get, User } from "@palbase/backend";
import type { UserT } from "@palbase/backend";

import { TodoStats } from "./dto/todo-stats";
import { StatsService } from "./stats.service";

@Controller("/stats")
export class StatsController {
  constructor(private readonly stats: StatsService) {}

  // The method name IS the generated call: `pb.stats.mine()`.
  @Get("")
  mine(@User() user: UserT): Promise<TodoStats> {
    return this.stats.forUser(user.id);
  }
}

// modules/stats/stats.module.ts
import { Module } from "@palbase/backend";

import { StatsController } from "./stats.controller";
import { StatsService } from "./stats.service";

@Module({
  controllers: [StatsController],
  providers: [StatsService],
  exports: [],
  imports: [],
})
export class StatsModule {}

For five minutes (ttl: 300 seconds) repeated calls serve the cached stats. When the entry expires the next request recomputes it, and every other request that arrives while that computation is running gets the same result rather than starting its own.

Where the cache actually lives

The engine builds the cache itself — makeMemoryCache() — and the runtime does not hand it another one. There is no Redis, no shared store, and no network hop. Three consequences follow, and all three are load-bearing:

• It is per process. Two processes serving the same Environment do not see each other's entries. Today an Environment runs one backend process, so in practice that is one cache per Environment — but the guarantee the API gives you is per process, and code that depends on more than that is code that breaks the day a second process exists.
• It does not survive a deploy. Every activated artifact is built into a fresh application object with a fresh, empty cache. A push is a cold cache, always.
• It is bounded. 50 000 entries, then eviction (below). It is not storage.

Warning: Cache is not a lock, a quota ledger, an idempotency store or a one-time-token table. Each of those has to be correct across every process that could serve the request, and this one is not. Use a table and Database.$transaction().

API

Every method is async because the interface is shared with clients that are not in-process; the memory implementation resolves immediately.

The 50 000-entry ceiling and what gets dropped

A set that would push the store past maxEntries (50 000) evicts first, in this order:

1. Every entry whose TTL has already passed.
2. If nothing had expired, the soonest-to-expire quarter of the store. Entries written with no expiry sort last, so the keys nearest the end of their life go first and keys that never expire go last.

Nothing warns you when this happens. A cache that is quietly evicting behaves exactly like a cache that is missing a lot, so size your keys — one entry per user is fine, one entry per user per request is not.

Note: incr does not evict. It writes its key directly, so a counter key space that grows without bound grows past the ceiling instead of being trimmed by it. Date-stamp or otherwise bound the keys you increment.

JSON-typed values

Values round-trip as JSON, so objects, arrays, numbers, booleans and strings all work, and get<T> returns your type rather than a string:

import { Cache } from "@palbase/backend";

export async function readSeats(): Promise<number | null> {
  await Cache.set("session-meta:42", { plan: "pro", seats: 3 }, 600);

  const meta = await Cache.get<{ plan: string; seats: number }>("session-meta:42");
  return meta ? meta.seats : null; // number — typed, parsed for you
}

Note: JSON serialization has the usual edges: Date objects come back as ISO strings, undefined fields are dropped, and class instances lose their prototype. Store plain data.

The generic is a compile-time convenience — the cache validates no shapes at run time. Keep each key family's writer and reader agreeing on the type.

getOrSet: per-process single-flight

A naive get → miss → compute → set sequence has a thundering-herd problem: when a hot key expires, every concurrent request recomputes it at once. getOrSet narrows that to one computation per key, per process:

• Hit → returns the cached value immediately.
• Miss → this process starts one call to fn, stores the result for ttl seconds and returns it.
• Concurrent misses in the same process → they receive the promise the first caller created. They do not call fn.
• Another process → has its own empty entry and its own fn call. Two processes mean two calls, not one.
• fn throws → the rejection is handed to every caller waiting on that fill, nothing is cached, and the in-flight entry is cleared so the next request tries again.

There is no distributed lock and no lock timeout. getOrSet never rejects because a computation elsewhere took too long — the only thing it can reject with is what fn itself threw.

Warning: The SDK's own JSDoc on CacheClient.getOrSet still describes a distributed lock, a single caller across every process, and a rejection when no value lands within the lock's TTL. None of that is implemented. Read this page, not that comment.

Warning: getOrSet caches whatever fn returns — including null. If fn returns null for "not found", that miss is cached for the full ttl. Return a sentinel or guard upstream when you do not want negative caching.

Idioms

Cache-aside with explicit invalidation

getOrSet handles the read side; pair it with del on the write side so an update shows up immediately instead of waiting out the TTL:

// modules/todos/dto/create.ts
import { z } from "@palbase/backend";

export const CreateTodo = z.object({ title: z.string().min(1) });
export type CreateTodo = z.infer<typeof CreateTodo>;

export const CreatedTodo = z.object({ id: z.string() });
export type CreatedTodo = z.infer<typeof CreatedTodo>;

// modules/todos/todos.service.ts
import { Cache, Database, Injectable } from "@palbase/backend";

import type { CreatedTodo } from "./dto/create";

@Injectable()
export class TodoService {
  async create(userId: string, title: string): Promise<CreatedTodo> {
    const row = await Database.public.todos.insert({ title, user_id: userId });
    await Cache.del(`stats:${userId}`); // invalidate — the next read recomputes
    return { id: row.id };
  }
}

// modules/todos/todos.controller.ts
import { Body, Controller, Post, User } from "@palbase/backend";
import type { UserT } from "@palbase/backend";

import { CreatedTodo, CreateTodo } from "./dto/create";
import { TodoService } from "./todos.service";

@Controller("/todos")
export class TodosController {
  constructor(private readonly todos: TodoService) {}

  @Post("")
  create(@Body(CreateTodo) body: CreateTodo, @User() user: UserT): Promise<CreatedTodo> {
    return this.todos.create(user.id, body.title);
  }
}

The TTL is the safety net — stale data can never outlive it — and del is the freshness path. Note what del reaches: the entry in this process. A second process holding its own copy keeps serving it until its TTL runs out, which is the honest reason to keep TTLs short on anything a write can invalidate.

Approximate counters

incr reads and writes one key in one synchronous step inside this process, so it never loses an increment to a concurrent caller here. It is still a per-process number:

import { Cache } from "@palbase/backend";

// A sampling gate, a hit tally, a "how noisy is this endpoint" counter —
// things that are useful when they are roughly right.
export function tally(): Promise<number> {
  return Cache.incr(`endpoint-hits:${new Date().toISOString().slice(0, 10)}`);
}

incr on a key that is missing, or holds something that is not a number, starts from 0 and returns 1. It preserves whatever expiry the key already had rather than clearing it, so a counter created by set(key, 0, 3600) keeps its hour.

Warning: Do not build a quota, an entitlement check or a rate limit on incr. A per-user export limit written this way over-admits in proportion to the number of processes serving the Environment, and it resets to zero on every deploy. For a real quota use a row and Database.$transaction(); for per-route HTTP rate limiting use the declarative rateLimit route option, which the engine enforces for you — see Controllers.

import { Database, inc, NotFound } from "@palbase/backend";

// Correct across processes and across deploys: Postgres does the read and the
// write in one statement, and the filter is the authorization check.
// The plan callback is SYNCHRONOUS — it builds the statement, it does not await.
export function countHit(sessionId: string, userId: string) {
  return Database.$transaction((tx) =>
    tx.public.sessions
      .updateWhere({ id: sessionId, user_id: userId }, { hits: inc(1) })
      .expectOne(new NotFound("no session with that id")),
  );
}

Caching a slow upstream

Wrap a third-party call so one slow fetch serves many requests:

import { Cache } from "@palbase/backend";

export function rates(): Promise<Record<string, number>> {
  return Cache.getOrSet<Record<string, number>>("fx-rates", 900, async () => {
    const res = await fetch("https://api.example.com/rates");
    return (await res.json()) as Record<string, number>;
  });
}

Note: That fetch goes through the outbound fence. If this Environment has an allowlist at all, api.example.com has to be on it or the call throws before it leaves the process — see Outbound Network.

Testing and local work

There is no in-process cache mock to install. @palbase/backend/test gives you api / createTestApi, an HTTP client that calls the release under test on a real Environment, so the cache your test exercises is the real one in the real process — see Testing.

Two things follow for anyone writing a test against cached behaviour. A test that asserts "the second call is served from cache" is asserting on process state that a redeploy resets, so key it on something the test itself wrote. And a test that asserts an entry is gone should use a short TTL or an explicit del rather than waiting for eviction, which only happens under pressure.

Note: palsvc — the platform-services process that runs beside your backend — keeps its own in-process cache, which is what replaced a Redis container there. It is not this cache, it holds none of your keys, and no API reaches it.

Related

• Database — what you are usually caching, and where anything that must be correct belongs
• Controllers — the declarative rateLimit route option
• Scheduled Jobs — recompute on a schedule instead of on a miss
• Outbound Network — the fence a cached fetch still passes through
• Testing — how a test reaches a real release