Optic-Driven Request Batching

Eliminating N+1 at the Optic Seam

The Bug You've Written More Than Once

You have a list of ids. For each one, you load the entity. The code is one line in a stream and looks blameless. In staging, with three rows, it flies. In production, with two hundred rows, it stalls, and the trace shows two hundred near-identical queries lined up in single file.

This is the N+1, and optics by themselves do not save you from it. A Traversal is the shape of the problem ("every element"); it does not have an opinion about how the per-element work runs. Hand a sequential strategy to a traversal of N foci, you get N round-trips.

The org.higherkindedj.optics.fetch package changes the strategy and leaves the optic alone:

sequenceDiagram
    autonumber
    participant C as Caller
    participant T as Traversal over N ids
    participant B as Backend

    Note over C,B: Without batching: N+1
    C->>T: modifyF(loadOne, ids)
    loop N times, one per focus
        T->>B: loadOne(id)
        B-->>T: entity
    end
    T-->>C: List of entities

    Note over C,B: With FetchApplicative: one round
    C->>T: modifyF(Fetch.fetch, ids, FetchApplicative)
    Note right of T: ap() merges pending request sets,<br/>so foci share a single round
    T->>B: batchResolver({id_1 ... id_N})
    B-->>T: Map of id to entity
    T-->>C: List of entities

Top half: the loop you didn't mean to write. Bottom half: the same traversal, the same source, the same backend, plus one new piece, FetchApplicative. One round, one batched call, every focus resolved.

The Pattern, Drawn

The pipeline has three pieces. The optic owns the shape. The applicative is the strategy. The runner is the boundary that actually talks to the backend:

flowchart LR
    OP(["Traversal<br/>(or any Optic)"]) -->|"modifyF(f, s, applicative)"| FA(["FetchApplicative"])
    FA -->|"produces"| F(["Fetch&lt;K, V, A&gt;<br/>a value, not an action"])

    F -->|"sync"| RC(["Fetch.runCached"])
    F -->|"async"| RA(["Fetch.runAsync"])
    F -->|"railway (Either)"| SC(["SafeFetch.runCached"])

    RC -->|"one keyset per round"| FN(["Function&lt;Set&lt;K&gt;, Map&lt;K, V&gt;&gt;"])
    RA -->|"one keyset per round"| BL(["BatchLoader"])
    RA -->|"fan out per source"| SR(["SourceRouter.routed"])
    RA -->|"cap dispatch size"| BC(["BatchLoaders.chunked"])

    classDef shape fill:#e5c890,stroke:#df8e1d,color:#232634
    classDef wire fill:#8caaee,stroke:#1e66f5,color:#232634
    classDef tier fill:#a6d189,stroke:#40a02b,color:#232634
    class OP,FA shape
    class F,RC,RA,SC wire
    class FN,BL,SR,BC tier

Everything else on this page is a variation on those three pieces.

// 1. The optic describes the shape (a list-traversal here).
Traversal<List<Integer>, Integer> ids = FocusPaths.listElements();

// 2. The applicative is the strategy: FetchApplicative batches.
var program = ids.modifyF(
    id -> FETCH.widen(Fetch.<Integer, Integer>fetch(id)),
    List.of(1, 2, 3, 4, 5),
    FetchApplicative.<Integer, Integer>instance());

// 3. The runner hands a whole round's keyset to the resolver in one call.
Fetch.RunResult<Integer, List<Integer>> result =
    Fetch.runCached(FETCH.narrow(program), backend::loadAll);

assertThat(result.rounds()).isEqualTo(1);          // one round
assertThat(result.backendCalls()).isEqualTo(1);    // one batched call

The trick is in step 2. FetchApplicative.ap merges the pending request sets of its two independent arguments. The optic walks the foci, applicative composition stacks them up, and the runner sees a single keyset by the time the dust settles.

When the Optic Can't Spell It: Id -> Entity

The codegen produces type-preserving optics: a Traversal<Team, UserId> is UserId in and UserId out. So how do you express "load each UserId into a User" when the focus type changes?

FetchOptics.fetchEach builds the type-changing list-traversal the codegen does not produce:

record Team(String name, List<UserId> memberIds) {}
record EnrichedTeam(String name, List<User> members) {}

Optic<Team, EnrichedTeam, UserId, User> memberFetch =
    FetchOptics.fetchEach(
        Team::memberIds,
        (team, users) -> new EnrichedTeam(team.name(), users));

var program = memberFetch.modifyF(
    id -> FETCH.widen(Fetch.<UserId, User>fetch(id)),
    team,
    FetchApplicative.<UserId, User>instance());

Fetch.RunResult<UserId, EnrichedTeam> result =
    Fetch.runCached(FETCH.narrow(program), userResolver);

The reader is the list-shaped field. The rebuilder reassembles the parent around the resolved values. One round, one batched call, regardless of how many members are in the team. The aggregate goes in Team, comes out EnrichedTeam.

When Keys Come From Several Backends

Real rounds are not tidy. You ask for a list of identifiers, and half of them are user ids and half are product skus, served by two different services. The naive shape is a switch statement inside the loader; the result is per-key calls again.

SourceRouter.routed composes per-source BatchLoaders with a classifier into one loader the runner can call. Each backend sees its own keys; the round is still one round; the per-source dispatches run concurrently:

BatchLoader<String, String> users    = /* user-directory loader   */;
BatchLoader<String, String> products = /* product-catalog loader  */;

BatchLoader<String, String> routed =
    SourceRouter.routed(
        key -> key.startsWith("u:") ? "users" : "products",
        Map.of("users", users, "products", products));

Fetch.RunResult<String, List<String>> result =
    Fetch.runAsync(FETCH.narrow(program), routed, new ConcurrentHashMap<>()).get();

BatchLoaders.chunked(loader, maxSize) caps a single dispatch's size if a downstream backend enforces a per-request limit (an $in clause cap, an HTTP query-string ceiling, a GraphQL batch limit). The substrate still sees one round; the loader splits the keyset into chunks behind the curtain.

When Failures Belong on the Value Channel

Exceptions are great when nobody else needs to know about them. The moment a failure has to flow through composition (partition successes and failures, retry only the failures, present the failures to the caller as data) they become a problem. SafeFetch wraps a run so that resolver exceptions, missing-key reports, loader failures, and deadlines become Either.left values instead of thrown exceptions. The run never throws, and the safe-async future never completes exceptionally:

Either<Throwable, Fetch.RunResult<UserId, User>> outcome =
    SafeFetch.runCached(program, failingResolver);

When a backend can report per-key failure without poisoning the whole round, the value type is Either<E, V> and SafeFetch.partition splits the result into successes and failures:

Function<Set<UserId>, Map<UserId, Either<String, User>>> partial = /* per-key Either */;

Fetch.RunResult<UserId, List<Either<String, User>>> result =
    Fetch.runCached(FETCH.narrow(program), partial);

SafeFetch.Partitioned<String, User> split = SafeFetch.partition(result.value());
split.successes(); // List<User>
split.failures();  // List<String>

A backend that returns no entry for a requested key is surfaced as MissingKeyException. This is on purpose: a silent null in the result list would be a worse signal than a typed failure.

The Wall: Where Batching Stops Working

Applicative composition collapses because the arguments are independent. flatMap cannot collapse, because the continuation's requests depend on the value the previous round produced. The library does not paper over this; it lays the boundary out where you can see it:

sequenceDiagram
    participant P as Program
    participant R as Runner
    participant B as Backend

    Note over P,B: Pure applicative: N foci, 1 round
    P->>R: ap(ap(ap(f, fetch a), fetch b), fetch c)
    R->>B: { a, b, c }
    B-->>R: { a:..., b:..., c:... }
    R-->>P: value

    Note over P,B: flatMap dependency chain: 3 rounds
    P->>R: fetch(a).flatMap(x -> fetch(x.next))
    R->>B: { a }
    B-->>R: { a:... }
    Note right of R: only now do we know<br/>what to ask for next
    R->>B: { a.next }
    B-->>R: { a.next:... }
    R->>B: { a.next.next }
    B-->>R: { ... }
    R-->>P: value

The practical rule is one line: anything you can express with map2, ap, or an optic traversal collapses to one round; every flatMap in a chain is another round. Express data dependencies as flatMap when you have one; do not reach for it when you don't.

What the RunResult Tells You

Fetch.RunResult is a record, so you can inspect, log, and assert against the run:

In tests, this is exactly what you want: assertThat(result.backendCalls()).isEqualTo(1) is how you prove that you actually killed the N+1 and didn't just hide it. The cache is per-invocation and in-JVM. A key requested again in a later round (say, across a flatMap dependency) is served from the cache and never re-fetched.

Limits, Stated Up Front

• Applicative-only batching. A flatMap data dependency costs an extra round. This is the Haxl law (see further reading), not a defect.
• Per-run cache. No distributed cache; concurrent runAsync calls must each be given their own cache map.
• No predicate pushdown. The backend receives a keyset, never the optic's filter expression. A filter placed before the fetch still shrinks that keyset, because filtered hands non-matching foci to applicative.of and never runs the fetch for them; what it cannot do is let the backend evaluate the predicate.

Previous: Optics Extensions Next: Plan Introspection and Guardrails