Taming JSON with Jackson

Declare the shapes a JsonNode can take, and read and write JSON through generated prisms with no casts.

"The art of programming is the art of organising complexity, of mastering multitude and avoiding its bastard chaos."

– Edsger W. Dijkstra

Dijkstra's words ring especially true when facing JSON: a nested, dynamically-typed structure with optional fields, variable array contents, and no compile-time guarantees. The imperative approach fights that with defensive code, null checks upon null checks, type assertions, deeply nested conditionals.

Spec interfaces take the other path. You declare the shapes the data might take, the processor generates type-safe prisms for them, and navigation becomes composition. Here is the destination first, on a real API response:

    List<String> emails = Traversals.getAll(USER_EMAILS, response);
    // [alice@example.com, bob@example.com, carol@example.com]

    List<Double> overForty = Traversals.getAll(USER_AGES.filtered(age -> age > 40), response);
    // [45.0]

    double page = PAGE.getOrElse(0.0, response);
    // 1.0

No null checks, no casts, no nested conditionals. And the write direction is the same paths, run backwards:

    JsonNode anonymised = Traversals.modify(USER_EMAILS, JsonApiBook::mask, response);
    // every data.users[].email is masked; response itself is unchanged

    JsonNode aged = Traversals.modify(USER_AGES, age -> age + 1, response);
    List<Double> ages = Traversals.getAll(USER_AGES, aged);
    // [33.0, 29.0, 46.0]

Why JsonNode Resists Auto-Detection

Point @ImportOptics at JsonNode and it has nothing to work with:

1. No sealed hierarchy. ObjectNode, ArrayNode and StringNode exist, but JsonNode is not sealed, so the processor cannot enumerate the variants.
2. No copy strategy to detect. ObjectNode and ArrayNode are mutable, and deepCopy() will isolate an edit, which is what JsonPaths.field below does before calling set. What is missing is anything declarative: no builder, no withers, no all-args constructor for the processor to rebuild a changed node through.
3. Predicate-based type checks. Jackson offers isObject() and isArray(), which auto-detection has no rule for.

Types like this need you to say what you want. That is a spec interface.

The Spec Interface

An interface extending OpticsSpec<S>, with one annotated abstract method per optic:

@ImportOptics
public interface JsonNodeOpticsSpec extends OpticsSpec<JsonNode> {

  @InstanceOf(ObjectNode.class)
  Prism<JsonNode, ObjectNode> object();

  @InstanceOf(ArrayNode.class)
  Prism<JsonNode, ArrayNode> array();

  @InstanceOf(StringNode.class)
  Prism<JsonNode, StringNode> text();

  @InstanceOf(NumericNode.class)
  Prism<JsonNode, NumericNode> numeric();

  @InstanceOf(BooleanNode.class)
  Prism<JsonNode, BooleanNode> bool();
}

@InstanceOf tells the processor to generate a prism that matches when the node is an instance of the given class, rebuilding through identity. From this declaration you get a class of static prisms:

Prism<JsonNode, ObjectNode>  object  = JsonNodeOptics.object();
Prism<JsonNode, ArrayNode>   array   = JsonNodeOptics.array();
Prism<JsonNode, StringNode>  text    = JsonNodeOptics.text();
Prism<JsonNode, NumericNode> numeric = JsonNodeOptics.numeric();
Prism<JsonNode, BooleanNode> bool    = JsonNodeOptics.bool();

A spec declares OpticsSpec<S> and each of its optic methods itself. Inheriting OpticsSpec<S> or an optic method from another interface is not supported yet.

Building Richer Tools

The generated prisms are primitives. Real JSON work wants field access, array traversal and value extraction, which are compositions of those primitives. Put them in an ordinary utility class:

  /**
   * A named field of an object node: empty when the node is not an object, or has no such field.
   */
  public static Affine<JsonNode, JsonNode> field(String name) {
    return JsonNodeOptics.object()
        .andThen(
            Affine.of(
                obj -> Optional.ofNullable(obj.get(name)),
                (obj, value) -> {
                  ObjectNode copy = obj.deepCopy();
                  copy.set(name, value);
                  return copy;
                }));
  }

  /** Every element of an array node, rebuilt as an array node after modification. */
  public static Traversal<JsonNode, JsonNode> elements() {
    return JsonNodeOptics.array()
        .andThen(
            Traversals.<ArrayNode, JsonNode>forIterableCollecting(
                list -> {
                  ArrayNode rebuilt = JsonNodeFactory.instance.arrayNode();
                  list.forEach(rebuilt::add);
                  return rebuilt;
                }));
  }

  /** The String behind a string node. */
  public static Affine<JsonNode, String> textValue() {
    return JsonNodeOptics.text()
        .andThen(
            Affine.of(
                node -> Optional.of(node.stringValue()),
                (node, value) -> (StringNode) JsonNodeFactory.instance.stringNode(value)));
  }

  /** The double behind a numeric node. */
  public static Affine<JsonNode, Double> numericValue() {
    return JsonNodeOptics.numeric()
        .andThen(
            Affine.of(
                node -> Optional.of(node.doubleValue()),
                (node, value) -> DoubleNode.valueOf(value)));
  }

Each is a prism composed with a hand-written Affine or Traversal, and the result types tell the story: field may miss (an Affine), elements may hit many (a Traversal).

A Real Pipeline: API Response Processing

The response this page works on:

{
  "status": "success",
  "data": {
    "users": [
      { "id": 1, "name": "Alice", "email": "alice@example.com", "age": 32 },
      { "id": 2, "name": "Bob",   "email": "bob@example.com",   "age": 28 },
      { "id": 3, "name": "Carol", "email": "carol@example.com", "age": 45 }
    ],
    "page": 1
  }
}

Name the paths once, at the top of the class, and the business logic reads as intent:

  /** Each user object inside data.users. */
  static final Traversal<JsonNode, JsonNode> EACH_USER =
      JsonPaths.field("data").andThen(JsonPaths.field("users")).andThen(JsonPaths.elements());

  /** Every user's email address, as a String: a Traversal followed by an Affine. */
  static final Traversal<JsonNode, String> USER_EMAILS =
      EACH_USER.andThen(JsonPaths.field("email").andThen(JsonPaths.textValue()));

  /** Every user's age, as a double. */
  static final Traversal<JsonNode, Double> USER_AGES =
      EACH_USER.andThen(JsonPaths.field("age").andThen(JsonPaths.numericValue()));

  /** The page number: a single value, so an Affine rather than a Traversal. */
  static final Affine<JsonNode, Double> PAGE =
      JsonPaths.field("data").andThen(JsonPaths.field("page")).andThen(JsonPaths.numericValue());

Those four constants are the whole abstraction. Traversals.getAll reads through them, Traversals.modify writes through them, and filtered narrows them, exactly as shown at the top of this page.

@InstanceOf versus @MatchWhen

@InstanceOf is for hierarchies whose variants are real Java subtypes:

@InstanceOf(ObjectNode.class)
Prism<JsonNode, ObjectNode> object();
// generates: node instanceof ObjectNode o ? Optional.of(o) : Optional.empty()

@MatchWhen is for libraries that expose a check-then-extract pair instead of subtypes:

@MatchWhen(predicate = "isObject", getter = "asObject")
Prism<Value, ObjectValue> object();
// generates: value.isObject() ? Optional.of(value.asObject()) : Optional.empty()

Both produce a Prism, so both compose the same way afterwards. Pick by what the library gives you: a type to test, or a method to ask.

@ImportOptics
interface ValueOpticsSpec extends OpticsSpec<Value> {

    @MatchWhen(predicate = "isString", getter = "asString")
    Prism<Value, String> string();   // rejected: a String is not a Value
}

Parameterised Targets

@InstanceOf takes a class constant, which is always raw, and the generated test runs after erasure. A parameterised target may therefore only be narrowed to the type arguments the source type pins down: the ones a value of that source type must already have had to reach the test at all.

flowchart TD
    accTitle: When a parameterised prism target is allowed
    accDescr: An InstanceOf prism onto Circle of T is accepted when the source S carries the argument Circle is reached under, as Shape of T does when Circle of X implements Shape of X, and javac then checks the test. When S is a Shape that declares nothing, it is rejected: declare Circle of a wildcard, or use MatchWhen.
    T@{ shape: hex, label: "@InstanceOf(Circle.class)<br/>Prism&lt;S, Circle&lt;T&gt;&gt;" } --> Q{"Does S carry the<br/>argument Circle is<br/>reached under?"}
    Q -->|"Shape&lt;T&gt;, and<br/>Circle&lt;X&gt; implements Shape&lt;X&gt;"| P["source instanceof Circle&lt;T&gt;<br/>javac checks it"]
    Q -->|"Shape, which<br/>declares nothing"| E["Rejected: declare<br/>Circle&lt;?&gt;, or use @MatchWhen"]

    classDef decision fill:#e5c890,stroke:#df8e1d,color:#232634
    classDef rw fill:#a6d189,stroke:#40a02b,color:#232634
    classDef tier fill:#a6d189,stroke:#40a02b,color:#232634
    classDef error fill:#e78284,stroke:#d20f39,color:#232634
    class T rw
    class Q decision
    class P tier
    class E error

A generic hierarchy pins its own argument, so the prism can promise it. T is the spec's own type parameter, and Circle<X> implements Shape<X> is what lets the test check it:

sealed interface Shape<X> permits Circle {}
record Circle<X>(X tag) implements Shape<X> {}

@ImportOptics
interface ShapeOpticsSpec<T> extends OpticsSpec<Shape<T>> {

    @InstanceOf(Circle.class)
    Prism<Shape<T>, Circle<T>> circle();
    // generates: source instanceof Circle<T> t ? Optional.of(t) : Optional.empty()
}

A base that says nothing about the argument pins nothing. Every instantiation passes the same test, so the same declaration is rejected:

class Shape {}
class Circle<X> extends Shape {}

@ImportOptics
interface ShapeOpticsSpec<T> extends OpticsSpec<Shape> {

    @InstanceOf(Circle.class)
    Prism<Shape, Circle<T>> circle();   // rejected: nothing checks T
}

Widened to the wildcard, which is what the test earns, it is accepted, and the spec needs no type parameter of its own once the prism stops promising one:

@ImportOptics
interface ShapeOpticsSpec extends OpticsSpec<Shape> {

    @InstanceOf(Circle.class)
    Prism<Shape, Circle<?>> circle();
    // generates: source instanceof Circle<?> t ? Optional.of(t) : Optional.empty()
}

An inner class of a generic class is narrowed by the same rule, its enclosing class's arguments included. A source that pins nothing of Outer's earns a Plain of some Outer:

class Node<U> {}

class Outer<X> {

    class Plain extends Node<String> {}
}

@ImportOptics
interface NodeOpticsSpec extends OpticsSpec<Node<String>> {

    @InstanceOf(Outer.Plain.class)
    Prism<Node<String>, Outer<?>.Plain> plain();
    // generates: source instanceof Outer<?>.Plain t ? Optional.of(t) : Optional.empty()
}

Where the source pins Outer's argument, through Node<X> in the class's own clause or through a source that is an inner class of Outer too, the test names it, as in Outer<String>.Sub. An inner class whose parameter's bound is one the source leaves free, as in Inner<Y extends X>, has no test that can be written, and the processor refuses it, as a bound the source leaves free explains.

A pinned argument must also be within its parameter's bound, since javac refuses the test otherwise. Gauge<Y extends Number> extends Node<Y> reached from a Node<U> is tested as Gauge<U>, so the spec declares U extends Number for the prism to be written:

class Node<U> {}

class Gauge<Y extends Number> extends Node<Y> {}

@ImportOptics
interface NodeOpticsSpec<U extends Number> extends OpticsSpec<Node<U>> {

    @InstanceOf(Gauge.class)
    Prism<Node<U>, Gauge<U>> gauge();
    // generates: source instanceof Gauge<U> t ? Optional.of(t) : Optional.empty()
}

Left unbounded, U makes the processor refuse the prism at the spec method, as it does where the source fixes the argument outside the bound, such as Gauge reached from a Node<String>. Where one bound will do, the message gives the declaration to write, as an argument outside its bound shows.

A class whose declaration and the source write two different types in one place, as Grid<Y> extends Node<Y[]> and a Node<int[]> do, is refused at the spec method too, since the test would never match. A test that would never match shows the fix.

Layering Domain Optics

The paths above are still shaped like the JSON. One more layer names them in the language of the domain, so the rest of the code never mentions field("data") at all:

public final class UserJson {

  private UserJson() {}

  public static Traversal<JsonNode, String> emails() {
    return JsonApiBook.USER_EMAILS;
  }

  public static Traversal<JsonNode, Double> ages() {
    return JsonApiBook.USER_AGES;
  }

  public static Affine<JsonNode, Double> page() {
    return JsonApiBook.PAGE;
  }
}

Now a service reads UserJson.emails(), and the wire format is an implementation detail of one class. (USER_EMAILS, USER_AGES and PAGE are package-private, so this facade sits in the same package as them; in your own code make them public or put the facade alongside.)

Generic Spec Interfaces

A spec interface can carry type parameters of its own, and the source type can name them:

@ImportOptics
public interface CartonOpticsSpec<U> extends OpticsSpec<Carton<U>> {
    @Wither("withLabel")
    Lens<Carton<U>, String> label();
}

generates public static <U> Lens<Carton<U>, String> label(). The parameters are the spec's, not the source type's, so you name them: Carton<T>'s own T never appears.

Each method declares the parameters its own source and focus types reach, which is what makes the two edges work:

The third row is worth noting: the source type need not be generic at all. A prism or traversal whose focus is parameterised brings the parameter in on its own.

One parameter is carried without being reached directly: one that a kept parameter's bound names, since the bound has to resolve. interface SubjectOpticsSpec<T, V extends List<T>> extends OpticsSpec<Carton<V>> focused through V generates static <T, V extends List<T>> Lens<Carton<V>, String> label(): T appears nowhere in the signature's source or focus, and is declared anyway so that V's bound means something.

An optic method cannot declare parameters of its own. The source type is fixed by OpticsSpec<S>, so nothing could ever bind them, and the declaration is rejected rather than generating a method no call could resolve:

@ImportOptics
interface CartonOpticsSpec extends OpticsSpec<Carton<String>> {

    @Wither("withContent")
    <X> Lens<Carton<String>, X> content();
}

The fine print: error reporting

A raw optic answers "is it there?" and nothing more: a missing field and a field of the wrong type both come back as an empty Optional. When you need to know which it was, the answer is not a special optic but the surrounding machinery:

• ValidatedPrism parses instead of matching, returning Validated<NonEmptyList<FieldError>, A> with a located error per failure.
• OpticOps validation methods run a validating function through a traversal, accumulating every failure.
• Mapping at the Boundary is the whole-boundary answer: a generated parse that reports every bad field at once, each located by path.

Reach for optics to navigate, and for one of those to diagnose.

Beyond Jackson

The same pattern fits any external type that resists auto-detection: Protocol Buffers (@MatchWhen on the generated hasX/getX oneof accessors, where the field is a message type rather than a scalar), XML DOM (prisms for element types), compiler or parser ASTs, and awkward legacy library types. The next page covers the other half of the story, the copy strategies (@ViaBuilder, @Wither, @ViaConstructor, @ViaCopyAndSet) that give you lenses rather than prisms.

PrismLaws.assertPrismLaws(JsonNodeOptics.object(), anObjectNode, aStringNode);

Previous: Optics for External Types Next: Database Records with JOOQ