Taming JSON with Jackson

Spec Interfaces for Jackson and Beyond

"The art of programming is the art of organising complexity, of mastering multitude and avoiding its bastard chaos."

– Edsger W. Dijkstra

Dijkstra's words ring especially true when facing JSON: a nested, dynamically-typed structure with optional fields, variable array contents, and no compile-time guarantees. The imperative approach fights that with defensive code, null checks upon null checks, type assertions, deeply nested conditionals.

Spec interfaces take the other path. You declare the shapes the data might take, the processor generates type-safe prisms for them, and navigation becomes composition. Here is the destination first, on a real API response:

    List<String> emails = Traversals.getAll(USER_EMAILS, response);
    // [alice@example.com, bob@example.com, carol@example.com]

    List<Double> overForty = Traversals.getAll(USER_AGES.filtered(age -> age > 40), response);
    // [45.0]

    double page = PAGE.getOrElse(1.0, response);
    // 1.0

No null checks, no casts, no nested conditionals. And the write direction is the same paths, run backwards:

    JsonNode anonymised = Traversals.modify(USER_EMAILS, JsonApiBook::mask, response);
    // every data.users[].email is masked; response itself is unchanged

    JsonNode aged = Traversals.modify(USER_AGES, age -> age + 1, response);
    // [33.0, 29.0, 46.0]

Why JsonNode Resists Auto-Detection

Point @ImportOptics at JsonNode and it has nothing to work with:

1. No sealed hierarchy. ObjectNode, ArrayNode and StringNode exist, but JsonNode is not sealed, so the processor cannot enumerate the variants.
2. No copy strategy to detect. ObjectNode and ArrayNode are mutable, and deepCopy() will isolate an edit, which is what JsonPaths.field below does before calling set. What is missing is anything declarative: no builder, no withers, no all-args constructor for the processor to rebuild a changed node through.
3. Predicate-based type checks. Jackson offers isObject() and isArray(), which auto-detection has no rule for.

Types like this need you to say what you want. That is a spec interface.

The Spec Interface

An interface extending OpticsSpec<S>, with one annotated abstract method per optic:

@ImportOptics
public interface JsonNodeOpticsSpec extends OpticsSpec<JsonNode> {

  @InstanceOf(ObjectNode.class)
  Prism<JsonNode, ObjectNode> object();

  @InstanceOf(ArrayNode.class)
  Prism<JsonNode, ArrayNode> array();

  @InstanceOf(StringNode.class)
  Prism<JsonNode, StringNode> text();

  @InstanceOf(NumericNode.class)
  Prism<JsonNode, NumericNode> numeric();

  @InstanceOf(BooleanNode.class)
  Prism<JsonNode, BooleanNode> bool();
}

@InstanceOf tells the processor to generate a prism that matches when the node is an instance of the given class, rebuilding through identity. From this declaration you get a class of static prisms:

JsonNodeOptics.object()   // Prism<JsonNode, ObjectNode>
JsonNodeOptics.array()    // Prism<JsonNode, ArrayNode>
JsonNodeOptics.text()     // Prism<JsonNode, StringNode>
JsonNodeOptics.numeric()  // Prism<JsonNode, NumericNode>
JsonNodeOptics.bool()     // Prism<JsonNode, BooleanNode>

Building Richer Tools

The generated prisms are primitives. Real JSON work wants field access, array traversal and value extraction, which are compositions of those primitives. Put them in an ordinary utility class:

  /**
   * A named field of an object node: empty when the node is not an object, or has no such field.
   */
  public static Affine<JsonNode, JsonNode> field(String name) {
    return JsonNodeOptics.object()
        .andThen(
            Affine.of(
                obj -> Optional.ofNullable(obj.get(name)),
                (obj, value) -> {
                  ObjectNode copy = obj.deepCopy();
                  copy.set(name, value);
                  return copy;
                }));
  }

  /** Every element of an array node, rebuilt as an array node after modification. */
  public static Traversal<JsonNode, JsonNode> elements() {
    return JsonNodeOptics.array()
        .andThen(
            Traversals.<ArrayNode, JsonNode>forIterableCollecting(
                list -> {
                  ArrayNode rebuilt = JsonNodeFactory.instance.arrayNode();
                  list.forEach(rebuilt::add);
                  return rebuilt;
                }));
  }

  /** The String behind a string node. */
  public static Affine<JsonNode, String> textValue() {
    return JsonNodeOptics.text()
        .andThen(
            Affine.of(
                node -> Optional.of(node.stringValue()),
                (node, value) -> (StringNode) JsonNodeFactory.instance.stringNode(value)));
  }

  /** The double behind a numeric node. */
  public static Affine<JsonNode, Double> numericValue() {
    return JsonNodeOptics.numeric()
        .andThen(
            Affine.of(
                node -> Optional.of(node.doubleValue()),
                (node, value) -> DoubleNode.valueOf(value)));
  }

Each is a prism composed with a hand-written Affine or Traversal, and the result types tell the story: field may miss (an Affine), elements may hit many (a Traversal).

A Real Pipeline: API Response Processing

The response this page works on:

{
  "status": "success",
  "data": {
    "users": [
      { "id": 1, "name": "Alice", "email": "alice@example.com", "age": 32 },
      { "id": 2, "name": "Bob",   "email": "bob@example.com",   "age": 28 },
      { "id": 3, "name": "Carol", "email": "carol@example.com", "age": 45 }
    ],
    "page": 1
  }
}

Name the paths once, at the top of the class, and the business logic reads as intent:

  /** Each user object inside data.users. */
  static final Traversal<JsonNode, JsonNode> EACH_USER =
      JsonPaths.field("data").andThen(JsonPaths.field("users")).andThen(JsonPaths.elements());

  // A Traversal composes with an Affine through asTraversal(): there is no
  // Traversal.andThen(Affine) overload, because the result is a Traversal either way.
  /** Every user's email address, as a String. */
  static final Traversal<JsonNode, String> USER_EMAILS =
      EACH_USER.andThen(JsonPaths.field("email").andThen(JsonPaths.textValue()).asTraversal());

  /** Every user's age, as a double. */
  static final Traversal<JsonNode, Double> USER_AGES =
      EACH_USER.andThen(JsonPaths.field("age").andThen(JsonPaths.numericValue()).asTraversal());

  /** The page number: a single value, so an Affine rather than a Traversal. */
  static final Affine<JsonNode, Double> PAGE =
      JsonPaths.field("data").andThen(JsonPaths.field("page")).andThen(JsonPaths.numericValue());

Those four constants are the whole abstraction. Traversals.getAll reads through them, Traversals.modify writes through them, and filtered narrows them, exactly as shown at the top of this page.

@InstanceOf versus @MatchWhen

@InstanceOf is for hierarchies whose variants are real Java subtypes:

@InstanceOf(ObjectNode.class)
Prism<JsonNode, ObjectNode> object();
// generates: node instanceof ObjectNode o ? Optional.of(o) : Optional.empty()

@MatchWhen is for libraries that expose a check-then-extract pair instead of subtypes:

@MatchWhen(predicate = "isObject", getter = "asObject")
Prism<Value, ObjectValue> object();
// generates: value.isObject() ? Optional.of(value.asObject()) : Optional.empty()

Both produce a Prism, so both compose the same way afterwards. Pick by what the library gives you: a type to test, or a method to ask.

@MatchWhen(predicate = "isString", getter = "asString")
Prism<Value, String> string();   // rejected: a String is not a Value

Parameterised Targets

@InstanceOf takes a class constant, which is always raw, and the generated test runs after erasure. A parameterised target may therefore only be narrowed to the type arguments the source type pins down — the ones a value of that source type must already have had to reach the test at all.

flowchart TD
    T["@InstanceOf(Circle.class)<br/>Prism&lt;S, Circle&lt;T&gt;&gt;"] --> Q{"Does S carry the<br/>argument Circle is<br/>reached under?"}
    Q -->|"Shape&lt;T&gt;, and<br/>Circle&lt;X&gt; implements Shape&lt;X&gt;"| P(["source instanceof Circle&lt;T&gt;<br/>javac checks it"])
    Q -->|"Shape, which<br/>declares nothing"| E(["Rejected: declare<br/>Circle&lt;?&gt;, or use @MatchWhen"])

    classDef decision fill:#e5c890,stroke:#df8e1d,color:#232634
    classDef tier fill:#a6d189,stroke:#40a02b,color:#232634
    classDef error fill:#e78284,stroke:#d20f39,color:#232634
    class T tier
    class Q decision
    class P tier
    class E error

A generic hierarchy pins its own argument, so the prism can promise it. T is the spec's own type parameter, and Circle<X> implements Shape<X> is what lets the test check it:

sealed interface Shape<X> permits Circle {}
record Circle<X>(X tag) implements Shape<X> {}

@ImportOptics
interface ShapeOpticsSpec<T> extends OpticsSpec<Shape<T>> {

    @InstanceOf(Circle.class)
    Prism<Shape<T>, Circle<T>> circle();
    // generates: source instanceof Circle<T> t ? Optional.of(t) : Optional.empty()
}

A base that says nothing about the argument pins nothing. Every instantiation passes the same test, so the same declaration is rejected:

class Shape {}
class Circle<X> extends Shape {}

@ImportOptics
interface ShapeOpticsSpec<T> extends OpticsSpec<Shape> {

    @InstanceOf(Circle.class)
    Prism<Shape, Circle<T>> circle();   // rejected: nothing checks T
}

Widened to the wildcard, which is what the test earns, it is accepted — and the spec needs no type parameter of its own once the prism stops promising one:

@ImportOptics
interface ShapeOpticsSpec extends OpticsSpec<Shape> {

    @InstanceOf(Circle.class)
    Prism<Shape, Circle<?>> circle();
    // generates: source instanceof Circle<?> t ? Optional.of(t) : Optional.empty()
}

Layering Domain Optics

The paths above are still shaped like the JSON. One more layer names them in the language of the domain, so the rest of the code never mentions field("data") at all:

public final class UserJson {

  public static Traversal<JsonNode, String> emails() { return JsonApiBook.USER_EMAILS; }

  public static Traversal<JsonNode, Double> ages() { return JsonApiBook.USER_AGES; }

  public static Affine<JsonNode, Double> page() { return JsonApiBook.PAGE; }
}

Now a service reads UserJson.emails(), and the wire format is an implementation detail of one class. (USER_EMAILS, USER_AGES and PAGE are package-private in the compiled example, so this facade sits in the same package as them; in your own code make them public or put the facade alongside.)

Generic Spec Interfaces

A spec interface can carry type parameters of its own, and the source type can name them:

@ImportOptics
public interface BoxOpticsSpec<U> extends OpticsSpec<Box<U>> {
    @Wither("withLabel")
    Lens<Box<U>, String> label();
}

generates public static <U> Lens<Box<U>, String> label(). The parameters are the spec's, not the source type's, so you name them: Box<T>'s own T never appears.

Each method declares the parameters its own source and focus types reach, which is what makes the two edges work:

The third row is worth noting: the source type need not be generic at all. A prism or traversal whose focus is parameterised brings the parameter in on its own.

One parameter is carried without being reached directly — one that a kept parameter's bound names, since the bound has to resolve. interface SubjectOpticsSpec<T, V extends List<T>> extends OpticsSpec<Box<V>> focused through V generates static <T, V extends List<T>> Lens<Box<V>, String> label(): T appears nowhere in the signature's source or focus, and is declared anyway so that V's bound means something.

An optic method cannot declare parameters of its own. The source type is fixed by OpticsSpec<S>, so nothing could ever bind them, and <X> Lens<Box<String>, X> content() is rejected at the declaration rather than generating a method no call could resolve.

The fine print: error reporting

A raw optic answers "is it there?" and nothing more: a missing field and a field of the wrong type both come back as an empty Optional. When you need to know which it was, the answer is not a special optic but the surrounding machinery:

• ValidatedPrism parses instead of matching, returning Validated<NonEmptyList<FieldError>, A> with a located error per failure.
• OpticOps validation methods run a validating function through a traversal, accumulating every failure.
• Mapping at the Boundary is the whole-boundary answer: a generated parse that reports every bad field at once, each located by path.

Reach for optics to navigate, and for one of those to diagnose.

Beyond Jackson

The same pattern fits any external type that resists auto-detection: Protocol Buffers (@MatchWhen on the generated hasX/getX oneof accessors, where the field is a message type rather than a scalar), XML DOM (prisms for element types), compiler or parser ASTs, and awkward legacy library types. The next page covers the other half of the story, the copy strategies (@ViaBuilder, @Wither, @ViaConstructor, @ViaCopyAndSet) that give you lenses rather than prisms.

PrismLaws.assertPrismLaws(JsonNodeOptics.object(), anObjectNode, aStringNode);

Previous: Optics for External Types Next: Database Records with JOOQ