# Traversal Generator Plugins ~~~admonish info title="What You'll Learn" - Which container types `@GenerateTraversals` supports out of the box - How to enable third-party collection support (Eclipse Collections, Guava, Vavr, Apache Commons, PCollections) - How the plugin discovery mechanism works - How to write your own generator for a custom container type ~~~ --- ## What Are Generator Plugins? When you annotate a record with `@GenerateTraversals`, the annotation processor needs to know *how* to traverse each field's container type. A `List` field requires different generated code than an `Optional` or an `Either`. Each container type is handled by a **generator plugin**: a small class that implements the `TraversableGenerator` SPI (Service Provider Interface). The processor discovers these plugins at compile time via Java's `ServiceLoader` and delegates code generation to whichever plugin claims support for the field's type. Higher-Kinded-J ships 31 generator plugins covering JDK types, HKJ core types, and five popular third-party collection libraries. --- ## Supported Types at a Glance ### JDK Standard Library (Always Available) These generators are always active. No additional dependencies are required. | Type | Focus | Behaviour | |------|-------|-----------| | `List` | Each element | Traverses all elements via `Traversals.traverseList()` | | `Set` | Each element | Traverses all elements in iteration order via `Traversals.traverseSet()`, keeping that order; elements that map to equal values collapse into one | | `Collection` | Each element | Rebuilds a set source as a set and any other source as a list, via `Traversals.traverseCollection()` | | `Optional` | 0 or 1 element | Applies function if present; returns unchanged if empty | | `Map` | Each value | Traverses values whilst preserving keys | | `A[]` | Each element | Converts to list, traverses, converts back to array | ### HKJ Core Types (Always Available) These types are part of `hkj-core`, which is always on your classpath. | Type | Focus | Behaviour | |------|-------|-----------| | `Maybe` | 0 or 1 element | Applies function to `Just`; passes through `Nothing` | | `Either` | Right value | Applies function to `Right`; passes through `Left` | | `Try` | Success value | Applies function to `Success`; passes through `Failure` | | `Validated` | Valid value | Applies function to `Valid`; passes through `Invalid` | ### Third-Party Libraries (Add to Your Dependencies) Generator plugins for third-party libraries activate automatically when the library is on the annotation processor's classpath. Simply add the library as a dependency; no further configuration is needed. #### Eclipse Collections ```kotlin dependencies { implementation("org.eclipse.collections:eclipse-collections:13.0.0") } ``` | Type | Notes | |------|-------| | `ImmutableList` | | | `ImmutableSet` | | | `ImmutableBag` | | | `ImmutableSortedSet` | Preserves natural ordering | | `MutableList` | | | `MutableSet` | | | `MutableBag` | | | `MutableSortedSet` | Preserves natural ordering | #### Google Guava ```kotlin dependencies { implementation("com.google.guava:guava:33.7.2-jre") } ``` | Type | Notes | |------|-------| | `ImmutableList` | Uses `ImmutableList.copyOf()` for reconstruction | | `ImmutableSet` | Uses `ImmutableSet.copyOf()` for reconstruction | #### Vavr ```kotlin dependencies { implementation("io.vavr:vavr:1.0.1") } ``` | Type | Notes | |------|-------| | `io.vavr.collection.List` | | | `io.vavr.collection.Set` | | #### Apache Commons Collections ```kotlin dependencies { implementation("org.apache.commons:commons-collections4:4.5.0") } ``` | Type | Notes | |------|-------| | `HashBag` | | | `UnmodifiableList` | | #### PCollections ```kotlin dependencies { implementation("org.pcollections:pcollections:5.0.0") } ``` | Type | Notes | |------|-------| | `org.pcollections.PVector` | Reconstructed via `TreePVector.from(Collection)` | | `org.pcollections.PStack` | Reconstructed via `ConsPStack.from(Collection)` | | `org.pcollections.PSet` | Reconstructed via `HashTreePSet.from(Collection)` | | `org.pcollections.PSortedSet` | Natural ordering only; custom comparators are not preserved | | `org.pcollections.PBag` | Reconstructed via `HashTreePBag.from(Collection)` | | `org.pcollections.PMap` | Value-focused; reconstructed via `HashTreePMap.from(Map)` | | `org.pcollections.PSortedMap` | Value-focused; natural key ordering only | --- ## Using Third-Party Types with @GenerateTraversals Once the library is on your classpath, usage is identical to JDK types: ```java import org.eclipse.collections.api.list.ImmutableList; import org.higherkindedj.optics.annotations.GenerateTraversals; @GenerateTraversals public record Warehouse( String name, ImmutableList products ) {} // The processor generates a traversal for the 'products' field automatically. // Use it exactly like a List traversal: Warehouse warehouse = new Warehouse("Depot", Lists.immutable.of("bolt", "nut")); Warehouse updated = Traversals.modify( WarehouseTraversals.products(), String::toUpperCase, warehouse ); ``` --- ## How Plugin Discovery Works The processor uses a three-layer mechanism to discover generators: 1. **SPI Interface**: `TraversableGenerator` in `hkj-processor` defines the contract. Any class implementing this interface can be discovered. 2. **ServiceLoader**: At compile time, the processors discover every registered implementation through `ServiceLoader.load(TraversableGenerator.class)`; `@GenerateTraversals` and `@GenerateFocus` load directly, and `@ImportOptics` loads through the shared registry. 3. **Avaje SPI**: Each generator class is annotated with `@ServiceProvider(TraversableGenerator.class)`. The [Avaje SPI](https://avaje.io/spi/) annotation processor automatically generates the `META-INF/services` files and validates that the `module-info.java` `provides` clause is complete. A missing entry causes a compile error with a copy-pasteable fix. ```text Annotation processors │ ▼ ServiceLoader.load(TraversableGenerator.class) │ ├── ListGenerator (supports List) ├── CollectionGenerator (supports Collection) ├── OptionalGenerator (supports Optional) ├── EitherGenerator (supports Either) ├── GuavaImmutableListGenerator (supports ImmutableList) └── ... 26 more generators ``` For each record component, the processor asks every loaded generator `supports(TypeMirror)`, and the highest-priority generator that answers `true` handles code generation for that field, wherever its registration entry lands. Generators rank themselves through `priority()`: `PRIORITY_OVERRIDE` (100) beats `PRIORITY_DEFAULT` (0) beats `PRIORITY_FALLBACK` (-100). Two generators claiming the same type at the same priority draw a compile-time warning naming both, and the first registered wins. The same resolution serves `@GenerateTraversals`, the Focus DSL's widening and `@ImportOptics`, so the three annotations always agree about which generator claims a type. The exceptions are the five types `@GenerateFocus` recognises by name, which it widens without asking a generator, and a primitive array, which it never widens. A component that holds elements but reaches no generator is not passed over silently. A `java.util.Collection` or `java.util.Map` subtype that no loaded generator supports — a `Deque`, a `SortedMap`, a Guava, Apache Commons or PCollections type whose plugin is not on the processor path — draws a compile-time **note** on the component, naming the unsupported type and the remedy: declare the component as a supported container, or put a `TraversableGenerator` for it on the annotation processor path. A container that is not a `java.util.Collection` or `Map` at all — Vavr's collections and Eclipse Collections' `Immutable*` types are `Iterable` only — is passed over silently, as is a `String` or an `int`, for the same reason a bare `Iterable` is not the bar: `java.nio.file.Path` implements it. --- ## Writing a Custom Generator If your project uses a container type that is not covered by the built-in plugins, you can write your own generator and register it as a service provider. ### The TraversableGenerator Interface ```java public interface TraversableGenerator { /** Return true if this generator handles the given type. */ boolean supports(TypeMirror type); /** * Declares the cardinality of elements in this container type. For a generator * that names an optic expression, it decides the path tier the Focus DSL gives * the field: * ZERO_OR_ONE → AffinePath, always (Either, Try, Validated) * ZERO_OR_MORE → TraversalPath under widenCollections, or when widenFields * names the field * * Default is ZERO_OR_MORE, which is correct for collection-like types. */ default Cardinality getCardinality() { return Cardinality.ZERO_OR_MORE; } /** * The optic @GenerateFocus composes to widen the field: an expression producing * an Affine for ZERO_OR_ONE, or an Each for ZERO_OR_MORE. Leave it empty and the * generator serves @GenerateTraversals and @ImportOptics only: under * @GenerateFocus the path stops at the container, as if no generator supported it. */ default String generateOpticExpression() { return ""; } /** The classes the optic expression names, imported where it is written. */ default Set getRequiredImports() { return Set.of(); } /** * Which type argument to focus on (0-indexed). * Default is 0. Override to 1 for types like Either * where the traversal focuses on the second argument. */ default int getFocusTypeArgumentIndex() { return 0; } /** * Rank among generators claiming the same type: the highest priority wins. * PRIORITY_OVERRIDE (100) replaces a built-in generator; PRIORITY_FALLBACK * (-100) yields to any default. Default is PRIORITY_DEFAULT (0). */ default int priority() { return PRIORITY_DEFAULT; } /** * Generate the body of the modifyF method. * Returns a Palantir JavaPoet CodeBlock. */ CodeBlock generateModifyF( RecordComponentElement component, ClassName recordClassName, List allComponents); /** * The same, for a body written into targetPackage. Override this one when the * body names a component's type; the default calls the form above. */ default CodeBlock generateModifyF( RecordComponentElement component, ClassName recordClassName, List allComponents, String targetPackage) { return generateModifyF(component, recordClassName, allComponents); } } ``` ~~~admonish note title="Naming a type in the body" A body that writes out a component's type, such as a local it declares, is not always written into that type's package: a companion generated under a `targetPackage`, or the optics generated for a type reached with `@ImportOptics`, land elsewhere. Name such a type with `ProcessorUtils.typeNameOf(type, targetPackage)` from the four-argument `generateModifyF`, and a type-use annotation that package cannot name, one package-private to the component's own package, is left off rather than copied into a file that could not compile with it. `BaseTraversableGenerator.getTypeArgumentName(component, index, targetPackage)` does the same for a type argument. A generator that names no type of its own needs neither. ~~~ ~~~admonish note title="Cardinality, the optic expression and @GenerateFocus" Only `@GenerateFocus` reads `getCardinality()` and `generateOpticExpression()`; `@GenerateTraversals` and `@ImportOptics` need neither. A generator takes part in Focus widening by naming an optic expression. Its cardinality then decides whether a field gives an `AffinePath` (zero or one element) or a `TraversalPath` (zero or more elements), in the static Focus method and the navigators alike. A generator with no expression takes no part: the field's path stops at the container, as if no generator supported it. ~~~ ~~~admonish tip title="Replacing a Built-in Generator" To take over a type a built-in generator already claims, implement `supports()` for that type and return `PRIORITY_OVERRIDE` from `priority()`. Registration order does not matter: the higher priority wins on every route that chooses a generator, so no fork of the built-in plugin is needed. Name an optic expression too, or extend the built-in generator to inherit its one: an override with none takes the type out of `@GenerateFocus` widening. `@GenerateFocus` asks no generator about `Optional`, `Maybe`, `List`, `Set` or `Collection`: it recognises them by name and widens them itself, so a build without the plugins widens them too. An override for one of those changes what `@GenerateTraversals` and `@ImportOptics` generate, not its Focus path. ~~~ ### Step-by-Step Example Suppose you want to add traversal support for a custom `NonEmptyList` type. **1. Create the generator class:** ```java package com.example.generator; import com.palantir.javapoet.ClassName; import com.palantir.javapoet.CodeBlock; import io.avaje.spi.ServiceProvider; import java.util.List; import javax.lang.model.element.RecordComponentElement; import javax.lang.model.type.DeclaredType; import javax.lang.model.type.TypeMirror; import org.higherkindedj.optics.processing.generator.BaseTraversableGenerator; import org.higherkindedj.optics.processing.spi.TraversableGenerator; import org.higherkindedj.optics.util.Traversals; @ServiceProvider(TraversableGenerator.class) public class NonEmptyListGenerator extends BaseTraversableGenerator { private static final String FQN = "com.example.NonEmptyList"; @Override public boolean supports(final TypeMirror type) { if (!(type instanceof DeclaredType declaredType)) return false; return declaredType.asElement().toString().equals(FQN); } @Override public CodeBlock generateModifyF( final RecordComponentElement component, final ClassName recordClassName, final List allComponents) { final String componentName = component.getSimpleName().toString(); final String constructorArgs = generateConstructorArgs(componentName, "newNonEmptyList", allComponents); return CodeBlock.builder() // Convert to java.util.List, traverse, convert back .addStatement( "final var javaList = source.$L().toList()", componentName) .addStatement( "final var effectOfList = $T.traverseList(javaList, f, applicative)", Traversals.class) .addStatement( "final var effectOfNonEmptyList = applicative.map(list -> com.example.NonEmptyList.of(list), effectOfList)") .addStatement( "return applicative.map(newNonEmptyList -> new $T($L), effectOfNonEmptyList)", recordTypeName(component, recordClassName), constructorArgs) .build(); } } ``` **2. Add the module-info.java provides clause:** ```java module com.example.generators { requires org.higherkindedj.processor; requires com.palantir.javapoet; requires java.compiler; requires static io.avaje.spi; provides org.higherkindedj.optics.processing.spi.TraversableGenerator with com.example.generator.NonEmptyListGenerator; } ``` **3. Add javapoet and Avaje SPI to your build:** ```kotlin dependencies { implementation("io.github.higher-kinded-j:hkj-processor:LATEST-VERSION") implementation("com.palantir.javapoet:javapoet:0.22.0") // the javapoet release hkj-processor is built with compileOnly("io.avaje:avaje-spi-service:2.17") annotationProcessor("io.avaje:avaje-spi-service:2.17") } ``` **4. Add your generator module to the annotation processor path in projects that use it:** ```kotlin dependencies { annotationProcessor("com.example:my-generators:1.0.0") } ``` The `TraversalProcessor` will now discover your `NonEmptyListGenerator` via `ServiceLoader` and generate traversals for any `NonEmptyList` field. It names no optic expression, so under `@GenerateFocus` a `NonEmptyList` field keeps a `FocusPath` on the list; override `generateOpticExpression()` with an `Each` for it to widen to a `TraversalPath` too. ### Implementation Tips - **Extend `BaseTraversableGenerator`** to inherit `getGenericTypeName()`, `getTypeArgumentName(component, index)` and `generateConstructorArgs()` helper methods. Read every type argument through one of the first two: they resolve a wildcard to the type it stands for, and a wildcard written into generated source does not compile. - **Construct the record through `recordTypeName(component, recordClassName)`**, not the class name you were handed: a generic record's traversal is generated in a method carrying its type variables, and naming the record without them constructs a raw instance. - **Name the effect through `effectVariable(component)`** wherever the body you emit writes a `Kind`: a record is free to declare a type parameter called `F`, and the generated method then declares the effect under another name. - **Map over what `f` returns as it is.** The body is written against `source`, `f` and `applicative`, and `f` is a `Function>` at the focus type, so its result needs no cast. A cast to `Kind` names the type it already has, and javac reports it under `-Xlint:cast` in every build that compiles your generated source. - **Use fully qualified names** in `supports()` to avoid false matches with similarly named types. - **Reuse `Traversals.traverseList()`** when your type can be converted to a `java.util.List`. Most third-party generators follow this pattern: convert to list, traverse, convert back. A map-shaped type hands itself to `Traversals.traverseMapValues()` instead, which keeps the keys and gives back a JDK `Map` to rebuild from. - **Override `getFocusTypeArgumentIndex()`** if your type's traversal target is not the first type parameter (e.g. `Either` focuses on index 1). - **Override `getCardinality()`** to return `Cardinality.ZERO_OR_ONE` for optional-like types (e.g. `Either`, `Try`, `Validated`). The default `ZERO_OR_MORE` is correct for collection-like types and does not need overriding. - **Override `generateOpticExpression()` and `getRequiredImports()`** for the field to widen under `@GenerateFocus`: name the `Affine` or `Each` that reads and rebuilds your container. Without them the Focus path stops at the container. - **Write integration tests** using Google's compile-testing library to verify generated code compiles and contains the expected statements. Compiling it with `-Xlint:cast,unchecked,rawtypes -Werror` and asserting `succeededWithoutWarnings()` catches a warning your users' builds would otherwise report in a file they cannot edit. --- ~~~admonish info title="Key Takeaways" * **31 built-in generators** cover JDK types, HKJ core types, Eclipse Collections, Guava, Vavr, Apache Commons, and PCollections * **Third-party support activates automatically** when the library is on the classpath; no configuration required * **The SPI is extensible**: implement `TraversableGenerator`, register it with `@ServiceProvider`, and the processor discovers it at compile time * **Most generators follow a common pattern**: convert to `java.util.List`, traverse with `Traversals.traverseList()`, convert back to the original type; map-shaped ones traverse with `Traversals.traverseMapValues()` and rebuild from the JDK `Map` it returns * **The optic expression opts in to widening, and cardinality picks the tier**: for a generator that names an optic, `ZERO_OR_ONE` produces an `AffinePath`, always. `ZERO_OR_MORE` produces a `TraversalPath` under `widenCollections = true`, or when `widenFields` names the field. Static Focus methods and navigator methods read the same answer ~~~ ~~~admonish tip title="See Also" - [Traversals](../optics/traversals.md) - Using generated traversals in practice - [Common Data Structures](../optics/common_data_structure_traversals.md) - Traversals for Optional, Map, and Tuple types - [Build Plugins](gradle_plugin.md) - The build plugin adds `hkj-processor-plugins` to your annotation processor path automatically ~~~ --- **Previous:** [Diagnostics](diagnostics.md) **Next:** [PCollections Integration](pcollections_integration.md)