For-Comprehensions
Write a chain of flatMap calls as a list of steps, with filters, independent steps and traversals.
- Turn nested
flatMapcalls into steps withFor.from,from,letandyield - Filter a chain with
when, given aMonadZero - Mark independent steps with
par, and say when they run at the same time - Traverse a collection inside a chain with
traverse,sequenceandflatTraverse - Choose between
For,ForPathandForState
The code on this page is ForBook.java: the page includes it, so the build compiles and runs it. ForBookTest.java holds what the page says each step does.
When nested flatMap calls get deep
A receipt needs three things in turn: the order, its customer's verified email, and a confirmation from charging the order's total. Each step needs what an earlier step found, so Monad's flatMap chains them. With one Optional you would nest flatMap calls inside one another, and the same happens over Kind.
For one container, an Effect Path chains the same steps with ForPath, such as ForPath.from(eitherPath). Application code stays on the Paths. This page is for For, a chain written once for any container that has a Monad. Written as nested calls, the receipt is:
// One receipt for any container F, as nested flatMap calls
public static <F extends WitnessArity<TypeArity.Unary>> Kind<F, String> receiptNested(
Monad<F> monad,
Kind<F, Order> found,
Function<Customer, Kind<F, EmailAddress>> verify,
Function<BigDecimal, Kind<F, String>> charge) {
return monad.flatMap(
order -> {
BigDecimal total = total(order);
return monad.flatMap(
email ->
monad.map(
confirmation ->
"receipt " + confirmation + " for " + total + " to " + email.value(),
charge.apply(total)),
verify.apply(order.customer()));
},
found);
}
The receipt is in the innermost lambda, and total has to be a local variable so that a later step can still see it. A For comprehension writes the same chain as a list of steps.
The code on this page uses these imports, with Order from the order records the Optics chapter introduces:
import static org.higherkindedj.hkt.either.EitherKindHelper.EITHER;
import static org.higherkindedj.hkt.instances.Witnesses.either;
import static org.higherkindedj.hkt.instances.Witnesses.list;
import static org.higherkindedj.hkt.instances.Witnesses.maybe;
import static org.higherkindedj.hkt.list.ListKindHelper.LIST;
import static org.higherkindedj.hkt.maybe.MaybeKindHelper.MAYBE;
import static org.higherkindedj.hkt.validated.ValidatedKindHelper.VALIDATED;
import java.math.BigDecimal;
import java.util.List;
import java.util.UUID;
import java.util.function.Function;
import org.higherkindedj.example.book.optics.cast.Customer;
import org.higherkindedj.example.book.optics.cast.EmailAddress;
import org.higherkindedj.example.book.optics.cast.LineItem;
import org.higherkindedj.example.book.optics.cast.Order;
import org.higherkindedj.example.book.optics.cast.OrderStatus;
import org.higherkindedj.hkt.Kind;
import org.higherkindedj.hkt.Monad;
import org.higherkindedj.hkt.MonadZero;
import org.higherkindedj.hkt.Semigroups;
import org.higherkindedj.hkt.TypeArity;
import org.higherkindedj.hkt.WitnessArity;
import org.higherkindedj.hkt.either.Either;
import org.higherkindedj.hkt.either.EitherKind;
import org.higherkindedj.hkt.expression.For;
import org.higherkindedj.hkt.instances.Instances;
import org.higherkindedj.hkt.list.ListKind;
import org.higherkindedj.hkt.list.ListTraverse;
import org.higherkindedj.hkt.maybe.Maybe;
import org.higherkindedj.hkt.maybe.MaybeKind;
import org.higherkindedj.hkt.validated.Validated;
import org.higherkindedj.hkt.validated.ValidatedKind;
Steps: from, let and yield
For.from(monad, source) starts the chain, and each later step adds values, filters the chain, or ends it:
| Step | What it does | Built on |
|---|---|---|
from(f) | Adds the value inside the container f gives back | flatMap |
let(f) | Adds a plain value worked out from the values so far | map |
when(test) | Gives the instance's empty value when test fails | MonadZero's filter |
par(f1, f2) | Adds two to four values that do not need one another | flatMap, then map2 to map4 |
yield(f) | Ends the chain with a result made from every value | map |
The first step's function receives the value bound so far. From the second value on, a step receives a tuple holding the values in the order they were bound, read as t._1() for the first, t._2() for the second, and so on. yield takes either the tuple or one parameter for each value. Written with For, the receipt reads top to bottom, and in it t._1() is the order and t._2() the total that let added:
// The same receipt, as a For comprehension: each step reads what the steps before it bound
public static <F extends WitnessArity<TypeArity.Unary>> Kind<F, String> receipt(
Monad<F> monad,
Kind<F, Order> found,
Function<Customer, Kind<F, EmailAddress>> verify,
Function<BigDecimal, Kind<F, String>> charge) {
return For.from(monad, found)
.let(order -> total(order))
.from(t -> verify.apply(t._1().customer()))
.from(t -> charge.apply(t._2()))
.yield(
(order, total, email, confirmation) ->
"receipt " + confirmation + " for " + total + " to " + email.value());
}
Run it with Either's Monad, once with an email that is verified and once with one that is not. ORDER is any order of two lines, totalling 50.00:
Monad<EitherKind.Witness<String>> eitherMonad = Instances.monad(either());
Function<BigDecimal, Kind<EitherKind.Witness<String>, String>> charge =
amount -> EITHER.widen(Either.right("PAY-1"));
Either<String, String> sent =
EITHER.narrow(
receipt(
eitherMonad,
EITHER.widen(Either.right(ORDER)),
customer -> EITHER.widen(Either.right(customer.email())),
charge));
// Right(receipt PAY-1 for 50.00 to ada@example.com)
Either<String, String> unverified =
EITHER.narrow(
receipt(
eitherMonad,
EITHER.widen(Either.right(ORDER)),
customer -> EITHER.widen(Either.left("email not verified")),
charge));
// Left(email not verified)
What happens when a step has no value is the instance's, as on the Monad page: Either's stops at the Left, and the charge never runs.
Filtering with when
when needs a MonadZero: a Monad with an empty value, such as an empty list or Nothing, to give back when a test fails. The type you start the chain from decides. Instances.monadZero(list()) gives a chain with when, and Instances.monad(list()) gives one without:
For.from(Instances.monad(list()), orders).when(order -> order.status() == OrderStatus.SHIPPED);
javac refuses it with "cannot find symbol", because a chain started from a plain Monad has no when. The instance decides what the empty value is: List's drops the element, and Maybe's gives Nothing:
MonadZero<ListKind.Witness> listMonad = Instances.monadZero(list());
List<UUID> shippedIds =
LIST.narrow(
For.from(listMonad, LIST.widen(List.of(ORDER, SHIPPED)))
.when(order -> order.status() == OrderStatus.SHIPPED)
.yield(Order::id));
// [00000000-0000-0000-0000-000000000002]
MonadZero<MaybeKind.Witness> maybeMonad = Instances.monadZero(maybe());
Maybe<UUID> notShipped =
MAYBE.narrow(
For.from(maybeMonad, MAYBE.just(ORDER))
.when(order -> order.status() == OrderStatus.SHIPPED)
.yield(Order::id));
// Nothing
Independent steps with par
par binds values that do not need one another. For.par(monad, …) starts a chain with two to five of them, and a step's par(f1, …) fans out two to four from the values bound before it. Its result is an ordinary step, so from, let, when or another par can follow.
par combines its values with the instance's map2 to map5, where from uses flatMap. So it does what the instance's Applicative does: with Validated's instance, par reports both errors, and from stops at the first. The step's par fans out from the order:
Monad<ValidatedKind.Witness<List<String>>> validatedMonad =
Instances.validated(Semigroups.list());
Kind<ValidatedKind.Witness<List<String>>, String> blankSku =
VALIDATED.widen(Validated.invalid(List.of("SKU is blank")));
Kind<ValidatedKind.Witness<List<String>>, Integer> noQuantity =
VALIDATED.widen(Validated.invalid(List.of("quantity must be positive")));
Validated<List<String>, String> independent =
VALIDATED.narrow(
For.par(validatedMonad, blankSku, noQuantity)
.yield((sku, quantity) -> quantity + " x " + sku));
// Invalid([SKU is blank, quantity must be positive])
Validated<List<String>, String> inSequence =
VALIDATED.narrow(
For.from(validatedMonad, blankSku)
.from(sku -> noQuantity)
.yield((sku, quantity) -> quantity + " x " + sku));
// Invalid([SKU is blank])
Either<String, String> fannedOut =
EITHER.narrow(
For.from(eitherMonad, EITHER.widen(Either.right(ORDER)))
.par(
order -> EITHER.widen(Either.right(order.customer().email())),
order -> EITHER.widen(Either.right(total(order))))
.yield((order, email, total) -> total + " to " + email.value()));
// Right(50.00 to ada@example.com)
par says the values are independent, and it runs them at the same time only where the instance's map2 does. No instance's map2 starts its values together: IO's and VTask's run them one after the other, and CompletableFuture's waits for futures that may already be running. To run two VTasks at the same time, use ForPath.par on two VTaskPaths, which calls Par.map2, or call Par.map2 yourself.
Traversing inside a chain
A step can run an effect on every element of a collection the chain holds, and bind the collected results:
| Step | What it binds | Built on |
|---|---|---|
traverse(traversable, extractor, f) | f run on every element, the results collected | Traverse's traverse |
sequence(traversable, extractor) | A collection of containers, turned into one container of a collection | Traverse's sequenceA, which is traverse with the identity function |
flatTraverse(traversable, innerMonad, extractor, f) | f run on every element, the nested collections flattened | traverse, then the collection's own flatMap |
traversable is the collection's Traverse, such as ListTraverse.INSTANCE, and innerMonad the collection's own Monad. extractor picks the collection out of the values bound so far; here the chain starts from it, so it is lines -> lines. LIST.widen puts the list in, and .map(LIST::narrow) takes the result out. priceInStock prices a line, or gives Nothing for a line with no stock, so one missing line makes the whole result Nothing. With maybeMonad from the when example:
Function<LineItem, Kind<MaybeKind.Witness, BigDecimal>> priceInStock =
line -> line.quantity() > 0 ? MAYBE.just(lineTotal(line)) : MAYBE.nothing();
Maybe<List<BigDecimal>> priced =
MAYBE
.narrow(
For.from(maybeMonad, MAYBE.just(LIST.widen(ORDER.lines())))
.traverse(ListTraverse.INSTANCE, lines -> lines, priceInStock)
.yield((lines, prices) -> prices))
.map(LIST::narrow);
// Just([40.00, 10.00])
Maybe<List<BigDecimal>> backordered =
MAYBE
.narrow(
For.from(maybeMonad, MAYBE.just(LIST.widen(BACKORDERED.lines())))
.traverse(ListTraverse.INSTANCE, lines -> lines, priceInStock)
.yield((lines, prices) -> prices))
.map(LIST::narrow);
// Nothing
Maybe<List<Integer>> quantities =
MAYBE
.narrow(
For.from(maybeMonad, MAYBE.just(LIST.widen(List.of(MAYBE.just(1), MAYBE.just(4)))))
.sequence(ListTraverse.INSTANCE, items -> items)
.yield((items, values) -> values))
.map(LIST::narrow);
// Just([1, 4])
Maybe<List<Integer>> flattened =
MAYBE
.narrow(
For.from(maybeMonad, MAYBE.just(LIST.widen(List.of(1, 2))))
.flatTraverse(
ListTraverse.INSTANCE,
Instances.monad(list()),
numbers -> numbers,
n -> MAYBE.just(LIST.widen(List.of(n, n * 10))))
.yield((numbers, flat) -> flat))
.map(LIST::narrow);
// Just([1, 10, 2, 20])
Which comprehension should you use?
| You are writing | Use |
|---|---|
| Application code over one Effect Path | ForPath, which takes and gives back Paths |
A method for any container, over Kind | For, on this page |
| A chain whose values are easier to name than to number | ForState, with named fields in a record, or toState from For, as ForState's two ways in shows |
| Steps that read a field, convert a value or match a case | focus, through and match, as For with Optics shows: through works on the first step only, and match needs a MonadZero |
A monad transformer, such as StateT, is a Monad over Kind, so For chains it as it does any other: the StateT page shows one.
Things people get wrong
- "
parruns the steps at the same time." It marks them as independent, and the instance'smap2decides how they run.ForPath.paronVTaskPathis the one that runs them at once. - "
Listhaswhen, so my chain does." Only a chain started from aMonadZero, such asInstances.monadZero(list()), haswhen. Started fromInstances.monad(list()), it has none. - "
Foris Java's version of Scala'sfor." It is a library builder, with no compiler support. Each step is a method call you can read in the source.
You can now write a chain of dependent steps once for any container, as receipt is, with plain values between the steps. You can filter it with when, mark independent steps with par knowing what each instance does with them, and traverse a collection inside it. The rest of this page is for how many values a chain holds, and for readers who know Scala's for.
Maybe's instance is a MonadZero. Why does this chain not compile, and what one change gives it when?
Monad<MaybeKind.Witness> monad = Instances.monadZero(maybe());
For.from(monad, found).when(order -> order.status() == OrderStatus.SHIPPED);
Answer and why
Answer and why
The declared type decides, and monad is declared a Monad, so the chain has no when and javac reports "cannot find symbol". Declare it MonadZero<MaybeKind.Witness>, and For.from starts the chain that has when.
Where this lives: Filtering with when.
With Either's instance, For.par starts a chain from Left("SKU is blank") and Left("quantity must be positive"). What does yield give?
Answer and why
Answer and why
Left(SKU is blank). par combines its values with the instance's map2, and Either's keeps the first failure. Only an instance whose map2 joins errors, such as Validated's, reports both.
Where this lives: Independent steps with par.
The fine print
Twelve values, and their tuples
A chain holds up to twelve values, from any mix of from, let, par and the optic steps. The first step is written by hand, and the steps that hold two to twelve values are generated when Higher-Kinded-J is built, so your build needs no processor for them. The twelfth has no room for another value, so it can only yield, call toState, or filter with when in a chain started from a MonadZero. Past four or five values, positions such as t._4() get hard to follow, and toState moves the values into a record with named fields.
Coming from Scala
Scala builds the same chain into its syntax:
for {
order <- found
total = order.total
email <- verify(order.customer)
confirmation <- charge(total)
} yield s"receipt $confirmation for $total to $email"
The compiler turns every <- but the last into flatMap and the last into map, an if line into withFilter, and = into a map that carries the value. For gives the same steps as methods: from, when, let and yield.
- Monad: the
flatMapthatfromis built on - Applicative: the
map2thatparis built on - ForPath Comprehension: the same steps over Effect Paths, with ForPath Traverse for collections
- ForState: a chain whose values are named fields, entered with
toState - For with Optics:
focus,match,ForTraversalandForIndexed - For with MTL Capabilities: reader, state and writer capabilities as steps
- Order Workflow: a longer chain, end to end
- Scala documentation, For Comprehensions: the syntax
Forfollows
Practise par on Paths in the ForPath Parallel journey (9 exercises), and traverse in Tutorial 03: For Traverse Comprehension (14 exercises).
Previous: MonadError Next: Check Your Understanding