Hello everyone,
Just to make things clear: this is not April fools 
Inspired by comonadic comprehensions proposal and the recent success of Comma McTraily I was thinking about how we could express applicative comprehensions in Scala. This could be useful not only for people using typelevel style (I suppose, I’m not one of them) but also for mere mortals like me and people to which the word “applicative” says absolutely nothing. And I think I may have found a simple and totally compatible way to do this.
Basic proposal
So here’s what I propose: let’s reuse the with keyword to express that some computations in a for comprehension are “independent”, which means that one does not depend on the result of another and therefore, they can be executed “together” or “in parallel”:
for {
a <- aa with
b <- bb with
c <- cc
d <- dd
} yield a + b + c + d
Desugaring of this syntax merges independent computations using product operation. The example above would be equivalent to:
for {
((a, b), c) <- aa product bb product cc
d <- dd
} yield a + b + c + d
The product operation merges two independent computations into a single computation whose result is a pair, i.e. (F[A], F[B]) => F[(A,B)]. It is yet to decide whether product is the best name - I chose it because I saw that Cats calls it like that.
Syntax details
Currently, the specification defines for comprehension syntax as follows:
Expr1 ::= `for' (`(' Enumerators `)' | `{' Enumerators `}')
{nl} [`yield'] Expr
Enumerators ::= Generator {semi Generator}
Generator ::= Pattern1 `<-' Expr {[semi] Guard | semi Pattern1 `=' Expr}
Guard ::= `if' PostfixExpr
My proposal would only change the definition of Generator to:
Generator ::= Pattern1 `<-' Expr {with Pattern1 `<-' Expr} {[semi] Guard | semi Pattern1 `=' Expr}
As we can see, the proposed change is minimal, and - more importantly - fully backwards compatible. The only potential doubt I had is the overloaded meaning of the with keyword (which already has two meanings). There are some corner cases in which it may be necessary to explicitly disambiguate between them, e.g.
for {
// we must use parens to disambiguate two meanings of `with` keyword
a <- (new A with B) with
b <- bb
} ...
Use cases
- Parallel
Futures inforcomprehensions
Currently, in order to use two or more concurrently executedFutures in aforcomprehension, we must first assign them to intermediate variables:
val af = Future { ... }
val bf = Future { ... }
val cf = Future { ... }
val sumf = for {
a <- af
b <- bf
c <- cf
} yield a + b + c
However, if we defined product for Futures simply as zip, we could instead write:
val sumf = for {
a <- Future { ... } with
b <- Future { ... } with
c <- Future { ... }
} yield a + b + c
I believe this would be a real benefit for most Scala programmers - you don’t need to know anything about applicatives to find this syntax useful.
- Batching of operations
When some computations are independent, they can often be batched into a single larger operation. For example, multiple independent network operations could be sent as a single network message.
As an example, here’s a hypothetical API for Redis transactions that could be possible to implement with the new syntax:
for {
_ <- multi() with
a <- get("a") with
set <- smembers("someSet") with
_ <- incr("b") with
_ <- exec()
_ <- doSomethingMore(a, set)
} yield ...
A Redis transaction consists of multiple Redis commands invoked atomically, as a whole. Therefore they cannot use each other’s results and it’s natural to send them all at once through the network. The new syntax allows us to express this type-safely and at the same time have result of every command in the transaction assigned to separate identifier so that we can access them without unnecessary ceremony.
Considerations
- As already mentioned, the overload of
withkeyword could possibly be confusing. Originally I considered to use comma but then realized that it may be error-prone and unreadable, because it may be hard to spot the commas. - Introduction of
productas magic method has some downsides. Collections already defineproductwith completely different meaning. Although applicative composition doesn’t seem very useful with collections, the name conflict may still cause very confusing compilation errors. So maybe we should choose different name or even completely different desugaring. - Maybe we should allow the
withkeyword to be also used after guards and assignments. Currently it’s not possible in order to keep things as simple as possible.
Implementation
Finally, the time has come for some hard arguments
I believe to have already implemented the proposed change. The POC pull request is here: https://github.com/scala/scala/pull/5819
I’m quite happy about it because the changes turned out to be simple. Unless I’m missing something, there’s also full backwards compatibility of scala-reflect module. Quasiquotes understand the new syntax, too. The most complex part to implement was the un-desugaring of for comprehensions used by reification.
—
I’m really eager to hear your feedback about this. I’m also aware that if this change were to make it into the language, this would require a SIP and all the procedure around it.
Cheers,
Roman
.