Pre SIP: `for` with Control Flow (an alternative to Scala Async, Scala Continuations and Scala Virtualized)

[yangbo]

@julienrf

@yangbo

• A for comprehension for (p <- e) yield e' , where e' is a DSL expression, is translated to e.flatMap { case p => wrap(e') } .

This rule seems to conflict with the current rule that desugars to map calls.

It does not conflict with the current rule, because the current rule is applied to e' only if it’s not a DSL expression. However, it is indeed problematic. In fact the following code will never compile as long as we translate them to flatMap/map/foreach member methods, because these member methods defined on collections and futures conflict with each other.

def asyncLoop(futureSeq: Seq[Future[String]]): Future[Unit] = {
  for (future <- futureSeq) {
    string <- future
    println(string)
  }
}

The solution would be to translate them to rebindable keywords.flatMap / keywords.map / keywords.foreach. I can write down another proposal for details. But now I just removed this part from this proposal.

@julienrf

@yangbo

• wrap(if (e1) e2 else e3) is expanded to keywords.ifThenElse(wrap(e1))

This rule is incomplete, we should also pass wrap(e2) and wrap(e3) as parameters to ifThenElse .

Updated.

@julienrf

Overall, the proposal seems very ambitious and introduces a lot of new things. Currently, explaining how for comprehensions are desugared is already a bit convoluted. I would be hesitant to generalize the desugaring as you propose.

Every proposal increases complexity. This proposal actually increases less complexity than alternatives. For example, neither Scala Continuations nor Scala Async has a set of standardized translation rules. They actually increase the hidden complexity and do not guarantee their behaviors. Also note that, unlike alternatives, this proposal does not introduce new keywords.

The complexity consideration is also the reason I would like to create multiple independent proposals to improve for expressions, not a single huge proposal.

@julienrf

Also, with these new rules is there a risk to introduce a runtime performance overhead when using the standard collections?

No, because existing for comprehension will not be affected. Rules introduced in this proposal is applied to DSL expressions only, which were invalid syntax before this proposal.

@julienrf

It would be interesting to compare your approach with what F# does.

Yes. Originally I thought the DSL interface should be minimal, e.g. generate code with only flapMap / pure, like what we did in Each, Dsl.scala, Monadless, Scala Continuations and Scala Async. But now I realized the F# approach may be better. The rules to translate control flow to functions are crystal clear, and they also allow libraries to create runtime that is optimized for a specific DSL, for example operator fusion in GPU programming.