PRE SIP: ThisFunction | scope injection (similar to kotlin receiver function)

Language Design
30

It is similar to my idea but without wrapping everything with ThisFunction[T,R]. You just expects that Builder’ll implement interface with apply() method that’ll be filled with anything in block.

  • + cost is potentially smaller. In both cases we need to create Builder or ThisFunction but in my proposition you’ll probably create T also (which is builder in your example).
  • - in my proposition ThisFunction is marker that say to compiler how to treat definition assigned to it (bit like PartialFunction i think). It could be simpler in implementation/ reason about.
  • - in your example you have not shown why HTML become inlined. We still need additional notation (like this notation from this proposal) to say to compiler that it should treat HTML differently.
//Why compiler decide to understand code like this:
html { body() }
//as shortcut for this
html( new HTML { override def apply():Unit = { body() }})
// instead of just expecting that some method body() returns HTML instance?
// what if there is method body() outside? what if it returns HTML instance?
31

I can suggest an option with ThisFunction :wink:

  trait ThisFunction[-T, +R] extends Function1[T, R] 

  def html(htmFuncl: ThisFunction[HTML,R]): HTML =  {
    val html = htmlFunc()
    html.set(...) //dependency injection
    html.apply()       //ThisFunction extends Function1 and we can call it as normal 
    html
 }

The main difference is in a place where HTML class will be constructed, and how apply \ body function will be implemented.

I think in my proposal there is no need to define ThisFunction. There will be enough to define Builder with annotation @FunctionalInterface
We can see the implementation of this approach in java lambda

But I have not enough competencies in scala internals to prove my words, sorry.

I just do not understand why It could be simpler

32 
def html(htmFuncl: ThisFunction[HTML,R]): HTML =  {
   //in my proposition htmlFunc will be (HTML) => R
   //not () => R. This is the differences. You cannot just call it. 
   val html:HTML = htmlFunc()  
   html.set(...) //dependency injection
   //below comment is misleading. i guess it is just copy-paste artifact but still. 
   html.apply()       //ThisFunction extends Function1 and we can call it as normal 
   html
}

In your proposal we don’t need ThisFunction because T type has been constrained to satisfy some conditions (and we’ll fill it automatically based on that conditions). In my implementation you don’t constraint any T type and put whole magic into implementation ThisFunction instance. In your solution context is natural (you are using all fields/methods/implicit from type T because you are inside body of T and it is natural) in my proposal it isn’t.

This are big differences, but not sure what is better.

33

May be there is no need for that constraint, I just assumed by analogy with the java.

I think advantages of my proposal are:

  • it is very simple to implement(compared to other),
  • we can manually create new anonymous classes if we need it

The disadvantage is

  • we cannot use custom \ external factories for the builders, so work with immutable classes is more difficult

So do I.
I prefer to have both option :), but it is too expensive

I will be happy with any implementation :wink:

But in such situations I prefer to choose more simple decision.

34

I prefer to have one. There is no need to have few ways of same thing.

It is very likely that none of them’ll land in scala. Maybe we could mimic similar possibilities using ImplicitFunction. I’ve some ideas, more constrained but still. Problem is that such big feature could be too problematic in maintenance and reasoning especially with other new features like ImplicitFunction.

Martin tries to not introduce features that are not necessary and in long term can harm whole community. I believe that this feature is game changer for scala but again… I belive. We should measure it somehow.

Same thing apply in yours and my proposal. Each comes with it’s own weight. It needs to be measured/estimated.

What needs to be done is to check what useful things can be done with this feature and is nightmare without it. We can take few examples of successful libraries/apis from kotlin that heavily uses receiver functions and try to translate them to dotty to see if it is possible, similar or harder to express it.

35

I had been thinking so for about five years.
But such checkpoints

give me hope

I remember how implicit function type was motivated.

Implicit function types are a surprisingly simple and general way to make coding patterns solving these tasks abstractable, reducing boilerplate code and increasing applicability.

So we just need to wait when DelayedInit will be deleted and there will appear enough boilerplate :slight_smile:

36

You are not quite right since we have

  • implicit conversions from A to B that actually can bring A’s names to the B’s scope;
  • explicit this.type type that can be B in an implicit conversion above.

In current Scala it should look pretty wordy; it can look like this:

trait ThisFunction[A, +R] {
  def apply(a: A)(implicit s: this.type => A = {_ => a}): R
}

Then, having e.g.

trait SomeBuilder {
  def dup: Int
  def str: String
}

we can define (again, very wordy for now)

val f: ThisFunction[SomeBuilder, String] = new ThisFunction[SomeBuilder, String] {
  override def apply(a: SomeBuilder)(implicit s: this.type => SomeBuilder) =
    this.str * this.dup
}

As you can see, SomeBuilder’s names are brought to the scope of defined “function” f (like you wanted). You can check that this really works.

(By the way, eliminating this in the body does not work, but it looks rather like a bug.)

But this is not exactly what was wanted. But, it seems that Dotty’s implicit function types really can solve this problem exactly in the manner that was requested.

(Notice that there is a special ImplicitConvertions type for implicit conversions in Dotty instead of just implicit function usage in Scala 2.)

We could have something like this in Dotty:

trait ThisFunction[A, +R] extends (A => (implicit ImplicitConverter[this.type, A] => R)) {
  override def apply(a: A)(implicit ic: ImplicitConverter[this.type, A] = { _ => a }): R
}

It means that definition of an instance of this type of function does not require a boilerplate with new ThisFunction and override def apply, so it could be done in a manner like this:

val f: ThisFunction[SomeBuilder, String] = { a: SomeBuilder => str * dup }

But at the moment it seems to be not working since at least strange currying rules: override def apply does not seem to override those apply from the A => (implicit ImplicitConverter[this.type, A] => R) type. This could probably be dictated by efficiency of compiled code (since def f(a: A)(b: B) = ... is not the same as def f(a: A) = {b: B => ...}), but it is not obvious that this should influence the language semantics. Thus, the fact that it’s not working now, can be a bug.

Nevertheless, my message is that implicit functions and implicit function type seem to be somehow helpful in your issue. And maybe Dotty will have enough for you to not to require syntactic changes and SIPs.

37

Sorry, I do not understand the practical usage of this example.

You are right.
But how will it look on real examples?
Let’s look for examle on ScalaFX or kjdbc

Of course we can use implicit conversions, but this conversions must be imported. It will cause name clashes:

Sorry, but I have been thinking about it for about a year, I just do not know how to make scalable library (like kdbc or scalafx ) with implicits :frowning:

1 Like
38

Of course it is possible. We can use prefixes for example :))

But use DelayedInit is much more simple.

39

there is one more usecase i found for this idea:

//with such simple code
implicit class OptionsSuperOps[T](d:Option[T]) {
  def ?[E] (with this c:T => E) = { d.map(c) }
  //def >[E] (with this c:T => E) = { d.map(c) }
  def ??[E] (with this c:T => Option[E]) = { d.flatMap(c) }
  //def >>[E] (with this c:T => Option[E]) = { d.flatMap(c) }
}



//instead of this
case class Name(first:String, middle:Option[String], last:Option[String])
val name:Option[Name]

User(
  userId = Some(id),
  firstName = name.flatMap(_.first),
  middleName = name.flatMap(_.middle).flatMap(_.headOption),
  lastName = name.flatMap(_.last),
)

//we could write this 
User(
  userId = Some(id),
  firstName = name ? first,
  middleName = name ?? middle ?? headOption,
  lastName = name ?? last,
)

it could be applied to much wider range of classes than Option (especialy using typeclasses ) but let stay with it for simplicity.
We have kotlin’s null checks for free :).

Maybe this is not big deal but still nice consequence of having some kind of scope injection in language.

As i know this cannot be encoded by implicit functions from dotty.

Gain is rather small. In current scala we can do this:

implicit class OptionsSuperOps[T](d:Option[T]) {
  def ?[E] (with this c:T => E) = { d.map(c) }
  def ??[E] (with this c:T => Option[E]) = { d.flatMap(c) }
}

User(
  userId = Some(id),
  firstName = name ? (_.first),
  middleName = name ?? (_.middle),
  lastName = name ?? (_.last),
)

//this won't compile
User(
  userId = Some(id),
  firstName = name ? _.first,
  middleName = name ?? _.middle,
  lastName = name ?? _.last,
)
// |missing parameter type
//  |
//  |The argument types of an anonymous function must be fully known. (SLS 8.5)
//  |Expected type: ?
//  |Missing type for parameter _$2
40

This does not compile because

  firstName = name ? _.first

means

  firstName = (x => name ? x.first)

i.e. scope of the underscore can be farther than you want. That’s why, bracketless syntax won’t compile ever with the meaning you want.

2 Likes
41

Implicit function can’t overload operator such as = (assign)

WHY DO WE NEED ASSIGN?

Let’s go on this example AJAX DSL.
It’s just assign HttpRequestWrapper.
It’s easy to use and clear.

1 Like
42

It looks more like an anti-pattern to me. Introducing a bunch of unchecked mutating stuff, only for making the call-site look slightly more interesting.
You can do virtually the same thing, but safer, with named and default arguments.

1 Like
43

Sure, assign a variable directly is a bad idea.
However, assign in Kotlin will call setter function instead of assign directly.

44

Builder in Dotty

45

So does Scala if you define a var_= method on a class. This could in theory also work with the automatically generated _= method for vars.
With ThisFunctions, your ajax example could work as follows:

class Ajax() {
  def url_=(url: String): Unit = { /* */ }
  def method_=(url: String): Unit = { /* */ }
  def success(f: String => Unit): Unit = { /* */ }
  def error(f: Exception => Unit): Unit = { /* */ }
}
def ajax(f: this Ajax => Unit): Unit = {
  val aj = new Ajax
  
  f(aj)

  // actually do the ajax evaluation
}

def testHttpOnError(): Unit = {
  val testUrl = "https://www2.baidu.com"

  ajax {
    url = testUrl
    method = "get"
    success { s =>
      println(string)
    }
    error { e =>
      println(e.message)
      Assert.assertTrue("connect timed out" == e.message)
    }
  }
}

with a translation of:

def testHttpOnError(): Unit = {
  val testUrl = "https://www2.baidu.com"

  ajax { _this: Ajax =>
    _this.url_=(testUrl)
    _this.method_=("get")
    _this.success { s =>
      println(string)
    }
    _this.error { e =>
      println(e.message)
      Assert.assertTrue("connect timed out" == e.message)
    }
  }
}
2 Likes
46

I support this proposal! And think it’s a much better solution than implicit function types, for the kinds of things implicit function types have been advertised for.

Here’s a quick list of other things that implicit function types cannot do:

  • ThisFunction can inject common types like String and Int
    • It is common folk wisdom that in Scala, you of course do not declare common types like Int and String to be implicit, but I think it’s worth pointing out that this is only a limitation of implicits, and not of ThisFunctions/ReceiverFunctions.

E.g.

trait Foo { def x: Int; def y: String } 

val f: ThisFunction[Foo, Int] = {
  x + y.length
}
  • ThisFunction can inject multiple members/values of the same type, unlike implicits

E.g.

case class Foo(x: Int, y: Int, z: Int, zz: Int, zzz: Int) {
  def zzzz: Int
}

val f: ThisFunction[Foo, Int] = {
  x + y + z + zz + zzz + zzzz
}
  • ThisFunction introduces capabilities into a scope in a DISCOVERABLE way. Consider the example typically given, with HTML builders:

class Table {
  val rows = new ArrayBuffer[Row]
  def add(r: Row): Unit = rows += r
  override def toString = rows.mkString("Table(", ", ", ")")
}

class Row {
  val cells = new ArrayBuffer[Cell]
  def add(c: Cell): Unit = cells += c
  override def toString = cells.mkString("Row(", ", ", ")")
}

case class Cell(elem: String)

def table(init: implicit Table => Unit) = {
  implicit val t = new Table
  init
  t
}

def row(init: implicit Row => Unit)(implicit t: Table) = {
  implicit val r = new Row
  init
  t.add(r)
}

def cell(str: String)(implicit r: Row) =
  r.add(new Cell(str))

table {
  row {
    cell("top left")
    cell("top right")
  }
  row {
    cell("bottom left")
    cell("bottom right")
  }
}

This example is taken from the Dotty website. However even this example is much better handled by ThisFunction’s. The problem with implicits here is that we are relying on the availability/unavailablility of many implicit values to tell us what is “in scope / available to use” and what isn’t. For example:


cell("top left")
// ^-- only because implicit search fails
//  do I find out this is wrong! `cell` is in scope, but
//  I would have to simulate implicit search to know that it 
//  can't be used here

row {
}
// ^-- same thing here

table {
  cell("top left")
  // ^-- and here
  row {
    cell("top left")
    cell("top right")
  }
  row {
    cell("bottom left")
    cell("bottom right")
  }
}

In Kotlin, or with ThisFunction, it is extremely obvious what is and what is not available.

  • With Implicit function types, even with all this working you still need to know where is the big import you need to perform. You’ll have to search around for where are all these little DSL functions in some package object somewhere and remember to import com.example.my.package.dsl._ whereas with ThisFunction, the exact right members are imported into scope in exactly the right places and only in the right places.

An other benefit: ThisFunction does not imply (and therefor commit to) any kind of ordering to the members injected. Suppose I have the following equivalent code under both ThisFunction and Implicit-Function-Types styles

trait Foo { def x: String; def y: Int }

val f: ThisFunction[Foo, Unit] = {
  print(x + y)
}
f(new Foo("", 1))

val g: (implicit x: String, y: Int) => Unit = {
  print(x + y)
}

g("", 1) // calling g explicitly

Now imagine I change the order around…

trait Foo { def y: Int; def x: String }

val g: (implicit y: Int, x: String) => Unit = {
  print(x + y)
}

^ here, the order of the parameters changing has broken the call to g, but f is perfectly fine.

@scalway thank you for this proposal! I really hope something like this can get into Scala. :tada:

47

After recently merged dotty extension methods pull request It become available to “mimic” something similar to what is expected by the means of specially arranged extension methods.
Actually I’ve already mentioned about this possibility in some previous post but that example is little bit obsolete (uses previous syntax), and probably not enough expressive.

So here is rewritten version of that crafts, which actually does something similar to
builder.scala
but using extension methods for some sort of scope semi-injection (here I call it “semi-injection” because “injected” names becomes available not “as is” but with some %-prefix)

// this is currently compilable/working code - works under current state of dotty master branch

object TestMain {

  def main(args: Array[String]): Unit = { Test.testDsl() }


  object Test {
    import builders.table
    import dslActivators.{dsl => %}

    val data =
      %table {
        %row {
          %cell("A1")
          %cell("B1")
        }
        %row {
          %cell("A2")
          %cell("B2")
        }
      }

    def testDsl(): Unit = {
      println(data)
      assert(s"$data" == "Table(Row(Cell(A1), Cell(B1)), Row(Cell(A2), Cell(B2)))")
    }
  }


  import scala.collection.mutable.ArrayBuffer

  object dslActivators {

    // dsl activating marker
    trait DslActivator

    // any "dummy" name that should serve as extension target (part of scope semi-injection)
    object dsl extends DslActivator

  }

  object builders {
    import models._
    import dslActivators._

    def (erased dslActivator: DslActivator) table (tableInit: implicit TableBuilder => Unit) : Table = {
      val tableBuilder = new TableBuilder
      tableInit(tableBuilder)
      tableBuilder.build
    }

    class TableBuilder {
      val target = new Table()
      def build: Table = target

      def (erased dslActivator: DslActivator) row (rowInit: implicit RowBuilder => Unit) : this.type = {
        val rowBuilder = new RowBuilder
        rowInit(rowBuilder)
        target.add(rowBuilder.build)
        this
      }
    }

    class RowBuilder {
      val target = new Row()
      def build: Row = target

      def (erased dslActivator: DslActivator) cell (payload: String) : this.type = {
        target.add(new Cell(payload))
        this
      }
    }
  }


  object models {
    class Table {
      val rows = new ArrayBuffer[Row]
      def add(r: Row): Unit = rows += r
      override def toString = rows.mkString("Table(", ", ", ")")
    }

    class Row {
      val cells = new ArrayBuffer[Cell]
      def add(c: Cell): Unit = cells += c
      override def toString = cells.mkString("Row(", ", ", ")")
    }

    class Cell(elem: String) {
      override def toString = s"Cell($elem)"
    }
  }

}

Here one can see that

    val data =
      %table {
        %row {
          %cell("A1")
          %cell("B1")
        }
        %row {
          %cell("A2")
          %cell("B2")
        }
      }

Is in fact

    val data =
      %.table {
        %.row {
          %.cell("A1")
          %.cell("B1")
        }
        %.row {
          %.cell("A2")
          %.cell("B2")
        }
      }

In “global” scope imported only builders.table (which is extension method applicable to %), other names from this snippet (%.row , %.cell) are not globally imported, instead they are “inject” into scope in some “fine way” by implicit functions.

So that this approach could be formally treated as some sort of workaround for issue highlighted by @AMatveev in his post related to DelayedInit vs implicit functions types

Continuing writing some other variation of aforementioned snippet one can also write something like

// this is currently compilable/working code - works under current state of dotty master branch

object TestMain1 {

  def main(args: Array[String]): Unit = {
    Test.testDsl()
    Test2.testDsl()
    Test3.testDsl()
  }


  object Test {
    import builders.table
    import dslActivators.dsl

    val data =
      %table {
        %row {
          %cell("A1")
          %cell("B1")
        }
        %row {
          %cell("A2")
          %cell("B2")
        }
      }

    def testDsl(): Unit = {
      println(data)
      assert(s"$data" == "Table(Row(Cell(A1), Cell(B1)), Row(Cell(A2), Cell(B2)))")
    }
  }

  object Test2 {
    import builders.table
    import dslActivators.dsl

    val data =
      dsl.table {
        dsl.row {
          dsl.cell("A1")
          dsl.cell("B1")
        }
        dsl.row {
          dsl.cell("A2")
          dsl.cell("B2")
        }
      }

    def testDsl(): Unit = {
      println(data)
      assert(s"$data" == "Table(Row(Cell(A1), Cell(B1)), Row(Cell(A2), Cell(B2)))")
    }
  }

  object Test3 extends dslActivators.DslActivator {
    import builders.table

    def testDsl(): Unit = {
      val data =
        Test3.this.table {
          Test3.this.row {
            Test3.this.cell("A1")
            Test3.this.cell("B1")
          }
          Test3.this.row {
            Test3.this.cell("A2")
            Test3.this.cell("B2")
          }
        }
      println(data)
      assert(s"$data" == "Table(Row(Cell(A1), Cell(B1)), Row(Cell(A2), Cell(B2)))")
    }
  }


  import scala.collection.mutable.ArrayBuffer

  object dslActivators {

    // dsl activating marker
    trait DslActivator

    // any "dummy" name that should serve as extension target (part of scope semi-injection)
    object dsl extends DslActivator

  }

  object builders {
    import models._
    import dslActivators._

    def (erased dslActivator: DslActivator) table (tableInit: implicit TableBuilder => Unit) : Table = {
      val tableBuilder = new TableBuilder
      tableInit(tableBuilder)
      tableBuilder.build
    }

    class TableBuilder {
      val target = new Table()
      def build: Table = target

      def (erased dslActivator: DslActivator) row (rowInit: implicit RowBuilder => Unit) : this.type = {
        val rowBuilder = new RowBuilder
        rowInit(rowBuilder)
        target.add(rowBuilder.build)
        this
      }
    }

    class RowBuilder {
      val target = new Row()
      def build: Row = target

      def (erased dslActivator: DslActivator) cell (payload: String) : this.type = {
        target.add(new Cell(payload))
        this
      }
    }
  }


  object models {
    class Table {
      val rows = new ArrayBuffer[Row]
      def add(r: Row): Unit = rows += r
      override def toString = rows.mkString("Table(", ", ", ")")
    }

    class Row {
      val cells = new ArrayBuffer[Cell]
      def add(c: Cell): Unit = cells += c
      override def toString = cells.mkString("Row(", ", ", ")")
    }

    class Cell(elem: String) {
      override def toString = s"Cell($elem)"
    }
  }

}

Here most remarkable looks that snippet

    def testDsl(): Unit = {
      val data =
        Test3.this.table {
          Test3.this.row {
            Test3.this.cell("A1")
            Test3.this.cell("B1")
          }
          Test3.this.row {
            Test3.this.cell("A2")
            Test3.this.cell("B2")
          }
        }
      println(data)
      assert(s"$data" == "Table(Row(Cell(A1), Cell(B1)), Row(Cell(A2), Cell(B2)))")
    }

If that issue related to this.extMethod() <==> extMethod() natural behaviour
would be fixed, then that snippet would become look like “true scope injection”

// this is not valid code in current state of dotty master branch since `this.extMethods()` != `extMethods()` for now

  object Test3 extends dslActivators.DslActivator {
    import builders.table

    def testDsl(): Unit = {
      val data =
        table {
          row {
            cell("A1")
            cell("B1")
          }
          row {
            cell("A2")
            cell("B2")
          }
        }
      println(data)
      assert(s"$data" == "Table(Row(Cell(A1), Cell(B1)), Row(Cell(A2), Cell(B2)))")
    }
  }

But any way it could not be treated as final solution, rather as some tricky workaround.
Something similar to Kotlin’s extension methods / extension functions / extension lambadas
may fit for this problem much better.

2 Likes
48

The link to

is this Extension Method

49

Thanks, yes, exactly that pull request (link fixed)