PRE SIP: ThisFunction | scope injection (similar to kotlin receiver function)

Nevertheless it is not the most comfortable thing to write
import some.word.that.i.always.forget._

The receiver function can help in such situation.

Of course the good programmer know by heart all that he needs. But I it is annoying situation for me.
There are some decisions:

But receiver function looks more comfortable for optional libraries

  doSomeAspect{
     ....
  }

instead of

    doSomeAspect{ it => 
       import implicit it 
   }
1 Like

Would scala-records solve your request for named tuples, if they were still published for Scala 2.12-2.13 (or if you’re using 2.11) ?

I think we’re all missing that this can already be enriched by scoping in Dotty, via implicit functions:

object StringScope {
  def (s: String) capWords = s.split(" ").map(_.capitalize).mkString(" ")
}

def stringScope[A](f: (given StringScope.type) => A): A = {
  f(given StringScope)
}

object TopLevelScope {
  def (`this`: Any) capWords(s: String) = StringScope.capWords(s)
}

def topScope[A](f: (given TopLevelScope.type) => A): A = {
  f(given TopLevelScope)
}

object App extends App {
  stringScope (
    println("hello world".capWords)
  )
  topScope {
    // this is ok
    println(this.capWords("hello world again"))
    // but not this...
    // println(capWords("hello world again"))
  }
}

The only restriction is that you HAVE to write this as a prefix to call an extension method, you can’t call it as a top-level name. I think this restriction is arbitrary, all scopes in Scala 2 so far have an implicit this, if extensions methods could apply to this when calling an unqualified method this would in effect allow adding new “top-level” methods by enriching (self: Any)

IIUC: There would be dynamic overhead which would be too significant for us.

Highly unlikely. Method calls on the JVM are themselves implemented by hashtable lookups, I.e. hashmap lookup should have the same cost as calling a method - the difference you see is likely purely artifacts of your profiler

May be.

import java.util

object RowPerformanceTest {
  val dsSize = 100000
  val rowSize = 20
  val repeatCnt = 1000
  var startTime:Long = _
  var endTime: Long = _
  val columnMap: util.HashMap[String,Int] = {
    val result = new util.HashMap[String,Int]
    var j = 0
    while(j<rowSize){
      result.put(s"column_$j",j)
      j=j+1
    }
    result
  }
  val dummyArray:Array[Int] = {
    val result = new Array[Int] (rowSize)
    columnMap.forEach{ (n,i) =>
      result(i)=i
    }
    result
  }
  val columnArray:Array[String] = {
    val result = new Array[String] (rowSize)
    columnMap.forEach{ (n,i) =>
      result(i)=n
    }
    result
  }
  val dataSet:Array[Array[Long]] = {
    val result = new Array[Array[Long]](dsSize)
    var i = 0
    while(i<dsSize){
      val array = new Array[Long](rowSize)
      result(i) = array
      var j = 0
      while(j<columnArray.length){
        array(j)=1
        j=j+1
      }
      i=i+1
    }
    result
  }
  def begin(): Unit ={
    startTime = System.nanoTime()
  }
  def end(): Unit = {
    endTime = System.nanoTime()
  }
  def main(args: Array[String]): Unit = {
    def testByKey(): Unit ={
      var i = 0
      var sum = 0L
      while(i<dataSet.length){
        var j = 0
        val row = dataSet(i)
        while(j<columnArray.length){
          sum = sum + row(columnMap.get(columnArray(j)))
          j=j+1
        }
        i=i+1
      }
      //println(s"testByKeySum:$sum")
    }
    def testByIndex(): Unit = {
      var i = 0
      var sum = 0L
      while(i<dataSet.length){
        val row = dataSet(i)
        var j = 0
        while(j<row.length){
          sum = sum + row(dummyArray(j))
          j=j+1
        }
        i=i+1
      }
      //println(s"testByIndex:$sum")
    }
    begin()
    testByKey()
    end()
    begin()
    for(i<-1 to repeatCnt){
      testByKey()
    }
    end()
    println("by key")
    val byKeyTotolTime =  BigDecimal(endTime-startTime)/1000000000
    println(s"  total time:$byKeyTotolTime")
    begin()
    testByIndex()
    end()
    begin()
    for(i<-1 to repeatCnt) {
      testByIndex()
    }
    end()
    println("by index")
    val byIndexTotolTime =  BigDecimal(endTime-startTime)/1000000000
    println(s"  total time:$byIndexTotolTime")
    println("ratio")
    println(s"  time:${byKeyTotolTime/byIndexTotolTime}")
  }
}

Output

by key
  total time:12.119882852
by index
  total time:1.641546377
ratio
  time:7.383210746777457631341718711612130

It is 7 times slower.

May be I have mistaken somewhere.
May be it is very unusual case.

When I see value.method(arguments) I expect that either method is defined on type of value or there is implicit conversion in scope that provides that method. It doesn’t seem helpful to add receiver methods to that list. I would need to search in three kinds of places instead of two.

Also, Rust compiler has no problems suggesting that you forgot to add:

import some.word.that.i.always.forget._

to enable certain extension methods. Look here:

Note that Rust has keyword use instead of import.

1 Like

I think you got it backwards. Actually:

  • marking something as implicit doesn’t add any named members to any scope
  • this.whatever(args) is no different than notThis.whatever(args) from implicit conversions / extension methods point of view

Therefore going from this.extensionMethod(args) to just extensionMethod(args) is a big leap and I’m against it (in this form at least).

2 Likes

No, it’s more consistent. whatever(a,b) desugars to this.whatever(a, b) if this.whatever exists, always, UNLESS it’s an extension method. This is an inconsistency with usual methods and this is the ONLY place where member methods are treated differently from extension methods. Applying extension methods here would remove an exception from the language and provide scope extensions with one stone.

Actually not. whatever is searched:

  • first in parameters of methods and functions we’re in
  • then it’s searched in this
  • then in outer classes
  • also it’s searched in imported members

If something is both imported explicitly and available from this or outer class then scalac fails with ambiguous error (unless the member imported is the same as the member without import). Specifying e.g. this.whatever(args) helps resolving that problem.

So generally in expression subject.member(args) the rules for finding subject are different from rules for finding member (and I haven’t yet touched extension methods / implicit conversions here).

2 Likes

Odersky has written:
-Third party serialization packages are typical examples of orphan instances. They require import implied.

To say the truth, I don’t know good decision for orphan tasks. It is not rare and currently we have writen our own base types for all primitives and excluding orphan is one of the Major aim. I can not say that receiver is more worse for such case.

Requiring import is not excluding. Orphan instances require import for sanity. Otherwise:

  • there would be compilation performance penalty not only during error reporting with import suggestion, but also during normal compilation passes (automatic orphan imports require scanning the whole classpath)
  • it would be very easy to have ambiguities. Let’s say you had only one Monoid[Int] on classpath and were happy with automatically imported orphan instances. Then you add some library to you app and that library brings another orphan Monoid[Int]. Suddenly all of your code that relied on automatically imported orphan Monoid[Int] instance breaks because of ambiguity.

Going back to receiver methods:

My stance is that Scala should encourage pure code over side-effecting code. Receiver functions are practically fully mutability oriented, i.e. all examples of receiver functions usage revolved around mutable builders or some other ugly imperative Javaism like that (and I haven’t switched from Java to Scala only to see more ugly imperative Javaisms).

Let’s also quote Kotlin docs https://kotlinlang.org/docs/reference/lambdas.html#function-literals-with-receiver to see how they work:

Therefore the syntax that I’ve proposed before:

receiver.function { this =>
  ... here we have new 'this'
}

closely matches what Kotlin’s receiver functions do.

This syntax introduces small penalty for receiver functions, makes them perfectly comprehensible and also allows users to opt-in or opt-out whenerver they want at use-site (receiver functions from Kotlin don’t have that flexibility).

Scala has penalties for mutability oriented code in other places, e.g.:

  • case class primary constructor parameters are vals by default - you have to add explicit var if you want mutability
  • methods and function parameters (and also intermediate values in for-comprehensions) are vals and you can’t change them at all - you need to copy them to some other vars explicitly
  • default collections available without prefix are (almost?) all immutable ones - you need to explicitly import the mutable ones
  • you can’t import from a var but you can import from a val
  • there’s no continue keyword, return works often by throwin exceptions (so it breaks in async code then), break is absent and you need to use scala.util.control.Breaks (which I never seen used)
  • etc there are plenty of such examples
  • therefore if you’re after mutability oriented code then you’ll want to avoid Scala anyway and Scala wants to avoid you :slight_smile: Mutability restrictions in Scala are not as tough as in Haskell (which outright rejects all mutable code no wrapped in IO type), but still Haskell is a strong inspiration (see scalaz, cats, etc)
4 Likes

You’re just saying that they’re different, which is known, and that’s exactly the inconsinstency I’m talking about - do they have to be different wrt extension methods?

That was reported yet Dec 10, 2018 on github - issue#5591 , along with issue#5588

1 Like

That’s good, esp Kotlin example in https://github.com/lampepfl/dotty/issues/5591 shows that Kotlin does resolve extension methods of this unqualified. So at least for designers of Kotlin it made sense that this.method and method are the exact same thing without weird exceptions…

You’re talking about inconsistency between two very different mechanisms. It doesn’t need a lot of effort to find multiple (potential?) inconsistences, e.g.:

  • there’s case object and case class, but no case trait - inconsistency
  • you can do import stableIdentifier._ but can’t do import unstableIdentifier._ - inconsistency
  • you can have class parameters, soon have trait parameters, but no object parameters in plan - inconsistency (this is actually absurd one, but still there’s some kind of inconsistency here)
  • you can apply var to constructor parameters but not to method parameters - inconsistency
  • you can import from any stable identifier but when you save stable identifier into a new variable it may not be a package (i.e. you can do val newSomething = stableIdentifier for any stable identifier that is not a package) - inconsistency
  • you need value.type to get type of non-literal, but for literals you don’t need that .type suffix - inconsistency
  • etc

Shall we solve all of them? Maybe, maybe not, we need to consider the consequences, the need for them, how they fit in the language (are they consistent with spirit of Scala?), etc

Sometimes this.member(args) is the same as member(args) but not always. Sometimes one form compiles and other don’t. Sometimes both compile, but refer to something different. That’s because there’s some overlap between them, but they are in fact governed by completely different rules.

A big counterargument for extension methods on this is that extension methods are mostly useful for achieving ad-hoc polymorphism for types you don’t control. E.g. you can enrich java.util.String only with extension methods (Dotty ones or Scala 2 ones, whatever). You can’t mix in additional traits to java.util.String as you don’t have control over it, i.e. you can’t edit it and add directly the methods you want. But (in current version of Scala) if you write code that uses this then this means you have full control over the class of which this is the instance. Therefore it’s much more natural to just add extra traits to that class instead of going through the contortions of adding extension methods.

Instead of this (extension methods on class you control):

  class X(val v: Int) {
    def hello(): Unit = {
      this.extensionMethod()
      // extensionMethod() // alternative potential syntax
    }
  }

  implicit class RichX(x: X) {
    def extensionMethod(): Unit = println(x.v + 5)
  }

you write this (ordinary OOP mixins):

  class Y(val v: Int) extends Mixin {
    def hello(): Unit = {
      methodFromMixin()
    }
  }

  trait Mixin { this: Y =>
    def methodFromMixin(): Unit = println(v + 5)
  }

In order for extension methods on this to make more sense, this would have to be of foreign type you don’t control, e.g.

library.method { this => // rebinding 'this' to an instance of foreign type
  // now this makes sense as we can't mixin anything to 'this'
  this.extensionMethod()
  // or the more concise but confusing syntax
  extensionMethod()
}

But how often such thing would happen in idiomatic Scala code? I think very rarely (but can be wrong here).

Kotlin is heavily oriented toward integrating with Java libraries and frameworks thus Kotlin programmers are often forced to deal with deficient Java APIs and ad-hoc extending them makes a lot of sense. OTOH, Scala APIs are usually pretty rich out of the box.

1 Like

Now that gets interesting with the recent proposal for self types syntax using val this: T. If you adopt that and the above suggestion to allow someType.baz { this =>, what you get is that the whole “auto prefix names with this“ is generalized so that rather than this meaning the current class scope, it means a variable in scope named this, which happens to default to the class scope.

(Note: I am not personally in favor of these, but if you’re going to look into them I think this would be more elegant.)

Interesting idea. I’m not sure how we should understand OuterClass.this in your interpretation, though I guess it could keep its usual interpretation as a way of referring to some potentially shadowed this that happen to be associated with a class name.

How are Java frameworks related to this? You can’t get a “foreign this” in Java either, there’s no way an extension method on this could help interface with a Java framework – it only makes sense with Kotlin’s own rebinding of this in scopes.

Thing is, implicit function types, together with the already implemented lexical scoping for implicits already implement scope enhancement – but bizarrely, only for methods – the only result of this inconsistency is that to introduce new top-level functions/values people will just add extension methods for constants or objects that are always visible, e.g. 't or 0. Instead of underscore.js, we’ll have zero.scala, let me demonstrate:

trait Fixture[A] extends Conversion[0, A]

trait TestFramework[A] {
  def (testName: String) in (test: given (Fixture[A]) => Unit): Unit = ???
}

This is enough to solve the problem of importing fixture members in test – all members of fixture are now accessible from 0. prefix inside the test:

trait Greeter {
  def greet(name: String): String = s"Hello $name"
}

case class MyFixture(name: String, greeter: Greeter)

object MyTest extends TestFramework[MyFixture] with App  {
  "say hello" in {
    assert(0.greeter.greet(0.name) == s"Hello ${0.name}")
  }
}

There are always literals or objects around to attach names to for scope injection and I expect that at least some DSLs, like Akka’s GraphDSL will migrate to implicit functions in Scala 3. However, the above is a hack, names “injected” onto literals like this are second-class, e.g. you can’t import from them and you can’t inject type names into scope this way, the fact that scope injection can be “almost done” using patterns just means that it should be provided in full power by the language instead, otherwise we’ll be stuck with zero.scala – I will 100% use this approach in dotty, because typing in { ctx => import ctx._ ; ... } in every test tires me.

Alternately, you could go the route that Scalatest has and implement your test DSL with intermediate objects and by-name parameters.

Aside from avoiding the scope injection issue, it’s much easier for users to extend your DSL if there are types they can add extension methods to.

To concretize this, your example would look like this in Scalatest:

trait Greeter {
  def greet(name: String): String = s"Hello $name"
}
class MyFixture {
  val name: String = ???
  val greeter: Greeter = ???
}

class MyTest extends WordSpec  {
  "say hello" in new MyFixture {
    assert(greeter.greet(name) == s"Hello $name")
  }
}