Would scala-records solve your request for named tuples, if they were still published for Scala 2.12-2.13 (or if you’re using 2.11) ?
I think we’re all missing that this can already be enriched by scoping in Dotty, via implicit functions:
object StringScope {
def (s: String) capWords = s.split(" ").map(_.capitalize).mkString(" ")
}
def stringScope[A](f: (given StringScope.type) => A): A = {
f(given StringScope)
}
object TopLevelScope {
def (`this`: Any) capWords(s: String) = StringScope.capWords(s)
}
def topScope[A](f: (given TopLevelScope.type) => A): A = {
f(given TopLevelScope)
}
object App extends App {
stringScope (
println("hello world".capWords)
)
topScope {
// this is ok
println(this.capWords("hello world again"))
// but not this...
// println(capWords("hello world again"))
}
}
The only restriction is that you HAVE to write this as a prefix to call an extension method, you can’t call it as a top-level name. I think this restriction is arbitrary, all scopes in Scala 2 so far have an implicit this, if extensions methods could apply to this when calling an unqualified method this would in effect allow adding new “top-level” methods by enriching (self: Any)
IIUC: There would be dynamic overhead which would be too significant for us.
Highly unlikely. Method calls on the JVM are themselves implemented by hashtable lookups, I.e. hashmap lookup should have the same cost as calling a method - the difference you see is likely purely artifacts of your profiler
May be.
import java.util
object RowPerformanceTest {
val dsSize = 100000
val rowSize = 20
val repeatCnt = 1000
var startTime:Long = _
var endTime: Long = _
val columnMap: util.HashMap[String,Int] = {
val result = new util.HashMap[String,Int]
var j = 0
while(j<rowSize){
result.put(s"column_$j",j)
j=j+1
}
result
}
val dummyArray:Array[Int] = {
val result = new Array[Int] (rowSize)
columnMap.forEach{ (n,i) =>
result(i)=i
}
result
}
val columnArray:Array[String] = {
val result = new Array[String] (rowSize)
columnMap.forEach{ (n,i) =>
result(i)=n
}
result
}
val dataSet:Array[Array[Long]] = {
val result = new Array[Array[Long]](dsSize)
var i = 0
while(i<dsSize){
val array = new Array[Long](rowSize)
result(i) = array
var j = 0
while(j<columnArray.length){
array(j)=1
j=j+1
}
i=i+1
}
result
}
def begin(): Unit ={
startTime = System.nanoTime()
}
def end(): Unit = {
endTime = System.nanoTime()
}
def main(args: Array[String]): Unit = {
def testByKey(): Unit ={
var i = 0
var sum = 0L
while(i<dataSet.length){
var j = 0
val row = dataSet(i)
while(j<columnArray.length){
sum = sum + row(columnMap.get(columnArray(j)))
j=j+1
}
i=i+1
}
//println(s"testByKeySum:$sum")
}
def testByIndex(): Unit = {
var i = 0
var sum = 0L
while(i<dataSet.length){
val row = dataSet(i)
var j = 0
while(j<row.length){
sum = sum + row(dummyArray(j))
j=j+1
}
i=i+1
}
//println(s"testByIndex:$sum")
}
begin()
testByKey()
end()
begin()
for(i<-1 to repeatCnt){
testByKey()
}
end()
println("by key")
val byKeyTotolTime = BigDecimal(endTime-startTime)/1000000000
println(s" total time:$byKeyTotolTime")
begin()
testByIndex()
end()
begin()
for(i<-1 to repeatCnt) {
testByIndex()
}
end()
println("by index")
val byIndexTotolTime = BigDecimal(endTime-startTime)/1000000000
println(s" total time:$byIndexTotolTime")
println("ratio")
println(s" time:${byKeyTotolTime/byIndexTotolTime}")
}
}
Output
by key
total time:12.119882852
by index
total time:1.641546377
ratio
time:7.383210746777457631341718711612130
It is 7 times slower.
May be I have mistaken somewhere.
May be it is very unusual case.
When I see value.method(arguments) I expect that either method is defined on type of value or there is implicit conversion in scope that provides that method. It doesn’t seem helpful to add receiver methods to that list. I would need to search in three kinds of places instead of two.
Also, Rust compiler has no problems suggesting that you forgot to add:
import some.word.that.i.always.forget._
to enable certain extension methods. Look here:
Note that Rust has keyword use instead of import.
1 Like
I think you got it backwards. Actually:
- marking something as
implicitdoesn’t add any named members to any scope -
this.whatever(args)is no different thannotThis.whatever(args)from implicit conversions / extension methods point of view
Therefore going from this.extensionMethod(args) to just extensionMethod(args) is a big leap and I’m against it (in this form at least).
2 Likes
No, it’s more consistent. whatever(a,b) desugars to this.whatever(a, b) if this.whatever exists, always, UNLESS it’s an extension method. This is an inconsistency with usual methods and this is the ONLY place where member methods are treated differently from extension methods. Applying extension methods here would remove an exception from the language and provide scope extensions with one stone.
Actually not. whatever is searched:
- first in parameters of methods and functions we’re in
- then it’s searched in
this - then in outer classes
- also it’s searched in imported members
If something is both imported explicitly and available from this or outer class then scalac fails with ambiguous error (unless the member imported is the same as the member without import). Specifying e.g. this.whatever(args) helps resolving that problem.
So generally in expression subject.member(args) the rules for finding subject are different from rules for finding member (and I haven’t yet touched extension methods / implicit conversions here).
2 Likes
Odersky has written:
-Third party serialization packages are typical examples of orphan instances. They require import implied.
To say the truth, I don’t know good decision for orphan tasks. It is not rare and currently we have writen our own base types for all primitives and excluding orphan is one of the Major aim. I can not say that receiver is more worse for such case.
Requiring import is not excluding. Orphan instances require import for sanity. Otherwise:
- there would be compilation performance penalty not only during error reporting with import suggestion, but also during normal compilation passes (automatic orphan imports require scanning the whole classpath)
- it would be very easy to have ambiguities. Let’s say you had only one
Monoid[Int]on classpath and were happy with automatically imported orphan instances. Then you add some library to you app and that library brings another orphanMonoid[Int]. Suddenly all of your code that relied on automatically imported orphanMonoid[Int]instance breaks because of ambiguity.
Going back to receiver methods:
My stance is that Scala should encourage pure code over side-effecting code. Receiver functions are practically fully mutability oriented, i.e. all examples of receiver functions usage revolved around mutable builders or some other ugly imperative Javaism like that (and I haven’t switched from Java to Scala only to see more ugly imperative Javaisms).
Let’s also quote Kotlin docs https://kotlinlang.org/docs/reference/lambdas.html#function-literals-with-receiver to see how they work:
Therefore the syntax that I’ve proposed before:
receiver.function { this =>
... here we have new 'this'
}
closely matches what Kotlin’s receiver functions do.
This syntax introduces small penalty for receiver functions, makes them perfectly comprehensible and also allows users to opt-in or opt-out whenerver they want at use-site (receiver functions from Kotlin don’t have that flexibility).
Scala has penalties for mutability oriented code in other places, e.g.:
- case class primary constructor parameters are
vals by default - you have to add explicitvarif you want mutability - methods and function parameters (and also intermediate values in for-comprehensions) are
vals and you can’t change them at all - you need to copy them to some othervars explicitly - default collections available without prefix are (almost?) all immutable ones - you need to explicitly import the mutable ones
- you can’t import from a
varbut you can import from aval - there’s no
continuekeyword,returnworks often by throwin exceptions (so it breaks in async code then),breakis absent and you need to usescala.util.control.Breaks(which I never seen used) - etc there are plenty of such examples
- therefore if you’re after mutability oriented code then you’ll want to avoid Scala anyway and Scala wants to avoid you
Mutability restrictions in Scala are not as tough as in Haskell (which outright rejects all mutable code no wrapped in IO type), but still Haskell is a strong inspiration (see scalaz, cats, etc)
4 Likes
You’re just saying that they’re different, which is known, and that’s exactly the inconsinstency I’m talking about - do they have to be different wrt extension methods?
That’s good, esp Kotlin example in https://github.com/lampepfl/dotty/issues/5591 shows that Kotlin does resolve extension methods of this unqualified. So at least for designers of Kotlin it made sense that this.method and method are the exact same thing without weird exceptions…
You’re talking about inconsistency between two very different mechanisms. It doesn’t need a lot of effort to find multiple (potential?) inconsistences, e.g.:
- there’s case object and case class, but no case trait - inconsistency
- you can do
import stableIdentifier._but can’t doimport unstableIdentifier._- inconsistency - you can have class parameters, soon have trait parameters, but no object parameters in plan - inconsistency (this is actually absurd one, but still there’s some kind of inconsistency here)
- you can apply
varto constructor parameters but not to method parameters - inconsistency - you can import from any stable identifier but when you save stable identifier into a new variable it may not be a package (i.e. you can do
val newSomething = stableIdentifierfor any stable identifier that is not a package) - inconsistency - you need
value.typeto get type of non-literal, but for literals you don’t need that.typesuffix - inconsistency - etc
Shall we solve all of them? Maybe, maybe not, we need to consider the consequences, the need for them, how they fit in the language (are they consistent with spirit of Scala?), etc
Sometimes this.member(args) is the same as member(args) but not always. Sometimes one form compiles and other don’t. Sometimes both compile, but refer to something different. That’s because there’s some overlap between them, but they are in fact governed by completely different rules.
A big counterargument for extension methods on this is that extension methods are mostly useful for achieving ad-hoc polymorphism for types you don’t control. E.g. you can enrich java.util.String only with extension methods (Dotty ones or Scala 2 ones, whatever). You can’t mix in additional traits to java.util.String as you don’t have control over it, i.e. you can’t edit it and add directly the methods you want. But (in current version of Scala) if you write code that uses this then this means you have full control over the class of which this is the instance. Therefore it’s much more natural to just add extra traits to that class instead of going through the contortions of adding extension methods.
Instead of this (extension methods on class you control):
class X(val v: Int) {
def hello(): Unit = {
this.extensionMethod()
// extensionMethod() // alternative potential syntax
}
}
implicit class RichX(x: X) {
def extensionMethod(): Unit = println(x.v + 5)
}
you write this (ordinary OOP mixins):
class Y(val v: Int) extends Mixin {
def hello(): Unit = {
methodFromMixin()
}
}
trait Mixin { this: Y =>
def methodFromMixin(): Unit = println(v + 5)
}
In order for extension methods on this to make more sense, this would have to be of foreign type you don’t control, e.g.
library.method { this => // rebinding 'this' to an instance of foreign type
// now this makes sense as we can't mixin anything to 'this'
this.extensionMethod()
// or the more concise but confusing syntax
extensionMethod()
}
But how often such thing would happen in idiomatic Scala code? I think very rarely (but can be wrong here).
Kotlin is heavily oriented toward integrating with Java libraries and frameworks thus Kotlin programmers are often forced to deal with deficient Java APIs and ad-hoc extending them makes a lot of sense. OTOH, Scala APIs are usually pretty rich out of the box.
1 Like
Now that gets interesting with the recent proposal for self types syntax using val this: T. If you adopt that and the above suggestion to allow someType.baz { this =>, what you get is that the whole “auto prefix names with this“ is generalized so that rather than this meaning the current class scope, it means a variable in scope named this, which happens to default to the class scope.
(Note: I am not personally in favor of these, but if you’re going to look into them I think this would be more elegant.)
Interesting idea. I’m not sure how we should understand OuterClass.this in your interpretation, though I guess it could keep its usual interpretation as a way of referring to some potentially shadowed this that happen to be associated with a class name.
How are Java frameworks related to this? You can’t get a “foreign this” in Java either, there’s no way an extension method on this could help interface with a Java framework – it only makes sense with Kotlin’s own rebinding of this in scopes.
Thing is, implicit function types, together with the already implemented lexical scoping for implicits already implement scope enhancement – but bizarrely, only for methods – the only result of this inconsistency is that to introduce new top-level functions/values people will just add extension methods for constants or objects that are always visible, e.g. 't or 0. Instead of underscore.js, we’ll have zero.scala, let me demonstrate:
trait Fixture[A] extends Conversion[0, A]
trait TestFramework[A] {
def (testName: String) in (test: given (Fixture[A]) => Unit): Unit = ???
}
This is enough to solve the problem of importing fixture members in test – all members of fixture are now accessible from 0. prefix inside the test:
trait Greeter {
def greet(name: String): String = s"Hello $name"
}
case class MyFixture(name: String, greeter: Greeter)
object MyTest extends TestFramework[MyFixture] with App {
"say hello" in {
assert(0.greeter.greet(0.name) == s"Hello ${0.name}")
}
}
There are always literals or objects around to attach names to for scope injection and I expect that at least some DSLs, like Akka’s GraphDSL will migrate to implicit functions in Scala 3. However, the above is a hack, names “injected” onto literals like this are second-class, e.g. you can’t import from them and you can’t inject type names into scope this way, the fact that scope injection can be “almost done” using patterns just means that it should be provided in full power by the language instead, otherwise we’ll be stuck with zero.scala – I will 100% use this approach in dotty, because typing in { ctx => import ctx._ ; ... } in every test tires me.
Alternately, you could go the route that Scalatest has and implement your test DSL with intermediate objects and by-name parameters.
Aside from avoiding the scope injection issue, it’s much easier for users to extend your DSL if there are types they can add extension methods to.
To concretize this, your example would look like this in Scalatest:
trait Greeter {
def greet(name: String): String = s"Hello $name"
}
class MyFixture {
val name: String = ???
val greeter: Greeter = ???
}
class MyTest extends WordSpec {
"say hello" in new MyFixture {
assert(greeter.greet(name) == s"Hello $name")
}
}
@morgen-peschke:
Here’s a quite neat overview of testing styles in ScalaTest: https://blog.softwaremill.com/fixtures-in-scala-three-simple-ways-to-reduce-your-test-boilerplate-5eb60ffe883
The way you presented has the advantage of easy composability new Fixture with ExtraData1 with ExtraData2 but for now they it suffers from lack of direct support for parametrization (traits do not have parameters yet, but will have in Scala 3) and you don’t get tear down (you would have to wrap the new Fixture with Whatever { <test code> } with some extra function to get e.g. withTeardown(new Fixture with Whatever { <test code > }).
@kai:
Extension methods are useful when the type interface is deficient. I was presuming that this would be more useful for Java types as Java e.g. for a long time lacked multiple inheritance of behaviour (Java 8 brought that) so making rich APIs was tedious.
As I’ve written before - marking something implicit doesn’t add any named member to any scope. That’s how it works in both Scala 2 and current Dotty / Scala 3. Look here if you want to read about scopes: https://www.scala-lang.org/files/archive/spec/2.13/02-identifiers-names-and-scopes.html
OTOH marking something implicit is sometimes required to make something else compile. If I have def myMethod[A: MyContext] ... then when invoking it I need to have implicit MyContext[A] instance in scope. That’s how it worked since the beginning of implicit contexts.
- I have already proposed:
function { this =>
<some code using members of new this directly>
}
It’s explicit, comprehensible and use-site configurable.
- I don’t see much improvement in:
"say hello" in {
assert(0.greeter.greet(0.name) == s"Hello ${0.name}")
}
over
"say hello" in { f =>
assert(f.greeter.greet(f.name) == s"Hello ${f.name}")
}
Using 0.something instead of f.something wouldn’t pay off when using IDE, as IDE would first suggest methods defined directly on Int and only after that you would see extension methods.
- Let’s not mix things together. Rebinding
thisis a completely different thing than unqualified extension methods / implicit views.
Here’s how you search for binding in current scope:
Bindings of different kinds have a precedence defined on them:
- Definitions and declarations that are local, inherited, or made available by a package clause and also defined in the same compilation unit as the reference to them, have highest precedence.
- Explicit imports have next highest precedence.
- Wildcard imports have next highest precedence.
- Definitions made available by a package clause, but not also defined in the same compilation unit as the reference to them, as well as imports which are supplied by the compiler but not explicitly written in source code, have lowest precedence.
Here’s how you search for implicit views:
- In a selection e.m with e of type T, if the selector m does not denote an accessible member of T. In this case, a view v is searched which is applicable to e and whose result contains a member named m. The search proceeds as in the case of implicit parameters, where the implicit scope is the one of T. If such a view is found, the selection e.m is converted to
v(e).m.- In a selection e.m(args) with e of type T, if the selector m denotes some member(s) of T, but none of these members is applicable to the arguments args. In this case a view v is searched which is applicable to e and whose result contains a method m which is applicable to args. The search proceeds as in the case of implicit parameters, where the implicit scope is the one of T. If such a view is found, the selection e.m is converted to
v(e).m(args).
How do you want to merge searching for implicit views on this to searching for binding in current scope? There has to be some precedence rules. Currently for e.m the precedence rules state that implicit views are tried last. That is good because searching for implicits is costly, so it should be done last. If we carry that rule to searching for binding in scope, then all packages, class members (including class members from outer classes), local variables and methods, functions and methods parameters, etc will have precedence over extension methods / implicit views on this.
Implicits are quite heavy, so we should rather strive to find a way to reduce their compilation performance impact rather than going to enable implicits everywhere. I don’t use scalaz / cats on regular basis but I remembers that when I was adding import scalaz._ ; import scalaz.Scalaz._ to classes a few years ago that slowed down IntelliJ IDE and Scala compiler considerably. Simply being more selective in implicits made performance much better, e.g. import scalaz.std.syntax.options._ (or something like that). Shapeless library is another proof that heavy use of implicits drag compilation speed down. People are inventing extra macros that use few implicits (if any) to implement things that could be done without that extra macros in Shapeless, but with much higher complation performance cost.