By-Type imports : are they necessary?

[som-snytt]

I agree that there are only two impossible tasks: naming things and not naming them.

An import is clearly about naming: import p.C does not distinguish between the type C and its companion term. The C is just a name.

So it is jarring that something other than names are subject to import. Types have more structure than a simple identifier. Types are subsequent to names: they are bestowed on named things.

It might be nice to have a different name or syntax for importing by-type. No doubt the current scheme is for concision: import p.{given C, C} shows me what I get from p. Moreover, import syntax is more like braceless package syntax, introducing a context that determines lookups for names and implicits. It would be more regular if we could import p.(given _) { code }. I wouldn’t mind if given were optional in parens: import p.(_). Things we say parenthetically are implicit.

My point is that import was never just a convenient shorthand (as I thought of it in Java): it is as structuring as various definitions which are named, even though imports themselves are nameless. This is more obvious in Scala 3, where implicit resolution relies more on nesting than naming.

I love anonymous givens, even though I always think “Robin Givens”. That is just an unfortunate consequence of naming.

Thinking differently about import might have been easier with contrasting syntax. An early proposal was import implicit, I don’t know if given import was ever considered. But I think adjusting how we think about imports is probably overdue.

I don’t know about IDEs, but the Scalafix folks complain that imports are not merely syntactic shorthand. It makes rewriting them non-trivial. Oh, I may have meant Scalafmt. Naming is so hard!

[sake92]

I’d also like import given .., similar to Java’s import static!
That way we could just Ctrl+F “import given” and see where all those givens come from.
With current scheme that’s a bit harder.

[soronpo]

@som-snytt

Thinking differently about import might have been easier with contrasting syntax. An early proposal was import implicit , I don’t know if given import was ever considered. But I think adjusting how we think about imports is probably overdue.

I proposed (without any feedback on that, BTW) that given import will import everything including the givens and import will import without the givens. So a simple rewrite rule will change the current import to given import and that’s it.

[rjolly]

@Katrix

The type tells you everything you need to know about something when you want to use it in givens

I have to disagree : you could have given reverseIntOrd as Ord[Int] with { // reverse order }. Then knowing what something is about is just a question of level in package nesting : import <...>.reverse.{given Ord[Int]} vs import <...>.reverseIntOrd. Personally I prefer the latter.

@Katrix

It also has the additional advantage that type names can never be “incorrect”. That cats import up there actually provides many more types than just Order . There is no way to know which instance the author wanted just from the name. With types import <...>.{given Order[Int]} and import <...>.{given Hash[Int]} are distinct, and shows the intent of the author much better.

I get your point, but choosing by type instead of by name is equivalent to up-casting, which you rarely have a compelling reason to do : val ord: Order[Int] = myComprehensiveInstance

[rjolly]

@som-snytt

My point is that import was never just a convenient shorthand (as I thought of it in Java): it is as structuring as various definitions which are named, even though imports themselves are nameless

Exactly. Imports in Scala are more like syntactic sugar for type Thing = org.myorg.Thing. Which is how it is done in Javascript by the way : Thing = Packages.org.myorg.Thing. Note that, as a consequence, they are not exactly nameless : import org.myorg.{Thing => MyThing} I think we have to come back urgently to something sane, that is : mere syntactic shorthand, as in Java, and use assignments as above for specific cases.

@sake92

I’d also like import given .. , similar to Java’s import static !

I like the idea. Note that it would be useful even if we do away with by-type imports, as import given could be mandated for given by-name imports.

[Blaisorblade]

Under the influence of Haskell, many people plan to use given instances to encode coherent typeclasses — in which case the type does tell you everything and reverseIntOrd cannot be made a given. In fact, AFAICS there is no such implicit in the Scala 2 library, only an explicit method.
From that point of view, import <...>.reverseIntOrd should just be forbidden. (EDIT: that is not my point of view, but it felt significant enough to mention).

However, Scala 3 still gives you some freedom here — which is useful for things like ExecutionContext (which is not a typeclass, so imports by type would not make sense).

[rjolly]

@Blaisorblade

From that point of view, import <...>.reverseIntOrd should just be forbidden

Well, it is equivalent to given reverseIntOrd = Ordering[Int].reverse. Would you forbid that too ? To address your point on coherent typeclasses, what if we only want the reverse implementation ?

[Blaisorblade]

@rjolly

Well, it is equivalent to given reverseIntOrd = Ordering[Int].reverse . Would you forbid that too?

Indeed, coherent typeclasses would forbid that.

@rjolly

To address your point on coherent typeclasses, what if we only want the reverse implementation ?

That choice is reserved to the library defining Ordering[Int], which is likely also the library defining Ordering. You can make that an instance for sth like Ordering[Down[Int]] or Ordering[Int @@ Down].

In this case, I doubt I’d enjoy code where sort does the opposite because of an implicit import or definition far away — fans of coherent typeclasses would argue that’s always a concern.

But I was unclear; I’m not that draconian and I don’t think Scala should enforce coherence for all typeclasses and all givens, nor is there an actual chance something like that happens. Coherent typeclasses would also forbid implicit ExecutionContext, after all.

What I should have said is that

1. imports by type are sufficient for coherent typeclasses
2. this use case is important enough to deserve special support
3. reverseIntOrd is a dubious use case.

[rjolly]

The language is probably going to support both coherent and non-coherent typeclasses. Naming implementations is appropriate in the case of non-coherent typeclasses. Whether Ordering should be coherent or not is a different issue (an interesting one, see below). In any case, dropping by-type imports in favor of by-name imports would not forbid coherent typeclasses. It would just be slightly redundant, as in that case I agree that just the type would suffice.

@Blaisorblade

You can make that an instance for sth like Ordering[Down[Int]]

Then we could downright define a new class for decreasing integers. But it defeats the whole value of typeclasses, which is to give new powers to existing types. We would be back in good old Java with something like class DescInt(value: Int) extends Comparable[DescInt] { // impl decreasing order }. On the other hand, with different typeclass implementations, you can plug a chosen ordering as you see fit and have everything work as usual.

[Blaisorblade]

@rjolly

In any case, dropping by-type imports in favor of by-name imports would not forbid coherent typeclasses. It would just be slightly redundant, as in that case I agree that just the type would suffice.

I wasn’t claiming otherwise, so we’re in agreement there :-). But for coherent typeclasses, many of your criticisms of import-by-type don’t apply; some others do, but they’re for other people to judge.

@rjolly

Then we could downright define a new class for decreasing integers. But it defeats the whole value of typeclasses, which is to give new powers to existing types.

Typeclasses were invented in Haskell where they are coherent, so “overriding” instances are not part of “pure typeclasses”. I like that you can define reverseIntOrd and pass it by hand; I don’t think it should be made implicit (except maybe in a very small scope), but I’d leave that call to code reviews.

[Jasper-M]

In Scala you could abstract over such things without wrapper classes like DescInt.

scala> object Ord {
     |   type Down[A] >: A
     |   implicit def reverseOrd[A](implicit ord: Ordering[A]): Ordering[Down[A]] =
     |     Ordering[A].reverse.asInstanceOf[Ordering[Down[A]]] // this should work without casting if you have opaque types
     | }
object Ord

scala> val list = List(4,2,1,3,5)
val list: List[Int] = List(4, 2, 1, 3, 5)

scala> list.sorted
val res0: List[Int] = List(1, 2, 3, 4, 5)

scala> list.sorted[Ord.Down[Int]]
val res1: List[Int] = List(5, 4, 3, 2, 1)

P.S. You can have it in Scala 2 without casting but it takes extra code to work around some issues: https://scastie.scala-lang.org/R93yT5yZQKuSL0VPKPW4zg

[rjolly]

While trying with an opaque type, I think I found a bug (0.22.0_RC1):

object Ints with
  opaque type DescInt = Int
  object DescInt with
    def apply(n: Int): DescInt = n
  given Ordering[DescInt] = Ordering[Int].reverse
import Ints.DescInt
import Ints.{given Ordering[DescInt]}
val list = List(4, 2, 1, 3, 5)
list.sorted
// List(1, 2, 3, 4, 5)
list.map(DescInt(_)).sorted
// infinite loop

[odersky]

Can you run with -Xprint:typer to see what this gets expanded to? Thanks!

[rjolly]

Here you are. It blocks on “summon”. I have added a thread dump.

object Ints with
  opaque type DescInt = Int
  given Ordering[DescInt] = Ordering[Int].reverse
import Ints.DescInt
import Ints.{given Ordering[DescInt]}
val s = summon[Ordering[DescInt]]

result of coherence/test.scala after typer:
package <empty> {
  final lazy module val Ints: Ints$ = new Ints$()
  final module opaque class Ints$() extends Object(), _root_.scala.Serializable
     {
   this: Ints.type =>
    opaque type DescInt = Int
    final given def given_Ordering_DescInt: Ordering[Ints.DescInt] = 
      Ordering.apply[Int](Ints.given_Ordering_DescInt).reverse
  }
  import Ints.DescInt
  import Ints.{given Ordering[Ints.DescInt]}
  final lazy module val test$package: test$package$ = new test$package$()
  final module class test$package$() extends Object(), _root_.scala.Serializable
     {
   this: test$package.type =>
    val s: Ordering[Ints.DescInt] = 
      {
        val x: Ordering[Ints.DescInt] = Ints.given_Ordering_DescInt
        {
          x
        }:
          {
            x
          }.type
      }
  }
}

"pool-7-thread-1" #721 prio=5 os_prio=0 cpu=1410.81ms elapsed=77.89s tid=0x00007f6024afb800 nid=0x6258 in Object.wait()  [0x00007f603d1ca000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.6/Native Method)
	- waiting on <no object reference available>
	at java.lang.Object.wait(java.base@11.0.6/Object.java:328)
	at dotty.runtime.LazyVals$.wait4Notification(LazyVals.scala:89)
	- waiting to re-lock in wait() <0x00000000e42b4d60> (a java.lang.Object)
	at Ints$.given_Ordering_DescInt(test.scala:3)
	at Ints$.given_Ordering_DescInt(test.scala:3)
	at test$package$.<init>(test.scala:6)
	at test$package$.<clinit>(test.scala)
	at java.lang.Class.forName0(java.base@11.0.6/Native Method)
	at java.lang.Class.forName(java.base@11.0.6/Class.java:398)
	at linoleum.Tools.run(Tools.java:109)

[rjolly]

Like this it works:

val reverseIntOrd = Ordering[Int].reverse
object Ints with
  opaque type DescInt = Int
  object DescInt with
    def apply(n: Int): DescInt = n
    given Ordering[DescInt] = reverseIntOrd
import Ints.DescInt
val list = List(4, 2, 1, 3, 5)
list.sorted
// List(1, 2, 3, 4, 5)
list.map(DescInt(_)).sorted
// List(5, 4, 3, 2, 1)

[odersky]

The problem is this line:

@rjolly

given Ordering[DescInt] = Ordering[Int].reverse

It gets expanded to

    final given def given_Ordering_DescInt: Ordering[Ints.DescInt] = 
      Ordering.apply[Int](Ints.given_Ordering_DescInt).reverse

which gives you an infinite loop. This is as expected. Inside object Ints, DescInt is known as an alias for Int, so summon[Ordering[Int]] will give you the local ordering for DescInt. That’s the most specific ordering since it is defined in the closest enclosing scope.

To fix this, you need to establish the Ordering[Int] outside object Ints. For instance like this:

val intOrdering = summon[Ordering[Int]]
object Ints:
  opaque type DescInt = Int
  ...
  given Ordering[DescInt] = intOrdering.reverse

[rjolly]

Yes, I saw that. Thanks !

[odersky]

No problem!

It’s not the first time I came across the issue that an implicit derivation caused a loop. It happens sometimes with Scala-2 implicits as well. Might be good to detect & diagnose simple patterns of this in the compiler.

[Ichoran]

Incidentally, having suffered under the limitations of coherent typeclasses while programming in Rust, I at least personally think they are far more trouble than they’re worth. Anything without a unique sensible implementation of a typeclass has to have doubled methods, one which has a trait bound, and the other of which takes an argument which is equivalent to the trait but allows alternate implementations. It’s really annoying, far more than occasionally getting the wrong implementation.

Even worse, if you have trait T defined in library A, and you have struct S defined in library B, you can’t provide trait T for S in your own code, because this pattern would allow inconsistency in typeclasses! So you have a really unpleasant loss of generality (worked around by a mess of pointless wrapper classes) when trying to assemble diverse functionality.

I think Scala’s design is immensely superior if we’re going to have just one way to do it. Of course, being able to ensure coherency is nice in certain circumstances (e.g. if you want to use reference equality to check whether your monoid zeros are the same, you’d better have coherency in your monoids!). But as the only way to do it, it’s one of those ideas that is nice in principle but in practice is incredibly limiting and painful compared to what it saves you from.

---

On topic again: I think by-type imports are great precisely because you don’t have to name things. That they suggest coherent typeclasses is not a downside for me, because they don’t actually require coherent typeclasses.

So for by-type imports! If I ever import things explicitly, that’s how I’m likely to do it (except for bizarre libraries that implement conflicting givens and require you to name a particular one to make the given functional).

[odersky]

Thank you for relating your experiences here. I suspected that coherence is troublesome in practice. It’s good to see that backed up by empirical evidence!