A counterargument to using convertibleTo or into instead of ~ is that it’s the same kind of modifier as => and *, which are symbolic. The maximum crypticity is reached when the 3 are used in conjunction:
def myMethod(x: => ~MyType*) = ???
but it would not be so often.
Oh, and ~'s usual meaning of “about” is convenient here. The method above would read “my method of x which returns about my type repeated”.
What evidence do we have for this statement? In fact, the one case in the standard library where implicit conversions were first considered a great idea and then afterwards were considered a terrible mistake (i.e. the Scala/Java collection conversions) is not of this nature.
Or that type is inferred…
Or some subpart of the type defines implicit conversions…
I have the impression that the discussion is drifting back to “implicit conversions are not so bad after all, once we use extension methods for method dispatch we should be OK”. I don’t agree with that. In fact, extension methods replace specifically implicit classes, and I had the impression that implicit classes were the least problematic part of the implicit conversion issue.
So I’d really like to get back to the discussion where we look at concrete use cases of implicit conversions and figure out the most restricted way to address those. And of course one of the escape hatches that will remain is that you can enable them with a language import. But you should not need to do that for straightforward, simple, Scala code.
This is a great demonstration why we want to restrict implicit conversions. In a cuttting-edge library, experimenting with mutual implicit conversions should be supported, and we have the language import for that. My experience tells me that in general such mutual conversions are a bad idea since you never know what your actual types are. In your case I assume representation does not matter since the two are equivalent. But I guarantee that the next person will set up something that will be super problematic, and will think it’s a great idea. I have seen too much of that.
Yes, but note that new style implicit conversions give a feature warning on use today, and I propose we keep that stable. What I’m trying to do is to identify the cases where the warning is not necessary.
That’s a refinement, not a reversal of decisions.
The other direction, which does restrict the language, is that we want to make the language import requirement for implicit conversions stricter over time: from a feature warning, to a regular warning, to an error. I agree that has to be clearly communicated.
The overarching narrative has also been the same for some time now:
What we are trying to do now is define the subset of implicit conversions where this strategy is simply too painful, where we need to support implicit conversions without requiring the language import. That should be the smallest possible subset, and where the possibility of an implicit conversion being inserted is made as explicit as possible.
So far, the only clear cases of this I have seen are conversions on method arguments and specific compiler-supported conversions such as numeric widening or scala.Function <-> js.Function conversions. Are there others?
Implicit coversions triggering when types are inferred seems reasonable to me. Scala leans heavily on type inference, and inferred types are as real as declared types, so anything that works on one should work on the other. That seems like a pretty fundamental property of the language that we wouldn’t want to change. I mean, even explicitly-typed method arguments often only know what method they are being passed to due to type inference on the receiver. Where do we draw the line whether a type is inferred or not?
For the sub-part thing, that also seems nicely symmetric with typeclasses: the implicit for F[T] could come from either F or Ts companion object, and there are good reasons for either case. Given implicit conversions are now (in scala 3) just typeclasses, it seems reasonable to follow suite. All the use cases in com-lihaoyi only need to search for implicit conversions in F, and in fact most (though not all) are meant to only be used that way (rather than being imported), but given the symmetry I would be reluctant to say that there aren’t use cases for searching for conversions in T.
My objection to special-casing method parameters for implicit conversions is that it breaks the symmetry of the language. Method arguments in Scala are generally not treated differently from any other expression, which is a pretty fundamental invariant to give up. If I can have def f(x: ~Foo) = ???; f(notfoo) why can’t I have val x: ~Foo = notfoo? or scala.Set.empty[~Foo] + notfoo? If I can have those, adding a whole new construct to the type system seems like a steep price in terms of complexity
Furthermore, someone who defines an implicit constructor for a type in its companion object is going to want it everywhere. Needing to remember to annotate the method adds a new layer of boilerplate and room for error, when the developer already specified exactly what they wanted. If they wanted it to be opt-in, they would have put it somewhere else as an “orphan” implicit and imported it explicitly to they places they wanted it enabled. Or they would define a wrapper type to convert to, to separate it from operations on the underlying type. Both of these are already possible today
Agreed. That’s why I think we need to decouple “permission to insert implicit conversion” from the actual types. Types can be inferred and can consist of many components and base classes and that makes it much harder to see where a conversion is expected.
It’s analogous to by-name => and repeated *. These are also reserved for method arguments. So if we conclude that conversions in method arguments are the only use case to be supported, it makes sense to introduce syntax to express this.
What about conversions from String to Seq[Char], Array[T] to Seq[T], Option[T] to Iterable[T]? Do we want them to be applied everywhere? Maybe, but such a design would not be minimal. So yes, annotating function parameters is more ceremony, but I don’t think unreasonably so. And when it comes to implicits the rule should be to be as explicit as possible.
At the very least, I think we would need some way to allow things like Set.empty[~Foo] + notfoo, where the method is not under our control but the type is and we want the type to be implicitly constructable. That’s not an uncommon use case where you want to put some values in a collection or something before calling the method later, but the collection operations are outside your control to annotate.
Code often flips back and forth between “call methods directly” and “put values in collection and call methods later” as it gets refactored over time. Having implicit constructors stop working in the latter case seems like a sure cause for confusion, and the workarounds (calling the implicit constructor manually, or define your own wrapper method/collection just to annotate the params as conversion targets) seem like they would only cause more confusion.
=> and * types already have this issue, but the workarounds are relatively lightweight (prepend the value with () => and use with x(), or wrap the multiple values in Seq(...) and use with x*). Perhaps if there was a similarly lightweight workaround for passing ~Foo types to non-~ external methods, the friction could be reduced?
The ceremony is one thing, but what I’m more worried about divergent behavior when people annotate vs not annotating the method parameters. This seems like something that method-author is sure to forgot some fraction of the time, even if the type-author intended the conversions to always apply, resulting in two different behaviors that method callers will get confused by.
Having the implicit annotation be on the method vs being metadata attached to the type (the status quo) feels very much like the difference between Java’s use-site variance and Scala’s definition-site variance. In both cases, the type author already has the best idea of how something should be used by the method callers, and asking the method authors to constantly affirm that decision is both tedious and error-prone.
A possible middle ground:
convertible (or some other name).convertible trait ConvertibleTo[T] (or convertible trait Into[T]) available in the standard library. By convention, if a library designer wishes to define an implicit conversion from T to U in a case where they control either T or U, they can define a Conversion[T, ConvertibleTo[U]] on the companion object of T or U. So the standard library could define a Conversion[String, ConvertibleTo[Seq[Char]] on the companion object of String of Seq, and then use ConvertibleTo[Seq[Char]] as it would have used convertibleTo Seq[Char].This way, it is very clear from an argument type’s declaration that it can accept implicit conversions. The compiler can safely avoid implicit searches unless an argument type is declared as convertible, or the magic import is available. And there is still a way for methods that accept types that are not themselves implicitly convertible to opt in to implicit conversions on those types.
I think there’s some thought about what happens in cases where the compiler can infer a convertible type. For example, if I define def foo[T](x: T): T = ??? and I have
val y: SomeTypeConvertibleToBar
val x: Bar = foo(y)
The compiler can infer that T = Bar, so does that mean that I the implicit conversion is available (without the magic import) if Bar is convertible? I think it would be acceptable to forbid this, and force caller to use the magic import, or the library writer to use def foo[T](x: ConvertibleTo[T]): T. I have not thought through the corner cases here.
You might worry that library authors will sprinkle around convertible too liberally. Again, this doesn’t worry me very much: if it is very clear from a type’s definition that it is can be produced via implicit conversions, remembering the rules for search for where those conversions can be defined is not so hard. The only downside will be potentially slower compile times, but it will at least be abundantly clear whose fault it is.
(Note: convertible is a bad name because it is ambiguous between “convertible to” and “convertible from”, but in the proposal, it would be “convertible from (some other type)”).
“If it makes you say shun, then shun it.”
The assumption is that the language import matters to people’s behavior.
Do we know whether this is actually true? I have dozens of imports in most files. The barrier to me adding another one that the compiler says I need to is approximately zero. It certainly wouldn’t be a barrier to me
If the compiler doesn’t warn me and things mysteriously don’t work right because of a missing import, that’s easy too: I just always include that import, in every single file.
Same deal as with givens. I never (intentionally) fail to import givens. Who knows if I might need them? Just import and don’t worry about it unless there’s a problem. Usually the companion object rules and such take care of it–I try to structure my code so they do. But can I be sure? Generally, no. So…
Anyway, personally, while I hope that the details about import-free conversions get nailed down to be optimal, as long as the “just add an import” option exists, I think the stakes are low.
Also, I think in cases where method argument annotation would be needed, people would instead adopt the best practice of using a Rust-style .into() at the call site (extension method with given conversion), because the chance that the annotation will be everywhere you need it in every library is probably roughly zero except in very unusual cases. (Those unusual cases might be important so it might still be worth defining the annotation.)
Regardless, I would imagine that the most common solutions will be to either import, or .into().
I have given this some thought but am not yet convinced. The alternative would be to demand
Set.empty[Foo] + notfoo.convert
Isn’t that clearer? Also if cases like this happen frequently, what’s to stop us from defining:
extension [T](s: Set[T]) def +~ (x: into T) = s + x
What would be a more complete example, where a case like this would arise and we do not have control over the addition method?
That’s completely fine. If people want maximum power they can have it. But there’s also work in teams where members are worried that the design becomes unmaintainable. Here a team lead could easily put a foot down and decree “no language imports” and that would be easy to check.
Also, teaching newcomers. It’s all well and good to say “implicit conversions are dangerous, use them sparingly” but without a clear control where they are inserted, this is not actionable. Requiring the language import helps. First, it means I need explicit opt in. Second, it means I can withdraw the language import and see by way of error messages where all the implicit conversions in my code are.
A meta-design point why we want to allow implicit conversions in arguments but not elsewhere: Implicit conversions are sometimes useful to support ad-hoc polymorphism. We already have two alternatives in our toolbox to address this: overloading and typeclasses. But overloading can lead to a combinatorial explosion of method definitions, typeclasses are more heavyweight, and neither can deal with variadic arguments. So allowing implicit conversions as a third way to support ad-hoc polymorphism seems reasonable.
My thesis is that that’s it. There is no other widespread, legitimate use case of user-defined implicit conversions in simple Scala 3 code. For more advanced code a language import is a reasonable requirement. Now, prove me wrong. 
Actually, the language import seems to be a good middle ground but I’m not sure about the exact behaviour of this import: Is it also required at use/call-site or only when defining an implicit conversion ?
If it is required at call-side, is the export feature compatible with this language import ? In the quoted project for example, there is no situation where the user would not enable implicit conversion while using this lib.
I have a class
case class Multiple[+A](value: A, num: Int){
def * (operand: Int): Multiple[A] = Multiple(value, num * operand)
//etc
}
object Multiple {
implicit def toMultipleImplicit[A](value: A): Multiple[A] = Multiple(value, 1)
The whole point of the class is to allow rapid data entry of serial values in code. Forcing the user of the class to add a “~” in very method where they use the class would sabotage its purpose.
This seems like the heart of the question to me. I have no problem with a language import at the definition site, but requiring one at the use site seems like it would be an unreasonable burden on libraries. One of the major selling points for Scala is the amount of power your can get from libraries (which helps justify a stdlib that is a lot slimmer than many languages); we shouldn’t introduce speedbumps that are basically non-sequiturs to the business logic that is using the library.
(Like @Iltotore I’m not clear on whether the proposal is to require such a use-site import or not; my point is just that we should keep library consumers in mind as a use case where such an import is inappropriate.)
The alternative .convert sounds plausible; it would thematically be similar to how we deal with => and * arguments already: as sugar for the library callsite, but in library “internals” often expanded to full lambdas/Seqs and passed around that way.
I’m not sure it’s better than the status quo, as in exchange for limiting implicits it would provide a whole new user-facing api they need to remember to call everywhere, and new confusing errors when it’s wrong, but it’s workable. These same annoyances exist with passing around =>s or typeclass-ed method parameters today, so nothing new.
The extension method wrapper approach on the other hand doesn’t work. Even if we wanted to “put things in collection and deal with them later”, there are dozens of collection operators on dozens of collections people use in different places, many of which are outside the standard library. Wrapping them all up front is impossible, and wrapping them on demand would be super tedious and messy
I guess my question here is: how widespread is this a problem these days? And does the proposed solution really happen?
If you look at Python, anyone and everyone can do crazy stuff with every feature and every internal piece of the runtime exposed, and yet life goes on without too much issue. Java is similar: anyone can pull up sun.misc.Unsafe and go to town. Both of these easily create as convoluted code as anything Scala language features can create!
I don’t see a similar “language import for advanced features” approach being used in any mainstream language. And at least in my professional and OSS experience, language imports have had zero role in determining the subset of Scala to use, even in orgs with relativey strict automatic enforcement of guidelines (e.g. ours, where we lean heavily on -Xfatal-warnings/@SuppressWarnings)
Are there really thousands of engineering teams and orgs out there relying on language imports to control their usage of the language? And what makes Scala so special that we need to take an approach that no other popular language community does here?
I agree with @odersky 's point about newcomers. Knowing that you have to add imports, but not really understanding why, leads to a lot of frustration for newcomers. Even worse, because of the interaction of inheritance, implicits, and name clashes, we have found ourself in a situation where if you include two common DSL imports in the same file, you will break compilation – but removing one of them (or in come cases, either of them) will compile just fine. Even worse, the error message says that an implicit conversion is missing when in fact the problem is that that implicit conversion is present twice because of inheritance.
For a newcomer, being able to copy-paste code have it compile is a very important ability. Of course they don’t expect that no imports are necessary, but for simple name imports, IDEs are usually pretty good at doing the work for you. Having to copy/paste all wildcard imports and crossing your fingers is not a good situation. At least with the magic import for implicit conversions, a newcomer knows that they should be able to copy-paste code with confidence if that import is not present.
I think right now I agree with most of this. The only concerns are:
Should ad-hoc polymorphism only apply to methods? Currently overloading and typeclasses only apply there, so it’s conceivable that we could make implicit conversions apply there too. Some added user-facing complexity, but not unprecedented. Everything I don’t like about limiting implicit conversions to methods, basically already applies to typeclasses today.
Are language imports the way to go to restrict things? I have personally seen zero evidence they work, despite writing code in a variety of environments. They could work, they are meant to work, but they just don’t. And there are other, better, ways to nudge developers in a direction you want (e.g. see the section “Linting Workflows” in Scala at Scale at Databricks)
What I don’t like about Set.empty[~Foo]: at first glance it looks like the type of the Set is not Foo but a type which is convertible to Foo. So, seeing
val s1 = Set.empty[~Foo]
val s2: Set[Foo] = s1
val s3 = s1 + notFoo
s3 + notFoo
It looks like (again at first glance, not after reading the docs) as if the assignment of s2 to s1 involves a conversion for every element in s1. Moreover, it is not clear to me what type s3 would have Set[Foo] or Set[~Foo].
Edit:
Also, If I get it right, then what is actually desired with Set.empty[~Foo] is kind of a zero overhead adapter in the sense of: apply some code (in this case a conversion) every time the type parameter occurs as parameter in the Set (what about if T occurs as return type?).
If we should support something like that then we should maybe strive for a more general solution.
Something like:
object ConversionToFoo:
inline convert[T](t: T)(using c: Conversion[T, Foo]): Foo = c(t))
val s1 = Set.empty[Foo pre-processed by ConversionToFoo.convert]
val s2 = s1 + notFoo
val s3: Set[Foo] = s1
which desugars to
val s1 = Set.empty[Foo]
val s3 : Set[Foo] = s1
val s3 = s1 + ConversionToFoo.convert(notFoo)
s3 + notFoo
which, after inlining etc. is more or less zero overhead. In this case I would assume s3 has type Set.empty[Foo] and calling s3 + notFoo would fail.
This leads to an interesting hypothetical: would fixing variadic arguments so they could support heterogeneous arguments and their corresponding type parameters + givens (and thus be amenable to typeclasses w/o implicit conversions) significantly change the situation?
Typeclasses are already considerably more lightweight now, with the way that extension methods are now imported, so if variadic methods can be fixed the use cases for implicit conversions could be shrunk further.
Just to expand on this idea a bit further, another possibility is to allow the convertible modifier to be placed on type aliases, such that convertible type T = U means that implicit conversions to T can be declared, but such can conversions can produce values of type U. In that case, you could just have type ConvertibleTo[T] = T in the standard library. Assuming this doesn’t cause too many compiler headaches, then you can use ConvertibleTo[Seq[Char]] with no .convert overhead inside the method.