Updated Proposal: Revisiting Implicits

Yes, each thing you need to import is a cost for users who want to access the feature. IIRC, referring to this as a “tax” on the feature originated in a talk from the 2011 Northeast Scala Symposium and a followup article.

As I said above, I’m not opposed to this for enabling additional syntax, I am primarily annoyed at the presumption of incompetence I received when pointing out that you do actually need to define and import a separate ops object to contain the extensions in some cases.

Your alternative suggestion to define them in the companion object and bulk import the givens is unworkable for two reasons:

  1. It’s the same number of imports.
    If you go back and check my example, you’ll note that I did not need to import the instances, only the syntax, so your suggestion is equivalent.
  2. The blanket import flattens the given prioritization, removing the ability to define relative priority.

The implementation is fine, though if the boilerplate could be reduced it’d be awesome. Being told that a simplistic example in the docs means I can’t possibly be correct about more complicated behavior is not.

1 Like
  1. Yes, they are equivalent, but I don’t understand how the import is a boilerplate. You are going to have to import something; I presume you’re not expecting the compiler to search for the entire classpath aimlessly.

  2. I’m not sure what you mean by that.

Wouldn’t allowing naming types at the use site mostly fix the problem here? For example 1.pure[F = Option]. Sadly AFAIK that proposal was dropped from Scala 3.

Import._ is expensive operation.

  • If someone does not remember it by heart he will have to go to documentation
  • if someone does not have a reflex to type it, he will feel annoyance and make an errors or be puzzled why it is not working at least until he get used to it.

In most cases an ide do the import automatically. But ‘import *.given’ is not such case at all.
It is not problem for main libraries, it is not problem for rare libraries. But I have to write decorators for middle cases because there are no other way to improve developers’ productivity.
So I don’t like ‘import *.given’ personally for that. Most people in our company do not work with it and thay dont care about it :slight_smile:
But I am not sure that it is good characteristic when even Martin says that it is good to avoid it.
Of course I can imagine that there are people that have no problem with it. But we had it and we try to avoid it.
IMHO: Typeclasses is a killer feature of scala and it is sad that it has such drawback.

2 Likes

No one told you that you “can’t possibly be correct”. Please be more measured and less combative in your tone on this forum (this isn’t the first time I’ve noticed it).

3 Likes

You mean expensive compared to import X (without _)? If so, then this has more to do with import-syntax than with implicits, because this applies to non-implicits as well.

I actually think that this “boilerplate” might be necessary to make implicits easier to comprehend and work with (that’s the main goal here, remember?). Making a “normal” import also include implicit interpretations (type instances, conversions, extension methods) is a side effect, as import is first and foremost a name resolution tool.

I usually do not do import X, a code assistant help me with that task.
So these things are not comparable.

I do not think so for many cases. But It is a question out of my competence level in any case :wink:

1 Like

My point was that the number of imports in your suggestion was equivalent, and moving the definitions into the companion isn’t doesn’t refute my original point that just defining them as extension methods in the trait doesn’t always work.

On of the primary techniques for prioritizing implicits in Scala 2 is nested traits, which still exists and works in Dotty. Bulk importing that flattens them into a single priority level, which is trouble if you have otherwise ambiguous definitions.

For example, assuming you have some typeclass Foo which can be derived from Order or Ordering and you want to provide a lift from these to Foo, you can’t do that by defining them both in the companion object. One needs to have lower priority, which is done by defining it in a trait which the companion object extends. import Foo.given undoes this.

You did, actually. Repeatedly.

For example, when I sketched out here my previous experience with being unable to get the simpler version you suggested to compile, including a link to the actual code exhibiting this problem, your response was only shades better than RTFM:

Your follow up also presumed that I hadn’t read the docs, nor tried their example, which I’d indicated I had:

Finally, when I minimized and zeroed in on the exact issues, your response was unpleasantly dismissive:

And yes, I imagine you’ve noticed my annoyance with “Why don’t you just use macros?”, and the similar attitude of “We think this is easy, why can’t you figure out these undocumented APIs?” which has been pointed in my direction multiple times. It grates on the nerves after a while. While I strive for respectful conversation, I’m not going to apologize for getting annoyed when talked down to.

1 Like

OK, let me clarify what seems to be root of the misunderstanding here. This thread is about the documents describing the implicit proposal (see first post), this is why I keep referring back to them. If you believe there’s something wrong with these documents (the behavior described is not ideal, too vague, …), then this thread is the correct place to have that discussion. But if you believe there’s something wrong with the implementation, then Issues · lampepfl/dotty · GitHub is the correct place to report that.

When I said:

I was not trying to be dismissive: we take every issue seriously, and we’re much less likely to forget them than a post in the middle of a giant forum thread.

The Scala Code of Conduct | The Scala Programming Language states “Be kind and courteous”, this is all I’m asking for here.

I would like us to get back on topic now, for further meta-discussion, please open a separate thread.

2 Likes

I agree with this comment, but I hope we are making great progress to address this problem!

Error messages produced by the compiler in case you missed to import something have been improved a lot recently. It now suggests you import clauses that might fix the compilation error.

You can see the test cases here: https://github.com/lampepfl/dotty/blob/46aec62f0bc326d85b65b7231bee16c0c5a3061a/tests/neg/missing-implicit.check

4 Likes

So, I just tried using the implicits as currently envisioned (given / using) to solve some actual coding problems, with attention to how they work.

In so doing–while editing the compiler to change the syntax to try out different things–I found that it doesn’t actually make a whole lot of difference to me what the given keyword is. It could be given, could be witness, could be provide, could be the, even. With a little practice you can get used to it. I personally like provide best out of the things I’ve tried. I think it’s a significant but modest improvement.


Aside

While editing the compiler, I realized that it’s really nice that GIVEN, Given, etc. can be a noun inside the compiler code. This is because flags and tokens and such are all items (i.e. nouns), so it’s nice if the identifier is a noun! However, I don’t think this should affect whether the language uses verbs or nouns or whatever in various places.

I did notice that nouns made the compiler code nicer to read, though!

End Aside


The bigger problem, though, is the order of types and values. This comes because both of these things are supposed to be valid:

given T = t
given t as T { def foo = "salmon" }

This is weird and hard to adjust to, at least for me. In the first case, you’re associating a preexisting type with a previously named term. In the second case, though, you’re declaring a new term by specifying a preexisting type and then creating an anonymous implementation of it. You have the same kind of problem you had with given/given: two considerably different constructs sharing the same terminology, which impedes mental switching.

I don’t yet have a good solution, but I think this is worth more thought. It pretty much bakes in an inherent conflict into the language. Changing as to for or is doesn’t really help; it’s the type, instance vs. newname, type { anonymous extension } ordering which is a pain.

So I really hope these things can be unified somehow.

One idea, which I don’t really like but maybe can be salvaged into something usable, is to introduce new terms or use pre-existing ones without a syntactic cue save for the braces after the type name.

given t as T

means that we can use an already named t when we need a T, while

given t as T { ... }

means that we define a new t for the cases where we need a T.

I don’t like this because it makes the introduction of new terms hard to distinguish from the usage of old terms. However, it does help a lot in making givens take on a predictable structure.

Alternatively, we could insist on the familiar old : notation to introduce new terms and either give up the abbreviated syntax entirely, or use an abbreviated new form:

given T = new { def foo = "salmon" }
given t: T = new { def foo = "perch" }

This has the downside of being irregular ( new { ... } means new Object { ... } normally), but it has the upside of preserving the usual expected syntax for declaring things. Now we always have

given [optional new term name :] Type = theAppropriateTerm

This is my favorite so far (haven’t played with it, since the compiler changes needed to get this to compile are beyond what I can do in a matter of minutes), but I still am not totally happy with it.

(Note: parameters can be handled fine, but it just gets more irregular:

given foo(i: Int)(using A): T =
  new(i, summon[A]) { def thingy = "halibut" }

)

Anyway, after using it, I’m now more worried about this than whether it’s called given or provide or paragon or archetypical or whatever. I really hope there’s a chance to fix this. I think it’s a substantial wart as it is. (I think provide t: T = new { ... } also works better than given.)

4 Likes

It’s worth noting that

given T = t

also possesses an explicitly named form which is

given u as T = t

and that, to consistency call things with respect to the other example, is actually

given t as T = x

Written next to the other one, that is

given t as T = x
given t as T { ... }

which is not confusing at all. Now we’re comparing comparable things: two explicitly named givens.

You can also compare the two corresponding anonymous givens:

given T = x
given T { ... }

which, again, is fine.

The confusion you observe comes only from the fact that you’re comparing side by side a named given and an anonymous one.

2 Likes

The forum needs an emoji reaction for “I’m not sure what you mean by that.”

The confused emoji, plus raised skeptical eyebrows.

Your example doesn’t compile surprisingly for me.

I did raise a dotty issue along the lines of, Do you really mean for implicit resolution to pick the opposite of what Scala 2 chooses? And the answer was yes.

So I think the team is very tolerant of tickets in that vein, if the ticket shows compilation results.

Those tickets serve as documentation for other confused people.

I haven’t followed all the discussions, but if this is in response to SIP proposals it’s not for “We think this is easy” reasons, but for things like keeping the language complexity down, and maintaining compiler speed, etc.

I don’t see @smarter talking down to you, but I can understand the frustration on having to uplift and relocate a discussion into another forum (a dotty issue), and I encourage you to.

With regards to macros and its API, I suspect it at this stage sending a PR to the dotty repo with your WIPs you might get you the guidance sorely needed.

1 Like

It would be more regular if as was mandatory, so that you would have to write:

given as T = x
given as T { ... }
1 Like

That would only be more regular if the : in type-inferred vals were mandatory as well, i.e.,

val x: = foo

instead of

val x = foo

From the current docs it seems as also must be used in:

given (using outer: Context) as Context

There is no name as Type there, so what are the rules for when as must be used?

1 Like

I was really hoping after @smarter’s last comment that this would simply die, but if comments are going to be added on the tag end of this thing without going back and reading the whole discussion, then perhaps it’s worth providing a summary.

I made this comment about the proposal referencing typeclass definition boilerplate:

That sparked a series of questions about what, exactly, I meant by that, which I did my best to explain. Concrete examples were requested:

After this point, things get weird. I attempted to provide those concrete examples, like this one containing a permalink to one class which has that behavior:

And this comment, which minimizes it down to something that would fit in scastie, were met with terse redirections to go read the docs. This one is pretty representative:

Note that, on the whole, the concrete examples appear to have been taken as intended, so I’m somewhat dubious that the problem is on my end. I’d link the responses, but they’re temporally adjacent so they’re easy to find, and I’d like to avoid dragging anyone else into this mess.

Apparently, @smarter thought I was kvetching about either the proposal or trying to report a bug, rather than trying to explain a misunderstanding about the current state of things that isn’t addressed by the proposal.

I can see how, from their point of view, it was intended as a helpful redirection back on topic. From my point of view the terse nature and off-topic suggestions came off as condescending and dismissive. I’m not sure how this disconnect between, “Concrete examples would be nice”, and “go look at the docs” happened, and I frankly don’t care at this point.

The bit about the macros has been covered to death in other threads. A good summary of why I find this unhelpful and a form of “talking down” can be found here.

For the record:

  • I like the current proposal
  • The definition boilerplate, is a thing, but it’s is less than in Scala 2
  • While it’d be nice to reduce it further, it just isn’t worth it to me to continue to engage

I believe efforts are being made to have better code assistance for implicits, in which case it seem to me that having an additional character or two is less of a problem.

The road to improving productivity is not just about faster code writing, but also about better code readability; we need to strive for a balance between the two.

Then I believe the real problem here is the lack of deliberate mechanisms for conflict resolution and prioritization. Having to rely on nested traits seem to me like a workaround that should be replaced with a dedicated syntax – one that does not increase the “import tax”.

Did you notice the very subtle (and undocumented) semantic difference between implicit val and object? Where one (object) is propagated to the 3rd scope which doesn’t directly import the implicit, while the other (val) does not?

There is an irregularity here. I’m not even sure if this difference is indeed between object and val or maybe there’s something else that is affecting the availability of the implicits in the 3rd scope.

How about implicit? To me it seems that the new proposal is almost identical to the old implicits. Moving from Scala 2 syntax to Dotty feels more like an anagram than a conceptual change, which only begs the question - how implicits are now simpler and easier to understand?

1 Like