Better type inference for Scala: send us your problematic cases!

Language Design
189

The following code compiles under Scala 2.13 but does not compile under Scala 3:

val s = new Serializable {
  val i = 11
}
println(s.i) // error in Scala 3: value i is not a member of Object

PS: This is also true if import scala.language.reflectiveCalls is added.

190

Scala 3 requires the parent to be a Selectable. For instance, this works:

import reflect.Selectable.reflectiveSelectable

val s = new Serializable with Selectable {
  val i = 11
}

It’s described here.

2 Likes
191

Is this pattern common for this type in particular ?

If that is the case, we should maybe special-case an error to provide a better error message

192

It’s not easy to give a better error message. In the original failed example s.i, s is of type Serializable whereas in Scala 2 it was of type Serializable { val i: Int } (and the access to i would require reflection). In the minimized example, we can see that s came from an anonymous class that contains a definition of i. But in general, all we have is that x: Serializable; that additional info is lost.

And we cannot very well warn on defining the anonymous class either, since in the overwhelming majority of cases nobody cares about members like i. People don’t expect them to be callable anyway.

193

I see, but it is possible to detect that i is not called in the body, and can’t be called from outside, and emit a warning

And that warning could direct the user to remove the member, or make the instance Selectable

(Not necessarily inside the compiler however)

194

I guess that should already be handled by -Xlint:unused or whatever warning/linting mechanism is available. i is effectively private[this] and unused so should be picked up by the regular unused variables detector.

195

Yes, that looks feasible and useful. Unlike unused, this could be on by default and specifically point to Selectable.

196

So I managed to find another type inference issue specific to Scala 3 in the context of pekko-streams. This one is a little more involved, but this version of the code which compiles fine for Scala 2

SplitAfterSize(chunkSize, chunkBufferSize)(atLeastOneByteString)
  .via(getChunkBuffer(chunkSize, chunkBufferSize, maxRetries)) // creates the chunks
  .mergeSubstreamsWithParallelism(parallelism)
  .filter(_.size > 0)
  .via(atLeastOne)
  .zip(requestInfoOrUploadState(s3Location, contentType, s3Headers, initialUploadState))

had to be changed to

val source1: SubFlow[Chunk, NotUsed, Flow[ByteString, ByteString, NotUsed]#Repr, Sink[ByteString, NotUsed]] =
  SplitAfterSize(chunkSize, chunkBufferSize)(atLeastOneByteString)
    .via(getChunkBuffer(chunkSize, chunkBufferSize, maxRetries)) // creates the chunks

val source2 = source1.mergeSubstreamsWithParallelism(parallelism)
  .filter(_.size > 0)
  .via(atLeastOne)

source2
  .zip(requestInfoOrUploadState(s3Location, contentType, s3Headers, initialUploadState))

In order for it to compile with Scala 3.3.0 as you can see in the PR at https://github.com/apache/incubator-pekko-connectors/pull/167/files#r1249747354

The error that Scala 3.3.0 compiler provides is

[error] -- Error: /Users/mdedetrich/github/incubator-pekko-connectors/s3/src/main/scala/org/apache/pekko/stream/connectors/s3/impl/S3Stream.scala:1184:11 
[error] 1180 |        SplitAfterSize(chunkSize, chunkBufferSize)(atLeastOneByteString)
[error] 1181 |          .via(getChunkBuffer(chunkSize, chunkBufferSize, maxRetries)) // creates the chunks
[error] 1182 |          .mergeSubstreamsWithParallelism(parallelism)
[error] 1183 |          .filter(_.size > 0)
[error] 1184 |          .via(atLeastOne)
[error]      |        ^
[error]      |bad parameter reference ([O] =>>
[error]      |  org.apache.pekko.stream.scaladsl.Flow[
[error]      |    org.apache.pekko.util.ByteString @uncheckedVariance, O,
[error]      |    org.apache.pekko.NotUsed @uncheckedVariance]
[error]      |) @uncheckedVariance[org.apache.pekko.stream.connectors.s3.impl.Chunk]#Mat at sbt-api
[error]      |the parameter is type Mat in class Flow but the prefix ([O] =>>
[error]      |  org.apache.pekko.stream.scaladsl.Flow[
[error]      |    org.apache.pekko.util.ByteString @uncheckedVariance, O,
[error]      |    org.apache.pekko.NotUsed @uncheckedVariance]
[error]      |) @uncheckedVariance[org.apache.pekko.stream.connectors.s3.impl.Chunk]
[error]      |does not define any corresponding arguments.
[error]      |idx = 2, args = org.apache.pekko.stream.connectors.s3.impl.Chunk,
[error]      |constraint =  uninstantiated variables:
[error]      | constrained types:
[error]      | bounds:
[error]      | ordering:
[error]      | co-deps:
[error]      | contra-deps:

My initial hunch is that this may be due to the @uncheckedVariance at https://github.com/apache/incubator-pekko/blob/580f12c29fb61b65758af9b0c31494af0af17175/stream/src/main/scala/org/apache/pekko/stream/scaladsl/Flow.scala#L69 not being properly propagated through

197

This thread (started almost 5 years ago!) was initially meant to gather feature requests related to type inference deficiencies in Scala 2 we hoped to improve in Scala 3. While it might still be useful for this kind of discussion, it has grown a bit unwieldy and I would encourage you to report issues on Issues · lampepfl/dotty · GitHub instead, especially when extra work would be needed like here to minimize the issue.

3 Likes
198

If thats the case it sounds like the best idea would be to either close this thread or rename so its intention is more clear/less confusing? I mean it is literally stated “send us your problematic cases” which implies its a bug tracker.

In the future I will make these kinds of reports on the official bug tracker, ticket made at Regression in Scala 3 for propagation of `@uncheckedVariance` via dependent types · Issue #18438 · lampepfl/dotty · GitHub

1 Like
199

@SethTisue Should we close this thread? I can’t do it myself.

1 Like