How does scala2 compiler cache/invalidate the typeChecking result for the given tree?

Tooling
1

Background

Hi, I’m currently trying to optimize the InteractiveDriver of Scala3.
Currently, every time developers hover on symbols in Metals in the Scala3 project, Metals asks InteractiveDriver to compile the unit with InteractiveDriver.run to make sure the compilation unit has the latest semantic information.

However, InteractiveDriver.run, especially the type checking, is CPU-intensive, which makes showing hover information in Metals a bit CPU-intensive, even though this feature is frequently used.

In Scala2 …

I realized it is the problem only in Scala3 projects, and Metals doesn’t consume so much CPU in Scala2 projects, even though Metals call typeCheck every time users hover on symbols in Scala2 projects.

I profiled using async-profiler and realized that scalac doesn’t run typed1 if the given tree is already typed, and that’s why typeCheck with scalac in IDE environment is blazingly fast.

Question

I also realized that when we modify the type in a dependent unit, scalac will run typed1.
For example, when we have two files A.scala and B.scala.

// A.scala
object A {
  val aaa = B.foo
}

// B.scala
object B {
  def foo: Int = ???
}

If users hover on A.aaa it shows, it’s type is Int. Then, modify the def foo: Int = ??? to def foo: String = ???.
Now, hover on A.aaa again, scalac takes some time to run typeCheck(unit) because now typed1 will be called. Which means when we modify a B.scala, alreadyTyped will be false.

Question is, how does scalac detect something has changed since the last typeChecking, and make alreadyTyped = false? around here https://github.com/scala/scala/blob/43ee5095c13b16aa2138c2b472b5a4a3da0a6ba6/src/compiler/scala/tools/nsc/typechecker/Typers.scala#L6032-L6047 ?

Additional Context

I was considering trying not to run InteractiveDriver.run if the “input hasn’t changed since the last compilation” https://github.com/lampepfl/dotty-feature-requests/issues/309 but if we could implement the same kind of optimization in Scala3, we don’t need to remember the compilation unit is up-to-date on Metals side.

5 Likes
2

It might be don in Metals side https://github.com/scalameta/metals/blob/d2ab724848b0c8b5c066694e5cf431c809715d34/mtags/src/main/scala-2/scala/meta/internal/pc/MetalsGlobal.scala#L690-L705 I’ll check more :bowing_man:

1 Like
3

Ahh, never mind, I realized this is done by Metals side + Scala2. I’ll describe later.

1 Like
4

Scala2 part

In Scala2 compiler, the Typer phase will skip typing if the given tree is already typed.

Metals part

Cache compilation unit

Therefore, if the content is the same, we reuse the same compilation unit that is already typed in the previous run. That’s why Scala2 will skip typing for the given unit.

Cache purge on change

That’s how Scala2 + Metals skip typing for the same input while cleaning up the compilation unit when we modify a file in the project.

It would be great if Scala3 also has a way to skip typer (or compilation) if the given tree is already typed (maybe it does?), we can implement this (kind of) cache mechanism in Metals.

3 Likes
5

As it looks like there’s no such thing in Scala3 (skip typechecking if it’s already typed) https://github.com/lampepfl/dotty/blob/5dee30b4a2006c4e26a35b48d723e6e0b3196fed/compiler/src/dotty/tools/dotc/typer/TyperPhase.scala#L40-L51
I would go for caching the compilation on Metals.

1 Like
6

I’ve done like 8 months ago, but I ended up caching on IDE side https://github.com/scalameta/metals/pull/4225 which boosted Scala3 + Metals performance (especially when we browsing without modifying any piece of code) :tada:

1 Like