How to check for broken symbols in Macro

giiita · June 3, 2021, 3:06am

c.typecheck and c.parse have been lost.
The scalatest assertCompile and others can certainly achieve something similar by calling compiletime.testing.typeChecks directly at test time.

However, there may be cases in other libraries where typechecks are desperately needed.
The goal is to avoid a compile error due to typecheck failure.

For example, Symbol in ScalaTestAntTask in scalatest refers to org.apache.tools, but since it is not bundled in scalatest, the compilation will fail when you access it with symbol.tree or something similar.

[error] -- Error: /src/dotty-test/src/test/scala/test/MacroCallSite.scala:16:25 
[error] 16 |      val result = inject[Foo]
[error]    |                   ^^^^^^^^^^^^^^^^^^
[error]    |undefined: new org.apache.tools.ant.Task # -1: TermRef(TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class tools)),object ant),Task),<init>) at typer

If you want to test this dynamically, there is no way to do so because compiletime.test.typeChecks cannot be used.

Is there a solution to this?

kai · June 3, 2021, 10:23am

I see at least two options for this case:

Check using Class.forName directly in the macro - if org.apache.tools is on the classpath, it will also be on the classpath for the macro.
You may check for missing symbols using implicits via “no more orphans” trick. I’ve checked that it keeps working on Scala 3 and use it in my libraries instances for other, not-bundled libraries. You can use summonFrom to try to summon an implicit written in this form from inside a macro - it will be found only if a symbol is accessible.

smarter · June 3, 2021, 3:23pm

Inside the compiler the getClassIfDefined API can be used for this but it’s not exposed from tasty reflect, feel free to open a feature request at GitHub - lampepfl/dotty-feature-requests: This repo holds feature requests for Dotty, bugs reports are at https://github.com/lampepfl/dotty (or even make a PR exposing it, it should look similar to how we expose def requiredClass(path: String) currently.

giiita · June 8, 2021, 11:06pm

Thanks for the suggestion.

Check using Class.forName directly in the macro - if org.apache.tools is on the classpath

Certainly this can supplement a non-compile error exception by referring to a corrupted class symbol, but it does not seem to be suitable for verifying the module class or object symbol.

kai · June 9, 2021, 12:22pm

Object class name just has $ at the end, e.g. for scala.Int object you may check Class.forName("scala.Int$")

giiita · June 9, 2021, 1:17pm

I verified this with dotty, but it was not possible to inspect the class definition itself for corruption.
If the definition is corrupted, the typer phase will generate an unsupplementable error: https://github.com/lampepfl/dotty/blob/2ce3e97cccb558feae079a87453d712694218e3a/compiler/src/ dotty/tools/dotc/core/SymDenotations.scala#L167

giiita · June 9, 2021, 3:10pm

Suppose we declare an arbitrary trait.

package foo.bar
trait Test

At this time, we can only see Test, so Class.forName("foo.bar.Symbol") will be successful.
However, it doesn’t seem to look that way to dotty.

// List(trait Test, object Test, module class Test$)
Symbol.requiredClass("foo.bar.Test").owner.declarations

Class.forName does not seem to work for this module class

java.lang.ClassNotFoundException: foo.bar.Test$
  | => cat java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:435)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:377)

I can’t seem to access the tree either.

[error]     |java.lang.ClassCastException: class dotty.tools.dotc.core.Types$NoType$ cannot be cast to class dotty.tools.dotc.core.Types$ClassInfo (dotty.tools.dotc.core.Types$NoType$ and dotty.tools.dotc.core.Types$ClassInfo are in unnamed module of loader sbt.internal.classpath.ClassLoaderCache$Key$CachedClassLoader @1c31b2bb)
[error]     |	at dotty.tools.dotc.core.SymDenotations$ClassDenotation.classInfo(SymDenotations.scala:1712)
[error]     |	at dotty.tools.dotc.ast.tpd$.ClassDefWithParents(tpd.scala:312)
[error]     |	at dotty.tools.dotc.quoted.reflect.FromSymbol$.classDef(FromSymbol.scala:36)
[error]     |	at dotty.tools.dotc.quoted.reflect.FromSymbol$.definitionFromSym(FromSymbol.scala:18)
[error]     |	at scala.quoted.runtime.impl.QuotesImpl$reflect$SymbolMethods$.tree(QuotesImpl.scala:2443)
[error]     |	at scala.quoted.runtime.impl.QuotesImpl$reflect$SymbolMethods$.tree(QuotesImpl.scala:2443)

And yet, it’s not NoSymbol.

Symbol.requiredClass("foo.bar.Test").owner.declarations.map(_.isNoSymbol) // List(false, false, false)

===
And we also confirmed that there is damage that cannot be detected by Class.forName(...).
As mentioned above, a class with a bad reference, such as ScalaTestAntTask, could be supplemented with NoClassDefFoundError, but if the corruption is internal to the class, such as org.scalatest.tools.ScalaTestFramework, the Class.forName(...) does not seem to work either.

Class.forName("org.scalatest.tools.ScalaTestFramework") // success
Symbol.requiredClass("org.scalatest.tools.ScalaTestFramework").tree // Compilation fail

java.lang.Exception: Stack trace
  | => cat java.base/java.lang.Thread.dumpStack(Thread.java:1379)
	at dotty.tools.dotc.report$.error(report.scala:72)
	at dotty.tools.dotc.typer.ErrorReporting$.errorType(ErrorReporting.scala:34)
	at dotty.tools.dotc.typer.TypeAssigner.assignType(TypeAssigner.scala:290)
	at dotty.tools.dotc.typer.TypeAssigner.assignType$(TypeAssigner.scala:19)
	at dotty.tools.dotc.typer.Typer.assignType(Typer.scala:106)
	at dotty.tools.dotc.ast.tpd$.Apply(tpd.scala:48)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1133)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1292)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.$anonfun$4(TreeUnpickler.scala:927)
	at dotty.tools.tasty.TastyReader.collectWhile(TastyReader.scala:137)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTemplate(TreeUnpickler.scala:930)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:857)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:776)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:122)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:167)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
	at dotty.tools.dotc.core.Types$NamedType.info(Types.scala:2174)
	at dotty.tools.dotc.core.Types$TermLambda.dotty$tools$dotc$core$Types$TermLambda$$_$compute$1(Types.scala:3528)
	at dotty.tools.dotc.core.Types$TermLambda.dotty$tools$dotc$core$Types$TermLambda$$depStatus(Types.scala:3541)
	at dotty.tools.dotc.core.Types$TermLambda.dependencyStatus(Types.scala:3555)
	at dotty.tools.dotc.core.Types$TermLambda.resultType(Types.scala:3494)
	at dotty.tools.dotc.core.Types$TermLambda.resultType$(Types.scala:3486)
	at dotty.tools.dotc.core.Types$MethodType.resultType(Types.scala:3600)
	at dotty.tools.dotc.core.Types$TypeMap.mapOverLambda(Types.scala:5274)
	at dotty.tools.dotc.core.Types$TypeMap.mapOver(Types.scala:5304)
	at dotty.tools.dotc.typer.Checking$NotPrivate$1.apply(Checking.scala:610)
	at dotty.tools.dotc.typer.Checking$.checkNoPrivateLeaks(Checking.scala:614)
	at dotty.tools.dotc.typer.TypeAssigner.avoidPrivateLeaks(TypeAssigner.scala:47)
	at dotty.tools.dotc.typer.TypeAssigner.avoidPrivateLeaks$(TypeAssigner.scala:19)
	at dotty.tools.dotc.typer.Typer.avoidPrivateLeaks(Typer.scala:106)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:889)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:776)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:122)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:167)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
	at dotty.tools.dotc.ast.tpd$.DefDef(tpd.scala:284)
	at dotty.tools.dotc.ast.tpd$.DefDef(tpd.scala:227)
	at dotty.tools.dotc.quoted.reflect.FromSymbol$.defDefFromSym(FromSymbol.scala:46)
	at dotty.tools.dotc.quoted.reflect.FromSymbol$.definitionFromSym(FromSymbol.scala:21)
	at dotty.tools.dotc.quoted.reflect.FromSymbol$.$anonfun$4(FromSymbol.scala:35)
	at scala.collection.immutable.List.map(List.scala:250)
	at dotty.tools.dotc.quoted.reflect.FromSymbol$.classDef(FromSymbol.scala:35)
	at dotty.tools.dotc.quoted.reflect.FromSymbol$.definitionFromSym(FromSymbol.scala:18)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$SymbolMethods$.tree(QuotesImpl.scala:2445)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$SymbolMethods$.tree(QuotesImpl.scala:2445)