Transition from `TypeKind` to `BType`

sschaef · February 3, 2017, 2:03pm

(Copied from https://gitter.im/scala/contributors?at=589204414c04e9a44e59f65f where it seems to get lost)

Mainly a question to @lrytz: I could need some help about the new backend. In scala-ide we depended on TypeKind in our 2.11 version. While migrating to a 2.12 version it had to be replaced by BType. Please see this commit: https://github.com/sschaef/scala-ide/commit/2cd885ec495efef0e190f4d9102d36135711c517 It introduces a call to typeToBType, which unfortunately throws a NPE at runtime.

The stack trace is something along:

java.lang.NullPointerException
	at scala.tools.nsc.backend.jvm.CoreBTypesProxy.primitiveTypeToBType(CoreBTypes.scala:361)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.primitiveOrClassToBType$1(BTypesFromSymbols.scala:177)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.typeToBType(BTypesFromSymbols.scala:192)
	at org.scalaide.core.internal.jdt.model.ScalaJavaMapper.javaDescriptor(ScalaJavaMapper.scala:246)
	at org.scalaide.core.internal.jdt.model.ScalaJavaMapper.javaDescriptor$(ScalaJavaMapper.scala:245)
	at org.scalaide.core.internal.compiler.ScalaPresentationCompiler.javaDescriptor(ScalaPresentationCompiler.scala:53)
	at org.scalaide.core.structurebuilder.ScalaJavaMapperTest.$anonfun$withTargetTree$2(ScalaJavaMapperTest.scala:81)
	at scala.tools.nsc.util.InterruptReq.execute(InterruptReq.scala:26)
	at scala.tools.nsc.interactive.Global.$anonfun$pollForWork$1(Global.scala:444)
	at scala.tools.nsc.interactive.Global.pollForWork(Global.scala:418)
	at scala.tools.nsc.interactive.PresentationCompilerThread.run(PresentationCompilerThread.scala:22)

In the REPL the following code works fine:

import genBCode.bTypes._
> typeToBType(typeOf[Int])
res0: $r.intp.global.genBCode.bTypes.BType = I

But in the IDE codebase it also throws an NPE

I found out that I can fix the issue by doing a genBCode.bTypes.initializeCoreBTypes() before I access the btypes stuff but that leads to another exception later on.

Should I call initialization routines by myself or could something else be wrong?

lrytz · February 3, 2017, 2:41pm

Hey Simon

Sorry I missed your post on gitter, I’m not following it, and it seems notifications don’t work - I used to get them, no idea what’s wrong.

You definitely need to call initializeCoreBTypes before using BTypes. You probably also need to call scalaPrimitives.init, see https://github.com/scala/scala/blob/2.12.x/test/junit/scala/tools/nsc/backend/jvm/opt/BTypesFromClassfileTest.scala#L27-L28.

The two calls need to happen in a late phase (exitingDelambdafy) to ensure all symbols have properly erased types. In general, you need to be careful not to pass any forntend types to typeToBType, i.e., when you extract a type from a symbol you need to do it at a late phase.

sschaef · February 3, 2017, 4:04pm

Ok, thanks. One problem is that we need to convert Scala types and symbols to JDT counterparts. This needs to happen on the PC thread and not in the backend. Therefore exitingDelambdafy would never happen.

If I call the two init methods on the PC thread, our tests all pass but I can see this exception thrown:

java.lang.AssertionError: assertion failed: Duplicate primitive method ==
	at scala.tools.nsc.backend.ScalaPrimitives.addPrimitive(ScalaPrimitives.scala:439)
	at scala.tools.nsc.backend.ScalaPrimitives.$anonfun$addPrimitives$1(ScalaPrimitives.scala:451)
	at scala.tools.nsc.backend.ScalaPrimitives.addPrimitives(ScalaPrimitives.scala:447)
	at scala.tools.nsc.backend.ScalaPrimitives.init(ScalaPrimitives.scala:222)
	at org.scalaide.core.internal.compiler.PresentationCompilerProxy.$anonfun$create$6(PresentationCompilerProxy.scala:172)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at scala.tools.nsc.util.InterruptReq.execute(InterruptReq.scala:26)
	at scala.tools.nsc.interactive.Global.$anonfun$pollForWork$1(Global.scala:444)
	at scala.tools.nsc.interactive.Global.pollForWork(Global.scala:418)
	at scala.tools.nsc.interactive.PresentationCompilerThread.run(PresentationCompilerThread.scala:22)

Can we skip some of the stuff that happens in init, if we have to run it on the PC thread?

lrytz · February 3, 2017, 7:58pm

Given that the init method first clears the primitives map (https://github.com/scala/scala/blob/v2.12.1/src/compiler/scala/tools/nsc/backend/ScalaPrimitives.scala#L196), it looks to me like the assertion (https://github.com/scala/scala/blob/v2.12.1/src/compiler/scala/tools/nsc/backend/ScalaPrimitives.scala#L439) can only fail if there are multiple concurrent invocations of init.

In the batch compiler we invoke scalaPrimitives.init and initializeCoreBTypes once per compiler run, in the beginning of the GenBCode phase. I think you also need to do it once per run, see the explanation here: https://github.com/scala/scala/blob/v2.12.1/src/compiler/scala/tools/nsc/backend/jvm/CoreBTypes.scala#L8-L28.

dragos · February 16, 2017, 1:29pm

Maybe it’s better to byte the bullet and rewrite that method to work on symbols and types directly. It was just laziness on my part to go via TypeKind. Note that your new code is missing the call to .erasure, which should take care of returning only erased types.

PS. Calling init in every run might be a bit difficult, since the presentation compiler creates new compiler runs on its own, see here.

wpopielarski · March 7, 2017, 7:22am

Hello guys, this is pretty simile problem described by Simon above. I call initializeCoreBTypes() but then I suffer with exception above. Could you give me just a cue what is wrong?

Appreciate your response
Wieslaw

2017-03-06 17:59:37,407 ERROR [main] - org.scala-ide.sdt.core - org.scala-ide.sdt.core - org.scala-ide.sdt.core - 0 - Throwable during asyncExec
java.lang.AssertionError: assertion failed: Bad superClass for class Object: class Any
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:342)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:345)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$typeToBType$1(BTypesFromSymbols.scala:177)
	at scala.collection.MapLike.getOrElse(MapLike.scala:128)
	at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
	at scala.collection.AbstractMap.getOrElse(Map.scala:59)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.primitiveOrClassToBType$1(BTypesFromSymbols.scala:177)
	at scala.tools.nsc.backend.jvm.BTypesFromSymbols.typeToBType(BTypesFromSymbols.scala:192)
	at org.scalaide.core.internal.jdt.model.ScalaJavaMapper.javaDescriptor(ScalaJavaMapper.scala:248)
	at org.scalaide.core.internal.jdt.model.ScalaJavaMapper.javaDescriptor$(ScalaJavaMapper.scala:245)
	at org.scalaide.core.internal.compiler.ScalaPresentationCompiler.javaDescriptor(ScalaPresentationCompiler.scala:53)
	at org.scalaide.core.internal.jdt.model.ScalaJavaMapper.mapParamTypeSignature(ScalaJavaMapper.scala:217)
	at org.scalaide.core.internal.jdt.model.ScalaJavaMapper.mapParamTypeSignature$(ScalaJavaMapper.scala:210)
	at org.scalaide.core.internal.compiler.ScalaPresentationCompiler.mapParamTypeSignature(ScalaPresentationCompiler.scala:53)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$DefOwner.$anonfun$addDef$1(ScalaStructureBuilder.scala:677)
	at scala.collection.immutable.List.map(List.scala:272)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$DefOwner.addDef(ScalaStructureBuilder.scala:677)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$DefOwner.addDef$(ScalaStructureBuilder.scala:664)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$Builder.addDef(ScalaStructureBuilder.scala:879)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$DefOwner.addDef(ScalaStructureBuilder.scala:662)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$DefOwner.addDef$(ScalaStructureBuilder.scala:662)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$Builder.addDef(ScalaStructureBuilder.scala:879)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.traverse(ScalaStructureBuilder.scala:950)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2$adapted(ScalaStructureBuilder.scala:957)
	at scala.collection.immutable.List.foreach(List.scala:378)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.traverse(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2$adapted(ScalaStructureBuilder.scala:957)
	at scala.collection.immutable.List.foreach(List.scala:378)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.traverse(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2$adapted(ScalaStructureBuilder.scala:957)
	at scala.collection.immutable.List.foreach(List.scala:378)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.traverse(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2$adapted(ScalaStructureBuilder.scala:957)
	at scala.collection.immutable.List.foreach(List.scala:378)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.traverse(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.$anonfun$traverse$2$adapted(ScalaStructureBuilder.scala:957)
	at scala.collection.immutable.List.foreach(List.scala:378)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser$TreeTraverser.traverse(ScalaStructureBuilder.scala:957)
	at org.scalaide.core.internal.jdt.model.ScalaStructureBuilder$StructureBuilderTraverser.traverse(ScalaStructureBuilder.scala:907)
	at org.scalaide.core.internal.jdt.model.ScalaCompilationUnit.$anonfun$buildStructure$5(ScalaCompilationUnit.scala:125)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at scala.tools.nsc.util.InterruptReq.execute(InterruptReq.scala:26)
	at scala.tools.nsc.interactive.Global.$anonfun$pollForWork$1(Global.scala:444)
	at scala.tools.nsc.interactive.Global.pollForWork(Global.scala:418)
	at scala.tools.nsc.interactive.PresentationCompilerThread.run(PresentationCompilerThread.scala:22)

lrytz · March 7, 2017, 8:00am

@wpopielarski a quick guess: you might be calling initializeCoreBTypes in an early compiler phase, at which point symbols still have non-erased types. See comments above, also https://github.com/scala/scala/blob/2.12.x/test/junit/scala/tools/nsc/backend/jvm/opt/BTypesFromClassfileTest.scala#L24-L28