What’s wrong with TypeTag? I’ve used it quite a bit and I’m not aware of any issues.
Nothing is wrong with it. The problem is that it won’t exist in Scala 3 anymore.
1 Like
Kinda figured it out, looks very similar to TypeTag.
2 Likes
- It has some serious concurrency issues, like it may show you
Set[Int]asSet[String] - It has a very obscure and esoteric API - essentially TypeTag is a set of internal compiler APIs and data structures available to the user
- It is not available in Scala 3 in any form (yet hopefully)
3 Likes
This may be an issue where perfect is the enemy of good.
What you’re suggesting is that a simplified runtime type system (for 80% of runtime type checks) is something you still want for your use case. scala-reflect meant to be a 100% solution, and wound up having LOTS of issues because of it. TypeTag is actually one of those issues.
The goal in Scala 3 is what we had before scala-reflect. For use-cases where you only need an 80% solution, you should be able to accomplish that with straight up Java reflection. If you need more, TASTY can provide you the basics. However, we don’t think a 100% solution is something folks need, and it’s unclear if there should be a “core” implementation that is not 100%.
The library you aimed to provide is an example of what we’d like to see come out of dropping scala-reflect. It’d be nice to have a runtime-reflection library that does not implement the EXACT scala type system, but is good enough for the 80% case. Additionally, it should fully document where it “cheats” and what limitations it has. You may have a good start at something that the community could build on for reflection. My only comment is that you might be able to make use of the TASTY data encoded in the class (plus some Java reflection) to grab your type tags from any object, without even needing to grab them in a macro if you wanted to be more reflective (although personally I prefer a tagged approach).
2 Likes
Scala 3 has the quoted package, with quoted.Expr as a representation of expressions and quoted.Type as a representation of types. quoted.Type esentially replaces TypeTag. It does not have the same API but has similar functionality. It should be easier to use since it integrates well with quoted terms and pattern matching.
2 Likes
What you’re suggesting is that a simplified runtime type system (for 80% of runtime type checks) is something you still want for your use case.
Not just this. There may be different views on the problem (including “nothing needs to be done”), so I just wished to make a discussion and explore what people think.
scala-reflect meant to be a 100% solution, and wound up having LOTS of issues because of it. TypeTag is actually one of those issues.
Actually scala-reflect was never an 100% solution - there were issues with PDTs at least - they were just unsound (and most likely there was no way to make them completely sound).
For use-cases where you only need an 80% solution, you should be able to accomplish that with straight up Java reflection.
Unfortunately no. Java reflection can’t even handle erasure, but in distage we need things like type tags for unapplied types - which even scala-reflect cannot provide out of the box.
If you need more, TASTY can provide you the basics.
But from what I can understand there is no way to access/manipulate TASTY data at runtime for now?
It’d be nice to have a runtime-reflection library that does not implement the EXACT Scala type system, but is good enough for the 80% case.
Okay, I made one and it is good enough to be useful…
Additionally, it should fully document where it “cheats” and what limitations it has.
… and it’s not a problem to document the pitfalls - it doesn’t handle PDTs properly and it ignores type boundaries. But I’m relying on undocumented behaviours! Actually I made a pretty sane support for boundaries but a necessary undocumented behaviour was broken in 2.13
you might be able to make use of the TASTY data encoded in the class (plus some Java reflection) to grab your type tags from any object
Could you give me any hint where should I look at to explore this?
1 Like
Thanks for pointing out. Though from what I can see at this point (0.17/0.18) Type does not provide any functionality. Also the following simple example works on 0.17 but doesn’t compile on 0.18, it can’t find QuoteContext for some reason:
import scala.quoted
import scala.quoted._
object Main {
def tag[F[_]](implicit t: quoted.Type[F]): String = {
t.toString
}
def main(args: Array[String]): Unit = {
println(tag[List])
}
}
Could you give me a hint where may I read/discuss future plans for this functionality? Also - is it going to be possible to get a Type instance for an unapplied type (List[_]) and then apply it to another type (Int)?
In general - seems like this is what we need, it just needs to be improved and equality/subtype checks needs to be implemented.
4 Likes
This QuoteContext context that is missing contains the reflection logic (i.e. some compiler logic). If you use it at runtime you can get a QuoteContext within a run or withQuoteContext (see http://dotty.epfl.ch/docs/reference/metaprogramming/staging.html#api). These will move to a different package soon, but no other API change will happen for now.
The code you wrote could be updated as follows
import scala.quoted._
// import scala.quoted._ // probable new package name for `run`, `withQuoteContext` and `Toolbox`
object Test {
delegate for Toolbox = Toolbox.make(getClass.getClassLoader)
def tag[F[_]: Type] given QuoteContext: String = {
the[Type[F]].show
}
def main(args: Array[String]): Unit = {
println(withQuoteContext { tag[List] })
}
}
It is possible to apply List[_] to Int. For example we could adapt the previous example to
def tag[F[_]: Type] given QuoteContext: String = {
val appliedF = '[F[Int]]
appliedF.show
}
The subtype check is currently possible trough the low level TASTy reflect API. Though it would be possible to have it as well in directly on quoted types. For example we could implement it like
import scala.quoted._
import scala.quoted.staging._
object Test {
delegate for Toolbox = Toolbox.make(getClass.getClassLoader)
def isSubtype[T1, T2] given (t1: Type[T1], t2: Type[T2]) given (qctx: QuoteContext): Boolean = {
import qctx.tasty._
val tpe1: qctx.tasty.Type = t1.unseal.tpe
val tpe2: qctx.tasty.Type = t2.unseal.tpe
tpe1 <:< tpe2
}
def main(args: Array[String]): Unit = {
println(withQuoteContext { isSubtype[List[Int], List[Any]] }) // true
println(withQuoteContext { isSubtype[List[Int], Vector[Int]] }) // false
}
}
6 Likes
That’s very interesting, thank you! Is there any chance it may work on ScalaJS (once it’s ready for Scala 3)?
1 Like
It would require a large subset of the compiler to work on on ScalaJS. That might be difficult or even impossible. Form the staging framework side it might be possible to use withQuoteContext to analyze the types could potentially work after I remove some reflective calls. But that would never work for run.
What would block you from using macros instead?
3 Likes
What would block you from using macros instead?
Complexity and correctness problems. As I said before, my macro works, but I don’t think it’s a right way to go.
1 Like
Please consider seriously providing scala-reflect in Scala 3. ScalaJack is a good example of a deeply reflective project/use-case (provides frictionless JSON serialization). It reflects deeply on Scala classes to understand their structure, types, and lots of other info in order to best understand how to serialize to/from JSON and other formats.
This information cannot be accomplished compile-time. I honestly can’t say if it can be done w/macros or not. (I can’t do it with macros at this point, as my understanding of them is not solid.) I’m open to learn, but do harbor suspicions. ScalaJack’s real power is that it doesn’t know what you’re going to throw at it until runtime. Today ScalaJack lives 'n dies by the TypeTag, not just equality checking (we use that too).
BTW, I’m 100% ok cleaning up scala-reflect to make it a little more friendly for use case such as mine. It works great, but sometimes it does feel like drilling for oil trying to divine the structure of complex types.
Simple example:
trait Pet{ name: String }
case class Cat(name: String, isPurring: Boolean) extends Pet
case class Dog(name: String) extends Pet
case class Pets(all:List[Pet])
val pets = Pets(List(Cat("Slinky",true),Dog("Spot"))
val sj = ScalaJack()
val js = sj.render(p)
// renders: {"all":[{"_hint":"com.mypackage.Cat","name":"Slinky","isPurring":true},{"_hint":"com.mypackage.Dog","name":"Spot"}]}
val orig = sj.read[Pets](js) // builds the original object hierarchy using the hints
ScalaJack doesn’t know until runtime the concrete types of the pets in the list until runtime, and it needs this to properly handle the type, in this case rendering a type hint field into the JSON.
This is a very simple example but this sort of thing is used extensively to support ScalaJack’s functionality over arbitrarily complex input types. Don’t even get me started on the internal gymnastics required to untangle all the different collections, especially now that CanBuildFrom is deprecated! Let’s just say that’s deep into the TypeTag weeds.
3 Likes
==> Update: I’ve spent a few days reading through the Macros and related docs for Scala 3. I’m still unclear now these mechanisms would support my use case. The essence of the “Pet problem” is this: (at runtime) given a list of Pet, generate JSON for each concrete Pet class, along with type hints. This means knowing the concrete type of each Pet in the list at runtime, and then extracting the case class constructor fields for these types. If I’m wrong and the new mechanisms can do that, can anyone point me in a good direction/sample?
A minor update:
- Izumi LightTypeTag has been published as an independent project under ZIO umbrella.
- We are going to port it to Dotty in foreseeable future, seems like there are enough APIs already.
Our TypeTag proven to be good enough for most of the possible usecases, so seems like it’s okay not to reimplement scala-reflect again in dotty 
7 Likes
I have a use-case in Quill which is very similar to your Pets example. If this LightTypeTag can be used to summon implicits and be passed around at runtime, I would be glad to use it. For Dotty-Quill, I specifically want to separate out Quotation from db-specific Contexts but that means I need to pass the types from quoted blocks (e.g. val quotedStuff = quote { stuff }) to executions (e.g. run(quotedStuff)). When quotedStuff is inline, I can just get types from the Tasty tree, when quotedStuff is runtime (e.g. for dynamic queries) that approach doesn’t work and the only way to go is to have runtime-passable TypeTags… which Dotty currently doesn’t have.
1 Like
I almost made LightTypeTag working, see https://github.com/7mind/dotty-typetag-research
Though there is a blocker: https://github.com/lampepfl/dotty/issues/8514
Any help will be appreciated!
2 Likes
Currently we have several blockers for complete izumi-reflect implementation (#8520, #8514, #8764). I really hope they can be fixed in foreseeable future.
Though right now our last release correctly handles =:= and <:< works for many simple cases when no type constructors with variant argument are involved.
2 Likes
tasty-reflect development is short-handed. If you think there’s a useful internal compiler method that should be exposed in the tasty-reflect API like baseClasses, I suggest making a PR yourself to add it instead of waiting for someone else to do it, here’s an example of how to add something to tasty-reflect: https://github.com/lampepfl/dotty/commit/7e8014ebb26d54bf3c589321e4f2c83de44339a0
3 Likes
Hi @pshirshov. I noticed that some of these issues have been resolved. Are there any updates around Type-Tag for Scala 3?