IIUC:
We can use Tasty inspector to process tasty files. But tasy inpector run compiler with several phases. It seems redundant for single file.
Why there are no way just to use tree from DottyUnpickler?
I think the right™ way to read TASTy is to use:
It doesn’t come with the whole compiler.
But AFAIK you need the whole classpath (even including JRE) from where the TASTy files originate. Why it’s like that, I don’t know. I guess you need to ask @sjrd.
Just using parts of the compiler does usually not work because everything depends on a “Context”, which is as far as I grok that a kind of fully configured and running compiler instance. Especially all operations that do anything with trees will always need “Context”. (The people with the knowledge please correct me if I’m wrong! I don’t know much about compiler internals)
1 Like
Tasty Query is thread-safe and immutable. It also works in Scala.js
IIUC:
When it creates a context, it scans and loads information about all class files, so it needs the full set of classpaths.
If that’s the case, then it’s not very good for reading several files.
I was actually talking about the “Context” in the Scala compiler. The compiler “Context” is little bit more involved than the one in TASTy-query, but the architectural idea is likely the same. (?)
Like said, as far as I know TASTy files can’t be read without the whole classpath. Detached TASTy files in general can’t be interpreted in a meaningful way. At least that’s what @sjrd said on a few occasions.
To be honest, I also think that’s somehow strange. Why can’t one just decode what’s inside a singular TASTy file, and just ignore links to external entities?
1 Like
In osgi bundles classpaths is quite complicated so I am not really sure that I can provide single context for all queries.
Presumably(I am not sure) I can make correct tasty query context for each query but, full scan is quite expensive in such case.
Try first, and complain after you measure it to be too slow? It’s not too slow for tasty-mima, nor for the other use cases I used it for. The Scala 3 debugger in Metals uses it as well, as it’s not too slow there as well. So perhaps it’s not actually too slow for your use case either.
There’s nothing meaningful you can do with a .tasty file in isolation. You need a full classpath. That’s just how it is.
Could you expand on that maybe a little bit?
That is also the part that I don’t understand. Why is a TASTy file in isolation “meaningless”? It has some bytes, and these byte can be decoded, no? Depending on what one wants to look up, why isn’t this enough?
I could imagine that one could for example get a list of class names defined in the corresponding source file, even without knowing anything about the rest of the class path. I would guess there are more such cases where at least some useful information can be extracted. Or where is the fault in my reasoning here?
I’ve used it once, and it was pretty fast. It’s not the compiler! 
In case of Scala 3 TASTy files you read an optimized binary format, which has almost all the info pre-computed, which is the whole point of TASTy. Such binary parsing is usually fast. (I think it needs to re-compute match types, maybe something more, and in case of Java classes these need to be scanned. But that’s quite fast I think.)
Try first, and complain after you measure it to be too slow?
If I can speed up requests in 3 times, will it worth it. The normal speed is a very difficult question. Is metals fast enough for a large project(4M + LOC)?
Necessity to gather full correct classpaths in an osgi environment makes a program fragile. How much does stability cost?
There’s nothing meaningful you can do with a .tasty file in isolation
IMHO: I can. For examle I can use macros to get type hierarchy and I can use isoltated tasty file to gather metadata from annotations and documentaion later it can speed up compilation up to 20 percent if my measurements is correct. Is it worth it? I think it worth it. Compilation speed is the main our headache.
I will try to use it. Thank you. I am just not sure I can provide correct classpaths.
IMHO it is not easy for an osgi environment.
This for sure true.
That’s actually an interesting question. I don’t know how TASTy-query would handle all the OSGi classloader magic. If it does at all?
But what do you actually want to do? From the previous posts it’s not really clear.
I guess I say nothing new when I say “modularization” and “caching”, right? 
But have you tested Bazel? It is said that it shines in the above two points, and can help especially with large and very large code bases. But that’s just what I’ve heard. I didn’t use it myself until now. It makes no sense in small or middle sized code bases. It’s quite complex and that only pays off when you really have a lot stuff to compile. But I think with 4 MLOC it could be worth some experiments.
1 Like
I have 2 main tasks.
- generate description of some classes in runtime(to speed up build)
- to speed up compilation by processing ast tree later
there is a slight misunderstanding, if you only want to traverse tree’s in some singular tasty file, then it is possible to have a small classpath, you just can’t perform semantic operations
2 Likes
tasty query doesn’t understand j.l.ClassLoader, its default loading mechanism requires you load all the classfiles ahead of time into byte arrays - but the interface does allow lazy loading so you could wrap URLClassLoader if you want (or however osgi does it)
why? because scala.js doesn’t support classloader so instead it interfaces with a very minimal virtual filesystem
2 Likes
It would be wonderful!
Is such behavior officially supported by the library? I mean is it true that this behavior will not be changed in the future?
Yes, that’s the methods that don’t take a (using Context) argument. Since they don’t have a context, they literally cannot look at other files, so it’s guaranteed not to change in the future by the very type system.
But there’s not much you’ll find there that you wouldn’t be able to get from parsing the source file with ScalaMeta. You can get the “surface syntax” of inferred types, but that’s pretty much it.
2 Likes
It works really well, except it seems there are no way to read documentation. I coudnt find something like docstring in scala 3 reflection.