Pre-sip scaladoc: search by type signature

Tooling
#1

I find myself using the search functionality in scaladoc quite a lot - one of the things I think would be invaluable would be to have the facility to search by type signature. Especially in larger libraries, I can’t always remember the name of a function, but I do know what type it takes as an input or output (being a functional programmer :slight_smile: ). Wouldn’t it be useful to be able to search by the type?

As an example, typing in Codec[Boolean] might filter all methods that take such a value as an argument or output. It would also be nice to be able to be even more specific by saying e.g. returns:Codec[Boolean].

Currently writing in a type (e.g. Unit) doesn’t match anything.

7 Likes
#2

Basically Hoogle-like functionality, yeah. I find myself wanting this all the time. Even within a single type, I often want to narrow down the members by signature…

5 Likes
#3

There’s something similar here: http://scala-search.org/.

I’d say this does not require a SIP, but rather someone that is willing to analyze the existing solutions and has the time to open a PR to scala/scala with a solid implementation.

I personally think that Scaladoc should be made independent from the compiler and be implemented in terms of tools like Scalameta (reading type information from the Semanticdb). So, if more work is to happen in this area, I would encourage contributors to try to implement it on their own :slight_smile:

4 Likes
#4

One annoyance one might run into, I suspect, is subtyping vs unification. Hoogle can use unification, and I’d guess it needs to (though I haven’t read the papers on which it’s based), but unification and subtyping don’t play nice with each other.
IIUC Dotty implements a constraint solver for subtyping (which I think is more robust), while Scalac does something else. Maybe you can stick to something Hoogle-like and pretend subtyping doesn’t exist, but then searching for List[A].?(A => List[B]): List[B] wouldn’t find List[A].?(A => Traversable[B]): List[B].

Martin also mentioned that Dotty’s constraint solver has some known incompleteness to avoid exponential slowdown, I don’t know if search should have the same tradeoffs; but I expect a first implementation should ignore such questions, otherwise this becomes a topic for a whole PhD.

#5

There is a related project built with Scalameta: Metadoc, a code browser with some IDE-like features. I think it could be useful for building a semantic search tool proposed here.

3 Likes
#6

Thanks for all the replies - learned about some really cool tolls out there. I particularly like how Hoogle allows you to enter the signatures … that would make a lot of sense for scala too Int => Codec[Bool] for example.

In terms of the implementation, my understanding is that hte current search functionality in scaladoc API is just some javascript in the generated html pages? Is that not correct?

I don’t believe there is any need to do any parsing or syntax analysis.

There are lots of great ideas here, but aren’t they a bit of overkill for something that you could probably implement by tweaking the js regex for the search functionality (assuming that’s how the search box works?)?

#7

I don’t know how Scaladoc works internally. I suggest you have a look, if you find a way of implementing this I think it’s worth a PR.

#8

Some things might be doable through plain regexes. What Hoogle does certainly not - you can write Int => Int and find the identity function (since it’s more general but can do what you want). But I agree a simple search would be better than nothing!

#9

Hey all. Sorry to revive this old thread from simpler times.

I really feel this is something that could be very useful as well. Given that it has been 3 years, does anyone know of new developments along these lines?

(I’ll be coming back to Scala after some time in Haskell/PureScript land, and I’d certainly do what I can to make this a reality).

6 Likes
#10

Scalameta and/or Semanticdb might be helpful here (and didn’t exist until recent years)

3 Likes
#11

This is for Kotlin, but written in Scala by some of the same folks working on scaladoc for Scala 3:

https://medium.com/virtuslab/how-to-write-hoogle-for-kotlin-in-scala-and-scala-js-8c98c1c303ff

Therefore, having Hoogle for Kotlin is like proof of concept for having a similar tool in Scala3 world.

3 Likes
#12

We’re in the process of analyzing the amount of work needed to implement/adapt Inkuire for Scala 3.

8 Likes
#13

Really awesome to hear about Inkuire being well placed to tackle this, @griggt and @KacperFKorban .

A few questions - looking over the issue tracker, it seems there isn’t a discussion of the mentioned evaluation of work needed to support Scala 3. Did I miss it, or would that be helpful?

I did see Migrate to Scala3 · Issue #169 · VirtusLab/Inkuire · GitHub, however, I assume that’s referring to migrating the codebase to Scala 3. If that’s the case, which would be better off to tackle first?

I also wonder to what extend non Scala-3 libraries might be supported. Down the road, it might be useful to have a way to crawl through JVM bytecode to extract signatures, for those interested in it (I realize not everyone will be, as not everyone is using Scala for the JVM).