I’m not sure how that would work but I think that would be very helpful.
I felt that was a great articulation of the problems with Scaladoc. Typically it’s up to the developer to “discover” the type classes they can use.
As a side note, I love and use this reference all the time, maybe I’m more of a visual person:
1 Like
I acknowledge lack-of-support support for TCs in Scaladoc til now is not a “lack of will” thing, but because it’s difficult. I’ll try to spell out in more detail how it could work…
Rather than the documenting methods available everywhere, unconditionally, the docs need to make a conditional statement:
when the import cats.implicits._ from org.typelevel % cats %% 2.1.0 is available, Int has a method |+|
There are many such statements for many different library/import contexts. So the idea of one API, known by the library creator, has to go away. Replaced by a series of contexts: when import X from library Y is in scope, methods Z1…Zn are available
TCs “superhero power” is to enable retroactive extension. Extending other people’s code after its written. But that also implies that API docs need to be extended, after they’re first defined.
HTML alone doesnt cut it. Docs probably need to include a series of structured modules of JSON data, that are collated dynamically to render the doc page. Each module describes one set of conditional methods. Additional JSON modules can be added into each directory, after publication, resulting in new methods appearing in the docs from a given context.
The API docs render as a list of contexts, which can be toggled off and on individually. Not using Cats library? Then turn off it’s methods…
3 Likes
If if understood correctly then it would be a very nice feature to have.
The data could be published by the libraries and downloaded by the doctool, which would only need to know about, for example, the library’s git repo url, or its name on Maven Central. 
It could also provide a library search based on Scaladex.
@benhutchison I think even a simpler version where a project can specify imports based on which extension methods should be added to its classes would be useful; it’s also more within reach. It should be feasible to implement such a feature with a reasonable amount of effort, just based on TASTy APIs. This is also a good opportunity to get ahead-of-time experience with TASTy, which is going to be the macro API when Scala 3 rolls around. I created an issue for this: https://github.com/lampepfl/scala3doc/issues/151, contributions are welcome!
1 Like
This is exciting! Thank you for sharing this update.
Has it been discussed whether the markdown documentation in Scala3doc should support executable examples? Similar to https://github.com/tpolecat/tut and https://scalameta.org/mdoc/
```scala mdoc
val x = 1 + 3
// x: Int = 4
```
It would be nice if the compiler could support this feature out of the box instead of relying on 3rdparty tooling.
15 Likes
We’d like to support executable code blocks both in doc comments and in Markdown files, yes. There’s some nice possibilities - we could, for instance, hyperlink the types in code snippets (again, Cats docs would really benefit from this). There are also benefits from a tighter integration with the compiler - we could, for instance, make incremental compilation compile both the actual source files of a project and the executable blocks. Ideally, we’d also support extensions that alter what compiled code blocks render to - for instance, an extension could render them to interactive online REPLs. I see mdoc has a similar idea and supports extensions along these lines.
This is all in a very early stage, though - for now we really need to focus on Scaladoc parity.
2 Likes
Actually @olafurpg: if you’d like to, it’d help if you could open an issue on Github about adding support for executable blocks and describe your experiences with mdoc - what features were actually important in your opinion, what made the tool work well, etc. It’d provide a nice starting point for the discussion on how we should support executable (compiled?) code blocks in Scala3doc.
3 Likes
Possibly a stupid question but… why not use mdoc? Its basically a pre-processing step of Markdown files, no?
1 Like
I didn’t think of that! If mdoc outputs Markdown files then you’re right that we could stack the tools like that. I’m not sure if we’d be easily able to hyperlink types in the code snippets this way though, and I don’t think we’d be easily able to support compiling code blocks in doc comments.
One observation here - Dokka actually can output Markdown files as an alternative to HTML files. If one wanted to, one could output the entire documentation as Markdown files and afterwards run mdoc on those files, uniformly transforming code blocks in standalone Markdown files and in doc comments. Just food-for-thought.
Maybe it make more sense to run an mdoc-like tool om the doc comments before the documentation engine even runs. Orchestration could then be done by the build tool. That may make it easier to provide the user with location information for the errors without having to create some sort of sourcemap.
2 Likes
Just to make it clear: I think that ideally, mdoc-like functionality should be integrated into Scala3doc. This’d mean that we can provide additional features (such as hyperlinking types in code blocks) and we are guaranteed to integrate well, i.e. we expand code blocks in doc comments and Markdown files in exactly the same way.
If we won’t be able to do that (and given the time constraints, we probably won’t implement it on Scala 3 release), someone from the community will still be able to integrate the two tools together, either by running mdoc on MD files and feeding them as input to Scala3doc, or by outputting Markdown from Scala3doc and running mdoc on the result - whichever one finds more preferable.
2 Likes
I’m not very concerned with the use of a third party renderer for the doc personally.
But I am somewhat concerned in that the doctool consists of a significant part of kotlin code and requires orchestrating a gradle build of the kotlin code as an sbt task. I’m worried that once the tool is more or less complete and the original authors will have moved on, finding people willing and able to maintain a tool that leverages scala, sbt, kotlin and gradle is going to be hard.
Piecing it all together is also hard: I haven’t been able to find API documentation for dokka and its plugin infrastructure that’s being leveraged either. I’m sure you could just read the source, but that that source is not scala also presents an extra hurdle.
4 Likes
I agree that depending on a large Kotlin codebase is a problem for core Scala tooling.
What I can tell you is that:
- Plugin support in Dokka is still in early stages and because of that, there’s little documentation. However, as far as I know, there are plans to fix this.
- The gradle project is necessary because some Dokka classes are only extensible with Kotlin syntax. Again, there are plans (from people who are working on Scala3doc) to fix this by making said classes extensible from Java.
- In the absolute worst case, we should be able to rewrite Dokka to Scala. The codebase is written in a style which isn’t far off from Scala, so it should “just” be a mostly mechanical translation. It’s not going to be effortless, but the option is there.
In any case, I’d like to emphasize that we benefit a lot from integrating with Dokka. We do pay some costs, yes, but we benefit from ongoing maintenance of Dokka and from the work which already went into making it a good documentation rendering engine. Dokka already supports features such as alternative output formats (HTML, Javadoc HTML, Markdown), documenting Java sources, and support for multiplatform code. Adding those features to Dottydoc on our own would be very much a non-trivial task, but with Scala3doc depending on Dokka, it looks perfectly within reach for post-release improvements.
1 Like
If there are guarantees that the people who wrote it so far will keep it maintained up to the point where those things have changed, then my worries are relieved. Are there though?
I genuinely do not know what to tell you other than: yes, it’s in the interest of every person working on Scala3doc right now to have it succeed, which includes making it as easily maintainable as possible. In particular, it’s in the interest of Dokka maintainers (some of whom are working on Scala3doc) to have Scala3doc succeed - both from a personal perspective and from the perspective that sees Scala3doc as a success story for Dokka plugin architecture. Seeing as said maintainers have written the Gradle sub-project and told me that they’d want to integrate it into Dokka, I think that it will actually be integrated into Dokka.
5 Likes
Hope it can make use of mdoc and scalafiddle
https://scalafiddle.io
and I like the elixir style too
https://hexdocs.pm/elixir/String.html#content
1 Like
I have remembered that a Scala tool already exists for transforming Markdown to formatted documentation. I have used it in conjunction with MDoc. It is called Laika. Documentation can be found here.
My question is, wouldn’t using Laika solve most of the issues. I think it would then be a question of automatically generating Markdown files for the API and passing that through Laika. Maybe this could be the first phase in the pipeline. Additional text can be added manually (Markdown and/or HTML). And code snippets are taken care of in a second phase via MDoc.
Of course this means an external dependency, but its Scala.
3 Likes
On the topic of documenting Scala, I think Metabrowse is a tool with huge potential and is a bit under-appreciated at present.
It’s a source code browser based on scalameta that properly understands the structure of Scala code and supports IDE-style click-navigation. Demo showing the standard lib.
Due to the limitations of Scaladoc around typeclasses and implicits (which may well remain in Scaladoc3), I rarely find Scaladocs useful for functional scala (eg Cats and friends).
Rather I typically read sourcecode to figure out APIs. I imagine this is a common experience. I envisage I would get a lot value from making Scala sources easier to navigate.
Eventual github integration is nirvana, but source navigation via doc-sites would also be valuable, especially if various ecosystem sites cross-linked, which I believe metabrowse is capable of.
7 Likes