Scala3doc - doctool for Scala 3

Scala 3 Release Projects
42

Hello everyone,

I think everyone’s due an update on the status of the doctool.

I’m happy to announce that we have actually already hit all three major milestones mentioned in the initial message!

That is, Scala3doc:

  1. was released together with Scala 3!
  2. has replaced Dottydoc as the engine for generating Scala 3 blog and documentation
  3. can document all Scala 3 definitions/types/modifiers, both old and new
  4. can document Scala 3’s standard library

If you want to try Scala3doc out, you can do that simply by adding useScala3doc := true to your SBT build, as long as you use at least Scala3 M1.

You can also see Scala3 website generated by Scala3doc on https://dotty.epfl.ch.

In addition, we also nearly on par with Scaladoc in terms of features! Which is to say that we:

  1. list implicitly-available definitions
  2. support Scaladoc-like member filtering (by name or by modifiers)
  3. render an inheritance graph for classes
  4. display known children, as well as class ancestors in initialization order
  5. have (nearly) Scaladoc-compatible definition linking

The only major Scaladoc features which we don’t yet have are @define variables (and we have a plan for those) and member grouping.

While there are still bugs to be fixed and visual improvements to be made, we will be able to release Scala3doc’s RC1 together with Scala3’s own RC1 with some time still to spare. This means that we’ll be able to spend some of our effort on planning what role exactly should Scala3doc plan in the wider Scala3 ecosystem. Look forward to a blog post on the matter!

6 Likes
43

Since we’ve announced Scala3doc many people have raised various concerns, asked questions and suggested ideas. We are incredibly grateful for it.

Scala3doc is much more mature than a few months ago and we are finally in a position to address raised points.

Dokka has a bad reputation, is slow and may not cover all Scala features

So far, we are able to document all features in Scala 3 using Scala3doc (that includes match types, givens and other features that don’t have any equivalents in Kotlin). In order to do that, we do not try to fit Scala constructs into the Kotlin model but rather create our own model. We still use some parts of the Kotlin model (function parameters, generics etc.), but we plan to migrate that away into our model as well (by properly implementing Signature and adding API there).

The latest releases of Dokka (1.4.x) are stable and well-received. Major Kotlin libraries are starting to adopt Dokka as well (e.g. kotlinx.serializaion ).

In terms of performance, it takes under 20 seconds to generate dotty.epfl.ch once code is compiled. It is also worth mentioning that we haven’t spent any time optimizing Scala3doc so far.

Dokka is written in Kotlin and it is outside our control

In the current version, we do not need any Kotlin code within Scala3doc. Previously static documentation was handled by dokka-site that was also written in Kotlin, but it was migrated into Scala3doc (and rewritten to Scala).

We still depend on Dokka and we do not plan to remove that dependency so far. However, we adopt a policy to isolate areas where we use APIs and constructs that are Dokka specific to have as little coupling as possible. In case something really bad happens on the Dokka side, these measures should make migration away from Dokka a much easier task.

Using existing tools in Scala3doc

Many great tools were suggested that Scala3doc can use (mdoc or Laika to name a few). We would love to have the freedom to use it within scala3doc. However, we are a part of the compiler ecosystem and we should avoid any Scala dependency. Depending on another Scala library is troublesome when releasing a new version of the language (to release Scala3doc for the new language version, we’d first need to migrate our dependency. Because of that, we were forced to give up all our Scala dependencies as well. This means that we needed to e.g. create a really trivial clone of scalatags.

We are aware that there is a need in the community to expand or consume documentation. We plan to introduce the ability to provide extensions for Scala3doc. However Scala3doc is not mature enough to provide any more details now.

We would love to hear from people interested in writing extensions what kind of extension point they would want to consume.

We are considering building in some features known from mdoc, scasite/scalafiddle, tut, Laika, metabrowse and other tools. Please let us know if there are other tools that we should take a look at.

Also, please let us know if there is an interest in consuming documentation as javadoc or .md files.

Documenting implicitly provided members

Scala3doc is currently able to document members implicitly provided by:

For now, we document mostly trivial cases, however, in some cases it may not be accurate. The current implementation is mostly a proof of concept, mainly to find a proper way to display such members from a UX/UI perspective.

We are actively exploring how we can provide full and precise information, including entries provided by dependencies as well. We hope that this mechanism combined with abilities similar to unidoc sbt plugin will make documentation more usable in the context of libraries like cats.

Documenting implicitly provided members is not a trivial problem so we can’t promise that we will be able to come up with a satisfying solution.

IDE integration

We are in touch with Scala Intellij Plugin and Metals developers to provide support for new syntax of documentation comments as well as we are exploring ways how Scala3doc can enhance the developer experience.

Review and community approval of the design

Initially, we decided to come out with our design of how documentation generated by Scala3doc looks and behaves. We wanted to create a usable version of the documentation that covers all features quickly. We also wanted to stabilize the way we create and render pages.

We are quite close to that goal and we have tooling in place that will allow us to quickly prototype new designs. We are really close to inviting everyone to review and discuss how future documentation for Scala 3 should look and behave (remaining issues are here).

8 Likes
44

Great work :clap:, looks promising.
I am still concerned about performance though. Someone tried to migrate Atrium (Kotlin library) during hacktoberfest to the new version and failed (see also https://kotlinlang.slack.com/archives/C0F4UNJET/p1603915431040000?thread_ts=1603915431.040000&cid=C0F4UNJET) if he included more than 27 submodules. Dokka always ran into a GC overhead (or similar). I didn’t not look into the issue myself but that sounds like the old dokka :pensive:

1 Like
45

I know that It took us quite a long to answer but a lot happened and I wrote 3 different versions of this reply but before each version was ready something changed and I needed to write it from the beginning.

So we’ve just refactored scala3doc so it does no longer depend on Dokka/Kotlin. In meantime, we’ve started to building docs for projects in community build and applies few simple optimizations in doctool itself (there is quite a lot work there still to minimize both time and size of generated docs).

Today it takes around 2 GB of RAM and around 60 seconds on my machine to generate documentation for scala 3 (all you can see here: https://dotty.epfl.ch/docs/index.html, compilation time is excluded). I think those are reasonable numbers at this stage of the project.

9 Likes