Discussion on the Scala Platform Process

The Scala Center has released the initial draft for the Scala Platform and the Scala Platform process. Official announcement is here.

With it, we intend to motivate library authors to join the Scala Platform, submit proposals and help us improve the governance rules and support that we provide to the future maintainers of the Scala Platform.

Some details of the process, as whether modules are allowed to have external dependencies or not, are intended to be worked out by the Community together with the Scala Platform Committee. If you’re interested in my personal take, I roughly explain some of my ideas in Gitter. Instead of encoding these ideas in the process, I think it would set a good precedent if we can discuss what’s the best approach as a community, likewise for future questions that may arise like a uniform place for documentation.

We want to hear from you, so if you want to bring up any suggestion, improvement, question or discussion, this is the right place to do it.

We need more description/discussion of how the Scala Platform will be used in practice by consumers not just library authors! Is it a badge of honor given to a set of dependencies that you still have to enter into build.sbt one by one? Are there meta-dependencies?

Personally I would prefer a variety of meta-dependencies that pull in useful subsets of the Scala Platform. (It would also be nice if SBT were smart enough itself to only download what you need, not the entire platform, if you tell it that you depend on the entire platform. Maybe this is a job for a scala.meta based tool–kind of like scalafix but for your build. “Refine dependencies.”)

TL;DR. When the standard library was modularized, you had separate artifacts, and a meta-artifact to get all at once, if you want. Assuming the main problem is that users are overwhelmed by the choices, that setup would be a good first step. Multiple meta-dependencies also sounds good.

I think that’s a cool idea, though we should be careful to not block the platform on technical infrastructure unless it’s crucial to the platform’s goals, or it can be produced quickly. (Scala.meta is not finished yet so it’s not obvious it can be used for this goal).
Ignoring feasibility, it’s hard to tell whether that’s crucial or not. But it’s not even obvious it can be done cleanly by SBT without ever downloading the dependencies—how do you know if scala maps to ch.epfl.scala in this code, if you didn’t download the library that provides ch.epfl.scala (I’m not sure on the details)?

import ch._
import epfl._
... scala.Platform ...

“Refine dependencies”, OTOH, could be easier to implement, but Java reflection would be another pitfall—you could basically reuse the dependency analyses from Proguard or similar (including any special handling for reflection), then map used classes to Maven/Ivy artifacts, and suggest changes to the build or even automatically modify the SBT file accordingly. At that point, you’d have a tool usable independently of the platform, though a scala-platform dependency would be an obvious use case.

I don’t mean for it to be blocking, just something to aspire to.

And you can’t download nothing, you just don’t need to download the entire jar. All you need to download is the entire symbol table, which should be 1-2 orders of magnitude smaller. For every random package out there, generating these things securely is a tall order. But for a curated set of modules in a platform it should be eminently doable. Absolutely not worth holding up progress for, though!

1 Like

9 posts were split to a new topic: Require C4 process for libraries?

The way I thought about the Scala Platform is just an artifact that depends on concrete versions of all the Scala Platform modules, so that when you add "org.scala-lang.modules" %% "platform" % "0.1" all the other Platform modules get automatically downloaded by sbt. I wanted also to contact the sbt maintainers to expose to user an easier way to depend on modules. But this is something that will be explored in the future.

There is a minor inconvenience for users with this approach because they need to remember the domain on which the classes live to import them (because every module has its own package). Note that forcing module maintainers to change all their package names by a “standard” Scala Platform package name is not the ideal solution. There’s an ongoing SIP proposal that will be submitted soon that aims at fixing this problem with package aliases. At the source level, module maintainers will be able to create package aliases so that the namer and symbol loader uses them interchangeably. The details are not fully worked out, but it seems like the best solution I’ve encountered so far.

1 Like

If anybody’s confused, I’ve forked the subthread about C4 to avoid mixing it with other subtopics, I hope nobody minds :slight_smile:

2 Likes

Whoa, cool subthread-forking feature, very nice.

Some I things I like very much about the SPP proposal:

  • Provides an easy path for publishing. Everyone wrestles with this — it would be great to have something both standard and simple. (Jorge, you might look at the Travis-based push-tag-to-publish-stuff we already have for Scala modules like scala-xml and scala-parser-combinators. Your thing may already be better, I don’t know.)
  • Provides nightly builds — most projects don’t even bother wrestling with this because it’s a pain. Getting this for free would be awesome.
  • Great committee — good recruiting there.

Some questions:

  • Is the sbt-platform plugin actually required to be part of the platform, or is it just an optional thing that projects can decide if they like? If it is required, might there be any reasons that members might hesitate to adopt it? Any compatibility concerns? Also, sometimes projects already have a lot of this type of infrastructure in place, and even if a superior alternative is offered, replacing the old stuff is work. (Maybe the concept of platform-release branch means member projects would have the option of leaving their main branches alone?)
  • Travis works fine for me, it’s free, and I already know how to use it. What’s the benefit of switching to Drone?
1 Like

Another question: will modules be in the scala. package? (Always, sometimes, never?)

Oh, I see my question about whether the sbt-platform plugin is required is already answered at https://scalacenter.github.io/platform-staging/faq.html. (The answer is no, not required.)