Metaprogramming configurability

Scala 3’s new metaprogramming capabilities are amazing!

One very important ability that seems to be missing, is the ability for downstream users to configure to upstream metaprogramming. When all library code is available at runtime, it’s easy: most libraries in the JVM(+) ecosystem allow downstream users to configure library behaviour by looking for a config file on the classpath, or reading env-vars/sys-props at runtime. We currently don’t have a mechanism for users to influence metaprogramming in Scala 3 libraries as the configuration needs to happen at compile-time.

Metaprogramming allows libraries to generate more-efficient, faster-executing code. For Scala.JS users there’s an extra, important benefit: metaprogramming allows us to prune code at compile-time and generate smaller code, resulting in smaller JS files that will ultimately be served to and downloaded by, all end-users of Scala.JS-based webapps.

Desired abilities

Imagine I’m publishing a library with an inline method / macro. I’m looking to provide the following to consumers on my library:

  1. The ability to use my library without any custom configuration, in which case the library uses a built-in default config.

  2. The ability for users to provide their own custom configuration and know that it will be applied globally.

    (In conjunction with the previous point (1), this rules out using implicits because with implicits, the onus is always on end-users to remember to add the import, and if they accidentally forget to import their implicit config, compilation would silently succeed using the library-default config; this would be very bad.)

  3. The ability for users to specify at the sbt/other-build-tool level, a setting like DEBUG, RELEASE, ENABLE_FEATURE_X (like C / C++ / Rust / etc).

Solution idea (part 1/2)

I think the easiest solution here would be to allow users to specify env-var-like settings via scalac flags. There’s already a -P:<plugin>:<opt> flag. It should be trivial to also support -E:<key>=<value> flags that build up a compile-time env which inline methods and macros could read to influence the code they generate.

For example, this could solve the DEBUG/RELEASE problem by allowing users to specify something like -E:scalajsreact.mode=RELEASE and have some code in scalajs-react like:

inline match compiletime.env.get("scalajsreact.mode")
  case Some("RELEASE") => ...
  ...

Using a scalac flag like above would also solve the global, import-less, config problem by allowing users to point to their config class. Something like -E:scalajsreact.config=com.example.ConfigDebug which, again, inline methods would query before deciding to reach for the built-in defaults or not.

inline def getConfig =
  inline match compiletime.env.get("scalajsreact.config")
    case Some(className) => Expr.xxx(className).valueOrError // or whatever
    case None            => LibraryDefaultConfig

Solution idea (part 2/2)

Library authors will need to define the set of config keys (i.e. fields), types and (optionally) defaults.
Easy enough in normal Scala but this all needs to be inline which doesn’t seem to work at the moment.

This doesn’t work, for many reasons:

// Defined upstream:
trait LibraryConfig {
  inline val featureOneEnabled: Boolean = true
  inline val featureTwoEnabled: Boolean = false
  inline val debugPrefix      : String  = "[%date% DEBUG] "
}
object DefaultConfig extends LibraryConfig

// Defined downstream:
object MyCustomConfig extends LibraryConfig {
  override inline val featureTwoEnabled = true
  override inline val debugPrefix       = "%date% -- "
}

In a perfect world I’d like to see something like this:

// Defined upstream:
inline trait LibraryConfig {
  val featureOneEnabled = true
  val featureTwoEnabled = false
  val debugPrefix       = "[%date% DEBUG] "
}
inline object DefaultConfig extends LibraryConfig

// Defined downstream:
inline object MyCustomConfig extends LibraryConfig {
  override val featureTwoEnabled = true
  override val debugPrefix       = "%date% -- "
}

I think this might actually be feasible if we tack on some restrictions. I’m thinking:

  • inline traits/objects are erased and only available to inline methods
  • only inline traits/objects can extend other inline traits, obviously
  • limit body statements to vals, defs, types
  • maybe now, or maybe later: also allow nested inline traits/objects
  • for all inline trait members, singleton types are widened (eg. trueBoolean)
  • for all inline object members, types are narrowed to the specific type of their values, even when not overridden (eg. Booleantrue)
  • no privacy modifiers - everything is public

(Note: I’ve never worked on scalac so take me talking about feasibility with some salt.)

Real use cases

Here are just two that come to mind.

  1. In scalajs-react there’s ScalaJsReactConfig which allows users to customise various generic behaviours in scalajs-react itself. An example is, in dev you probably want to see names for your components like com.example.myapp.HelloComponent, and in prod might want those component names to all be blank. It’s pretty hard to do as is and in fact there’s an open bug that this selective-component-name behaviour doesn’t seem to be working in all cases :frowning:

  2. ScalaCSS allows you to create CSS that’s both safe in terms of content; and safe in that all styles get their own field, making it impossible to reference a non-existant style, and easy to detect unused styles as they become stale. It’s quite configurable for some good reasons: here are the default dev and prod settings. The problem is that because all the CSS construction happens at runtime, the additional JS it generates ends up being huge. We’re talking something like 100kb of minified JS or something crazy like that just for safe styles! If users were able to provide configuration details at compile-time, we could probably have the library do all of its work at compile-time too, so that the generated code ends up just being like the example below. That would result in a 99% reduction in its JS footprint, not to mention it’d be lightning-fast at runtime.

    object Styles {
      // Code after inline/macro expansion:
      val header = Style(className = "_ax", css = "._ax{color:red}")
      val footer = Style(className = "_ay", css = "._ay{margin-top:1ex}")
    
      // One library case class with two string fields.
      // That would be an effectively non-existent runtime/JS-size footprint
    }
    
12 Likes

Why not depend on an implicit configuration context?

1 Like

That’s explained in Desired abilities points (1) and (2) :slight_smile:

I’ve now raised an issue and a draft PR to implement the first part of this:
config though scalac flags.

3 Likes

Is the proposed -E flag to inlines essentially what -Xmacro-settings was to macros in Scala 2?
We use the latter extensively in Scala 2 to pass information about build environment to compiled code like this scalacOptions += s"-Xmacro-settings:is-ci=${insideCI.value}"