Pre SIP: scala-cli as new scala command

SBT plug-ins do not work out of the box with Artifactory in enterprise environments due to not following POM consistency

In my experience, in the corporate environments that I’ve been to, sbt worked well. But the Artifactory/Nexus instance admins had to disable the strict consistency check. Then everything worked well.

  • ensuring scala-cli can itself be downloaded via Maven

This is an important point! scala-cli developers, please make note of this. It can be very important for some people in some specific circumstances to be able to download scala-cli via Maven Central (or through an internal corporate Artifactory/Nexus proxy).

2 Likes

Speaking about scala-cli configuration, Configuration | Scala CLI, this seems unfeasible and un-scalable to me for any codebase larger than a HelloWorld, would it be possible to read and aggregate directives bottom-to-top from the scala.conf files? This way we could e.g. define the scala version in the root folder, and main and test library dependencies on the sub-folder level.

Using directives are aggregated from across all the files provided, so it would be perfectly fine to define a config.scala top level and config.scala in main and test directories. However, since tests inherits config from main (so no need to define a Scala version in both) I would suggest having conf.scala and conf.test.scala top level with all configuration.

Using directives has a dedicated guide and hopefully, it will explain a bit more. In case of any further questions, we have both Discord channel and Github discussions to explain, help or discuss changes and suggestions.

I have opened a SIP PR here.

2 Likes

I don’t know if this helps anything, but I’ve been (slowly) writing some new Scala 3 books, and Scala-CLI is a terrific tool for books, especially intro books. It’s so nice to be able to tell the novice Scala user to just install one tool and then you can run .scala files, .sc files, and include third-party libraries:

//> using scala "3"
//> using lib "com.softwaremill.sttp.client3::core::3.7.2"

import sttp.client3.*

@main
def doPost() = 
    val backend = HttpURLConnectionBackend()
    val response = basicRequest
        .body("Hello, world!")
        .post(uri"https://httpbin.org/post?hello=world")
        .send(backend)
    println(response)

Running a simple command like this:

$ scala-cli Post.scala

is so much better than having to explain sbt or Mill to a new developer whose brain is probably already swimming, especially when they just want to run one file that happens to require dependencies.

(But I also understand that I just look at things from the “book writing” and “introducing new developers to Scala” use cases.)

15 Likes

That use case is for sure important when it comes to Scala adoption and the long term growth of our community.

4 Likes

There is a revived discussion on the syntax of what follows the //> directives marker in scala-cli here:

If you have views on this, you are very welcome to contribute to the discussion there.

3 Likes

Hello Contributors!

We are discussing the directives syntax after the magic comment //> related to SIP-46 in this discussion and I would like to summarize the discussion here to get your feedback.

Problem and trade-off

A. The current keyword using in directives after //> overlaps with the Scala keyword, while having another different meaning.

B. Changing the keyword now affects tooling and people’s code but changing it after that SIP-46 is elevated from “experimental” to “stable” may be even worse. Depending on how fast the decision process connected to SIP-46 is governed, we may risk a delay in introducing scala-cli as the new Scala.

Proposed solution

Change the keyword using to use in directives for scala-cli after //> and allow for a “grace-period” when the old keyword still works and deprecation warnings are given etc.

Questions to you

  1. Do you think we should change the keyword using to something else to avoid the overlap with Scala’s using?

  2. Are you fine with use as keyword instead of using after //> ?

By this post I risk opening up to “eternal bike-shedding”, as what is the “best” keyword is a subjective matter. What we can say more objectively is that we want it short and that it should not overlap with other keywords, and the proposed use instead of using fulfills that.

If we can settle on the use keyword fast, while agreeing that the change solves a real problem, there does not have to be much of a delay.

5 Likes

+1 for use

4 Likes

In my opinion using is perfectly fine here and the similarity to the Scala 3 using was an added bonus, which is why it was specifically chosen. It does set the context for running the program. Besides, it has a lot of adoption already in docs and scripts.

I would love to hear exact reason on why it would be wrong to use using.

Some that I can think of don’t really convince me:

  • it will be harder to google → neither use nor using is particularly good at that and I am not convinced first thing a beginner would do is to google using Scala
  • harder to communicate → one would be using directive and the other just using or really most of times they would say using keyword

I can’t really find any solid objective arguments and I think in the long run this will make no difference.

5 Likes

The objective argument is that context of a build and implicit arguments are different things. If that is “solid” is what is debated here :slight_smile:

What do others think? Is it a big deal or not to keep the using while some think it might be confusing to mix up the configuration of the build with arguments to functions?

I’m on the fence myself: on one hand I can live with using on the other hand I think use is more direct and I like that we don’t overload meanings.

(The aim of this thread now is to provide input to the SIP meeting on March 17.)

I think using is fine. Both the implicit using and the comment using record an assumption about the context in which the computation runs. So they are quite analogous from that point of view.

1 Like

If we already have a special comment syntax //> do we need an additional using or use keyword at all? Why not just something like below? Presumably the //> syntax is enough of a marker to discriminate between directives and normal user commentw

//> scala 2
//> options -Xfatal-warnings
//> dep com.lihaoyi::upickle:2.0.0
12 Likes

It ought to say given instead, as it is defining values instead of receiving them

//> given scala 3

By contrast, to consume whatever scala version is provided implicitly,

//> using scala

which could complain that no scala config was provided anywhere.

If there is load or call syntax,

//> load myscript.scala using scala 3
3 Likes

using sounds fine. If the worry is about using using in more than one way, we already have _ doing that. That doesn’t seem to be a big source of confusion.

But I like @lihaoyi’s suggestions to use //>

One other thing I would strongly recommend considering is using a more standard metadata format. What ScalaCLI is doing isn’t so special that it needs an entirely new language made for it. YAML or TOML would be well suited and there is plenty of precedence using it in tooling e.g. YAML frontmatter for Markdown files or TOML config in Rust.

Using a more common language would simplify onboarding of experienced developers, simplify tooling support, avoid the typical problems that appear in ad-hoc languages, and simplify any programmatic queries or transformations that someone may want to perform on their scala directives across their codebase (e.g. bumping dependencies like scala-steward does)

8 Likes

The directives currently are essentially key value, the added using is omitted. We could for sure make it optional, but I am worried that //> is very symbolic and would get us back to the days of an overuse of symbolic operators.

1 Like

I agree that only //> is cryptic and that a keyword makes it more explicit and telling what it is all about in a better way than only symbols. Also a keyword lend itself well to syntax highlighting, hoovering, error marking when misspelled, auto-completion, searching the internet, etc.

3 Likes

I don’t think examining the intuitiveness and explicitness of each potential syntax in a vacuum is the right way to frame the discussion. Rather, we should look at the menu of possible syntaxes that people elsewhere use to serve this purpose, and pick or adapt the one we like the best. In particular, we should explicitly avoid being too novel: the innovation here is in the scala-cli runtime and the way it provides a batteries-included way to get started with Scala. We shouldn’t be in the business of making innovative new metadata formats.

Examples of other languages doing similar things include:

Kotlin file annotations

@file:Repository("https://maven.pkg.jetbrains.space/public/p/kotlinx-html/maven")
@file:DependsOn("org.jetbrains.kotlinx:kotlinx-html-jvm:0.7.3")

import kotlinx.html.*
import kotlinx.html.stream.*
import kotlinx.html.attributes.*

Java/Scala Annotations:

@AnnotationName()
@AnnotationName(value = "elementValue")
@AnnotationName("elementValue")

Javadoc:

/**
 * @author      Firstname Lastname <address @ example.com>
 * @version     1.6                 (current version number of program)
 * @since       1.2          (the version of the package this class was first added to)
 */

Markdown/HTML front matter

---
title: YAML Front Matter
description: A very simple way to add structured data to a page.
---
<h1> {{ title }} </h1>
<p> {{ description }} </p>
Page content here...

Go compiler directives:

//go:norace

C++ directives:

#define TABLE_SIZE 100

Python file encodings

# coding: pyxl

Swift Annotations

// MARK: - View Life Cycle

override func viewDidLoad() {
    super.viewDidLoad()

    // Setup View
    setupView()

    // FIXME: Fix Bug
}

Overall, there are roughly 3 categories: @-based syntax (Java/Javadoc/Kotlin), YAML-based syntax (Markdown/HTML), and comment-ish-based syntax (Go/C++/Python).

I don’t have a strong opinion on whose lead we should follow, but I do feel strongly that we should follow/adapt some existing convention and not come up with our own thing. The current proposal //> using ... prefix for annotations, while workable, manages to look exactly like no other syntax anywhere else in the programming community, and also looks like no other syntax someone would find in the Scala language (even the re-use of using is questionable: the syntax is different, as is the meaning). That’s an unnecessary stumbling block for onboarding developers, whether they have prior experience in Scala or in other languages.

IMO, adapting any of the existing conventions would be an improvement over the current proposal, if only due to the familiarity it would provide. e.g.:

Kotlin/Java-annotations-style

@file("utils.scala")
@dep("org.scalatest::scalatest:3.2.10")
@jvm("11")

println("Hello World!")

Javadoc-style

// @file utils.scala
// @dep org.scalatest::scalatest:3.2.10
// @jvm 11

println("Hello World!")

Front-matter style

/***
file: utils.scala
dep: org.scalatest::scalatest:3.2.10
jvm: 11
***/

println("Hello World!")

These are all strawman examples I just made up, and may need some tweaking. But IMO they would be much more familiar to the broader programming community, which is a plus if we hope Scala-CLI would help us reach people who are not already deeply embedded in the Scala way of thinking

6 Likes

Thanks for the great comment @lihaoyi !

We decided to use the comment style mostly due to the fact that it doesn’t require changes in the language itself, so we need to stick with that. Not sure if we really need to use an existing style, those either are used for something else (javadoc) or are almost the same as the current directives we have.

Currently what we have is just key value configuration, which what all of these formats have in common. The format is not that complex as it would be something people would have a lot of difficulties working with. And I am not even sure if all of those formats are that well known to developers, some of that I see for the first time (totally anecdotal I know).

I think from the user perspective the biggest challenge will be figuring out what they can configure and we can provide tooling in both Metals and Intellij to autocomplete the directives so that users would have minimal overhead figuring things out. This is where we should focus our efforts, users will anyway be lost irrelevant of the format we use if we don’t provide them with a proper tooling.

Also, unfortunately I think it’s a bit too late to rethink the whole using directives and this time we should focus on the final smaller changes in order to make the SiP progress.

Lastly, in my personal opinion the using format is the best in terms of visibility, the rest looks really too close to regular comments for my taste:

//> using file "utils.scala"
//> using dep "org.scalatest::scalatest:3.2.10"
//> using jvm "11"

//> start gives us a lot of opportunities for tooling, which is another thing I really like. using is probably redundant for sure, but might still be useful. We can make it optional, this is not a problem at all, but I would not want change the whole format at this point.

5 Likes