Pre-SIP: program Foo = { println("Hello world!") }

Language Design
41

I would rather not introduce a new language construct for that specific purpose.

I’m sympathetic to the idea of supporting top-level def definitions, and my suggestion is to change the entry point detection algorithm to support not only top-level objects having a main method taking an Array[String] and returning Unit, but also top-level methods named main (optionally taking an Array[String]). Here are some examples:

// example1.scala
def main = println("hello, world")

// example2.scala
package foo
def main(args: Array[String]) = println(s"hello, ${args(0)}")

// example3.scala
package foo
object Main {
  def main(args: Array[String]) = println(s"hello, ${args(0)}")
}

Invoking scala example1.scala would execute the entry point, if exactly one entry point is found.

Last, the JVM-linker should expose a means of configuring the name of the wrapping object, when the entry point is a top-level main def:

// build.sbt for example2.scala source file
Compile / entryPoint := EntryPoint.topLevelMethod("foo.main", jvmWrappingClass = "foo.Main")

The default would be to use the default wrapping object, which is <empty>$package, foo$package, and foo.Main, in the above examples.

3 Likes
42

I’m against new keyword. There is no real problem for it and new people in scala should understand pretty quickly ... def main(a:Array[String]):Unit = .... Most of stuff is important while learning and anyone that uses scala needs to understand it anyway in few hours. New keyword (even soft one) should be universal somehow, not special case used once in whole codebase.

But… scripting capabilities are important for anyone that want’s use scala without learning it (lots of people are using js in such way :slight_smile: ). Data Scientist, DevOps, admins and lot of people that just want’s to solve problem RIGHT NOW and they try to do it as fast as possible. They don’t wan’t to learn scala… they wan’t to learn very small subset of language that’ll be helpful.

Here is why amm shines. It is such tool, it allows you to write scripts that are Python-like. My proposition is to not add anything to standard *.scala files and just consider to upgrade *.sc capabilities or bless some stuff from amm as standard scala scripting.
I thing most interesting are:

2 Likes
43

For easy of starting out, new users, and me for new tasks, I just want to write my code.

println("hi Mum")

This should just work. Frankly, I don’t care if i have to put it into a .sc file to tell the tooling it’s a scala script. I just want this zero biolerplate version to behave. Saying program = bla is boilerplate. It’s there to give the compiler the warm and fuzzies. But I don’t care about the compiler when I’m scripting. I care about it when I write library code. Being able to sit down with a colleague and for us to just start writing down a script, without all the ceremony is a huge win. Particularly if they don’t know scala, and perhaps if they are used to scripting in bash or python.

44

Hear, hear. Frankly, just in terms of mindshare, this would be fabulous. It seems like a silly little thing to many of us, but AFAICT that little bit of ceremony contributes disproportionately to Scala’s reputation of being more hassle than it’s worth for quick little tasks. And given the number of people who only do “quick little tasks”, that matters.

I’m pretty neutral in terms of exactly where this happens (Ammonite, compiler, whatever), but having an officially blessed (that is, talked up on scala-lang.org), well-supported, easy-to-use approach for zero-overhead scripting that can easily pull in libraries would be great from a “sales” perspective…

1 Like
45

I’d love to see a real world example of such a “quick little task”, so that I can understand why for some people wrapping their code into a main method is considered such an incredible burden.

And then I would love to see a comparison with writing this in another popular language after you have added the best practice boilerplate in that language.

46

Suppose I want to list, in alphabetical order, the set of first tokens from lines of a file.

with Open("myfile","r") as f:
    tokens = [line.split()[0] for line in f]
    for i in sorted(list(set(tokens))):
        print(tok)

I can run it in the python interpreter or drop it in a file and python thing.py and it runs. (Well, it should run; something very much like this runs. I didn’t test it.) If I want to switch the file from hard-coded to command line, I add import sys and change the string to sys.argv[1]. Kinda arcane, but super compact.

With my library in Scala, here’s the code I’d enter in the repl:

import kse.eio._
"myfile".file.slurp.foreach{ lines =>
  lines.flatMap(_.split(" ").head).
    toSet.toList.sorted.foreach(println)
}

If I were to make a main function and everything, it would easily double the size (and I have to to get the command-line arg). If I were to add a build file to get the dependency, it would quadruple the size, make me work with twice as many files (up to 5x if you follow “best practices”), and two sets of tools instead of one. If I were to try to take the name of the file as a command-line argument.

So, despite Scala being in code alone (with decent libraries) comparable to Python (widely regarded as a compact powerful language) for tasks like this, Scala is swamped by boilerplate as you try to move from a quick thing you tried out in a REPL to a tiny utility.

1 Like
47

Thanks for the example. Some thoughts:

(1) The script-ish code is 4 lines, the pre-SIP would be 6 lines, and with main method, it would be 8 lines. True, 8 is double of 4. But is 8 lines of code so terrible?

(2) Is this a real world example? Is there a need out there to “list, in alphabetical order, the set of first tokens from lines of a file”? Just wondering? I have used Python to mung text files. Typically, these are tab-separated, so there is a library (csv), which I import and then I create a reader object. They also have header lines.

(3) As you point out, the real burden is not to write the code, but to specify how to build it, include dependencies, deploy.

(4) Languages like Perl or Python are more oriented towards tasks like processing text files than Java or Scala. Let’s say you want to process text files in Scala with the least effort. You could write a tool that accepts a Scala fragment and puts it into a wrapper that provides not only a main method, but also imports libraries helpful for manipulating Files and Strings, then compiles and runs the resulting Scala file. That would make it truly as easy and powerful as a Perl or Python interpreter. But that wouldn’t be a language feature, but a specialized tool.

48

Yes, 8 lines is pretty bad. Not only is it wasting half my time, it’s also wasting half my attention attending to things that are completely incidental to the problem I want to solve.

Python, R, Julia, Ruby, Haskell, etc., all don’t do this to you. Rust kinda does, but it has exactly one universal tool (cargo), and it at least writes all the boilerplate for you. Etc…

Very close. I actually needed to pick out the ones that were numbers and list them in numeric order. I could have picked a CSV example instead; this happened not to be CSV.

import $ivy.`com.github.ichoran::kse:0.8.0`, kse.eio._

isn’t much worse if that’s all you have to do. (Yes, having to remember the path is a pain.)

Sure, but you asked for a minimal example, and this one fit. I could have picked adding time intervals instead, or finding how long it takes for a biexponential function to reach 5% of the original value, or listing all the files in a directory that were created last week.

Yeah, that’s exactly the problem. Perl and Python have made their core tooling such that you can use them, unaltered, for many “specialized” cases. You don’t need anything extra; they work for a bunch of stuff out of the box with minimal overhead.

There is no fundamental reason why Scala can’t do the same. There are some tradeoffs that maybe we don’t want to make, but mostly it’s not been a priority to compete on that level, which is a shame.

49

ad (1) No it isn’t bad, but it discourages writing scripts. Ammonite has best scripting capabilities I’ve ever seen (even better than python in my opinion) and this is the only reason why I’m in favor of be more scripts friendly in native scala. I’ve seen how it could be made well :).

ad (2) I’ve made similar short one shot scripts many times. It is common that you just need to answer “How many times we are using each category in this file?” or “how much money we spend for each project?”. Even if it could be made using few bash commands it is very nice to use language which you know. Here it is not so interesting example but lets assume that you just need to use groupBy once and it become much harder challenge in pure bash.

ad (3) This is more controversial part. Ammonite has it’s magic import $ivy.`organisation::somelib:version` and it works there. This is extreamly powerful tool (See for example play framework server in one file)

ad (4) I’m very happy with processing texts in scala. I’ve made few pretty complicated parsers using Idea worksheets and ammonite scripts. This is place where scala could be used, and even if it is not main target it is nice addition.

50

Basically, other languages, especially Perl and Python, have fat standard libraries.

You could, for example, use Ammonite and predef a whole bunch of libraries you like, then you wouldn’t have to mention them extra in your scripts.

51

Does ammonite support import $ivy in predef.sc? Because if you use certain libraries a lot you might add them there.

52

Yes it does, and yes, that’s what you can do with it.

53

If someone is on a such early stage to be not able to comprehend a single entry point then probably he would benefit from scripting syntax. What having program keyword solves in his case? If beginner comes from Java then he expects main method with args and he is productive with organizing code into classes. If beginner comes from scripting language then he’ll wonder why do we need to wrap everything in object or class and can’t just execute instructions one by one, which he is extremely used to and finds it natural.

Going for Scala scripting (*.sc instead of *.scala) instead of program keyword will make both guys happy. Java programmer will use his OOP skills in *.scala files, while script programmer will prefer to write *.sc scripts. I think it would be good to give this kind of choice in Scala programming tutorials, instead of weird mix of Javaisms and scriptisms.

4 Likes
54

(1) The script-ish code is 4 lines, the pre-SIP would be 6 lines, and with main method, it would be 8 lines. True, 8 is double of 4. But is 8 lines of code so terrible?

Yes! It’s enough to stop people from doing it at all. They had to spend half their file writing things that weren’t what they wanted to do. That is a huge hike to the cognitive cost of writing your (probably disposable) script.

(2) Is this a real world example?

It’s representative of common scripting tasks. Imagine all the horror shows people write in sed/awk. I wrote a script yesterday to 1. run BLAST (a bioinformatics tool) and pipe a string to stdin, collect the stdout into a string 2. parse the CSV output 3. group by the value in the first column 4. print out the max of the last column per group.

(3) As you point out, the real burden is not to write the code, but to specify how to build it, include dependencies, deploy.

The burden shifts as the size/complexity of the task grows. However, we should be working hard to minimise the burden at each and every step. At every stage, somebody gives up. Let’s have fewer people give up.

(4) Languages like Perl or Python are more oriented towards tasks like processing text files than Java or Scala.

This is fair. But it comes down to what is in the stdlib, the prelude, and just how fat it is. As you point out, this can be handled by tooling that runs the script.

55

I am also in the camp that thinks that explicit main methods are not an acceptable alternative for extends App. I resent if the compiler forces me to write stuff I don’t care about, and everything having to do with main falls into that category. It’s pure boilerplate. There’s nothing in

  object Foo {
    def main(args: Array[String]): Unit = ...
  }

that tells me more than I"want to run the code in ... as program Foo". Furthermore, the annoyance is very visible for new and occasional users of the language. It would feel like a big step backwards to them from the previous Foo extends App idiom.

That said, and after thinking some more about it, program Foo { ... } feels like a large language footprint for something that is comparatively minor. Sure, program will be a soft keyword, and the construct is very easy to implement. But still, it feels like we spend a new language construct on something very specialized.

So, how about the following alternative instead? Allow a @main annotation on arbitrary methods. E.g.

    @main def hi() = println("hello world!")

The @main annotation would be permitted on arbitary toplevel methods. The example above would expand to something like:

    def hi() = println("hello world!")

    class hi {
       static def main(args: Array[String]) = hi()
    }

so you could call it with java hi.

Advantages:

  • The generated code could come from a macro or, if annotation macros are not available, from a later phase in the compiler. It’s pure code generation. The typer would not know about the main method and the class it is in.

  • Since this has to do with host interop (i.e. what method is callable form the host environment?) an annotation seems the right way to express this. Similar to how we express bean getters and setters, say.

  • It makes it clear that the program code is run in a method, not as the initialization code of an object.

  • We can provide for many ways to pass arguments. Either no arguments at all, or as an array, or as something else (immutable array, list, or finite number of strings). I.e. the following could all work:

  @main def hi(args: Array[String]) = ...
  @main def hi(args: IArray[String]) = ...
  @main def hi(args: Seq[String]) = ...
  @main def hi(arg1: String, arg2: String) = ...

That last point is to discuss. The point is, we have a lot of flexibility what kind of methods can be annotated with @main.

7 Likes
56

@odersky basically everything you’ve described with @main methods has been implemented in Ammonite scripts. It works very well!

57

The ammonite @main functionality and the way it maps the arguments of the main method to program arguments (in an extensible way since it’s based on a Read type class) is really amazing.

In repeat.sc:

@main def repeat(str: String, i: Int = 3) = str * i

In terminal:

$ amm repeat.sc hi
Compiling .../repeat.sc
"hihihi"
$ amm repeat.sc hi 4
"hihihihi"

It does seem like a lot of effort went into making Ammonite so user-friendly and powerful, so making it the standard, advertized way of doing Scala scripting would seem appropriate (as long as it stops silently sending logs over the internet by default).

I also agree that having a post-typer macro annotation that generates a main class, using the same conventions as ammonite, would already go a long way towards making Scala applications more pleasant to write.

2 Likes
58

It hasn’t done this for a while now, but nobody noticed

1 Like
59

Cool, good to hear.
(Also, how do you expect someone to notice, when it was silent in the first place? :sweat_smile:)

60

The only reason to use main function is that the java command line tool requires the signature, along with some complicated -cp, -X, -XX, and -D arguments.

My thoughts:

  1. Aren’t people from Java world who know those complicated arguments of the java command line tool also understand the convention of main? Which is more natural to them, def main or program Main?
  2. Are there other approaches to launch Scala applications for people who don’t use java command line tool? Is an average Scala starter expected to understand the java command line tool？