Proposal: Simplifying the Scala getting started experience

That sounds good to me, how would this affect mill where the build files are just every .sc file at the root of your project ? Would you also make .scala files at the root part of your build ?

1 Like

Just wanted to say I think this is indeed a super important topic to get right! I am keeping an open mind so far on what’s the best way to achieve this. Some random thoughts

  • thinking out of the box is good.
  • compilers are underrated. There’s a lot to be gained from re-using the one tool that has to be true to the language standard (and essentially defines it).
  • compilers could be more open.
  • the idea of having different languages expressed by different file extensions (.sc) vs (.scala) is interesting.
  • always think of the IDE experience. Most people consume Scala through IDEs, including beginners.
7 Likes

I agree that fulfilling Scala’s promise as a “scalable” language in its tooling is an important goal.

I often write scripts to interact with legacy java codebase.

The “default” or “built-in” REPL should always demonstrate the compiler behavior for syntax and semantics.

The other use case is other shells such as Spark.

I think a simpler default REPL with a fuller-featured amm with a shared backend would be a useful way to pool the work.

There are other under-utilized or under-supported modes such as scalac -d app.jar.

I’m currently tutoring a small group of devops coworkers who’ve expressed an interest in learning Scala, and I’m very happy to see this proposal, it accurately points out a problem that is very real.

I use mill extensively on my personal time, and ammonite is my go-to REPL, and have used .sc scripts several times. The main reason why I don’t recommend ammonite to the tutoring group is that the editor support for it is lacking at best. Intellij has superficial support for it but falls short as soon as ivy dependencies are involved.

I’d love to be able to tell people who are getting started “just a .sc file in vscode/sublime/atom and start typing”. With that in mind, the gap I’d like to see bridged would be one between https://scalameta.org/metals/ and ammonite, which could probably achieved by implementing the BSP semantics in ammonite, and for the semanticDB project to be able to feed off ammonite-compatible .sc files. Either tasks don’t seem trivial, and I can definitely see how some sort of “official endorsement” from the scala language representatives towards ammonite would help attracting contributors.

I also don’t think that such an endorsement would prevent dotty and ammonite to converge towards the mutual goal of a lower entry barrier, as ammonite could benefit from some built-in functionality from the compiler whilst not creating a tight coupling (ivy resolution, etc).

In the latest version intellij understands $ivy imports in ammonite scripts, however it can use some polish.

Sometimes it shows a pop-up notification offering to add the dependencies, otherwise you have to press alt-enter on it, which gives two ways to add it, not sure the exact difference. It may need to already by downloaded so you may need to run it first and let ammonite download it.

I do wish they would publish changelogs in the plugin update info, it might make it easier to stay on top of all the progress that happen…

Nobody mentioned the most important thing - setting up the environment.

Imagine someone is coming from Python. All he needs to run Python on a new Windows machine is to download Python installer, run it, write *.py files and execute python script.py. That’s it. This even covers multiple source code files as importing in Python is easy - no upfront compilation is needed, neither setting any classpaths.

With Scala many things can go wrong, e.g. JDK installation and configuration, IDE installation and configuration, SBT installation and configuration, SBT project definition, placement of files and so on. Getting to the point where Scala compiler just works and resulting program runs fine can be many times more difficult than writing ten different main methods.

3 Likes

Welllllll, kinda. Then you find you need pip, and then conda, and then your versions are wrong, and then everything seems to work but the thing you were using before fails at runtime, and then…

I think the startup story with Python is sometimes very nice, but it is a very thin veneer over occasional unspeakable horrors. In contrast, though Scala is a little work to set up (without an IDE; it’s more work with an IDE), I find it relatively painless (run the Java installer, which is WAY more reliable than the Python installers, and grab mill or sbt) compared to Python.

Not to say we couldn’t try to make it even nicer, but I don’t think Python is the language to beat.

4 Likes

How far can you go without pip and conda? Python’s standard library is quite extensive, so you could do quite a lot with it alone and we’re talking about getting started experience.

1 Like

These statements are true, but I’d argue largely irrelevant to the end user. A new Scala programmer doesn’t care that Ammonite needed a simple compiler plugin to fix the line numbers to refer to pre-wrapping code, or that Ammonite needed its own Scala parser to do code transformations because the Scalac parser is too tangled up with other concerns.

Sure it would be nice for me, as the person writing Ammonite, to have to do less work, but in a lot of these cases that work is already done: the workarounds have been written, they work well enough, and unless the core language is going to be fiddling with the spec on a weekly basis I don’t think there’ll be any problem keeping up.

Getting us onto more shared technical infrastructure, while a worthy goal, is largely orthogonal to the end user experience.

This is true, but probably a solvable problem. Of all the ways we can improve the Scala onboarding IDE integration, fixing the bugs in the Intellij-Ammonite integration is definitely the easiest.

That is why this proposal is a lot more to do than changing the syntax of main methods. In fact, a lot of the problems you describe are SBT problems that this proposal explicitly addresses. It is very much mentioned!

By default, Ammonite gets rid of project definition, configuration, and placement of files. If we bundled Ammonite with a JVM, or even bundled Ammonite with IntelliJ-Scala (or made it automatically download it where necessary) that would bring us down to a single step (IDE installation).

3 Likes

Agreed. We should try to improve both.

https://xkcd.com/1987/

2 Likes

that would bring us down to a single step (IDE installation).

We already have something this simple, or very close. If we just focus on writing, compiling, and running code, you only need IntelliJ with the Scala plugin. It’s been a while since I installed it, but I believe it comes with its own JDK and fetches Scala for you. You create a Scala project. Not an SBT, Gradle, or other project; a native IntelliJ Scala project. You create a source file in the src directory and off you go. This seems perfect for a beginner.

2 Likes

This is true. I myself learned Scala working entirely within IntelliJ, and even wrote and published libraries using IntelliJ. It’s a good question to ask: why is this not enough?

I think there are a few reasons:

  • Working 100% within an IDE is not production ready: nobody wants to install IntelliJ on their CI server to compile/test/run their code, no matter how easy it is to develop locally, or use IntelliJ to build and run their code on a production server.

  • Similarly, Scala-in-IntelliJ doesn’t play well as a member of a larger unix ecosystem, where you may want your scripts to be run by other scripts. This is common in many professional-novice environments, where your code is but one part of a larger ecosystem of tools and workflows.

  • Automating things within an IDE, while possible via IntelliJ plugins, is several orders of magnitude more difficult than automating things via standalone scripts or command-line tools.

  • Working within an IDE does not scale well across a team: IDE config files are generally not clean enough to check into source control and share across a team. This leaves each individual setting up the project manually each time, which is tedious and error prone.

Thus, while Scala-in-IntelliJ is good enough for “just learning Scala”, it definitely falls short for anything beyond that, in particular all the professional-novice workflows, where someone who wants to work on a simple - but production ready - data processing program, website, automation script, etc. definitely wants their code to play well with the world outside of their getting-started IDE.

So it’s still a good goal to try and get a beginners setup that is IDE-independent. IDEs can easily play catch-up to the non-IDE workflows, and have done so with IntelliJ/VSCode/Atom sprouting support for every build tool under the sun. The converse, an IDE-driven workflow becoming a better member of the broader ecosystem, has basically never happened as far as I know.

1 Like

Minimal SBT project:

Give it a name, for example “smurf”

Create folder smurf as project root folder.

In root folder, create file src/main/scala/smurf/app/Smurf.scala with content:

= = = = = = =

package smurf.app

object Smurf {
def main(args: Array[String]): Unit = {
println(“Let’s smurf!”)
}
}

= = = = = = =

Back in the root folder smurf, type

sbt run

Watch it print:

Let’s smurf!

Oliver’s recipe also works if you put Smurf.scala at the root level, without bothering to nest it all the way down inside src/main/scala/smurf/app:

% mkdir demo
% cd demo
% echo 'object Smurf extends App { println("hello world") }' > Smurf.scala
% sbt run
[warn] No sbt.version set in project/build.properties, base directory: /Users/tisue/tmp/demo
[info] Set current project to demo (in build file:/Users/tisue/tmp/demo/)
[info] Updating ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/tisue/tmp/demo/target/scala-2.12/classes ...
[info] Done compiling.
[info] Packaging /Users/tisue/tmp/demo/target/scala-2.12/demo_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] Running Smurf 
hello world
[success] Total time: 3 s, completed Mar 27, 2019 6:11:29 PM
% 

-warn cuts down on the noise level:

% sbt -warn run
hello world
1 Like

To say the truth I do not think it is simple.
Here would be simple :wink:

But I think it does not matter.
In real life we spend about half a day to set up our workstations in new os. The real simplicity in such complicated stack(java, sbt, bloop, ide) can be achieved only by thin client(web browser, ssh terminal) and cloud server wich is alredy deployed somewhere.

Here’s the minimal Python project:

In smurf.py write

print("Let's smurf!")

and type python smurf.py.


Here’s the minimal R project:

In smurf.r write

cat("Let's smurf!\n")

and run Rscript smurf.r


Minimal Julia project:

Create a file smurf.jl containing

println("Let's smurf!")

and run julia smurf.jl


Minimal Rust project

Pick a name, like smurf. Type cargo new smurf.

Enter the smurf directory.

Inside src/main.rs change the line

    println!("Hello, world!")

to

    println!("Let's smurf!")

Run with cargo -q run (for a quiet run).


Minimal sbt doesn’t look so great in comparison; half of that is sbt’s fault and half is Scala. We should fix both.

3 Likes

You can write in Smurf.sc:

println(“Let’s smurf!”)

and then run it with

amm Smurf.sc

where amm is the executable of Ammonite.

No, Scala with SBT is not quite as simple, but I think it is pretty good considering that it is optimized for larger projects. Like, projects where you have five or more library dependencies, where you have tests, where your project might itself be used as a library dependency by some other projects and you want to publish it to an artifact repository, and perhaps you build a web service, or use an external database, etc.

Scala is supposed to mean “Scalable Language”. I’m assuming that means scaling up, not down, right?

If you want to use Scala as if it was Python, don’t make the language more complicated, use a specialized tool instead, like Ammonite.

You may hate the application object and the main method, but whatever you choose, on the JVM it will be an object and a main method, and they will be visible on various occasions, so trying to pretend they don’t exist just adds a layer of obfuscation.

Except that’s not official, and very well could go away if Li Haoyi stops paying attention (depending on whether anyone wants to pick it up to keep it going as the Scala compiler evolves). And it’s awkward to bridge that way and the build-tool way to do things.

Thus this proposal: this, or something much like it, should be standard.

Sometimes obfuscation in service of simplicity is a good thing.

The same logic argues for doing away with case classes (magic methods that you pretend don’t exist), objects (more magic), for-comprehensions (entirely magic), old-style closures (yet more magic), and so on.

Pretending that something doesn’t exist is very handy when it doesn’t matter.

Except nothing prevents it from handling small projects better save history and inattention. So…yes…it’s pretty good at what it does, but that doesn’t really argue whether or not it should do something else better. (Well, one could also argue: in a big project we don’t want new users involved because they’ll likely muck stuff up, so the tooling should be formidable enough to scare most of them away until they’ve learned enough to be helpful rather than a danger. But I don’t think that’s what you were intending to argue?)

So I’m not sure how the point is relevant to this thread which is specifically proposing ways to simplify the Scala getting started experience.

2 Likes

Why do you think it is possible at all?
I think dynamic type languages always will be able to be more simple than static type languages. (if one is not java script :)) )

It is always script in practice.
But as you have said.

Why do we have such situation?
I think static type languages does not suite well for small programs.

If I were linux administrator I would be quite scared the need to recompile all scripts just for of one bug fix in some core script :))

So I just can not see any real motivation for propagation tools like “Ammonite”

I like “ammonite” but I think without “dynamic invocation” it always be something around scala users.

So I just cannot see which power can make scala practical usage more simple than Intellij idea can :wink: