See what happens when I’m not fully paying attention?! 
Thanks.
This is a misnomer… Dotty is published, sure. To say that it is released, which has a stigma of being a stable, production-ready library? That’s frustratingly obtuse.
1 Like
I believe that these experiences actually strengthen my arguments:
-
Perl 6: Was in announcement state forever. Scala 3 will go into feature freeze this summer, will have developer previews this year and the final release will be out next year.
-
Python 2/3: Suffered exactly the “all pain and no gain problem” I was writing about. People did not get why they should suffer the pain of migration for Unicode strings. Now that Python 3 has more interesting features, Python 2 is going away. Besides, I am confident we can avoid Python’s migration problems through a combination of static typing and binary compatibility.
3 Likes
I would personally consider “dotty” as “released” when scala 3.0.0 is released.
2 Likes
Is the syntax stable yet?
Correct me if I’m wrong, but Scala 3 will have been in development for 7 years if the release goes by the timeline that you suggest. Sure, it’s not the 15 years that Perl 6 went through, but it is significant. Why should we assume that Dotty is immune to this treatment?
Large projects take time, and sometimes they take longer than we’d like. The difference is that Dotty was just Dotty, the research project, until last year. We announced Scala 3 then and we will deliver it next year. That seems like a reasonable time frame.
1 Like
I think the biggest missed opportunity is compile times: I think that if we can provide significant compilation time wins, people will migrate to Scala 3, and eat the migration pain, without any new features at all! In case people have forgotten, in the recent Scala Developer Survey (marginally) more people wanted faster compile times than every other language feature/improvement combined.
Different work from trying to judge new features on their “elegance” or “simplicity”, but if you can show people Scala code compiling 2x as fast (which is not unreasonable: that was about the improvement from 2.10.0 to 2.12.8) people will jump through as many hoops as you’d like them to. Reactive-Akka developers, Scalaz/Cats developers, Better-Java developers, Scala-JS-React developers may disagree on the elegance of implicit function types or the new typeclass syntax, but everyone would agree that improved compile times would be a huge carrot that is hard to turn down, especially in professional environments with large codebases.
If we think the Dotty Scala codebase is so much easier to work with (I presume we do, because otherwise why would it be worth spending a decade rewriting NSC), and it has not been micro-optimized-to-the-moon like Scalac has, then it should be significantly easier to get significant wins. The fact that the language is simpler for humans to reason about should surely also make it simpler for machines to analyze.
I don’t think metaprogramming migration is going to be that hard. The main difficulty is just getting something that we can migrate to, which has not existed for the past 5 years. I write a lot of macros, in a lot of OSS libraries, and it shouldn’t take me more than a week or two to re-write all my macros using whatever meta-programming library you’d like, as long as it supports the same use cases.
The last “supports the same use cases” line is key. Past work on Metaprogramming and macros (at least post scala-reflect) has been hamstrung by the “let’s think up some elegant setup from first principles” approach. I think that is entirely the wrong approach: a feature is nothing without use cases, and if you want a serious alternative to scala-reflect, you need to start with deeply understanding everyone’s use cases for macros, how you might want to support them, and then consolidating that into a coherent API.
If I was the one working on the project, I would start by taking a dozen or so popular OSS macro libraries and making sure their use cases are fully supported by-hook-or-by-crook, before even considering the elegance of the API. Until that happens, it’s not a “migration” issue, it’s just a “new thing is strictly worse than old thing” issue.
I wouldn’t ask anyone just-getting-started to work with Scala 3, unless they had Martin Odersky and team to teach them. The top priority for someone getting started is to just do what “everyone else” is doing, to minimize the number of irrelevant issues they may encounter while learning, and everyone else is very much not doing Scala 3. “Spark works on Scala 3 (in class at EPFL)” is very different from “Spark works on Scala 3 (at 2am on Saturday night when your biggest customer just called)”
13 Likes
Humans are deeply psychological creatures. The pain they feel can have a lot to do with their expectations. If you tell people its going to be an epic change then you’re setting yourself up for trouble. There is a simple way to ease the transition from Scala 2 to 3. Rename 2.13 as Scala 3, 2.14 becomes 4 and the first Dotty becomes 5.
I would like to see us go over to yearly releases. This seems the trend that a lot of software is going. Using this approach if a feature / change is not ready, it just gets put back to the next release. When releases are regular, this is not the end of the world. More frequent releases would make it much easier to implement a proper deprecation cycle. Ideally when a feature is replaced, the old mechanism should not be removed in the release that its replacement is introduced.
I feel the text book issue is unavoidable. Our software is in continual evolution, the teaching of it must be in continual evolution. Surely one thing that the history of Scala has taught us, is that language creators can never predict exactly how the capabilities of the language will be used. Text books start to date before they are even printed.
Written from my Kubuntu 18.04 desktop, kernel 4.15, systemd 237.
1 Like
Sorry but I think it’s the other way around, it’s revisionism to say that performance was not a goal. When Dotty was originally announced talks and blog posts about it focused primarily on its performance and removal of features from Scala 2. Since the later Scala 2.12 patch releases matched Dotty’s performance and many new features were added to Dotty since then nobody is trying to sell it that way any more, that’s all.
1 Like
Agreed, and we’ve spent a significant amount of our cycles working on performance (recent example: Reduce some allocations by odersky · Pull Request #5748 · lampepfl/dotty · GitHub), but there’s still no free lunch in sight (except maybe [WIP] Parallelize the compiler via two-pass compilation by smarter · Pull Request #4767 · lampepfl/dotty · GitHub if everything works out), and meanwhile there’s a ton of other things to do to get to the feature freeze, so this isn’t a priority currently.
That’s basically the approach we’re taking now, starting with scalatest (much trickier than you’d think!): ScalaTest macros · Issue #5491 · lampepfl/dotty · GitHub, and recently monocle: Implement Monocle macros by liufengyun · Pull Request #5944 · lampepfl/dotty · GitHub.
1 Like
What I meant to say was: if you’re new to Scala, you don’t have to worry too much about which exact version you’re starting with, because most of what you’re going to do will be the same across language versions, Spark working with Dotty being an illustration of that. I’m not saying anyone should use Dotty in production today.
Aha, the goal of Dotty was to be faster, but it is not actually faster. Thanks for the history lesson!
1 Like
Yes and just from a learning perspective its much safer to keep with the herd. The new thing may be better, but virtually all the other early adopters will be fully conversant with the old thing. All the early discussions, early blogs etc are geared to people upgrading them from the old thing. When I came to Scala, I found it quite challenging because familiarity with Java, the JVM and the Java ecosystem was the starting place for most discussions.
I really do feel that gearing Scala 3 transition around teaching Scala to beginners, is putting the cart before the horse. The experienced Scala 2 users and library writers need to become conversant with the new features before we can finalise how to teach Scala 3 from scratch. How to develop students from scratch must follow not lead language development.
Yes I’m sure carrot and stick, pain and gain need to be somewhat balanced in a version upgrade, but that doesn’t mean upgrades can not be broken down into smaller chunks. My favoured compromise would be yearly version releases but no special long term support releases. Promising extended long term support for Scala 2.last would be a mistake in my opinion.
I would suggest again to consider forking the Dotty compiler, one fork’s goal would be the fastest possible convergence with and replacement of Scala 2, the other fork would support few dependencies, give few guarantees, but allow extensive experimentation.
3 Likes