What an interesting topic!
Thanks for posting, and thanks to people who actually shared their solutions!
Introduction:
Most of my points are going to relate to this talk by Richard Feldman. I don’t agree on everything he’s stating, but that’s a great talk (and he’s a fantastic speaker).
(I’ll describe the points he’s making in a different order than the one in the presentation)
-
Platform exclusivity
Not sure it’s a path Scala can take.
Obviously, the JVM is a huge fantastic platform shared by a lot of languages. So there’s no exclusivity there.
Scala.js could possibly be a way to go? But the Front-End world finally has some great functional languages for building applications, namely Elm, ReasonML and probably Purescript.
Webassembly maybe? not sure about that.
For sure people have made a great work with Scala.js, but I’m not sure it’s a “platform exhaustivity” Scala could go for in the future.
-
Quick upgrade
This is both true and not true.
Scala doesn’t have a “quick upgrade” path from Java as some of its competitor can have (Kotlin, Vavr, for instance).
But they don’t bring as much.
I once had to draw what Scala’s learning curve would look like to me, and inspired from a talk (I can’t find anymore, and Alvin Alexander’s writings):
- it has a steep learning curve towards writing “better Java” (= less-verbose, better types, pattern matching, function composition). Mostly because:
- syntax, different from most of the languages in the JVM-world,
- concepts like eta-expansion and
val/def
- confusing stuff like “should I write
( or {? Oh, both work actually? So they’re the same? Oh, in fact no.” “Can I omit the . here? Or maybe I shouldn’t. Oh I guess it’s not working because I omitted the dot, so I’ll add one, and in the future, in a pure cargo-cult manner, I’ll add dots everywhere the compiler is giving me this error”
- some concepts that are used everywhere in “Scala codebases you’re using” but that can mean really different things: namely:
implicits.
- then, it has a very smooth curve towards “Scala/FP beauty” (pure functions programming, function composition, immutability everywhere, extracting power from case classes / objects / traits, replacing clunky inheritance by mixin traits, using implicits correctly, etc.)
- you start refactoring your own code, relying on the compiler, adding more types you’ve heard about, wrapping stuff in
Options or Try
- you start being quite good at using implicits only where they’re really useful, and not harmful
- eta-expansion helps you write curried functions for better composability
- you finally get to the point where you’re building powerful opaque-types that can be
unapplyd properly, safely: so, easy to deal with for people using your code, while being easy to understand.
- until naturally you start hearing, and getting more&more interest in “Advanced FP constructs” (Group, Monoids, Functors, Monads, etc.)
- I think that’s where you’re hitting your second huge learning-curve barrier (that I didn’t manage to go over this one, yet).
- Ok, I think I finally got the Functor/
flatMap thingy
- Ok, I understand effects can be harmful, I’ve seen it a lot of times already. If every other functional construct has the same kind of benefits (Lenses maybe?). I should start learning it. Moreover it’ll help me understand part of the docs where it’s taking
Monoid or Semi-Group examples
- So, Should I start using Cats? There’s probably a library I’ve been using that’s using Cats right? I’ll read its source code.
- Oh… what the hell are these cabalistic symbols? Oh well, I’m lost, thought I had made progress but don’t understand anything… Well, I guess I’ll what some parts of the doc has to say about these constructs.
- And from here you’re back to reading examples in the dotty doc, for instance, full of Semi-Groups etc. Without even understanding the problems it tries to solve.
I really hope dotty will help on flattening/softening this learning curve. And I really think that’s the most powerful lever we can act on. Functional Programming is already getting more and more popular. As an example: coming from a telecom/network academic background: FP wasn’t even taught for us in high-school (only Pascal, C, Java, and Matlab obviously), it was only in the pure “computer-science” cursus. It’s no longer an exclusivity now, both cursus are taught ML-family languages as first languages. (M. Feldman talks about this from Elm’s point of view in the presentation above).
The ultimate reward of learning Scala is a fantastic/powerful reward, I think it really should stay the same: from writing better Java towards using powerful FP constructs and types that will separate computation from effects, making the impossible stuff impossible to write, and modeling complex problems in the most safe way possible, while targeting a well-known, battle-tested, runtime.
But we can make efforts with the learning curve / path. From bridging with “now-common” Java language constructs (method references) to writing Scala/Vavr cross tutorials, by adding very simple, real-life examples where advanced constructs are used. (Example for HKT: “try to write the signature of a single Java method that takes a specialized collection, sorts it, and returns the exact same type of collection” i.e. not referring to the “most common ancestor”. It sounds pretty silly, but it shines at evidencing "Oh, right, if C is the type of the collection, and T the type of the elements in it, I simply can’t write C<T>, oh, and in Scala C[T] is possible, I think I get it now!).
For me, the major points on having a smoother leaning curve would be:
- not having as many ways to write stuff, it’s so confusing. And as a beginner you tend to “lose confidence” (like "oh this is probably not compiling because I used
{ instead of (")
- having a more tight relationship between keyword <=> functionality (just Google “what does implicit mean in Scala”, hopfully you’ll find one of the most useful answers in Stackoverflow for Scala beginners, going into every case, one by one). Distinction between loops and comprehensions is another (less “huge”) example. Typeclasses through implicits are adding complexity to this problem. I really think having a sound "
where keyword => type class concept => another way of thinking what you’re calling inheritance" => just google “type class” and you’ll probably see the point" would help having a less steep learning curve.
- still introducing advanced concepts in the docs, but probably in a less-generic, (maybe less-academic
) more “real-life” oriented examples. This one is a really hard one, but some people managed to (most book writers do, example Nilanjan Raychaudhuri when explaining how to turn the strategy design pattern into functional programming in Scala in Action: it’s still helping me to this day, Alvin Alexander also does, in its “Simplified Functional Programming” book).
- people love languages that “make them better programmers” (Haskell is probably the most known language to do so, lately, Elm started getting this reputation, too). I really really think that’s what will make Scala a popular language. For now, its reputation is too much “Scala is a language that makes you write incredibly clever code no-one else actually understands”. If we can move this to: “Scala makes you write battle-proof code easy to read, understand, and maintain, yet using powerful academic constructs”. That would be a guaranteed success
- Killer App
That’s a major point. The only “niche” I could find, would be: strongly-typed Machine Learning applications.
Scala had one niche (kindof) at some point with Apache Spark. And I saw a lot of people diving heads-down into Scala in order to build Spark applications. (Akka could be mentioned as a killer-app, too, but since you can use both in Java now…).
A lot of people are getting into ML, Python has been the default choice for a lot of them.
Python has great stuff built-in, but I think (sorry if I’m mistaking, that’s pure assumption) its massive adoption in the ML world has come from the fact:
- it has very advanced “math-computation” libraries (SciKit, etc.)
- it is quite single-function-friendly (Lambdas as in FAAS, not as in lambda-calculus)
- it’s easy to experiment with (dynamic-typing, etc.)
But from my perspective, the ML-world starts turning from “experiment-oriented” (a bunch of scripts where these adoption arguments really matter) to “factory-oriented” (an application we have to maintain in the long-run, with business rules/logic in it) where other kind of features matter more.
And I really think that’s the field where both functional languages and an advanced type-system really shine: they make some stuff easier to conceptualize, describe, model, and maintain.
Every “bug” detected by the compiler (through the type-system for instance) is something we don’t need to check for or write a test for.
Advanced constructs like Opaque-types etc. can help “making impossible states actually impossible” (another great talk by M. Feldman, actually). We could even be harsher with nulls and Exceptions in Scala, that’d be for the best.
As a “baby-Scala” developer, Scala helps me A TON with this. Pattern matching exhaustivity, Option/Either/Future etc, composing pure functions, etc.
It’s a great tool for turning a bunch of “useful-but-dirty scripts” or “small prototypes” into in industrial application. (Vavr is another one, btw).
So I guess it could be a niche, a potential killer app for Scala. (things like Tensorflow-Scala, etc.). A goal could be: “Java developers leaning towards ML should be embracing Scala in the process instead of Python.”:
- they (and their Ops teams) are already confident with the JVM, its debugging/monitoring tools, its optimizations, so they’re “safe”
- since they’re coming from Java, they’re already relying on a compiler to “check stuff for them” (refactoring, for instance). Pushing this approach even more in a world where you’re not confident-enough (you just start creating ML-applications) is an obvious advantage
- provide a Python alternative to either FP lovers, or people being used to deal with advanced type constructs
- now that GraalVM can build native images, “functions” can be written in JVM languages. Using a functional language to write complex functions makes total sense.
Conclusion
Finally, lets look at Richard Feldman’s “Considerations” slide (the considerations people have in their mind, according to him, when looking at the language):
-
Benefits
Scala should emphasize its benefits, really.
The first time I read about Scala I discovered a blog article writing about “Libra” that could basically make dimensional equations, and assert dividing a measure by time would end-up in velocity. Love at first-sight for me!
Such great examples are not that frequent, although I think a lot of the community has examples to share about that. Scala helps modeling complex / advanced logic, has a powerful type system that helps designing programs that are not going to make absurd computations. Starting learning Scala is starting a journey towards academic functional programming. A very hard/long journey, but that’s a huge benefit.
-
Familiarity
It’s bad, I’m sorry to say it like that, because I like Scala, but “familiarity” for Java/Javascript/Kotlin/Groovy/C programmers is not good. It’s really hard (even for me who is a baby Scala programmer) to remember the “unfamiliar” stuff because Scala tends to be a language with one of the most important survivor bias. (because of the benefits). But things like someKeyValue.map(_+_) even though extremely useful and noise-reducing are terrible for beginners. (I’ve seen people go absolutely crazy with this very example). Brackets for generics have been hard to figure out for me during the first month (hopefully I wrote it at the time, because survivor bias erased this from my memory). def/val and eta-expansion where other JVM languages tend to use method references for the same needs. “Should I write val? def?”
Not sure how this can be worked on without restarting from scratch though. Sorry I don’t have any real solution here apart from working with experienced Java/Javascript/Kotlin/Groovy/C++ developers wanting to learn Scala, and asking them all-day-long how they feel about the code they’re reading.
Cabalistic symbols in libraries don’t help with “familiarity” or “first-impression”, too. “What the hell is ~>, is this a language construct?” “Oh, so |@| is an operator? Nice! Oh actually not, it’s a method defined by cats”.
-
Learning Curve
I went into way too much details already
-
Ecosystem access
Keep it as it is! It’s really rich, can deal with Java libraries, has great Scala libraries (scalatest), integrable with other build tools than sbt so that it’s not “all-brand-new”. IntelliJ support is quite good. Compiler could be faster but… It’s fine for me to be honest.
Thanks for the topic, and the opportunity to express these.
Hoping this can be useful to the community.
I think it can be overcome by good business task oriented documentation. IMHO scala has a lack of such documentation. The current documentation is too philosophical in high level(