Since more or less forever there has been a kind of tension accompanying the design of the language between, on the one hand, what makes Scala a great choice for teaching and, on the other hand, the needs of advanced power users.
Both sides have very strong arguments, but the goals of both sides seem contradictory.
Compromises, as always, often leave both sides quite dissatisfied. That’s the nature of compromise.
Personally, I think neither side can be dismissed:
Success in the area of teaching is the be-all and end-all of a language’s spread and prevalence. If you want your language to grow, you have to make it easy for people to learn it. If you scare off newcomers, all is lost…
At the same time, Scala is especially successful among experienced and advanced developers. A small team of talented people who know what they’re doing can do wonders with all the powerful features Scala has to offer. Very often people choose Scala exactly because it offers features that are not available in more “industry-oriented” languages.
Scala is sort of the C++ of the JVM; for the good, and frankly for some of the bad. Powerful features come with complexity, and complexity clearly has a price. Be it just the problem of hiring sufficiently experienced people.
I would like to propose a partial solution, and a way to ease the tension between the two valid points of view.
In fact, it’s not even an entirely new idea at its core. A long, long time ago, somewhere I can’t remember, @odersky proposed something he called “language levels”. The idea was quickly rejected, for reasons I think are understandable. It was a bit over the top, and most importantly, not actionable in any way. I think the proposal included something like ten “language levels”, rather arbitrarily tied to specific language features. It would probably be impossible to write any real-world application without using up to the most “powerful” levels. Which would make the whole concept moot.
But the idea of “language levels” as such is actually quite good! It just needs to be simplified and made at least somewhat workable.
So I would like to propose the introduction of three semantic “language levels”: “basic”, “standard” and “advanced”.
“Basic” Scala would include everything needed for teaching beginners. It would leave out quirky and confusing parts. No magic. But enough to get things done. Maybe think “Python, non-expert level”.
“Standard” Scala would include besides everything in “basic” some of the more rough but commonly needed parts of Scala. Think “everything an average professional developer needs on a daily basis”.
“Advanced” Scala would include “standard”, but also the more obscure parts of the language not everyone is familiar with, and of course all of the most advanced features.
I don’t want to discuss in detail which feature belongs where as this is a classic bikeshed topic. Those details can be figured out, if needed… (Also, moving things around after the fact would be possible anyway, I guess).
The idea, of course, is not just to have these feature category definitions on paper. The idea is, in a next step, to apply a mechanism to these categories, similar in part to a mixture of the “experimental” and “language import” features in the compiler.
“Language levels” may need to be explicitly declared. Depending on some compiler switch, the use of features in the “advanced” category, or even the use of “standard” features, would have to be manually enabled: Some “language level annotation” would be required at least on a per compilation unit basis, or maybe even (IDK what’s best at the moment) on a package level (like e.g.
advanced package foobar).
The use of “language levels” should probably also be some kind of “viral” like “experimental” is: If you directly use dependencies of a higher level than is required to be manually enabled the compiler should yell at you if you don’t annotate your code with the appropriate “language level”.
But “language levels” should not apply directly to expressions unlike “experimental”, or “language imports”. This is to keep them useful, and to avoid micromanagement. It’s a kind of “all or nothing” declaration for larger units of code. That’s because these units are usually used (and read, and hopefully understood) together as a whole. This would also enable more useful metrics, imho. If you have a lot of “advanced” packages (or compilation units?) in your codebase this is a much clearer signal that you’re writing complex code than a few counts of maybe only very loosely scattered “language imports” here and there but possibly everywhere.
For teaching beginners, the compiler would usually be switched to a mode where it only allows “basic” unannotated Scala. The compiler would be able to automatically check if only “basic” features are used, which might be interesting for teachers.
More interestingly, “advanced” Scala could be restricted to only some modules of an application. Organizations could also set limits on the relative number of such “advanced” modules in a code base. At the same time, you could check whether marking things as “advanced” is really necessary, e.g. if a larger module contains only a handful of usages of advanced features (but still needs to be annotated as a whole of course). This could be a signal to refactor the module and get rid of or move the “advanced” parts elsewhere, and free the whole module from the “above average complexity” label.
The point is: Such “language levels” would be actionable because they are quite coarse-grained.
Of course, and as always, adding stuff doesn’t make things simpler, quite the contrary. To keep the balance, I would suggest getting rid of “language imports” at the same time. The features would overlap anyway.
“Language imports” never made sense to me. They’re just an annoying micromanagement chore. They’re not actionable in any way. They don’t provide any insightful metrics. They don’t let people retreat form the usage of certain features. Beginners, in particular, will just hit the yellow light bulb and be cool that the snippet from Stackoverflow is now compiling.
But if you have to tag your entire compilation unit / package / module (?) with something that is considered “problematic” under some aspects, that is a much more serious decision. Maybe introducing “advanced” components requires sign-off from the team lead / some seniors? Maybe your professor won’t accept such a solution? “Language levels” would be a much more powerful tool to enforce some guidelines! Which “language imports” can’t really be, because in practice you can’t micro-manage things in such detail. It’s too much context dependent.
So the TL:DR would be something like this: Replace “language imports” with coarse-grained semantic “language levels”, which are categories of language features. Have the compiler check for proper usage; add tooling that can provide “language level” metrics for a code base.
What needs to be figured out, of course, is which feature goes where. Also, the level of granularity isn’t clear to me yet. My gut is leaning towards whole packages, but I’m not sure. I hope someone has some arguments for a direction here. (It shouldn’t be finer than compilation units, though, I’m pretty sure of that).
This idea doesn’t have a high priority. It’s just an idea for the future. Deciding which features belong in which “subset” of Scala would require a review of the whole language, and that’s a lot of work (and bikesheding). Currently there are more important issues still open. Maybe the right time to think about this proposal in more detail would be when reviewing books for a fully polished Scala 3 at some point in the future?
But I really think this is something worth considering. If you had a fairly reliable switch to confine the power of Scala to just a few dedicated places, and mark those places in a very clear way so that especially automatic tools could flash some warning lights when approaching, this would imho ease the tension between having an adequate language for newcomers and at the same time a mighty tool for power users.
This here needs to end:
Imho it’s possible to end this with this “language levels” proposal. Just draw a clear line in the sand and agree that crossing it requires some kind of pass. Let’s finally stop pulling in two opposite directions.
Scala is for everyone. It’s a big tent!
Without any “pass” (“language level annotation”) you get “standard” Scala (or in case the compiler is in “beginner mode” respectively “basic” Scala). Most people will work with the “standard” code category most of the time, and this should be the “blessed way” communicated to the world. The dedicated experts can take care of the “advanced” components. Beginners can be restricted to “basic” Scala. All this can be easily monitored and enforced.
At the same time, this proposal would, imho, lower the barrier to entry for new features into the language considered “advanced”, as you could more effectively fence off those features from general use. (Something that language imports don’t do!)
Of course, marking things as “advanced” isn’t a bad thing in itself! It just needs to be clearly communicated that this stuff is for more experienced users, and if you commit to too much of this stuff, you may get into trouble at least in terms of finding the appropriate talent to maintain your stuff.
Of course, some amount of “advanced” code is likely to be needed almost everywhere. At least some of your deps will have this mark. But now you can see clearly where the (language) complexity is hiding, and fence off those parts if necessary.
What do you think? Does this make sense? Do you think it would make things better? Could it happen?
Oh, and while we’re at it; not really related, but something that could be done when touching these parts of the language anyway: How about renaming “experimental” to “unstable”? I think that fits the intent much better. But maybe this is just a ticket in the Scala wishlist bug tracker and not a real discussion topic…