So I heard many times that the OOP features of Scala are best used as a module system. Packages also are a kind of module system. Would it be possible to unify these? It sounds aesthetic, not sure how feasible or desirable though. Any thoughts?
I guess what I’m saying is that having both packages and objects feels redundant, but leaving my post open-ended in case any other suggestions on the general modular programming topic come up that could add value to the language.
It’s hard because the JVM itself has a lot to say about what packages are and how they behave. Scala isn’t really free to redefine the concept of package too much, or we lose bidirectional interop with Java.
As an interesting data point, Ammonite scripts unify the concept of “source file” and “object” by making all source files objects. This gives a pretty Python-like experience where the name of the file you put things in becomes part of the import path to use those things, which can makes things easier to find, especially if you don’t use wildcard imports. In Normal Scala code, the name of the file is meaningless and you have to search through ~all files in the package to try and find what you want.
I believe Dotty’s support for top-level definitions uses the same technique (with the same limitations), though only as an implementation detail.
Honestly I prefer Ammonite’s Python-like style than the default Scala style of just having everything dumped into the enclosing package. Not sure how to get from here to there though, apart from making Ammonite’s script runner more fully featured so it can be used in more places
Controversial opinion: so what? 
Just saying, even if I had zero fingers, I could count on my hands how many times I’ve seen Java code depending on Scala libraries in the wild. IMHO, this goal has always held Scala back.
On the one hand, remember that much of the Lightbend stack is consumed heavily from Java code.
Even more importantly, though, it’s utterly routine for Scala code to use Java libraries, frequently many of them. That side of the equation is certainly central to the ecosystem…
No doubt! My comment was about the goal of making Scala code usable from Java – which it arguably already isn’t, unless the Scala code in question is basically Java. But removing that restriction would certainly open up a lot of opportunities for Scala.
Also, not to stir up anything, but since you brought up Lightbend… I don’t think they care all that much about Scala, and that’s recently become even more apparent. Why should the language be beholden to their needs? If there’s any other community examples of Java depending on Scala I’m all ears, but from my experience Java code already avoids Scala libraries like it’s COVID-19. How can they hope to deal with implicits and variance and macros and all the other things that make Scala better than Java in the first place? If you want to make a library for Java programmers, you’ll write it in Java. Otherwise you’re asking for trouble.
The two-way interop is most interesting perhaps not with libraries, but when you have a mixed codebase. Then you have arbitrary code calling back and forth between Java and Scala.
Lack of interop would largely prohibit Java-based mocking and dependency injection libraries (which is a good thing IMO as compile time reflection for me is a bad thing and Scala is good enough to do all the type machinery at compile time).
Interop is important not only for Lightbend projects like Akka or Play Framework, but also for projects like Apache Kafka or Apache Spark (both of which mix Scala and Java a lot). Spark (and other big data libraries) is considered a major catalyst bringing Java developers to Scala. Even a simple thing like dropping a special syntax for symbols ('something) which Martin wanted to repurpose fully to quotes and splices caused a major uproar from Spark developers. Play Framework started as Java project and then migrated to Scala, keeping the Java API. Do we want to make such migrations hard?
Scala does treat packages as some kind of objects. So I would claim there is already some unification going on, and Scala 3 goes a bit further than this than Scala 2:
private[p].This is also reflected in the Scala compiler: A package is represented as an object with a special flag.
The main difference between a package and an object is that a package is open – different files can declare members of by virtue of having a package clause.
What other capabilties of objects would people like to see in packages?
I’m going to hazard a guess at
Package objects can inherit, but they are phased away IIRC. How to implement linearization order for traits and classes from which a package would inherit without enforcing a single package object?
In addition to what’s been said, how about packages parametrized on packages (or maybe also other language constructs (see below some related thoughts))?
How about another angle at unification? Classes and objects. Is there ever any benefit from having different instances of a class whose ctor takes no arguments? Unless the ctor has side effects the answer might be no, but a side effecting ctor is probably an antipattern anyway. So maybe we could get away with just one language construct for these?
What I don’t know though is what would be an alternative to expressing companion objects.
I’m probably getting carried away a bit here, but all these language constructs we have, packages, classes, objects, but even vals and defs, they right now feel so similar to me. Classes’ bodies are constructors, right? so basically defs. Defs and vals (methods and functions) are mostly interchangeable. All of these can be almost arbitrarily nested. Then the current discussion on packages, classes and objects…
wouldn’t be surprised if much more unification could be done than I originally asked about. Maybe even collapsing it all into a single concept would be possible, but not necessarily that far. Maybe it would even be desirable.
It would be very different though. Still, if it’s practical improvement and simplification and doesn’t cause loss of expressivity I’d be happy to see it happen in say Scala 4. I can imagine it would be.
I can also imagine it wouldn’t. This sounds a bit like I’m going backwards and this way we would end up with something like just the core DOT, so that this already exists actually in this way. But I’m not sure if it would be a bad thing, given the amount of similarity between our different language constructs.
It also sounds a bit Lispy, especially blurring the line between data and code, classes (and by extension e.g. ADTs based on classes) and methods or functions. But that line is already blurred, functions are objects, class bodies are ctors… maybe the fact renders some of our syntax redundant and not actually adding expressivity.
Seconded. I would not have been able to adopt Scala in the first place without this.
Yes: when the class has interior mutability.
Actually lol’d at the thought that I didn’t realize this, didn’t take mutability into a account at all. Thanks for pointing this out.
Anyway, that whole post is pretty extreme in probably both impractical and unfeasible ways, most I can hope for is that it serves as inspiration to people who can come up with ideas that can actually be made to work.
I like the general idea, if any much smaller than proposed steps are ever made in that direction I think it will likely be nice.