I’m saying it’s not useful because there’s no way to verify that it’s correct to infer Serializable unless you actually attempt serialization.
For example:
Welcome to Scala 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_121).
Type in expressions for evaluation. Or try :help.
scala> :paste
// Entering paste mode (ctrl-D to finish)
class NotSerializable(val value: String)
sealed abstract class Example
case class ErrorMessage(msg: NotSerializable) extends Example
case object Done extends Example
// Exiting paste mode, now interpreting.
defined class NotSerializable
defined class Example
defined class ErrorMessage
defined object Done
scala> val serializableInferred = List(ErrorMessage(new NotSerializable("oops!")), Done)
serializableInferred: List[Product with Serializable with Example] = List(ErrorMessage(NotSerializable@136ccbfe), Done)
scala> val os = new java.io.ByteArrayOutputStream()
os: java.io.ByteArrayOutputStream =
scala> val oos = new java.io.ObjectOutputStream(os)
oos: java.io.ObjectOutputStream = java.io.ObjectOutputStream@4ea1aa52
scala> oos.writeObject(serializableInferred)
java.io.NotSerializableException: NotSerializable
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at scala.collection.immutable.List$SerializationProxy.writeObject(List.scala:476)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
... 28 elided
The problem in general with Serializable is that it is a tag type – it adds no capabilities.
There is nothing you can do with a Foo & Serializable that you can’t do with a Foo. Calculating the intersection with it in a LUB is not useful in the sense of static typing and proving correctness during compilation.
I propose that tag types are not inferred unless the alternative is Any.
A tag type is one that adds no capabilities – no methods or members.
Everything you do with a tag type is a runtime check, so if you want to screen for one, use a pattern match.
I don’t think Serializable should be removed from case classes, I think Scala’s inference needs to change. The root of the problem is inference of unfriendly LUB types.
As for Product this is a truly useful trait in many cases, but just noise in others. ADTs automatically extending it could make cleaner LUBs. The ‘protected extends’ proposal or some sort of annotation to provide inference ‘hints’ are appealing to me. The question in my mind is whether that declaration goes with the definition of the type, or at the use site where it is extended from, or some combination of both.
My gut feeling is that the case for the former is strong with tag types or anywhere that the type has too little connection to static type safety.
I think that inferring Any is still preferable, because you have things like -Xlint:infer-any. The problem with Serializable and friends is often that it prevents this warning from firing.
If you suggest that the compiler automatically detects if a type adds no members and then doesn’t infer it, I’m not sure that’s a good idea since you could still have extension methods.
I would prefer the compiler not to infer Any. At least in my code Any as LUB is pretty much invariably an error. Just because the JVM offers silent boxing doesn’t mean that the Scala language has to use it.
Up until now Jvm Scala has been canonical Scala. And that makes sense with Scala.js, because to a large degree the capabilities of the Js platform are a subset of the capabilities of the Jvm. That won’t be the case with Scala Native. Scala Native is potentially not one platform, but many, as well as Linux, Mac and Windows it could eventually support WebAssembly via LLVM. So for example I would like canonical Scala to offer fully powered Structs / Compound value types, that could be just ignored and treated as normal AnyRef classes on Js and also on the Jvm. If Vallhalla arrives with limited Compound Value types then Scala would hopefully be able implement them on the Jvm with its limitations without having to limit Scala capabilities on all platforms.
Anyway I can see an argument for allowing AnyRef to be inferred, although even there I would tend to prefer it to be explicit, but not for inferring Any or AnyVal.
I think that inferring Any is still preferable, because you have things like -Xlint:infer-any . The problem with Serializable and friends is often that it prevents this warning from firing.
That is a good point. My thought process had been that inference of a tag type is no worse than Any, and potentially very slightly better. I’d rather be forced to ascribe Any if I’m not in a REPL and have the compiler fail rather than infer it, and its probably the same with a naked tag type that is basically a runtime distinguishable alias for Any or AnyRef.
If you suggest that the compiler automatically detects if a type adds no members and then doesn’t infer it, I’m not sure that’s a good idea since you could still have extension methods.
A tag type has nothing useful at compile time other than the type itself, no capabilities. It is mostly a runtime tag, for dynamic checks. Dotty has opaque types now, which seems to cover most of the compile time needs for a use case where one wants to create one type from another without adding capabilities.
Other JDK tag types (marker interfaces) besides Serializable are Cloneable and Remote. If these were made today, they would probably be implemented with annotations, but the JVM did not have annotations at the time they were created. A special exception could be made for only these, but I’m not convinced that is better than the general case.
Does anyone have any real world use cases of what I’m calling a tag type that are not dependent on runtime checks of the type? Its possible in Scala to do some things at compile time based on this type, but none that I can think of are actually useful given the other tools available, like opaque types.
I don’t do it often, but there are useful examples, largely having to do with state. An example I sometimes teach goes something like this:
case class Book()
trait CheckedIn
trait CheckedOut
def checkout(book: Book with CheckedIn): Book with CheckedOut = ...
def checkin(book: Book with CheckedOut): Book with CheckedIn = ...
It’s different from opaque types, because I’m not trying to hide anything about Book – I’m just literally adding a compile-time tag to it, so that the compiler can enforce that only something in an appropriate state can be passed to certain functions.
A sealed trait or abstract class has the compile time property of exhaustive match checks. When unifying ADT cases, even if some are non-case classes or objects, I do want the common sealed ‘type tag’ to be inferred.
A sealed trait or abstract class has the compile time property of exhaustive match checks. When unifying ADT cases, even if some are non-case classes or objects, I do want the common sealed ‘type tag’ to be inferred.
In Dotty, one would use an enum for an ADT, which would not qualify as a tag type.
As for sealed traits in general, they should not qualify. Although they are not adding capabilities, they are adding restrictions. The same is true of an abstract class with no members or methods – it represents a restriction. What I’m after can be mixed in with anything, not much more than Any. A sealed trait is more than that.
There are probably more restrictions needed to define what I’m after – essentially what Java calls a marker interface.
Re: opaque types – yes, if you don’t want to hide anything they aren’t a good fit.
Given the features available in Dotty, would you still teach the approach above? There is nothing preventing “new Book with CheckedIn with CheckedOut” for example. I would expect an ADT here, or some other structure if one needs to check in and check out various things and can not impose extra restrictions on Book.
The api above implies that one can not check out a naked book, but only one that has the CheckedIn state on it.
Inferring List[CheckedIn | CheckedOut] is not especially useful, if it might have Movies or Books in it, for example.
sealed trait LibraryItem
trait CheckedIn extends LibraryItem
trait CheckedOut extends LibraryItem
case class Movie()
case class Book()
Obvious uses of the above would indicate we need a few more rules:
inferring a sealed type in an intersection is OK
inferring a type that extends from anything other than Any / AnyRef (AnyVal?) OK.
In summary, I think I have to redefine what I’m looking for as a tag trait:
A trait
with no parameters, fields or type parameters
with no supertype – its free to mix in arbitrarily
that is not sealed
To use the Book case above and gain inference on CheckedIn and CheckedOut, you would have to make the types a member of a sealed trait or extend another trait. Using a sealed trait would help the use case in any event.
Alternatively, there could be a way to allow such traits to be inferred in intersections by default, something like @inferrable trait CheckedIn, or the opposite @noinfer trait Serializable.
EDIT:
Summary of my position: Serializable and I suppose Clonable and other Java marker interfaces are examples of use cases where the type is really only useful for a runtime check of some sort. Inferring these when LUBbing is not helpful, and its worse if the LUB contains part of an ADT. I suspect there is a way to categorize these in a more general way than making them special cases, but I may be wrong. I would want to simply not infer them when calculating a LUB. They can be inferred otherwise without issue, as far as I can tell.
Syntax like case trait X that would expand to trait X extends Product with Serializable would solve the problem in easy way.
case trait Base // expands to: trait Base extends Product with Serializable
case class A(x: Int) extends Base
case class B(y: String, z: Char) extends Base
case object C extends Base
val seq = Seq(A(5), B("dd", 'z'), C) // inferred to be Seq[Base]
seq would be inferred to Seq[Base] and not Seq[Base with Product with Serializable] because Product and Serializable are already in Base.
It would certainly solve the issue with inferring Product.
For Serializable, I’d really like for a case class or case object to only extend Serializable if all it’s fields explicitly extend Serializable. It’s not perfect, but it’s a reasonable middle ground between our current situation where Serializable is effectively useless, and the correct-but-infeasible solution that would require explicitly extending Serializable where applicable.
While case trait is more literate than the Scala 2 solution of reminding people to have their ADT rootextends product with serializable, it’s hardly any more effective at preventing the LUB problem. Library code could neglect it, or you could be extending classes other than the ADT root.
Also, we already have a new keyword for marking an ADT root; enum occupies that role.
I would like the opposite be practicioned - only make case classes of (immutable and thus) serializable data. Case classes are a good fit for immutable value classes not for e.g. avoiding new keyword for class with side effects.
Computing LUB has many quirks, I don’t think working around Product and Serializable will solve all LUB surprises. Sample REPL session:
Welcome to Scala 2.12.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_201).
Type in expressions for evaluation. Or try :help.
scala> abstract class A extends Ordered[A]
defined class A
scala> abstract class B extends Ordered[B]
defined class B
scala> List(null: A, null: B)
res2: List[scala.math.Ordered[_ >: B with A <: scala.math.Ordered[_ >: B with A <: Object]]] = List(null, null)
res2 got a crazy and useless inferred type.
It doesn’t matter how many supertypes a case class extends as long as it extends at least one common case trait (or analogously: as long as it extends one common trait <name> extends Product with Serializable).
scala> trait T extends Product with Serializable // common trait shared among case classes
defined trait T
scala> case class A() extends T with Ordered[A] with Iterator[Int] { def hasNext: Boolean = ??? ; def next(): Int = ??? ; def compare(that: A): Int = ??? }
defined class A
scala> case class B() extends T with Ordered[B] with Iterator[String] { def hasNext: Boolean = ??? ; def next(): String = ??? ; def compare(that: B): Int = ??? }
defined class B
scala> List(null: A, null: B)
res5: List[T with Iterator[Any] with scala.math.Ordered[_ >: B with A <: T with Iterator[Any] with scala.math.Ordered[_ >: B with A <: T]]] = List(null, null)
res5 got a crazy LUB, but neither Product nor Serializable is explicitly present in it, because T absorbed it.
case is not a new keyword. It’s already used for marking ADT leaves. Marking ADT root with it would make the whole syntax more regular.
While I agree with the principle that a case class should only contain immutable data, I really dislike the idea of attempting to enforce that by requiring all fields to extend Serializable. This would tightly couple a core language construct with a dying API, and require any non-case class immutable classes to manually extend Serializable if we want to use them as a field in a case class, even in cases where this capability will never be used.
It also wouldn’t enforce anything, as Serializable doesn’t actually require anything beyond the tedium of adding extends Serializable and generating serialVersionUID. Serializable is also fully compatible with mutable objects, so even if all the fields were legitimately serializable, it wouldn’t mean they’re actually immutable.
case trait DoNotEverDoThis
case class ThisWillSummonCthulhu(data: java.util.Vector[Int])
Yep, that’s 100% serializable, 100% mutable, and 100% a bad idea, but hopefully illustrates my point.
I meant that rather as an guideline than a strict enforcement, so there shouldn’t actually be any compiler warnings or errors about serializability, perhaps unless e.g. a specific linting rule is enabled.
Immutable does not imply serializable. All the usual examples of process and thread IDs, file handles, transactions, etc. can be represented as e.g. an Int in a case class, and there’s nothing wrong with doing that - but serializing it would be wrong.
Physically, any runtime value can be serialized using reflection, with no exceptions. Logically, you can’t deduce the advisability of serializing something from its type.
I just went and read the Scala 2.12 specification in detail. And in the section defining “Case Classes” (5.3.2 Case Classes), there is no mention whatsoever of the two traits “scala.Product” nor of “java.io.Serializable”: