SIP Suggestion: Add ?: and ?. syntactic sugar for more convenient Option[T] usage

Language Design
1

Option[T] is a great & principled way of handling missing / null values. However, it can be syntactically quite verbose.

Inspired by other languages that have chosen a foo(s ?: String) approach in favor of foo(s: Option[String]), how about introducing a little bit of syntactic sugar to smoothen the use of Option:

  1. def foo(s ?: String) can be called with either an Option[String], or with a String. If it’s called with a String, then the argument is wrapped in Option.apply().

    The type of s is Option[String].

  2. foo?.bar() is shorthand for Option(foo).map(_.bar()) if foo is not already an Option, otherwise it’s foo.map(_.bar()).

    If bar() returns a String, then the result of foo?.bar() is Option[String].

Advantages:

  1. Easier interop with Java null-returning libraries while retaining all of Option's benefits.

  2. Shorter & simpler yet (IMHO) clear syntax.

Disadvantages:

  1. May lead to culture wars about the most principled / correct way of doing things.

  2. May exacerbate operator overloading write-once-never-understand-again line noise.

Precedent:

  1. There’s already special syntax for having an arbitrary number of parameters passed to a function that leads to a non-shown type being used: foo(s: String*) type erasing to foo(s: Seq[String])

  2. There’s already a mechanism for having default parameters, i.e. making compile-time modifications to the method parameters.

Issues:

  1. How do we handle methods in JARs? Is foo(s: Option[String]) identical to foo(s ?: String), or does there need to be a special annotation included in the compiled code to indicate that it’s ?: not Option?

    Use same / similar mechanism as foo(s: String*)?

  2. How will it work with generics where it’s not known at compile time if a type T is an Option[U] or something else? What if T <: Option[String]?

  3. It’s possible to name a variable a_?, which would be somewhat confusing when mixed with this syntax, and force the introduction of separating whitespace: a_? ?.foo().

Apologies if this has already been suggested and discarded - couldn’t find anything when searching for it.

2 Likes
2

We could do this in a generalized way by introducing “postfix type” syntax. That would allow us to define a type alias for ? such that String? expands to Option[String]. I’m not sure how the type alias would be written. In order to sidestep that question let’s assume it works with an annotation. So you could do

@postfix type ?[+A] = Option[A]

An alternative that might be interesting once dotty supports explicit nullability would be

@postfix type ?[A] = A | Null

We could then say that vararg syntax (String*) is not a special grammar, but simply a special postfix type alias.

If you want to define things that can be used with Option-wrapped or plain values, that can be solved in other ways. For instance, you define a new type constructor, OptionOrRaw, with implicit conversions from Option and raw values. And of course you could alias your postfix type to OptionOrRaw.

Also, instead of special ?. syntax, you could define a custom operator like ?: that does the same thing.

3

I’m far more concerned about the appalling performance of Option than the syntax. By coincidence I was thinking about Option overnight as I’d been writing an A* pathfinder. The solution obviously needs to be efficient and the unnecessary boxing of Option is unacceptable. I was wondering whether to write my own

Opt[A <: AnyRef]

or if there is a standard solution out there for Scala. Option[Int] and Option[Double] are boxed twice I believe, when one boxing should be sufficient. However I haven’t encounter any situation so far where the performance of this actually mattered.

1 Like
4

I do believe there is a standard solution for Scala: https://github.com/sjrd/scala-unboxed-option

1 Like
5

Unboxed options have other problems, like

scala> UOption(1) == 1
res1: Boolean = true
scala> UOption(null) == null
res2: Boolean = false
6

Yes, although I’ve been wondering if you could get unboxed option completely right using the new opaque types. Seems to make sense at first blush; I don’t know if it’s been tried seriously yet…

7

@nafg: Using def foo(s: String?) to more closely match def foo(s: String*) sounds fine from a syntax perspective.

OptionOrRaw: could you give a small code example of how that would work?

?. / ?: custom operator: I’m not entirely clear on how that would work?

def ?[T](a:T) = Option(a) is possible, but doesn’t do the .map(_.bar()) part and isn’t an operator so clearly not what you meant :wink:

8 
sealed class OptionOrRaw[+A](val opt: Option[A])
object OptionOrRaw {
  implicit class Opt[+A](val self: Option[A]) extends OptionOrRaw[A](self)
  implicit class Raw[+A](val self: A) extends OptionOrRaw[A](Some(self))
  // implicit def toOpt[A](oor: OptionOrRaw[A]): Option[A] = oor.opt
}
def test(flag: OptionOrRaw[Boolean] = None) = flag.opt
test()
test(true)
test(Some(false))

It wouldn’t be possible to implement an exact equivalent to ?. but I suppose you could do this:

class Elvisish[A, B](a: A)(implicit asOor: A => OptionOrRaw[B]) {
  def ?:[C](f: B => C): Option[C] = asOor(a).opt.map(f)
}
implicit def toElvisish[A, B](a: A)(implicit asOor: A => OptionOrRaw[B]): Elvisish[A, B] = new Elvisish(a)(asOor)

1 ?: (_.toLong) // Some(1L): Option[Long]
Some(1) ?: (_.toLong) // same result
None ?: (_.toLong) // None

Not as terse as 1?.toLong. I don’t think it would be possible to implement it without either changing the language, or a bit of Dyamic+macro craziness (and even then not certain).

9

@jxtps Thank you for writing up your proposal. I’ll start with answering a few individual points that arose in the discussion.

This is normal. It is analogous to the fact that Option(null) != Some(null), but Option(1) == Some(1).

Note that USome(null) == null. In general, USome(x) == x for all x.

No you can’t do any better than unboxed-option, because the implementation of unboxed-option is already basically what an opaque type would provide. It’s just very hard to correctly implement without opaque types baked into the language, but in this specific case it can be done.

I see this thread as an opportunity to remind people of one very important thing: Option is not a better way to deal with null! It is a better way to deal with optional values. null itself is a poor way to deal with optional values. It is not type parametric, as you cannot represent an optional value of a type that can itself hold null. I have said that countless times, and for the most part I have stopped trying to explain this. But this thread might be the first time someone actually suggests this in a post including “SIP” in the title, so maybe it’s worth trying one more time.

For that reason, aliasing ?[A] to A | Null is a terrible idea. It’s a very common idea, unfortunately. Even Martin wanted to do that. The big problem if you do that is that it will be easier to write A? than Option[A], so people will write A?. But A | Null is not type parametric, and that will cause countless obscure bugs. We’ll have regressed to Java-like unsafety about null, perhaps even worse than Java because of the false sense of security in doing so. If at all, A? should desugar to Option[A]. Yes, Option boxes, but at least it is type parametric.

And that brings me to the original proposal.

This kind of “if it has this type shape, do that, otherwise do that” has no precedent in Scala, and it would be dangerous. Because if I have an A and I use this operator, I don’t really know whether null is supposed to be considered as “empty” or as “valid value that happens to be null”. This operator would end up turning too many nulls into None. It’s bad enough that Option(x) is named as it is (it should be Option.fromNullable(x) or something like that), but giving it a short alias like that is going to make it be used even more. But this constructor is completely overused already. How many have I seen people writing Option(x) instead of Some(x)?

So, at the very least we should scrap everything that implicitly converts from A to Option[A] in the proposal. If we do that we’re left basically with

  • a: String? as a parameter is something that
    • at call site, can be given a String, or can be not given at all.
    • at call site it would also make sense to allow passing x: _? when x is an Option[String], which corresponds to x: _* for varargs when x is a Seq[String].
    • at definition site, inside a method, a is an Option[String].
  • an “Elvis” operator x?.f, which is actually nothing more than syntactic sugar for x.map(_.f)

When you look at it that way, the two main bullets are really separate proposals. The second one probably wouldn’t hold its ground, because it is redundant wrt. for comprehensions, and does not provide a solution for the flatMap case. The first bullet is defensible.

13 Likes
10

2 out of 3 sub points are effectively equivalent to default parameter x: Option [String] = None with only difference being an ability to pass String without explicit wrapping. I don’t think it’s worth the effort…

The only thing I can say after recently working with typescript is that X | undefined and X | null are extremely painful to work with without Elvis operator but Option is far superior to both of these approachesand doesn’t need one.

3 Likes
11

I’m conflicted about the Elvis operator. On the one side, it is way too much cost for way too little gain. On the other hand, it feels like step in the right direction. I’m not satisfied with for-comprehensions. I would like to be able to write:

val x: Option[Int] = 1

val y: Option[Int] = 2

val z: Option[Int] = x + y

and end up with z being Some(3). Oh, and I don’t want Option to be privileged, I want the same to also work with Either or Future. I know, I’m demanding.

12

That’s somehow possible with current Scala + cats/scalaz as Option[T] forms a semigroup for every T that forms a semigroup itself. So you can write x |+| y and get what you wish.

Whats more important, I think it’s also possible for every other Applicative and any operation with mapN. Maybe we could have some better syntax for applicatives…

Edit:
Or maybe we already have. Just found on Twitter : https://github.com/ThoughtWorksInc/each/blob/3.3.x/README.md

1 Like
13

Of course, one could create classes that extend or work like Option and have a + operator or similar. But that’s not what I want. I want a language improvement where, if:

(1) I’m trying to apply a method on a reference of type M[T]

(2) M does not have that method

(3) T has that method

(4) M has a map method that takes a function T => U

it would automatically try to rewrite it using map. And analog for flatMap.

14

This strikes me as being kind of like implicit conversions – powerful, and kind of useful, but also dangerously “magical”. It would definitely cut down on code length, but I worry that that may be at the cost of comprehensibility…

2 Likes
15

The resulting code is as easy to understand as 1+2=3. :wink:

16

It’s easy to understand what’s the outcome, it’s hard to understand what’s going on under the hood when something doesn’t work.

2 Likes
17

I think it would be great to have optional arguments that do not require explicit wrapping on the call site.

I regularly want that, because the difference between foo("ab", arg = "cd") and foo("ab", arg = Some("cd")) is important IMHO, the latter having too much syntactic overhead. The only way to achieve this currently is to use a parameter with a default value of null, but this is obviously very unsatisfying.

There is also another use case for optional parameters: being able to give a proper function type to a method with default arguments, like in TypeScript. I have explained that point further in the following ticket: https://github.com/lampepfl/dotty/issues/4275#issuecomment-427654517

4 Likes
18

I stay unconvinced about the usefulness of non-wrapped optional parameters but the second argument is strong enough to convince me on its own. This makes language more regular and eta-expansion more comprehensible.

19

Failure modes would be way easier to understand than, say, a missing implicit.

20

This is not as easy to understand as 1+2=3. There is definitely room for complexity and confusion. In Scala we can already pass bare values as Options if we add an implicit conversion, so we can explore this.

implicit def toOption[A](a: A): Option[A] = Some(a)

And I have a function that takes an Option.

def f(s: Option[String]): Unit = s.foreach(println)

Now you can call f with a bare String.

scala> f("hi")
hi

Suppose I want all my Ints to be Strings.

implicit def intToString(i: Int): String = i.toString

Here’s a place where I see it becoming “hard to understand what’s going on under the hood when something doesn’t work.”

scala> f(3)
<console>:18: error: type mismatch;
 found   : Int(3)
 required: Option[String]
       f(3)

What?! I specifically requested Scala make my Ints into Strings. Now we have to add even more implicit conversions to help users avoid this unexpected scenario. I expect the result to be a huge mess of surprising implicit conversions and surprisingly missing implicit conversions.

2 Likes