Make Duration and FiniteDuration general?

How would that help?

Despite the shortcomings that many attribute to Future, there is one fundamental, objective thing going for it: it matches the semantics of the JavaScript built-in Promise. As such, it is extremely well supported in Scala.js, and lots of APIs, including core things like asynchronous tests in testing frameworks, build on top of it.

Moving Future out of the stdlib would have a severe impact on all of that.

3 Likes

Could you clarify this connection? As far as I’m aware, Scala.js support and matching the semantics of JavaScript built-ins isn’t really a driver in Future being the foundation of so many APIs.

I’d expect that Future would still be the default choice for concurrency in Scala, even if it were fundamentally incompatible with JavaScript’s concurrency model, simply because it’s the default concurrency primitive in the stdlib.

Oh no, if it had been completely incompatible with Scala.js, no testing framework would have used Future for their async test features, because the testing frameworks added async test features because of Scala.js. On the JVM you can always work around the lack of such support with an Await, but on Scala.js, if your testing framework doesn’t support async, then you simply cannot test anything async with it.

The small testing frameworks that were born soon after Scala.js, to support Scala.js, all added async support very quickly because of that. Bill Venners of ScalaTest told me he had started adding async support without taking Scala.js into account, had almost abandoned the endeavor, and had only completed it because he was made aware that it was a requirement for Scala.js support. Newer testing frameworks are expected to support Scala.js too, so they all have Future-based async support as well.

So if testing frameworks have async support today, it’s because of Scala.js, and clearly they wouldn’t have chosen an abstraction incompatible with JavaScript’s concurrency model.

4 Likes

Thanks for the clarification wrt async testing :+1:

Could you provide similar clarification for the broader assertion?

Well, I don’t have as precise or powerful examples as with the testing frameworks. But any other library that wants to cross-compile and offer a consistent API across JVM and JS chooses something that is not incompatible with JavaScript concurrency model.

Now, the fact that it is Future is for a number of them a direct consequence of Future being in the stdlib, as opposed to all the other similar-ish constructs out there. That choice is not due to Scala.js and the corresponding semantics with Promise, of course. They could equally use any other Task-like abstraction from libraries that cross-compile for JS, and some do.

My point is, if the thing in the stdlib had been fundamentally incompatible with Scala.js, a lot less libraries would have used that. And we would have needed to add another abstraction at the top of the dependency graph for use by testing frameworks, at the very least. And if it had required us to pour 100s kLoC to implement the one that is in the stdlib, we wouldn’t have either, and so we would still have had to invent a different abstraction for use the testing frameworks. So we would still have a situation where, at the top of the dependency, with a status very close to being in the stdlib, we would have had something similar.¹

I understand my initial sentence:

suggested a broader assertion. I should have decomposed it better, perhaps as follows: As such, it is extremely well supported in Scala.js. And therefore a lot of APIs, including core things like asynchronous tests in testing frameworks, could build on top of it, rather than use or invent something else with Scala.js support.


¹ For those who doubt that the testing frameworks would have adopted whatever we came up with, look at what happened with the use of run-time reflection that testing frameworks do: they do use portable-scala-reflect or a custom reimplementation thereof (not to have the actual dependency).

3 Likes

Here’s another option I’d like to throw into the ring:

  • Deprecate scala.concurrent.duration

  • Slowly migrate usage onto java.time.Duration

  • Implement java.time.Duration and other non-timezone-related APIs directly in Scala.js, just enough to replace the use cases of scala.concurrent.duration

  • Leave a full implementation of the broader java.time package in Scala.js for third party libraries that users could plug in if necessary

@sjrd would this work?

1 Like

Scala 2.13 finally has https://github.com/scala/scala/blob/2.13.x/src/library/scala/jdk/DurationConverters.scala to convert between Scala and Java durations so that alleviates the problem a lot IMO.

4 Likes

How does Scala Native fit into this? My personal preference would be to look towards maximum compatibility across all 3 platforms.

1 Like

Do you want to make a study of all the transitive dependencies of java.time.Duration?

  • Start with java.time.Duration
  • Add all the classes that are mentioned in there
  • Recursively add all the classes mentioned in those classes, until you add nothing more

Once a class is touched, even for just one field or method, the entire class must be implemented in the core, otherwise the remaining bits cannot be implemented in the non-core plug-in.

Isn’t linking done on a per method basis? Meaning it would be OK if there are methods on the class that reference other classes not provided, as long as those methods never get called at fastOpt or fullOpt time?

So as long as we can get feature-parity with scala.concurrent.duration without touching any timezones, that would be enough. This doesn’t seem like an obviously invalid approach.

1 Like

Could someone re-state what problem we are looking to solve here?

3 Likes

@lihaoyi It’s not about linking. Here it’s about the possibility to implement the missing pieces in an opt-in library. See:

2 Likes

As far as I am concerned, there is no problem with the status quo.

2 Likes

Could someone re-state what problem we are looking to solve here?

I’ll try, though IMO it’s more of a situation that could use improvement than a problem (if it is, it’s a nuanced one) :slightly_smiling_face:

s.c.Duration is essentially the defacto general time duration primitive used in the Scala ecosystem, at least in my experience and based on the numbers in the original post.

We don’t know why this happened, but my best guess is that its API is nice and plays well with the rest of Scala, and its implementation is general enough for most use cases, the implementation is short and doesn’t leak concurrency-related details, and it’s cross-platform + cross JVM compatible.

All would be fine except:

  • There is a scary warning at the top of the s.c.Duration Scaladoc warning ominously about using it as a general time duration abstraction.

  • Library authors need to support s.c.Duration, perhaps due to its adoption “in the wild” as a general duration primitive, but it feels weird to given :arrow_up:.

  • j.t.Duration, introduced with Java 8 and meant for general usage on the JVM, straight taking over for s.c.Duration has its downsides: the Java API doesn’t play nicely OOTB with Scala, necessitating imports and conversions (and associated instantiations).

    There’s also the fact that this option seems undersirable from a Scala-as-a-cross-platform-lang PoV (explained much better by @sjrd)

    • Side note, scala-java8-compat’s DurationConverters module converts java.time.Duration to-and-from s.c.Duration (link)

For writing Scala code, it feels like the situation could be improved, maybe by offering a simple general-purpose Scala-native Duration. This could be one which other, more-specific Durations can be built (e.g. Rust has std::time::Duration and a super-set Chrono lib that has chrono::Duration).

3 Likes

To me it seems that chrono::Duration now is largely redundant. It doesn’t bring anything anything useful except methods like Duration.weeks(x) but it then lacks floating-point division and multiplication (i.e. scaling by a factor). I think Duration.weeks(x) is rather of small importance - how often do you set timeouts in weeks? Never? std::time::Duration was progressing slowly and maybe that’s the reason chrono::Duration was created. scala.concurrent.duration.FiniteDuration is (and long was) more featureful than both Rust types. That means that if Scala’s Duration is not general enough, then Rust’s Durations are even worse. Also the conversions between that two mentioned Rust Durations result in Result type which is an equivalent to Scala’s Try which means you need to unwrap that on every conversion. Look how they convert their Durations:

    #[test]
    fn try_to_std_duration() {
        assert_eq!(StdDuration::try_from(0.seconds()), Ok(0.std_seconds()));
        assert_eq!(StdDuration::try_from(1.seconds()), Ok(1.std_seconds()));
        assert!(StdDuration::try_from((-1).seconds()).is_err());
    }

My only point with std::time::Duration and chrono::Duration is to demonstrate that you don’t need a std-lib Duration to solve all possible time-related use-cases: if needed, it can be built on top of. Specifically to Chrono: the crate itself serves a need that std::time does not (ISO 8601, TZs, efficiency); that it has its own Duration is due mostly to historical reasons IIUC.

IIUC then chrono::Duration isn’t built on top of std::chrono::Duration but has conversions. Similarly Scala’s FiniteDuration isn’t built on top of Java’s Duration but has conversions.

IIUC then chrono::Duration isn’t built on top of std::chrono::Duration but has conversions.

Correct. “can be built on top of” referred to the potential of a superset lib being built on top of std lib definitions. Specifically for Chrono

Chrono does not yet natively support the standard Duration type, but it will be supported in the future.

In any case, it’s a glimpse of how things can potentially be done, including with conversions. The point remains the same: IMO it would be nice std lib can provide a simple, general-enough Duration that can be built on top of; having it be BC with lots of existing code and cross-platform would be even better.

Adding support for something at some (not yet defined) point in time is the same as being built on top of it? Seems dubious. Rust’s std::thread::sleep takes std::time::Duration only, it doesn’t accept chrono::Duration. That doesn’t look like an extensible design (i.e. Rust is not any better than Java or Scala in this regard).

Scala has basic support for durations in its stdlib and Rust has basic support for durations in its stdlib. Extra support is provided by Java stdlib in Scala’s case and by chrono crate in Rust’s case. Conversions are explicit in both languages.

Scala could have generic support for durations by having a specific typeclass for it in stdlib, like it has for numeric datatypes (scala.math.Numeric). But is it worth the extra implementation and maintenance burden?

Scala has basic support for durations in its stdlib and Rust has basic support for durations in its stdlib.

One of the issues at hand here is whether the one that is included in the stdlib, s.c.Duration, currently commented as not meant as a general purpose representation of time, is indeed something one would count as “basic support for durations”, in the same vein as the one in Rust’s stdlib.

Some would argue, given that warning, and indeed, the package, that it is not. The usage of it in the wild, and some people’s perception of it (including your’s, @tarsa) indicates otherwise.

If it is a general duration, it would be nice to make that explicit by removing the warning comment and/or aliasing it in a more general package.

If it isn’t, decide whether it would be desirable to have a Scala-Native duration (for which good arguments have already been made in this thread), and if so, how to go about doing so.

1 Like