Could Cats core be imported / merged with the standard library?

Scala Standard Library
1

As I understand it a significant part of Shapeless is being imported to the Scala language. Could Cats core be imported / merged with the standard library? What would be the costs / downsides? I wondered if the new collections library would be more amenable to this.

1 Like
2

I’m a cats contributor and I think this would be a mistake to try to put all of cats core into the standard library.

Dependencies are not that costly if a big consideration is given to binary compatibility. The main offering of the standard library in my view is binary compatibility across an entire version of scala. In fact, cats has already done this: version 1.x are backwards compatible and 2.x are still compatible for scala 2.12 (which has more forgiving rules around traits).

I think we should move the other way: a smaller standard library and better tooling for maintaining binary compatibility for libraries. For instance: it would be interesting to have a way to do small binary breaks if we could easily see that virtually no downstream would be broken by that. Or perhaps tasty could offer us much more forgiving binary compatibility. Or of course, we could investigate more source based dependencies where incompatibilities become compile time breaks where they belong rather than runtime breaks. Or we could imagine a linker that runs on binary outputs and finds (or warns) of possible binary incompatibilities statically. Or…

A lot could be done here, So far almost all we have is MiMa, which is really great, but we could do more in this space.

9 Likes
3

I don’t think this is correct. My understanding is that many of the parts of it that were previously powered by macros will now be powered by language features, but as for the actual content of shapeless, it remains a separate library

4

What benefit would that have?

1 Like
5

A shared concept of basic category objects (functors, monoids, etc). Implementation and design spae are now rather stable (they weren’t a couple of years ago).

Still, I’m not sure at all it would be a good idea, I’m just statting what is the main benefits of having something in stdlib. And the main cost is stagnation, even if not-best-in-category implementation is in stdlib (read: “what scala still has an unique horrible XML library when it has a dozen good JS framework, iterated from a tens of ancestors”).

My personnal desire would be a more minimal stdlib with interoperable features lib. For example, the situation is OK in JS (for ex we made circe and liftweb-json talk to each other with a couple of mappers). And even for more fundamentals concepts like the one in Cats, it’s something that can be reached: zio interoperability with cats is excellent (and certainly no small work).

4 Likes
6

I don’t remember who said it, but I once read something along the lines of, “The standard library is where things go to die.”

7

While I like this idea in principle, one worry I have is the chatter over in Proposal: Implied Imports appears to be amiably hostile (for lack of a better term) towards orphan typeclass instances, and as you break out the standard library by spinning off various bits of functionality, you increase the number of times you’ll need to implement something like cats.Functor[org.json4s.JValue].

1 Like
8

So obviously objections could come from at least two places. Cats maintainers might feel that inclusion in the standard library would limit and compromise their vision, that it would hamper development and evolution. They might feel that Cats is not mature enough yet for the standard library.

On the other hand Scala main developers may feel their are problems with Cats that would make Scala main worse. Perhaps it would compromise compile times or not compose effectively with the existing standard libraries trait hierarchy. I know Cats developers are committed to pure functional code, would inclusion of Cats compromise the code of those that wanted to use mutation in a less structured or differently structured way?

However if Cats and Shapeless are to remain outside of the standard library, just as a user it would be good to know why? Cats and Shapeless are not just some domain specific libraries, they are not like a testing framework that can be bolted on after the fact. I presume those that are into Cats, use Cats in every project.

It seems to me that new Scala developers may well come across Cats if not Shapeless fairly early on in Scala development, so I feel it would be good to have a short comparison of the claimed advantages and disadvantages of using Scala with Cats and using Scala without Cats. Similarly with Shapeless. Scala is supposed to be becoming more opinionated, so it would good to know what that the Scala compiler and Standard library team don’t like about Cats and Shapeless. I don’t mean that in a bad or negative way, it would be just good to have a simple enunciation of the differing principles, priorities and preferences.

9

I think the obvious thing to mention here is the idea of Scala Platform. Sadly this idea seems to be dead as it was not picked up properly by SIP committee or Scala Center.

1 Like
10

I really think you’re proceeding from the wrong assumptions. You seem to be saying, “If this is good, then it should be in stdlib”. That’s exactly the reverse of the general sentiment I’ve been seeing in the community in recent years, which is that stdlib should only be for things that are essentially universal, and everything else should be stripped out – that stdlib should be getting smaller, not larger. And while, yes, Cats is a Very Good Thing, it’s far from universal – many, many projects don’t make use of it. (And that goes triply for Shapeless.)

This has nothing to do with Cats and Shapeless being disliked. It has to do with them being inappropriate for stdlib. Very different matter…

5 Likes
11

My question was what benefit the OP is looking for, specifically

12

OTOH it seems like there are some concepts that the stdlib is missing that could greatly enrich it and which have remained stable for several years across the existing FP libraries, i.e. some kind of Monoid thing. We already have type classes in there (e.g. Ordering).

I still think it’s worth discussing whether some subset of cats-kernel could end up in the stdlib in the future. I agree with @oscar when he says that it would be a mistake to put something like cats-core there.

Another topic that I don’t think has been discussed for a while is the idea of a scala-kernel module that contains only a bare minimum of classes required to compile Scala code, i.e. Unit, Tuple1-N, Product, Function1-N and so on.

13

The functionality, or at least a good chunk of it, that Cats offers does strike me as universal, its for use across all problem domains, perhaps it is not useful to every application , but I would imagine that it would provide something useful to virtually every at least medium sized project.

I haven’t used Cats up until now, because the useful functionality that I was aware of, was small things, easy to replicate, which didn’t seem to justify the complication of an extra dependency. However I would have used that pre designed, written and tested functionality, if had been in the standard library. The killer use case for me was Functor instances for the construction instances for instances of my own own type classes. So I will start using Cats myself once I’ve moved to 2.13.

But why would we not want to have things like biMap, map2 and flatMap2 in the standard library? You say many projects don’t make use of Cats. Why is that? Maybe in part its just because its not part of the standard library? Maybe they would use it if it was there. Perhaps they have developed their own implementations of much of this functionality. But is this necessary? Are they better for these developers use cases and technology stack or they just different?

I’m not saying that Cats Core or a substantial part of it should be in StdLib, I can speculate a number of reasons why that is currently not the case. Issues with the ScalaZ community prior to Cats and TypeLevel even existing, when the inclusion of implementations for the main Category theory abstractions was discussed before. Incompatibly with CanBuildFrom. Perhaps if Cats Core was made part of StdLib this would create problems for the ScalaZ, ecosystem or for the Twitter library stack?

What I am saying is that the case for not including this functionality in the StdLib in 2019, for me, has not been made. That as Scala is seeking to be more opinionated, and as the implementations in Cats have matured, the justification for leaving it to community libraries needs to be stronger.

14

I believe the primary reason for not using an FP library is the combination of not needing one, and preferring polymorphism and aggregation of capabilities through inheritance over implicit enrichment through typeclasses.

As for it not being included in the stdlib, well, I don’t see that much value in distributing stuff through the stdlib that don’t need to be in there – I prefer a more modular approach. Indeed, the stdlib is already splitting off modules, that aren’t that widely used, like parallel collections. Dependencies are not free, and that cats core jar is pretty heavy. I would argue that the futures and promises would also be better suited for a module than the stdlib, but I’m not sure how many people would agree to that.

2 Likes
15

Having Future and Promise in the stdlib helped immensely with Scala.js interopt, also in general Future is the generic async type in Scala. Having Future in the stdlib has helped tremendously with Scala’s ecosystem.

16

I’d like to see cats-kernel be merged into the standard library. This just includes the basic typeclasses Eq / Order / Hash / Monoid etc. so that they can be used in scala.collection.

I’m in favor of deprecating the following and shift to cats alternatives:

  • scala.math.Equiv => cats.Eq
  • scala.math.PartialOrdering => cats.PartialOrder
  • scala.math.Ordering => cats.Order
  • scala.util.hashing.Hashing => cats.Hash
4 Likes
17

To go off on this tangent, that sounds suspect to me – how is scalajs interopt harder with Future and Promise implementations in a module rather than in the stdlib?

It never was in a module, so we don’t really know whether having it in the stdlib rather than in a module has helped tremendously with the ecosystem, whatever helping with the ecosystem means exactly.

2 Likes
18

Binary compatiblity that is strictly tied to the version of Scala. Considering the history of all other Task/Future/IO types which break binary compatibility multiple times within the same Scala version, it would have been an absolute nightmare especially considering that async is a core feature in Javascript (unless you wanted to emulate callback hell in Javascript which isn’t even a pattern that any Scala developer uses).

Quoting one of the libraries that I maintain GitHub - zalando-nakadi/kanadi: Kanadi is a Nakadi client for Scala

Using another IO/Task/Future library “X”. Minimal friction with the ecosystem is a design goal of Kanadi which means we need to consider the lifecycle and dependencies of certain task types (to date none of the other IO/Task/Future types have had a good history when it comes to binary compatibility or lifecycle longevity). Although Future has its drawbacks, its ultra stable and portable across the Scala ecosystem.

With this library I am already have to maintain 3 branches due to binary compatibility, if Future also broke binary compatibility many times during its lifecycle the dependency matrix would have exploded. I would have loved to use something like Task, but it wasn’t an option for the reason stated.

Yes it was, it used to be part of Akka, see Futures (Scala) — Akka Documentation vs Futures (Scala) — Akka Documentation . It was then merged into Scala stdlib.

Believe it or not, for something as core asynchronous computation you need some abstraction in the stdlib. It doesn’t need to be perfect (although I would argue that using strict async abstraction like Future is better wrt interopt for other platforms like Scala.js since they all are strict by default so it make things a lot easier when wrapping libraries)

1 Like
19

Being a module rather than part of the stdlib doesnt affect binary compatibility of scalajs.

I also don’t agree that being part of Akka can be reasonably called being in a separate module, at least, not in good faith, especially when talking about modularizing to avoid heavy dependencies.

Believe it or not, for something as core asynchronous computation you need some abstraction in the stdlib.

I’d have a much easier time believing that if a truthful argument were presented for that.

2 Likes
20

It’s pointless to speculate whether or not the development of Scala.js would have been compromised or made more difficult if Futures and Promises had been in a module instead of the stdlib. I don’t think anyone can evaluate that specific question better than me (except perhaps Tobias), and I cannot tell whether or not it would have been the case.

I can say that it was convenient that they were in the stdlib, but I cannot tell whether it would have been fundamentally more difficult if they had not.

5 Likes