Standard Library: Now open for improvements and suggestions!

Hmm. Very mixed feelings about (1). On the one hand, I definitely get where you’re coming from; in principle, I suspect you’re right. OTOH, I have a nasty feeling that the Scala ecosystem as a whole has the Seq == List equivalence baked in in a billion places. (In this instance, I’m even more worried about applications than libraries.)

If it does happen, it would need (IMO) to be accompanied by some sort of hard-assed scalafix utility that looks for inappropriate casts. (If that’s even possible – I haven’t done enough with scalafix to know if that would be a straightforward rule to write.) Otherwise, I worry about people hitting mysterious runtime exceptions.

Also, I worry a bit about serialization problems. I don’t know how common it is for people to serialize raw Seq, but it wouldn’t surprise me if it’s fairly common, and it might well have incorrect assumptions baked in.

1 Like

I am mostly concerned people are using Seq just because it is short and easy to type. And then they might be even indexing into it, forgetting that it is O(n) and trashes all the caches to top it off.

It is all about incentives. Someone (maybe even Martin Odersky) said that asInstanceOf is so long and hard to type because your are not supposed to do it often. We should make the correct data structures be easy to type and use.

One more thing, I have recently seen we have introduced IArray in Scala 3, which is further confusing, as it seems to be very similar to ArraySeq.

I feel all this needs to simplified / unified.

3 Likes

I regularly try Vector to replace List in some code that has a significant time use in my profiler, and Vector never clearly wins. Its often slower or has no measurable benefit. I don’t believe the guaranteed[1] cache locality of Vectors outweighs the generally more expensive operations.

This clearly does not mean that List is always better, just that changing the default might be quite invasive on programs that do use Seq.


  1. List may also have cache locality, if the elements are allocated closely together, which is likely with the way ListBuilder works. Also, I believe the JVM GCs can and will consider access patterns when moving memory around to improve cache locality on hot paths. ↩︎

1 Like

Early on we tried to change List in the Scala compiler to Vector. It got ~ 10% slower. YMMV, but that also shows it’s not a clear-cut win.

1 Like

How “early on” was that? Modern Vector (since Scala 2.13.2 in 2020) is a rather different beast than older Vectors. In particular, small vectors are more compactly represented.

HotSpot is a moving target as well, of course. We’ve just moved to 17+, and users who value performance should be (and likely are) on 21 or 25.

2 Likes

I guess it was around 2016? Not sure. Agreed that thuings might have changed since then.

  • Martin
1 Like

Vector is not the only option. If you only do filter, map, flatMap, like what you would expect on Seq, ArraySeq / IArray / Array should be a clear winners.

Also I assume for comprehensions unnecessarily wrap Array and IArray, so I hope this is fixed someday.

normally no, map, flatMap and withFilter on arrays are extension methods that return Array/IArray - if you do see an example that doesnt follow this then it might be an implicit conversion for a function that wasnt special cased

2 Likes

Another suggestion: sumBy(f)

I have checked my codebases and almost never I use .sum alone. It is always in combination with .map(f).sum. I know foldLeft exists, but it is not as readable.

This is especially useful on Set[T], because you can introduce a bug if you just call map, you must use .iterator.map(f).sum to avoid deduplication.

6 Likes

we merged in a change to make -> fully inlined from 3.10.0!

11 Likes

more merged PRs for 3.10:

7 Likes