Proposal for programmatic structural types

Scala Improvement Process
21

Reducing the feature set is to get back to acceptable complexity. This is for spec, implementation, and users trying to understand the library.

There’s also another aspect here. So far, the lions share of proposals that are discussed here and that will be discussed in the future were invented, spec’ed, and implemented by myself. I can’t clone myself, so have to prioritize what I can do. I can say with confidence that re-designing legacy structural types again is not something I will do, nor will I impose this on somebody else as a task.

So, any wish to change structural types from what they are now would have to come with a firm commitment to do the work.

22

I guess my real question here is: why were these features drop in the reimplementation in dotty? To the eye of someone who has never looked at the dotty Typer (i.e., my eyes), at least vars and overloaded methods don’t seem like they would impose any additional work (maybe an if here and there, but nothing fundamental). If I’m right, then I suppose I could even implement them myself. If I’m wrong, maybe you have a short enough explanation of what’s problematic?

23

Refinements in Dotty are name-based. I.e. a refinement is of the form

T { name: S }

Structural types are built from such refinements. It’s awkward to encode vars or overloading in this framework. In Scala-2, refinements were essentially some sort of anonymous class, so it would have been harder to drop these features than to keep them.

24

I think there are a lot of disadvantages:

  • It seems very useless to use records without var
  • I think it is very important having a way to access record’s data very quickly(like java invokedynamic)
  • I think the real killer feature of dynamic invocation is absence of binary incompatibility.

So I do not understand why this proposal is for type, I think It should work for trait either.
For example

   def newInstance[T](implicit tag: TypeTag[T]):T&Dynamic =  new DynamicProxy(tag)

In such implementation it will be very useful at least for me.

25

I would like to express my strong support for the proposal.

Building on the proposal, Olof Karlsson, a former student of mine, implemented extensible records and performed a comprehensive experimental evaluation, comparing with (1) old-style structural types, (2) case classes, (3) trait fields, (4) Shapeless, and (5) Compossible.

You can find the full results in the following paper:

Olof also gave a talk about this work at Scala Symposium '18:

What’s exciting about our records design is that it supports width and depth subtyping as well as type-safe extensibility also in a polymorphic context, using context bounds. Example:

  def center[R <: Record : Ext["x",Int] : Ext["y",Int]](r: R) = r + ("x", 0) + ("y", 0)

Here, the context bound : Ext["x",Int] expresses the requirement that it is safe to extend a record of type R with an “x” field of type Int.

Right now, our implementation requires a small extension of Dotty which synthesizes instances of an Extensible type class, which is used in the above context bound using the Ext type lambda:

  type Ext[L <: String, V] = [R <: Record] => Extensible[R, L, V]

Our records implementation easily supports more than 200 fields (important in some enterprise settings), in a scalable way, outperforming even case classes in some benchmarks (with more than 60 fields). See the above paper for microbenchmarks as well as a case study parsing JSON-encoded commit events from the GitHub API into typed records and processing them.

Implementation of the Dotty extension:

The performance evaluation uses a new benchmarking source code generator, called Wreckage, built on top of the JMH microbenchmarking harness:

5 Likes
26

I know this isn’t an ideal response, but I have never found a solution that’s best served with structural types. It’s always been either myself doing it wrong, or it’s been a case of a missing typeclass that represents the structure. I have used extensible records and similar mechanisms, and I’ve used Dynamic.

So my genuine question is if we actually need structural types in dotty? I know we’d need them for no-think porting of scala2 code to dotty, but are they actually required for dotty in and of itself?

27

Of course there are use case for it.
For example:

   //We have prototyped something like this in macros. . 
   def doSql[T](sql:String):Result[T] = ???
   
   def main():Unit = {
          doSql[{id:String;name:String}]("select '1' id, '2' name").foreach{r => 
                println(s"${r.id},${r.name}");
           }
    }

But in large datasets I prefer to use index access, so I do not use such method.
And for me, the dynamic invocation for traits(with var) would be more useful.

28 
def main():Unit = {
          case class IdName(id: String, name: String)
          doSql[IdName]("select '1' id, '2' name").foreach{r => 
                println(s"${r.id},${r.name}");
           }
    }

I’m not saying this is by definition an improvement, but it achieves the same thing without the structural type.

29

I will not argue about static or dynamic linking. I would prefer to use dynamic one in this case but it is just a holy war.
I have a question. Why do not we drop anonymous classes, functions etc. I am sure any language would be turing complete?

30

Defining a class per query is unnecessary. We can use literal types to write record types. SIP-23 gave this example:

Under this proposal we can express the record type directly,

type Book =
  ("author" ->> String) ::
  ("title"  ->> String) ::
  ("id"     ->> Int) ::
  ("price"  ->> Double) ::
  HNil

The query example then becomes:

doSql["id" ->> String :: "name" ->> String]("select '1' id, '2' name") ...

1 Like
31

It seems, such complicated abstraction for simple pojo very good ilustrates some lack of dynamic capabilities.

1 Like