Thoughts about ignoring def 'instructions' when typing

Language Design
1

I very heavily use - some say overuse - block initializations and block scoped methods (primarily methods within methods). Generally, my rule of the thumb is to put everything in as narrow scope as possible, which sometimes results in those nested definitions taking a considerable amount of space. This results in code akin to:

  val myField = {
    def initializer() :Int = {
        ...
    }
    initializer()
  }

In such cases, I’d much rather write

  val myField = {
    initializer()
    def initializer() :Int = {
    }
  }

Local methods can already call methods defined after them in the same scope.
I can’t, however, because the type of the above block is Unit. This is a degenerate example, so the case isn’t as pronounced, but with several helper methods and the actual initializing expression being something more than calling one of them, it really hinders readability. I sometimes resort to code as in the first example, with an artificial initializer method to bring the actual initializing expression closer to the field declaration, but that could be also argued as being just additional noise, especially as the reader can only trust
that the initializer method is called only once, at the end of the block.

I thus wonder if it wouldn’t be better if, when determining the type of a block expression, all def declarations were ignored. It would also help catch bugs such as code in the second example, were the type of the whole expression is inferred to be Unit because of a def.

  1. Is there a good reason for why the things are as they are, or couldn’t be as proposed here?
  2. Do you think it would actually be a change for the better?
2 Likes
2

On one hand, I kind of agree. On the other, it would become harder to see which is the expression that is being returned. Now it is unambiguous because it is always the last thing in the block whether there are local functions or not.

5 Likes
3

I asked dotty for begin and enhanced end for improved visibility.

To find where your code starts,

def f =
  def g = ...
  def h = ...
begin f
  g * h
end f

and to specify a result expression,

def f =
  def g = ...
  def h = ...
  def run = g * h
end f with run

This is a natural way to scope code and also reduce method size. But I agree some syntax tweak would help for clarity. I had not looked through the telescope in the direction you suggest.

What if end could come earlier in a block, before the defs? But isn’t that just return?

scala> :pa
// Entering paste mode (ctrl-D to finish)

def f = {
  return g
  def g = 42
}


// Exiting paste mode, now interpreting.


  return g
  ^
<pastie>:2: error: method f has return statement; needs result type

Probably there is a way to make that work, under the rubric, “Terminal return requires no result type.”

That would go a long way to rehabilitate the reputation of return in the language.

3 Likes
4

I don’t like the repetition in def f/begin f/end f, and essentially as presented here, it doesn’t address my qualms about the separation of the return statement from the method signature. Good catch with return, though. I am ashamed to admit I completely suppressed awareness of its existence, but if it were used only as here: as the only exit point from the method (how far the compiler would go to verify it could be up for discussion), with no ‘real’ dead code following it, I think it would suit me perfectly without stepping on Jasper’s toes, either. Somewhat amusing, as it is a close relative of one of the very few cases in C where goto is the solution: packing all error handling at the end of the method, after the return statement.

5

I am also often in this situation, a for-comprehension is at the end of a def and before there are a lot of def:

def a = {
  def b = ...
  def c = b * 2
  ...

  for {
    rc <- c
    rd <- d
    ....
  } yield rz
}

Personally, I want to see the relevant part (the for-comprehension) first when reading the method and the rest afterwards. Likewise, I don’t like that I have to place def b before def c just because b calls c. What I basically want are nested def which support forward reference usage.
I could define the nested def as normal private methods but I don’t like this for two reasons:

  1. they are very specific to this method and are most likely not reused in others.
  2. in this sense they would pollute my private API making it harder for other devs to see the relevant private methods in code completion etc.

I am not a scala compiler expert. I thinks that nested def are finally translated to local lambdas stored in a variable and there’s no forward reference usage allowed for local variables. In some situations I make use of this feature, something like

def a(p: P) = {
  var z = p.a.b.z
  def b = z.a() + 1
  def c = z.b().mkString("...")
}

So I prefer a solution which supports both situations. I propose a different direction to solve both needs. How about one can define private def inside a def where those are finally translated to real private def (and thus support forward reference usage but no local usage). The compiler would make sure that they cannot be called from another method on a scala level (preventing calls via reflection are out of scope IMO). So the following:

def a(p: P) = {    
  for {
    rc <- c(1)
    rd <- d
    ....
  } yield rz
  
  private def c(i: Int) = b * 2
  private def b = p.b() + 1
}

would translate to something like the following where name mangling and argument order could be different of course.

def a(p: P) = {    
  for {
    rc <- c(p, 1)
    rd <- d
    ....
  } yield rz
}
private def a$c(p: P, i: Int) = a$b(p) * 2
private def a$b(p: P) = p.b() + 1

For simplicity, I would not allow local variable usage in those functions, so the following would be disallowed:

def a(p: P) = {
  var z = p.a.b.z
  private def b = z.a() + 1
                  ^ local usage not allowed 
}

and think about later if we want to allow usage of val (which would be passed automatically as additional parameters).
Thoughts?

6

Here’s an ugly workaround, that may nonetheless be nicer to use than alternatives:

def foo(x: Int): Int = let(new:
  val result = helper(x)
  
  def helper(x: Int): Int = helper(x.toDouble)
  def helper(x: Double) = x.toInt
)

where you need to define:

abstract class WithDefs[A] { val result: A }
def let[A](wd: WithDefs[A]): wd.result.type = wd.result

Proper language support for this would of course be better.

2 Likes
7

IIRC that’s already how local defs are translated:

def instance(a: Int) = {
  val b = 42
  def local = a * b
  println(local)
}

<=>

def instance(a: Int) = {
  val b = 42
  def local = a * b
  println(instance$local(a, b))
}
private def instance$local(a: Int, b: Int) = a * b
8

That’s great news, in this case we are not that far away from a technical perspective I guess. And I see now that I was restricting myself unnecessarily. Local def can call other local def which are defined later on (i.e. forward reference usage is allowed). However, something like the following is not allowed:

def a = {
  val r = b
      ^ forward reference to method b defined 
  
 def b = 1 
  r
}

Why? I must miss something

1 Like
9

I just realized this could be greatly simplified, to a point where I’d almost consider it a practicable solution:


def foo(x: Int): Int = new:
  return helper(x)
  def helper(x: Int): Int = helper(x.toDouble)
  def helper(x: Double) = x.toInt

foo(1)

Unfortunately, Scala 3 currently complains with:

return outside method definition

Probably because the return ends up in constructor code in the generated class. Scala 2 also complains. But in principle, it could be made to work with non-local returns.

1 Like
10

You can make it a lazy val for order of execution hangups.

@LPTK I guess “ugly” is what smart people call what others think of as “clever”.

11

Nice!
What does new: do here though? I thought it creates an anonymous class extending the type expected by the compiler?

12

Yeah you’re right. The hope was that the unconditional return would let the compiler realize the new: was never returned, but that’s not really how Scala’s type checker works.

13

you’re right but it’s quite ugly IMO because it creates unnecessary overhead but more important, it does not need to be lazy. If local def are in the end private def then there is no forward reference. Shall I open a bug?

14

Not an expert, but I suppose it would be a feature request to look at what is captured before erroring. The conservative rule is easy to understand and averts

scala> def f = {
     |   val x = g
     |   val y = 42
     |   def g = y
     | }
         val x = g
                 ^
On line 2: error: forward reference to method g defined on line 4 extends over definition of value x

I don’t see related feature requests, but there are a couple of bugs, so it may not be a feature that pulls its weight in LOC? Dotty’s feature-request repo is forgiving about requests which are wish lists, or at least I’ve used it that way.