[p-dev] Closures and scopes

[Paul Bone]

In C-like languages a new block can create a new scope for variables.  But
that's not how things work for Plasma normally.  For example:

    if (...) {
        # complex computation over many lines
        x = ...
    } else {
        x = ...
        y = ...
    }
    i_can_use(x)
    # I can't use y, it is only available in the else branch.

But it could be useful to have the kinds of scopes you get in C like languages, particularly because Plasma is single-assignment.  We can add a special scope blog for this purpose.

    scope {
        x = ...
        i_can_use(x)
    }
    # x is out of scope.

Michael Richter suggested adding an additional feature to these scopes.  The
same scope could be used to "hide" a variable so you can have a fresh one in
a local scope.

    y = 1
    scope hide (y) {
        # y is not in scope.
        y = "quax"
        # y is now "quax", it's totally different from 
        # the y above, as if it had a different name, 
        # so of course it can have a different type 
        # and value.
    }
    # The original y is back in scope, it is 1.

One thing is missing, if I want some variables to be available after this
block (like if-then-else) but still want x to have that smaller scope, I'd
need to list either the ones that get the smaller scope or those that get
exported. So I could add more syntax for that.

    y = ...
    scope (x) hiding (y) {
        # Can't use y here.
        x = ...
        z = ...
        
        y = "New y if I want"
    }
    i_can_use(x)
    # Can't use z here.
    z = "but I can make a new one"

I think I like this idea, it might be something that's used rarely but when
it is desirable it's going to be _very_ useful.

This led to another idea.  When I posted about this on reddit[1]
u/PegasusAndAcorn gave me the idea that such scoping rules could also be
used for closures.  Either by requiring all captured variables to be
explicitly listed, or having multiple types of closures, one which requires
all captured variables to be explicitly listed.  We discussed this for a
while on IRC and Michael came up with the following regarding the first
option. I've paraphrased these.

    Pluses:
     + Explicit, it makes it clear exactly what's getting captured, useful
       for large closures.
     + States intent clearly, if you see a closure that explicitly closes
       over 'a' but doesn't use it, that's a bug, the compiler could raise
       it.
    Minuses:
     + Verbose, making things normally concise such as lambdas more verbose
       removes some of their utility.
     + Cognitive load, programs will need to think more about code as they
       write it.
       Ed: this can be spun as a "plus" since it leads to better discipline,
       but Michael is pointing out that this will be met with resistance.
    Interesting:
     + This may be a way to encourage programmers to think more explicitly
       about their code, not just closures.

    Note (Ed):
     + The implementation cost is minor so this wasn't included as a minus.

There are actually a few different options here.

 1. We could require that all closures have this restriction.
 2. Add a special 2nd closure syntax that enables this restriction.
 3. Have this for "normal" closures (those with the syntax of nested
    functions, but not for lambda expressions.
 4. Don't change closures, instead allow people to get this behaviour by
    wrapping them in scopes like those above.

Note that this feature doesn't make sense for partial application, so that
would be the same in all cases.

Some of these are opt-in and therefore avoid some of the minuses.  The last
one avoids making this a concious part of people's programming, unless it
becomes a well-known idiom which I wouldn't bet on, so these loose the
explicit plus.  Personally I like #3.  It avoids the verbosity minus for
lambdas, which is when you want things to be concise, and otherwise has the
same benefits.

So while function declarations normally look like, with optional resource
clause (omitted here):

    func name(arg1 : Type1, arg2 : Type2, ...) -> RetType1, RetType2
    {
        ...
    }

We can add a new "with" "scope" or "closure" clause and require it when
functions are nested within other functions.

    func name(arg1 : Type1, arg2 : Type2, ...) -> RetType1, RetType2
        scope var1, var2, ...
    {
        ...
    }

Since this almost always creates a closure, we could change the "func"
keyword into "closure" also.  So that things say what they are.  A closure
that captures no variables is not a closure and can be labeled "func"
instead.  This difference is important because a closure usually requires a
memory allocation, but a function does not.  This difference would not be
reflected it lambda expressions (ones that are closures and ones that
aren't) which is a small drawback, but a different issue.  (Partial
application always creates a closure.)

I'm curious if anyone has any feedback, but I also wanted to capture these
ideas in one place that I can find later.  If you made it this far through
the e-mail, thank you.

--

 1. https://www.reddit.com/r/ProgrammingLanguages/comments/6nk6b1/block_to_the_future/dkb9k2l/?utm_content=permalink&utm_medium=front&utm_source=reddit&utm_name=ProgrammingLanguages


-- 
Paul Bone
http://paul.bone.id.au