Capture Checking in Scala 3.4

Written by Mark Hammons

Several new features have been introduced in Scala 3, but not all of them have been completed yet. If one peeks at the Scala 3 reference, one'll find a section detailing experimental features that are still in development which usually require either an @experimental notation or a nightly Scala version to use.

Capture checking is one of these experimental features, but in Scala 3.4 it was made available to anyone who merely imports language.experimental.captureChecking, opening it up to usage for anyone who is using Scala 3.4.

What is capture checking though? What does it do, and why is it being developed?

How Scala handles resource management

Typically, resource management in Scala nowadays involves automatic resource management that looks like the following pattern:

import scala.util.Try
import java.io.FileOutputStream

trait Releasable[A]:
  def release(a: A): Unit

object Releasable:
  given [A <: AutoCloseable]
      : Releasable[A] with
    def release(a: A) = a.close()

object ResourceManager:
  def apply[A, B](
      inst: => A
  )(fn: A => B)(using
      releaser: Releasable[A]
  ): Try[B] =
    val resource = Try(inst)
    val res = resource.map(fn)
    resource.map(
      releaser.release
    )
    res

object TypicalResourceManagement:
  val use = ResourceManager(
    new FileOutputStream("hello")
  )(_.write(5))

This automatic resource management pattern is good, handling the acquisition and release of a resource, but it doesn't guarantee much safety. It falls victim to the following weaknesses:

1. Resources can be exfiltrated from the manager

2. Resources can be closed prematurely
   

The second can be solved with some elbow grease in present-day Scala, but the first is not solvable within the constraints of Scala's type system. This is because the usage pattern above allows a generic return; no matter what one tries, that return can include the resource that needs to be managed.

The simplest example of how easy exfiltration is is the usage of the identity function.

val resource: Try[FileOutputStream] = ResourceManager(new FileOutputStream("hello"))(identity)

Here, the FileOutputStream has been directly exfiltrated using the identity function, giving access to the resource outside of the scope that should manage it. This is typically not a big issue for FileOutputStream because using it after closure will result in an error. However, not all resources will be closed or put into an unusable state when this scope exits; some return to a usage pool, or are put into a different state that may cause weird errors or bugs when they're used outside of their managed contexts.

In Scala 2, there were ways of defining a Not[A] type class, a type class that is only summonable if an instance of A is not available in context. This technique could be used to stop simple exfiltration of the resource via the identity function by adding a guard like this to ResourceManager.apply

def apply[A, B](
      inst: => A
  )(fn: A => B)(using
      releaser: Releasable[A]
  )(using Not[A =:= B]): Try[B] =

This guard would stop the identity function from working by making the Not cease to exist for A =:= B since A =:= B can be proven for the identity function.

This technique is not enough to stop the exfiltration of resources though, as they can accidentally or maliciously be embedded in another type, letting them be smuggled out. For example:

class Smuggler(fos: FileOutputStream)
//escaped
ResourceManager(FileOutputStream("text.txt"))(fos => Smuggler(fos))

As can be seen, the problem of resource exfiltration is intractable with Scala's current-day type system. If only something like def apply(inst: => A)(fn: A => B{inst}) existed, to indicate that B contained inst.

Oh, that's what capture checking does!

Capture checking: a fundamental enhancement to Scala's type system

Scala 3 has had some minor changes to its type system compared to Scala 2, including the introduction of new types, but capture checking is a new concept for all types in Scala 3.4. When one combines a type with syntax like ^{a,b,c}, one gets a type that is allowed to capture the capabilities of a, b, and c. a, b, and c here are parameters or local variables that the type is allowed to embed in itself. That is, the ^operator indicates that a type can capture something, and the elements encased in {} and separated by commas are the capture set of the type. An example would be Object^{a,b,c,d}, which is an Object that is allowed to capture a, b, c, and d.

When thinking about types with capture sets and types without, a type with a capture set is a supertype of a type without them, and a type with a subset of another's capture set is a subtype of the other. Finally, a type with just the ^ is allowed to capture anything and everything it wants (it has the universal capture set).

So, the gist of capture checking is that it adds a method of detecting the capture of parameters or local values to Scala's type system. There are caveats however that need to be kept in mind when using this system.

What can be checked? What does it mean to allow capture?

Take a look at the following function:

import language.experimental.captureChecking

class CaptureCheckingDemos:
  class Test1(obj: Object)
  def fn1(obj: Object): Object = Test1(obj)

In this code, an instance of Test1 captures obj, and is returned as an Object, but in the previous section, it was noted that types without a capture set cannot capture anything. So will this compile?

The answer is yes, this code still compiles even with capture checking enabled. That's because any type without a capture set is considered pure, and pure types are not considered by the capture checker at all. If one wants to track the capture of a parameter with capture checking, one must define a capture set for the parameter's type. However, in this code, there's nothing specific to track for capture. In fact, it's doubtful one would care if the input to this method has captured anything. That's where the universal capture set comes into play. ^ appended to a type by itself indicates that the parameter has the universal capture set, and is allowed to capture anything.

def fn2(obj: Object^): Object = Test1(obj)

This new approach results in a compilation error, but not because Test1 captures obj. The compilation error is related to Object^ not being Object. Thinking back to the previous section, it was noted that Object^ is a supertype of Object. This effectively means that Test1 cannot be constructed from Object^ because Test1expects to be constructed from a pure object. Defining a new wrapper type fixes this variance incompatibility:

class Test2(obj: Object^)
def fn2(obj: Object^): Object = Test2(obj)

Now capture checking is doing its work. It complains that Test2(obj) has the type Object^{obj}, but the return type defined for the method is just Object. Basically, it's prevented the capture of obj by Test2.

Going back to the resource manager defined before, it's possible to redefine it in a way that prevents resource leakage:

trait Releasable[A]:
    def release(a: A^): Unit

  object Releasable:
    given [A <: AutoCloseable]: Releasable[A] with 
      def release(a: A^): Unit = a.close()


  object ResourceManager2:
    def apply[A, B](
        inst: => A^
    )(fn: A^ => B)(using
        releaser: Releasable[A]
    ): Try[B] =
      val resource = Try(inst)
      val res: Try[B] = resource.map(fn)
      resource.map(
        releaser.release
      )
      res

  class Smuggler(fos: FileOutputStream^)

  //doesn't compile
  val test = ResourceManager2(FileOutputStream("test.txt"))(a => a)

  //doesn't compile either
  val test2 = ResourceManager2(FileOutputStream("test.txt"))(fos => Smuggler(fos))

With this new definition, Scala rejects these attempts at smuggling the resource out of the new resource manager.

Is Capture Checking ready for use?

Capture checking is still a work in progress. While it's improving day by day, it still has bugs, and it has a nasty habit of crashing build servers like bloop. While it may be tempting to try to integrate this feature into your codebase, keep in mind that it's extremely experimental, and all of its features and syntax haven't been solidified yet. Still capture checking is an incredibly powerful new addition to Scala's type system, and hopefully, it will blossom into something truly great in the future.

In the meantime...

Happy Scala hacking!

__

To submit a blog, please contact Patrycja