Kirk Rader  1.0-SNAPSHOT
Lexical Closures

Lexical environments in Scheme.

The observant reader may have noticed that examples of mutually recursive lambda expressions in a number of places were shown using disjoint sets of variable names, as in the basic example of functioncomposition". This was done mainly for clarity of exposition: it is very obvious which variables "belong" to which functions when they all have different names where their lexical or dynamic scopes overlap.

This is not actually a requirement of Scheme, where normal lambda bindings are lexically scoped. For example, the same example of function composition could have be written with both functions having overlapping sets of variable names with no change in meaning or outcome:

((lambda (f x) (f x)) (lambda (x) (+ x 1)) 2)

It simply would have been a bit harder to describe what was going since in this case there are two different variables both named x.

Historically, not all Lisp dialects have been lexically scoped in this way. Bindings for all variables existed in a single global environment in very early implementations of Lisp. This fact meant that there were times that the programmer actually did have to take care to use disjoint variable names for mutually recursive functions to avoid "name collisions" that could affect the outcome of a computation. That "bug" was sometimes exploited as a "feature" by some algorithms, so modern lexically scoped Lisp dialects often have alternative versions of special forms like lambda, let etc. that share a single, global name space. Some Scheme dialects, for example, have defined things like a fluid-let special form that works like ordinary let but whose bindings are in a single "fluid" (a.k.a. "dynamic") environment. Guile, which is used for the examples in this document, adopts the mechanisms defined by SRFI-39.

But wait! There's more!

If all there were to a lambda function's variable bindings that they are lexically scoped, they would not be substantially different from, say, pointers to functions in languages like C or anonymous functions in languages like Java. The lexical environments that are created by lambda binding in Scheme (and other modern dialects like Common Lisp) represent "closures" over their entire run-time context such that a function object, when treated as a data value, "remembers" the lexical environment in which it was created. Mutliple invocations of the same such function create distinct such "closed" environments, also known as "closures." Consider:

(define (make-foo a)
  (lambda () a))

(define foo1 (make-foo 1))

(define foo2 (make-foo 2))

After evaluating the three preceding expressions in a Scheme environment, you will have three global variables, all of which are bound to functions.

The function make-foo returns an anonymous function whose lexical closure includes a definition for the local variable a. Each anonymous function returned by make-foo "remembers" the value which was passed to make-foo when it was invoked with its own "closed over" binding of a. Both foo1 and foo2 are bound to functions made using calls to make-foo, but with different values for a. In particular, invoking foo1 returns 1 while invoking foo2 returns 2:



What this means is that anonymous functions in Scheme can be used in many cases where you might need to define some custom data structure in other programming languages. In fact, lexical closures are so powerful that there is a saying among Scheme programmers to the effect that "objects are a poor man's closures."

Here is an extended example, demonstrating the point:

;; -*- geiser-scheme-implementation: guile -*-
;; Copyright 2016 Kirk Rader
;; Licensed under the Apache License, Version 2.0 (the "License"); you
;; may not use this file except in compliance with the License. You
;; may obtain a copy of the License at
;; Unless required by applicable law or agreed to in writing, software
;; distributed under the License is distributed on an "AS IS" BASIS,
;; implied. See the License for the specific language governing
;; permissions and limitations under the License.
;; Function: (make-drill)
;; Create a simulated drill press.
;; Usage: (let ((drill (make-drill)))
;; (drill 'start-motor!)
;; (drill 'move-bit! 50)
;; (drill 'move-bit! 0)
;; (drill 'stop-motor!))
;; A drill has two attributes:
;; motor-state: 'stopped or 'running (initially 'stopped)
;; bit-position: integer between 0 and 100 (initially 0)
;; A bit position of 0 indicates that the bit is in its highest
;; position while 100 indicates that the bit is in its lowest
;; position.
;; It supports the following messages:
;; 'get-motor-state - returns current motor-state
;; 'get-bit-position - returns current bit-position
;; 'move-bit! new-position - simulates a stepper motor controlling
;; bit height
;; 'start-motor! - set motor-state to 'running
;; 'stop-motor! - set motor-state to 'stopped
;; the message handlers collectively perform checks to enforce the
;; following constraints:
;; * the bit position must be an integer between 0 and 100 (inclusive)
;; * the motor can only change state when the bit position is 0
;; * the bit can be greater than 0 only when the motor is running
(define (make-drill)
(let ((motor-state
;; state of the main motor
;; value determined by a stepper motor
;; controlling the height of the bit
;; A drill is a function that takes a message and additional
;; arguments that depend on the particular message
(lambda (message . arguments)
(letrec ((signum
;; return 0, 1 or -1 based on the sign of the given
;; number
(lambda (x)
(cond ((negative? x) -1)
((positive? x) 1)
(else 0))))
;; instruct the stepper motor to move the bit to the
;; specified height
(lambda (new-position increment)
(display "stepping bit position from ")
(display bit-position)
(display " to ")
(display new-position)
(display " by ")
(display increment)
(let loop ((done (= bit-position new-position)))
(unless done
(set! bit-position (+ bit-position increment))
(loop (= bit-position new-position))))))
;; set the value of bit-position or throw an exception
;; if constraints would be violated
;; see set-bit-position!
(lambda (new-position)
(cond ((not
(and (integer? new-position)
(>= new-position 0)
(<= new-position 100)))
(throw 'drill "invalid bit position" new-position))
((and (eq? motor-state 'stopped)
(> new-position 0))
(throw 'drill "motor not running" new-position))
(signum (- new-position bit-position)))))))
;; Set the value of motor-state to 'running or throw
;; an exception if constraints would be violated
(lambda ()
(cond ((eq? motor-state 'running)
(throw 'drill "motor already running"))
((not (= bit-position 0))
(throw 'drill "bit position not zero" bit-position))
(display "started motor")
(set! motor-state 'running)))))
;; Set the value of motor-state to 'stopped or throw
;; an exception if constraints would be violated
(lambda ()
(cond ((eq? motor-state 'stopped)
(throw 'drill "motor not running"))
((not (= bit-position 0))
(throw 'drill "bit position not zero" bit-position))
(display "motor stopped")
(set! motor-state 'stopped))))))
(case message
(unless (null? arguments)
(throw 'drill "invalid-arguments" (cons message arguments)))
(unless (null? arguments)
(throw 'drill "invalid arguments" (cons message arguments)))
(unless (= 1 (length arguments))
(throw 'drill "invalid arguments" (cons message arguments)))
(move-bit! (car arguments)))
(unless (null? arguments)
(throw 'drill "invalid arguments" (cons message arguments)))
(unless (null? arguments)
(throw 'drill "invalid arguments" (cons message arguments)))
(throw 'drill "unsupported message" (cons message arguments))))))))

The variable names and embedded comments make the preceding example fairly self-explanatory. Suffice it to say here that make-drill is a function that returns "instances" of "objects" that are really just anonymous functions created by evaluating a lambda expression, where the object's attributes are stored in a closed lexical environment. The drill "instances" provide functionality that is accessed by invoking them as functions (since that is all they really are), as in:

(let ((drill (make-drill)))

  (drill 'start-motor!)
  (drill 'move-bit! 50)
  (drill 'move-bit! 0)
  (drill 'stop-motor!))