What’s going on?
You say: feel like there is nothing special is going on here. The outer foo in bar increments the global x, and foo surrounded by let in bar increments the shadowed x. What’s the big deal?
The special that’s going on here is that LET can shadow the value of *x*. With lexical variables that’s not possible.
The code declares *x* to be special via the DEFVAR.
In FOO now the value of *x* is looked up dynamic. FOO will take the current dynamic binding of *x* or, if there is none, the symbol value of the symbol *x*. A new dynamic binding can, for example, be introduced with LET.
A lexical variable on the other hand has to be present in the lexical environment somewhere. LET, LAMBDA, DEFUN and others can introduce such lexical variables. See here the lexical variable x introduced in three different ways:
(let ((x 3))
(* (sin x) (cos x)))
(lambda (x)
(* (sin x) (cos x)))
(defun baz (x)
(* (sin x) (cos x)))
If our code were:
(defvar x 0)
(let ((x 3))
(* (sin x) (cos x)))
(lambda (x)
(* (sin x) (cos x)))
(defun baz (x)
(* (sin x) (cos x)))
Then X were special in all three above cases, because of the DEFVAR declaration, which declares X to be special – globally for all levels. Because of this, there is the convention to declare special variables as *X*. Thus only variables with stars around them are special – by convention. That’s a useful convention.
In your code you have then:
(defun bar ()
(foo)
(let ((*x* 20))
(foo))
(foo))
Since *x* has be declared special via the DEFVAR above in your code, the LET construct introduces a new dynamic binding for *x*. FOO is then called. Since inside FOO the *x* uses dynamic binding, it looks up the current one and finds that *x* is dynamically bound to 20.
The value of a special variable is found in the current dynamic binding.
Local SPECIAL declarations
There are also local special declarations:
(defun foo-s ()
(declare (special *x*))
(+ *x* 1))
If the variable had been declared special by a DEFVAR or DEFPARAMETER, then the local special declaration can be omitted.
A lexical variable directly references the variable binding:
(defun foo-l (x)
(+ x 1))
Let’s see it in practice:
(let ((f (let ((x 10))
(lambda ()
(setq x (+ x 1))))))
(print (funcall f)) ; form 1
(let ((x 20)) ; form 2
(print (funcall f))))
Here all variables are lexical. In form 2 the LET will not shadow the X in our function f. It can’t. The function uses the lexical bound variable, introduced by the LET ((X 10). Surrounding the call with another lexically bound X in form 2 has no effect on our function.
Let’s try special variables:
(let ((f (let ((x 10))
(declare (special x))
(lambda ()
(setq x (+ x 1))))))
(print (funcall f)) ; form 1
(let ((x 20)) ; form 2
(declare (special x))
(print (funcall f))))
What now? Does that work?
It does not!
The first form calls the function and it tries to look up the dynamic value of X and there is none. We get an error in form 1: X is unbound, because there is no dynamic binding in effect.
Form 2 would work, since the LET with the special declaration introduces a dynamic binding for X.