LISP FUNDAMENTALS

Math operations in Common Lisp are probably different than what you're used to.

Common Lisp math operations use prefix notation: the math operator is a function that comes at the beginning of the parenthesis, and all numbers afterward evaluated using that operator, from left to right. In school, we learn math using infix notation, where the operators are placed between each number.

(+ (* 5 5 5) (/ 18 2) (- 20 3)) ; using infix notation: 5 * 5 * 5 + 18 / 2 + 20 - 3

Let's look at a slightly more complicated example of Common Lisp syntax:

(defun cube (x)
  (* x x x))

After the opening parenthesis comes defun, a built-in macro. defun has different evaluation rules compared to regular function calls. It's used to define a function. The function above is given the name cube. After giving the function a name, we define its parameters, the data it takes as arguments to run the operations inside the function. The function takes one argument: x.

Notice that cube isn't evaluated as a variable. The x in (x) isn't initially interpreted as a function to be called.

defun's first argument is a symbol designating the name to give the function, and the second argument is a list of arguments that the function is expected to receive when it's called. The forms that follow afterward are called the body of the function.

Common Lisp supports both globally and lexically scoped variables. There are several ways to define variables.

Use defvar to introduce a global variable.

(defvar *debug* nil)
*debug*

NIL

To rebind a value to a defvar variable, you need to call setf on the variable. To change the initial value, you can't simply redefine the defvar and recompile the form.

(defvar *debug* t)
*debug*

NIL

Still nil.

You have to use setf to modify the value of a defvar variable.

(setf *debug* t)
*debug*

T

Use defparameter to introduce a global variable with a starting value that can be modified by recompiling the defparameter form.

(defparameter *debug* nil)
*debug*

NIL

(defparameter *debug* t)
*debug*

T

Global variables defined with defvar and defparameter are surrounded with * by convention.

Use defconstant to introduce a global variable that has a value that can't be modified.

(defconstant +pi+ 3.14)
+pi+

3.14

(setf +pi+ 42)

+PI+ is a constant and thus can't be set.

If you try to recompile the defconstant with a different value, you'll be given the option to redefine the constant via a restart.

(defconstant +pi+ 42)

The constant +PI+ is being redefined (from 3.14 to 42)

Constant variable names are surrounded with + by convention.

Use let to introduce or reassign a local variable.

(let ((name 'micah))
  name)

MICAH

You can also reassign a global variable temporarily.

(defparameter *var* 'outside)
*var*                   

OUTSIDE

(let ((*var* 'inside))  
  *var*)              

INSIDE

And the value outside the let will remain.

*var*

OUTSIDE

If you need to use a variable defined within a let within the same let form then you need let*.

This let form will break:

(let ((data 'data)
      (clean-data data))
  clean-data)

The variable DATA is unbound.

Change the let to let* to make it work properly:

(let* ((data 'data)
       (clean-data data))
  clean-data)

DATA

While variables are an important kind of symbol, they aren't the only kind of symbol.

One example of a non-variable symbols are keywords. Keywords are any symbol that begins with : (colon). Keywords evaluate to themselves; they don't hold some other value.

:this-is-a-keyword

:THIS-IS-A-KEYWORD

Keywords are used especially in property-lists and associative-lists as the key in a key/value relationship.

Function names, class names, etc. are all symbols, too. Symbols (besides keywords) are themselves a data structure and can hold more than one piece of information.

There are several functions for getting at the data saved in a symbol.

While most of the time you'll use defparameter, let, defun, etc. to define symbols, sometimes you'll need to do symbol surgery, constructing and interning symbols in manually. Some of the above functions will come in handy for those kinds of operations.

There are several different kinds of parameters you can define. The most common will be required parameters. Parameters defined without any other options are required.

(defun my-fun (this-is-required))

If you don't want a parameter to be required, you can use the &optional lambda list keyword to make it optional.

(defun fun-with-optional (a &optional b)
  (format nil "The values passed: ~a and ~a.~&" a b))

(fun-with-optional 5)

The values passed: 5 and NIL.

The default value for optional arguments not passed is nil. You can set the default value.

(defun fun-with-optional (a &optional (b 10))
  (format nil "The values passed: ~a and ~a.~&" a b))

(fun-with-optional 5)

The values passed: 5 and 10.

You can create a predicate that will return whether the value was supplied or the default is being used.

(defun fun-with-optional (a &optional (b 10 b-supplied-p))
  (if b-supplied-p
      (format nil "The values passed: ~a and ~a (passed by user).~&" a b)
      (format nil "The values passed: ~a and ~a (default).~&" a b)))

By convention the predicate is named *-supplied-p, with * being the name of the parameter. Its value is t if the user supplied an argument in that position, otherwise it defaults to nil.

Using the default value of the optional argument:

(fun-with-optional 5) 

The values passed: 5 and 10 (default).

With a value passed to the optional argument position:

(fun-with-optional 5 10)

The values passed: 5 and 10 (passed by user).

All parameters following the &optional lambda list keyword will be optional. This is true of all lambda list keywords.

(defun fun-with-lots-of-options (&optional (a 1) (b 2) (c 3))
  (+ a b c))

If you want an argument to be both optional, but also want the user to be able to set the argument in any position by name, then you can use the &key keyword.

(defun fun-with-keys (&key a b c)
  (+ a b c))

(fun-with-keys :a 1 :b 2 :c 3)         

As with &optional, you can specify default values.

(defun fun-with-keys (&key (a 1) (b 2) (c 3))
  (+ a b c))

*-supplied-p parameters can also be specified. If specified, a *-supplied-p parameter will be either t or nil depending on if the user passed a value for the keyword.

(defun fun-with-keys (&key (a 1 a-supplied-p) (b 2 b-supplied-p) (c 3 c-supplied-p))
  (format nil "~a (~a) + ~a (~a) + ~a (~a) = ~a~&"
          a a-supplied-p
          b b-supplied-p
          c c-supplied-p
          (+ a b c)))

(fun-with-keys :a 5 :c 7)

5 (T) + 2 (NIL) + 7 (T) = 14

You can collect arbitrary arguments into a list with &rest.

(defun add (&rest args)
  (apply #'+ args))

(add 1 2 3 4 5)

15

It is possible to mix the different kinds of parameters.

As a general rule, you don't want to mix &optional and &key parameters.

(defun do-not-mix (&optional (a 1) (b 2) &key (c 3))
  (+ a b c))

(do-not-mix :c 5)                       

The value
  :C
is not of type
  NUMBER

Use one or the other, but not both.

You can mix &optional and &rest. Mixing &key and &rest will result in an error if you put &key before &rest, but it's okay to put the &rest before the &key.

(defun key-before-rest (&key my-key &rest args)
  (print my-key))
(key-before-rest :my-key "&key before &rest, you'll have a bad time.")

Execution of a form compiled with errors.
Form:
  #'(SB-INT:NAMED-LAMBDA KEY-BEFORE-REST
      (&KEY MY-KEY &REST ARGS)
    (BLOCK KEY-BEFORE-REST (PRINT MY-KEY)))
Compile-time error:
  misplaced &REST in lambda list:
                               (&KEY MY-KEY &REST ARGS)
   [Condition of type SB-INT:COMPILED-PROGRAM-ERROR]

(defun rest-before-key (&rest args &key my-key)
  (print my-key))

(rest-before-key :my-key "&rest before &key, you're okay.")

&rest before &key, you're okay.

If you want named arguments in your &rest args, you can supply key-value pairs in the &rest argument and manually parse them yourself.

(defun rest-args-as-plist (&rest args)
  args)

(rest-args-as-plist :a 1 :b 2 :name 'micah)

(:A 1 :B 2 :NAME MICAH)

We'll see a bit later what's going on in loop below, but this is just to say that you can have an arbitrary number of key-value pairs in your &rest argument.

(loop :with args := (rest-args-as-plist :a 1 :b 2 :name 'micah)
      :for (k v) :on args :by #'cddr
      :when (eql k :name)
        :do (return v))

MICAH

In Common Lisp, the value returned by the last expression called in a form will be the return value.

(defvar *some-var*)
(defun do-a-bunch-of-stuff ()
  (setf *some-var* "I assigned a value to *some-var*.")
  (let ((num 5))
    (square-then-double num)
    (random 10)
    "I am the value that this function will return"))
(do-a-bunch-of-stuff)

There are cases, however, when you might want to do an early return, especially when looping. When using loop, you can do early returns with return. In other cases, use return-from.

(defun early-return-example ()
  (let ((nums '(2 2 4 4 6 6 7 8 8 8)))
    (dolist (n nums)
      (when (oddp n)
        (return-from early-return-example (format nil "~a is not even." n))))))

(early-return-example)

7 is not even.

It's possible to return multiple values using the values function.

(defun return-a-bunch-of-stuff ()
  (values
   (+ 2 7)
   (* 7 7)
   (/ 100 25)))
(return-a-bunch-of-stuff)

9, 49, 4

A simple let won't bind all of the values returned by a function that returns multiple values. Instead, only the first value will be bound.

(let ((val (return-a-bunch-of-stuff)))
  val)

9

If you need to bind multiple values, use multiple-value-bind.

(multiple-value-bind (a b c)
    (return-a-bunch-of-stuff)
  (format t "~a * ~a * ~a = ~a" a b c (* a b c)))

9 * 49 * 4 = 1764 => NIL

Sometimes you might want to take a list and break it into pieces–called destructuring. For that, there's destructuring-bind.

(destructuring-bind (a b c)
    (list 1 2 3)
  (format t "~a * ~a * ~a = ~a" a b c (* a b c)))

1 * 2 * 3 = 6 => NIL

This destructuring can be done on arbitrarily deep trees of cons cells.

(defparameter *deep-tree* `(defun function-name (a lambda-list)
                            (let ((b :some-value))
                              b)))

(destructuring-bind (the-defun the-function-name (the-a the-lambda-list)
                     (the-let ((the-b the-value-bound-to-b))
                      the-b-at-the-end))
    *deep-tree*
  the-value-bound-to-b)

:SOME-VALUE

When a variable is passed to a function, a new, local variable is introduced with the value of the variable passed to the function.

(defvar *some-num* 5)
(defun add-5 (num) ; local variable num is introduced
  (setf num (+ num 5)))

num is assigned the value of *some-num*.

(add-5 *some-num*)

*some-num*

5, not 10

If you want to modify the value of the global variable, you need to use setf on the global variable directly.

(defun add-5 ()
  (setf *some-num* (+ *some-num* 5)))

(add-5)

*some-num*

10

Common Lisp functions are first-class. Many of Lisp's functions can take functions as arguments.

To pass a function as an argument, you have a few options.

You can use function:

(mapcar (function square-then-double) '(1 2 3 4 5))

Or, more commonly, use the reader macro #':

(mapcar #'square-then-double '(1 2 3 4 5))

The funcall and apply functions take a function name for their first argument, applying the function to the rest of the arguments.

(funcall #'+ 1 2 3)

6

(apply #'+ '(1 2 3))

6

You can use anonymous functions with lambda:

((lambda (x) (* x x)) 5)

25

Lambdas are useful when using functional programming functions like mapcar or remove-if-not.

(mapcar (lambda (x) (+ (* x x) (* x x))) '(1 2 3 4 5))

(2 8 18 32 50)

Because lists are so fundamental in Lisp, there are many functions for manipulating lists.

length returns how many items are in the list:

(length '(Gerald Sussman stole my wife and kicked my dog that skallywag))

11

reverse returns a new list that has the order of the elements of the initial list reversed:

(reverse '(1 2 3 4 5))

(5 4 3 2 1)

first, second … tenth get items in a list based on their position in the list:

(let ((my-list '(common lisp is a general purpose multi-paradigm programming language)))
  (list (first my-list)
        (second my-list)
        (ninth my-list)))

(COMMON LISP LANGUAGE)

member searches for a single item in a list. If it finds the item, it returns a list where the first item is the item searched for, and then the rest of the items that come after it:

(member 'general '(common lisp is a general purpose multi-paradigm programming language))

(GENERAL PURPOSE MULTI-PARADIGM PROGRAMMING LANGUAGE)

nth selects individual items by their index:

(nth 4 '(common lisp is a general purpose multi-paradigm programming language))

GENERAL

position tells you the index of an item in a list:

(position 'general '(common lisp is a general purpose multi-paradigm programming language))

4

append combines two lists:

(append '(this list is a list) '(a list))

(THIS LIST IS A LIST A LIST)

Lists can be used as key/value pairs, called property lists, using keywords.

(defvar *micah* (list :name 'Micah :age 40))

*MICAH*

Use getf to get the value for a certain key.

(getf *micah* :age)

40

Combine setf with getf to reassign a value for a key.

(setf (getf *micah* :age) 99)
*micah*

(:NAME MICAH :AGE 99)

Lists are a very simple data structure capable of making other data structures. Consider the linked list of cons cells above:

(cons 'defun (cons 'sum (cons (cons 'x
                                     (cons 'y nil))
                               (cons
                                (cons '+
                                      (cons 'x (cons 'y nil)))
                                nil))))

(DEFUN SUM (X Y) (+ X Y))

While it is simply a linked list of cons cells, it's useful to think of it another way: a tree. Each cons cell has two parts: a car and a cdr. The car holds a leaf in the tree, whereas the cdr holds either another branch (in the common case) or another leaf. When working with lists as a tree, you can use car and cdr to access those two positions.

(defparameter *my-tree* (cons 'defun (cons 'sum (cons (cons 'x
                                     (cons 'y nil))
                               (cons
                                (cons '+
                                      (cons 'x (cons 'y nil)))
                                nil)))))

Now call car on the tree:

(car *my-tree*)

DEFUN

(cdr *my-tree*)

(SUM (X Y) (+ X Y))

copy-tree returns a copy of a tree of cons cells:

(copy-tree *my-tree*)

(DEFUN SUM (X Y) (+ X Y))

subst returns a new list that substitutes leaves in the tree:

(subst 'z 'y *my-tree*)

(DEFUN SUM (X Z) (+ X Z))

sublis does the same with multiple leaves in the tree:

(sublis '((sum  . subtract)
          (+    . -)
          (x    . a)
          (y    . b))
        *my-tree*)

(DEFUN SUBTRACT (A B) (- A B))

sublis takes a special type of list for its first argument: an association list, otherwise known as an alist or table. An alist is a list with nested lists.

(defparameter *en-to-ja-table* '((one   . ichi)
                                 (two   . ni)
                                 (three . san)
                                 (four  . yon)
                                 (five  . go)))

With alists, the car is a key and the cdr is a value. You can search a table by either key or value using assoc and rassoc.

(assoc 'two *en-to-ja-table*)

(TWO . NI)

(rassoc 'ni *en-to-ja-table*)

(TWO . NI)

You can get at the value using cdr.

(cdr (assoc 'two *en-to-ja-table*))

NI

You can either use a dotted-list–a cons cell that has a non-nil value in the cdr position as in *en-to-ja-table* above–or a regular list.

(defparameter *en-to-ja-table* '((one   ichi)
                                 (two   ni)
                                 (three san)
                                 (four  yon)
                                 (five  go)))

The only difference is that you need to use cadr to get at the value if you don't use dotted-lists.

(cdr (assoc 'two *en-to-ja-table*))

(NI)

(cadr (assoc 'two *en-to-ja-table*))

NI

Lists can also be treated like sets–an unordered sequence of unique elements.

With adjoin, you can add a single unique element.

(adjoin 'three '(one two three))

(ONE TWO THREE)

Notice that 'three only occurs once. Since it was already in the "set", it wasn't added.

(adjoin 'four '(one two three))

(ONE TWO THREE FOUR)

Contents of two sets can be compared to form new sets:

(defparameter *pizza* '(salty sweet cheese sauce round carbs))
(defparameter *cake* '(sweet chocolate brown carbs round))

intersection returns a set of unique elements that are present in both *pizza* and *cake*:

(intersection *pizza* *cake*)

(CARBS ROUND SWEET)

union returns a set that combines all unique elements of both *pizza* and *cake*:

(union *cake* *pizza*)

(SAUCE CHEESE SALTY SWEET CHOCOLATE BROWN CARBS ROUND)

set-difference returns a set that includes all of the elements in *cake* that are not present in *pizza*:

(set-difference *cake* *pizza*)

(BROWN CHOCOLATE)

Or vice versa:

(set-difference *pizza* *cake*)

(SAUCE CHEESE SALTY)

subsetp returns t if a set is a subset of some other set or nil otherwise:

(defparameter *cheese-pizza* '(salty cheese sauce round carbs))
(subsetp *cheese-pizza* *pizza*)

T

In the section on variables, setf was used above to modify the value of a variable defined with defvar. While modifying variable values is a typical use, setf is used for modifying places in many other situations.

For example, it can be used to modify the value of an item in an alist. Say we have an alist like this:

(defvar *some-alist* (list (list 'a 1) (list 'b 2) (list 'c 3)))
(second (assoc 'c *some-alist*))

3

We can modify the value associated with the c key using setf:

(setf (second (assoc 'c *some-alist*)) 7)

7

Now let's check the value of c again.

(second (assoc 'c *some-alist*))

7

In BEYOND LISTS, we will see how to use setf to modify the values of items in other kinds of objects beyond lists.

For symbols, variables, lists, and other objects, you need to use one of eq, eql, equal, or equalp. Each of them tests equality of different degrees. If you come from Python or JavaScript you'll usually expect functionality similar to equal or equalp.

eq will test equality of identity. Do these two objects share the same place in memory?

(defparameter *deez-nums* '(1 2 3 4 5))
(defparameter *your-nums* '(1 2 3 4 5))
(defparameter *gods-nums* '(one two three four five))

(eq *deez-nums* *your-nums*)                ; NIL, two different lists.
(eq *deez-nums* *deez-nums*)                ; T, same list.
(eq *gods-nums* '(one two three four five)) ; NIL
(eq 'one 'one)                              ; T, symbols are reused when they
                                            ; are from the same package.
(eq '(1 2 3 4 5) '(1 2 3 4 5))              ; NIL, two lists are constructed
                                            ; separately.
(eq #\a #\a)                                ; T
(eq #\a #\A)                                ; NIL
(eq "hello" "hello")                        ; NIL, strings are arrays of
                                            ; characters and are constructed
                                            ; separately.
(defparameter *greeting* "Hello, world!")
(eq *greeting* *greeting*)                  ; T, same array of characters.

eql is the same as eq, except that if the arguments are characters or numbers of the same type then their values (not their places in memory) are compared.

equal tests structural similarity.

(equal '(1 2 3) '(1 2 3))               ; T
(equal '(1 3 2) '(1 2 3))               ; NIL
(equal "hello" "hello")                 ; T
(equal "HELLO" "hello")                 ; NIL
(equal 1 1.0)                           ; NIL, different types

equalp is further lenient:

(equalp #\a #\A)                        ; T
(equalp "hello" "HELLO")                ; T, good for case-insensitive testing
                                        ; of characters or strings
(equalp 1 1.0)                          ; T, good for testing numbers across
                                        ; number types
(equalp '(1 2 3) '(1 3 2))              ; NIL

Characters and strings have their own equality operators: char-equal and string-equal.

(char-equal #\a #\A)                    ; T, same as (equalp #\a #\A)
(string-equal "hello" "HELLO")          ; T, same as (equalp "hello" "HELLO")

Additionally, there are tests like char-greaterp, string=, char<, etc.

For more detail you should look at the Hyperspec.

Common Lisp has the typical logical operators.

and tests if two or more forms are true. All forms are evaluated left-to-right, and evaluation stops if any inner forms return nil. The and form returns the value returned by the last form inside it if all forms in it evaluate to non-nil values.

(and 't 5)          ; => 5
(and 't 5 'hello)   ; => HELLO
(and 'nil 5 'hello) ; => NIL

or returns the value of the first form that evaluates to true. If no form passed to it returns a true value, it returns nil.

(or 5 't 'nil 'hello)                  ; => 5
(or 'nil (> 1 5) (eq "hello" "hello")) ; => NIL

or will stop evaluating forms on the first form that evaluates to true.

(or 'nil (> 5 10) (= 1 2) ; all evaluate to nil
    (print "Evaluated, returns true.")
    (print "Not evaluated."))

Evaluated, returns true.

not will return t if the inner form returns nil, and nil if that form returns t.

(not (= 1 1))  ; => NIL
(not (oddp 2)) ; => T

There are a number of different conditional forms. Of course, there is trusty ol' if. if is a special operator. It takes a test argument. If the test returns true, the next form is evaluated. If the test returns false, then the form after that is evaluated.

(if (> 10 1)                   ; if
    '10-is-greater-than-1      ; then branch
    '10-is-not-greater-than-1) ; (optional) else branch

If you don't need a second (else) branch, you can use when or unless:

(let ((num 0))
  (when (<= 1 num 10)
    (format t "~&FROM WHEN EXPRESSION: ~a is between 1 and 10" num)
    (print num)
    (print (+ num num)))

  (unless (<= 1 num 10)
    (format t "~&FROM UNLESS EXPRESSION: ~a is not between 1 and 10" num)
    (print num)
    (print (+ num num))))

(unless test ...) is equivalent to (when (not test) ...).

One of the upsides of not having an else branch is that you can do multiple operations after the test with when and unless.

cond takes lists of tests and forms to evaluate if the test returns t. The parentheses can be tricky here.

(cond (test form-to-evaluate-if-test-returns-t)
      (test form-to-evaluate-if-test-returns-t))

Here is a practical example:

(defparameter *monster-happiness-meter* 49)
(let ((mhm *monster-happiness-meter*))
  (cond
    ((>= 0 mhm)      (format t "~&Get this monster lifting weights at the gym, now!")    'take-monster-to-gym)
    ((<= 70 mhm 89)  (format t "~&This monster is pretty happy.")                        'hang-out-with-monster)
    ((<= 50 mhm 69)  (format t "~&This monster is feeling a little down.")               'invite-monster-to-lunch)
    ((<= 30 mhm 49)  (format t "~&Get this monster's mommy on the phone.")               'call-monsters-mommy)
    ((<= 1 mhm 29)   (format t "~&Did this monster's grandma die or something?")         'console-monster)
    (t               (format t "~&This is the catchall fallback expression.")            'fallback)))

CALL-MONSTERS-MOMMY

cond will stop evaluating on the first form that returns t.

case takes a key expression and then some clauses. It evaluates the clauses in order. If the value of the first item of a clause is eql to the value returned by the key expression, the rest of the items in the clause are evaluated.

case is part of the family of case forms: case, ccase, ecase, typecase, ctypecase, and etypecase. If you know case statements from other languages, you understand the basic idea.

case and typecase return nil if no match is found. If you want to trigger errors when no match is found (rather than providing an fallback clause), use ecase and etypecase. If you want the option to provide a value for the key expression and continue the program, use ccase and ctypecase.

(defparameter *env* :DEVELOPMENT)

(defun check-environment ()
  (case *env*
    (:DEVELOPMENT "logging EVERYTHING")
    (:PRODUCTION "locking in and locking down")))

(check-environment)         ; => "logging EVERYTHING"

(let ((*env* :PRODUCTION))
  (check-environment))      ; => "locking in and locking down"

(let ((*env* :AWS))
  (check-environment))      ; => NIL

case will return nil when none of the other cases match. You can set the fallback case using t or otherwise.

(defun check-environment ()
  (case *env*
    (:DEVELOPMENT "logging EVERYTHING")
    (:PRODUCTION "locking in and locking down")
    (otherwise "you gotta set your *env* to :DEVELOPMENT or :PRODUCTION")))

(let ((*env* :AWS))
  (check-environment))

you gotta set your *env* to :DEVELOPMENT or :PRODUCTION

If you want to return an error if the argument falls through all the checks, use ecase.

(defun check-environment ()
  (ecase *env*
    (:DEVELOPMENT "logging EVERYTHING")
    (:PRODUCTION "locking in and locking down")))

(let ((*env* :AWS))
  (check-environment))

:AWS fell through ECASE expression.
Wanted one of (:DEVELOPMENT :PRODUCTION).
[Condition of type SB-KERNEL:CASE-FAILURE]

Restarts:
0: [RETRY] Retry SLY evaluation request.
1: [*ABORT] Return to SLY's top level.
2: [ABORT] abort thread (#<THREAD tid=11923 "slynk-worker" RUNNING {70071BE463}>)

If you want a continuable error, use ccase.

(defun check-environment ()
  (ccase *env*
    (:DEVELOPMENT "logging all errors")
    (:PRODUCTION "locking in and locking down")))

(let ((*env* :AWS))
  (check-environment))

:AWS fell through CCASE expression.
Wanted one of (:DEVELOPMENT :PRODUCTION).
[Condition of type SB-KERNEL:CASE-FAILURE]

Restarts:
0: [STORE-VALUE] Supply a new value for *ENV*.
1: [RETRY] Retry SLY evaluation request.
2: [*ABORT] Return to SLY's top level.
3: [ABORT] abort thread (#<THREAD tid=11955 "slynk-worker" RUNNING {7007B83833}>)

Notice that now you can store a value in *env*. If you do, it will retry the case check with that value.

The typecase family works the same, but will check the type of value returned by the key expression.

(let ((x 5))
  (typecase x
    (list 'this-is-a-list)
    (number 'this-is-a-number)
    (function 'this-is-a-function)
    (otherwise 'i-dont-know-what-this-is)))

THIS-IS-A-NUMBER

handler-case is another member of the case family that works on conditions and errors. We'll look at it more in the chapter on Errors & Conditions.

You can iterate by using a functional programming style mapping. You've seen it in action already:

(mapcar #'square-then-double '(1 2 3 4 5))

(mapcar #'first '((1 2 3 4 5) (one two three four five) (what is going on here) (once
 upon a time)))

(1 ONE WHAT ONCE)

There are several varieties of mapping functions. mapcar is the most common. The most general mapping function is map.

;;; Go through each character in the string and upper case it. Return a string.
(map 'string #'char-upcase "hello")

"HELLO"

;;; Take an item from each of the lists and multiply them, returning a list.
;;; Finishes at the end of the shortest list.
(map 'list #'* '(2 4 6 8) '(3 5 7 9 11 13 15) '(10 10 10))

(60 200 420)

You can reduce multiple values down to one value by applying some function to successive items in the sequence with reduce.

(reduce #'+ '(2 2 2))

6

(reduce #'* '(2 2 2))

8

(reduce #'append '((2 2 2) (3 3 3) (4 4)))

(2 2 2 3 3 3 4 4)

If you define a function to use for reducing, it needs to take two arguments:

(defun sum (x y) (+ x y))

(reduce #'sum '(1 1 1 3 4 5 6 7))

28

You can filter the contents of a sequence with remove-if-not:

(remove-if-not #'oddp '(1 2 3 4 5 6 7 8 9 10))

(1 3 5 7 9)

You can define your own predicate and pass it to remove-if-not:

(defparameter *dangerous-animals* '(lion tiger bear snake shark))

(defun safe-animal-p (animal)
  (not (member animal *dangerous-animals*)))

(remove-if-not #'safe-animal-p '(dog cat monkey lion hamster shark snake bear
 koala frog))

(DOG CAT MONKEY HAMSTER KOALA FROG)

(defun greater-than-50-p (num)
  (> num 50))

(remove-if-not #'greater-than-50-p '(40 50 30 90 80 10 70 100 25 60 55 2))

(90 80 70 100 60 55)

You can also use remove-if:

(defun dangerous-animal-p (animal)
  (member animal *dangerous-animals*))

(remove-if #'dangerous-animal-p '(dog cat monkey lion hamster shark snake bear
 koala frog))

(DOG CAT MONKEY HAMSTER KOALA FROG)

There are several iteration forms in Common Lisp that involve "doing". Among them, the most popular are dotimes and dolist.

dotimes is for doing something a set number of times.

(dotimes (i 5)
  (if (= i 4)
      (print "I'M NOT CRAZY!!!")
      (print "I'm not crazy!")))

"I'm not crazy!" 
"I'm not crazy!" 
"I'm not crazy!" 
"I'm not crazy!" 
"I'M NOT CRAZY!!!"  => NIL

dolist will simply iterate through a list. If you're doing simple stuff with lists, this is probably what you want.

(dolist (i '(yet another beautiful short list))
  (format t "~&~a" i))

YET
ANOTHER
BEAUTIFUL
SHORT
LIST => NIL

dotimes and dolist also take an optional result-form argument. The result-form will be returned at the end of the iteration.

(dotimes (i 5 (format t "~%Finished."))
  (print i))

0 
1 
2 
3 
4 
Finished. => NIL

The most common use of the result-form is to return data that is collected/transformed during the iteration.

(defun my-reverse (list)
  (let ((result nil))
    (dolist (item list result)
      (push item result))))

(my-reverse '(this list will be returned in reverse order))

(ORDER REVERSE IN RETURNED BE WILL LIST THIS)

Another common use is to return t

(defun all-even-p (nums)
  "Return T if all NUMS are even, NIL if any are odd."
  (dolist (i nums t)
    (format t "~&Looking at ~a..." i)
    (when (oddp i)
      (format t "~&Oops, this is odd!")
      (return nil))))

(all-even-p '(1 2 3 4 5))

Looking at 1...
Oops, this is odd! => NIL

(all-even-p '(2 4 6))

Looking at 2...
Looking at 4...
Looking at 6... => T

It's not strictly necessary to have a result-form. You can just manually return it.

(defun my-reverse (list)
  (let ((result nil))
    (dolist (item list result)
      (push item result))))

(defun my-reverse-manual-return (list)
  (let ((result nil))
    (dolist (item list)                 ; result not set as result-form
      (push item result))
    result))                            ; reversed manually called at the end of
                                        ; the let block

Notice the use of push here. It is a destructive function that modify lists by adding items. push puts items at the front of the list, returning the modified list.

The corresponding pop function removes the first item and returns that item.

(let ((my-list '(1 2 3 4 5))
      (result-list nil))
  (dolist (item my-list (list my-list result-list))
    (push (pop my-list) result-list)))

(NIL (5 4 3 2 1))

do is the most general and powerful of the do family. It is also a bit complicated and difficult to read/understand.

(do ((var1 init1 [update1])
     (var2 init2 [update2])
     ...)
    (test action-1... action-n) ; base case
 body)

(defun all-even-do-p (nums)
  (do ((n nums (cdr n)))
      ((null n) (return t))
    (format t "~&Looking at ~a..." (first n))
    (when (oddp (first n))
      (format t "~&~a is odd." (first n))
      (return nil))))

(all-even-do-p '(2 4 6 7 8 10))

Looking at 2...
Looking at 4...
Looking at 6...
Looking at 7...
7 is odd. => NIL

Unlike dotimes and dolist, which take care of incrementing the counter or stepping through the list, do requires you to specify the step/update at the end of each iteration. It also requires you to take care of specifying the base case–the conditions for ending the loop.

do is useful especially for people who are comfortable with iterating recursively because recursion also requires the programmer to specify the stepping function and base case.

(defun check-all-even-recursive (nums)
  (cond ((null nums) t)
        ((oddp (car nums)) (format t "~&~a is odd!" (first nums)) nil)
        (t (format t "~&~a is even." (car nums))
           (check-all-even-recursive (cdr nums)))))

(check-all-even-recursive '(2 4 6 7 8 10))

The step function is (cdr nums). Using cdr to step through a list is called "cdring down" the list. The similarities between the recursive method and the do method make it both powerful and often unergonomic. As a reader of code, when you see either a recursive or do iteration, you have to check the step function and base case–something you don't have to do with dotimes, dolist, mapcar, remove-if-not, etc.

The loop iterator is much more widely used than the do iterators (although for small tasks dotimes and dolist are still common), so if you wish for most other Lispers to understand your code and collaborate with others, you should generally prefer loop.

If you prefer a more applicative/functional style, then the map family, remove-if and remove-if-not, and reduce are the way to go.

The most popular iterating construct in Common Lisp is the loop macro:

(loop :for item :in '(one two three)
      :do (print item))

loop is a macro that uses its own syntax. If you come from Python or JavaScript, it doesn't look so strange, but it looks different from typical Lisp code.

(loop :for n :in '(1 2 3 4 5)
      :collect (* 2 (sqrt n)))

(2.0 2.828427 3.4641016 4.0 4.472136)

The loop macro will get a fuller treatment in a later chapter.

If you only need to test the sequence, check-all-even could be rewritten using every.

(every #'evenp '(2 4 6 8 10))
                                        ; => T
(every #'evenp '(1 4 6 8 10))
                                        ; => NIL

every runs a test on all items of a sequence and returns t if the test returns t for every item.

Similarly, some tests all items of a sequence, but will return t as soon as the test returns t for an item.

(some #'oddp '(2 4 6 8))
                                        ; => NIL
(some #'oddp '(1 4 6 8))
                                        ; => T

notevery runs a test on all items of a sequence and returns t if the test returns nil for one item.

(notevery #'oddp '(1 3 5))
                                        ; => NIL
(notevery #'oddp '(1 2 3 5))
                                        ; => T

notany runs a test on all items of a sequence and returns t if the test returns nil for every item.

(notany #'oddp '(2 4 6))
                                        ; => T
(notany #'oddp '(1 2 4 6))
                                        ; => NIL

There are times when it's necessary to do an early return. We saw an example with all-even-p:

(defun all-even-p (nums)
  (dolist (i nums t)
    (format t "~&Looking at ~a..." i)
    (when (oddp i)
      (format t "~&Oops, this is odd!")
      (return nil))))

The result-form set for dolist is t, meaning that when we get to the end of the list we should return t.

However, when we spot an odd number, we need to return from the dolist loop early using (return nil).

Strings in Common Lisp are vectors of characters. As a result, all operations that can be used on sequences can be used on strings or vectors (explained in a later chapter).

concatenate combines two or more sequences–meaning it can combine lists, vectors, or strings. The first argument specifies the output type.

(concatenate 'list '(#\h #\e #\l #\l #\o #\space) '(#\w #\o #\r #\l #\d))

(#\\h #\\e #\\l #\\l #\\o #\\Space  #\\w #\\o #\\r #\\l #\\d)

(concatenate 'vector '(#\h #\e #\l #\l #\o #\space) '(#\w #\o #\r #\l #\d))

#(#\h #\e #\l #\l #\o #\Space  #\w #\o #\r #\l #\d)

(concatenate 'string '(#\h #\e #\l #\l #\o #\space) '(#\w #\o #\r #\l #\d))

"hello world"

(concatenate 'string "hello " "world")

"hello world"

length can not only tell you the number of items in a list, it can also tell you the number of characters in a string.

(length "hello world")

11

reverse can place characters in a string in reverse order.

(reverse "YOU WILL REWRITE THAT THING IN COMMON LISP IMMEDIATELY")

"YLETAIDEMMI PSIL NOMMOC NI GNIHT TAHT ETIRWER LLIW UOY"

map is the most general of the mapping functions. If you pass string as the result type, you can run character operations on the string and get back a new string.

(map 'string #'char-upcase "hello")

"HELLO"

There are also functions specialized to work on strings.

The previous map call can be simplified using string-upcase.

(string-upcase "almighty")

"ALMIGHTY"

You can probably guess what string-downcase does.

string= and string-equal can be used to test if two strings are the same. string= is case-sensitive, string-equal is case-insensitive.

(string= "almighty" "almighty")         ; => T
(string= "almighty" "Almighty")         ; case-sensitive => NIL
(string-equal "almighty" "ALMIGHTY")    ; case-insensitive => T

There are other string comparison operators: string>, string/=, string-not-greaterp, etc.

Streams are either a source or destination for some data. Common Lisp uses streams for reading and writing files, etc.

Use with-open-file to create a block where a streaming connection with some file is active.

(with-open-file (stream #P"io-test.txt" :direction          :output
                                        :if-exists          :supersede
                                        :if-does-not-exist  :create)
  (format stream "~&Put this in the test file.~&This will be on a new line.~&"))

with-open-file is a macro providing a shortcut to using the lower-level functions open and close combined with unwind-protect. It ensures that the connection to the file is closed before leaving the block.

An exhaustive explanation of all the options open and with-open-file can take are in the Hyperspec.

If you want to read a file, set :direction to :input and use one of read, read-line, or read-char.

(with-open-file (stream #P"io-test.txt" :direction :input)
  (loop for line = (read-line stream nil nil)
        while line
        do (format t "~a~&" line)))

Common Lisp functions for working with file systems, paths, etc. are generally not portable between Lisp implementations. The package UIOP is the defacto-standard source of portable path and filesystem utilities. Even with UIOP, pathnames can still be tricky.

format is used to write strings to output streams. The first argument is the stream. If set to t, then it will send the input to *standard-output*, which is the output stream to the REPL.

(format t "~&Almighty Lisp~%")

If the first argument is set to nil, then it returns a string.

(format nil "~&Almighty Lisp~%")

However, as in the with-open-file examples, you can also use format to write to file streams, etc.

format has an extensive set of control-string directives used for customizing how text is formatted. All of the directives begin with tilde, such as ~a, ~&, etc. They also have an extensive set of modifiers. Complex format strings are vaguely similar to regular expressions and tend to get just as hairy.

I will cover the bare minimum here. Refer to the Hyperspec to learn all of the directives. I will explain any other directives as necessary.

Tilde a will output the data in human readable, "Aesthetic" format.

(format t "~a" (aref "hello" 0))

h => NIL NOTE: not #\h

format takes an arbitrary number of arguments after the format string. For each argument, you need to supply another Tilde a or some other directive.

Tilde % and Tilde & output newlines. Tilde % will always output a newline. Tilde & will only output a newline if the output stream is not on a newline already.

(defparameter *alist* '((:micah male 40 married japan)
                        (:takae female 35 married japan)
                        (:mom female 71 married america)
                        (:papa male 75 married america)))

(loop for row in *alist*
      do (format t "~&~a: ~a~%" (first row) (rest row)))

MICAH: (MALE 40 MARRIED JAPAN)
TAKAE: (FEMALE 35 MARRIED JAPAN)
MOM: (FEMALE 71 MARRIED AMERICA)
PAPA: (MALE 75 MARRIED AMERICA)
 => NIL

Tic-tac-toe works like this:

• There are two players.
• Each take turns putting their "piece" on the board–either an X or an O.
• If either player gets three of their pieces in a row vertically, horizontally, or diagonally, they win.
• If all spaces are filled without any player winning, it's a draw.

The game is simple, so the data representation can be simple.

(defvar *board*)

(defun reset-board ()
  (setf *board* (list 'board
                      0 0 0
                      0 0 0
                      0 0 0)))

(defparameter *player-one* 1)
(defparameter *player-two* 10)

The board is a flat list of 0's representing empty spaces. board is a filler symbol to make later code more intuitive to understand. We use setf to assign the globally scoped *board* variable to the value '(board 0 0 0 0 0 0 0 0 0).

We represent player "pieces" as 1 and 10.

• Type the above code into a buffer named tic-tac-toe.lisp.
• Compile each form individually with Sly->Compilation->Compile Defun
• Without typing anything more, practice evaluating the symbols *player-one* and *player-two* with Sly->Evaluation->Eval Defun.
• Type (reset-board) in the buffer and then evaluate that form. What is the value of *board*?

Now that we have our data representations settled, we need a way to manipulate it.

(defun place-piece (board piece position)
  (setf (nth position board) piece)
  board)

Using setf with nth here is similar to doing something like var[n] = my-value in other languages. You setf to a place, such as a variable, a hashtable key, list position, array index, etc.

Here we assign the place in our *board* to the value of piece. The last value returned by the last form evaluated in a function becomes that function's return value. We return the board to be able to show the updated board to the players.

We also need a way to map positions on the board to all possible winning positions–Touretzky calls them triplets.

(defparameter *triplets* '((1 2 3) (4 5 6) (7 8 9) 
                           (1 4 7) (2 5 8) (3 6 9) 
                           (1 5 9) (3 5 7)))       

The first three triplets are horizontal winning positions, the second three are vertical, and the last two are diagonal.

Next, we need a way to calculate the state of those triplets.

(defun sum-triplet (board triplet)
  (+ (nth (first triplet) board)
     (nth (second triplet) board)
     (nth (third triplet) board)))

(defun compute-sums (board)
  (mapcar (lambda (triplet) (sum-triplet board triplet)) *triplets*))

mapcar is one of the many built-in functions that takes a function as an argument. mapcar will apply the function to each item in the sequence and collect them into a list, returning the list.

Let's test it out by placing some pieces manually.

(reset-board)
(place-piece *board* *player-one* 1)
(place-piece *board* *player-two* 2)
(place-piece *board* *player-one* 5)
(place-piece *board* *player-two* 9)
(place-piece *board* *player-one* 7)
(place-piece *board* *player-two* 3)
(place-piece *board* *player-one* 4)

(BOARD 1 10 10 1 1 0 1 0 10)

Now let's test compute-sums.

(compute-sums *board*)

(21 2 11 3 11 20 12 12)

We can see now that player one, represented as 1s on the board, occupies all three spaces in a triplet. We have a winner, but our program doesn't know that yet. Let's add win and tie detection.

(defun winner-p (board)
  (let ((sums (compute-sums board)))
    (or (member (* 3 *player-one*) sums)
        (member (* 3 *player-two*) sums))))

(defun tie-p (board)
  (not (member 0 board)))

Test them on the current board:

(winner-p *board*)

(3 11 20 12 12)

(tie-p *board*)

NIL

member is called a semi-predicate: it searches a sequence like sums for an item like (* 3 *player-one*). If none is found, it returns nil. If one is found, however, it doesn't return t; it returns a list of the item plus the rest of the list after the item.

We can add pieces to the board, calculate the state of the board, and detect a winner. If we make an interface for two human players, we can have a game.

We need a way to show the board to players using format. Instead of trying to do everything at once, we'll break it down into pieces, starting with converting player pieces from numbers to letters

(defun convert-to-letter (piece)
  (ecase piece
    (0  " ")
    (1  "X")
    (10 "O")))

(defun opponent (piece)
  (if (= piece 1)
      10
      1))

Test convert-to-letter:

(convert-to-letter 1)

X

(convert-to-letter 10)

O

Now let's print a row from the board.