DEPLOYING
At some point, you gotta deploy your app. It's the most important step, yet also the one most fraught with options, uncertainty, and complexity.
How you plan to deploy affects the way you write the code. For example, we'll be deploying the tic-tac-toe game we made earlier. As it's written, it is only fit as a text-based app. That's why we'll be deploying it as a CLI application.
On the other hand, if you are writing a web application, that entails a whole swarm of other complications: in addition to the web-specific coding that you'll need to do, there's also the complicated step of choosing how to deploy it. Do you use some niche web hosting service that provides native support for Common Lisp? Do you use a VPS or cloud server? If you do, do you run your application in a container, or do you create an executable? What's your CI look like? And on and on it goes.
In this book, we'll keep things simple and give you one way of doing things as a starting point, but I'm sure you'll have lots of fun learning new and better deployment environments and workflows once you're done here.
EXECUTABLE CLI APPS
Common Lisp can make executables. There are several ways to make executables. Which method you use will depend on the nature of your project.
Tic-tac-toe
Our tic-tac-toe game is the simplest example. We will use SBCL's
sb-ext:save-lisp-and-die function.
(sb-ext:save-lisp-and-die #P"path/to/the/binary"
:top-level "ttt:play-game-with-computer"
:executable t)
The first argument is the path to the binary you want to create.
The second argument is the package-qualified name of the "main" function you want to call when running the executable.
The third argument tells SBCL to return an executable, not a Lisp image.
In order to call sb-ext:save-lisp-and-die, you of course need to have some
code compiled and loaded; however, you can't run it in a Sly REPL. Instead, you
need to run it in the command line. That would look like this:
sbcl --load tic-tac-toe.lisp --eval "(sb-ext:save-lisp-and-die #p\"ttt\" :toplevel #'ttt:play-game-with-computer :executable t)"
First, load the lisp file, then evaluate the call to sb-ext:save-lisp-and-die.
You can make the process more accessible to users by creating a Makefile to run the above:
build:
sbcl --load tic-tac-toe.lisp --eval "(sb-ext:save-lisp-and-die #p\"ttt\" \
:toplevel #'ttt:play-game-with-computer :executable t)"
But, if you try to run this, you'll have a problem: the package ttt doesn't
exist because we hadn't learned about packages before making the tic-tac-toe
game. So let's go add a package definition to the tic-tac-toe.lisp file.
(defpackage #:ttt
(:use :cl)
(:export #:play-game-with-computer))
(in-package #:ttt)
With that, you can run make build. Once the executable is created, you can
type ./ttt in your command line and enjoy a spirited game of tic-tac-toe
against the computer.
almighty-kaikei
For very simple programs, the above is good enough. However, if your project has
an .asd file, then you have the option of making an executable from a whole
system.
The simple version of this process looks like this:
- Configure the system to build an executable.
- Load the system with
asdf:load-system. - Run
asdf:makeon the loaded system.
This process uses ASDF, rather than calling sb-ext:save-lisp-and-die directly.
Configure System
First, we need to configure the system. In our case, we want to load the
almighty-kaikei-examples system, since it actually uses the library.
(defsystem "almighty-kaikei-examples"
:author "Micah Killian"
:version "0.0.1"
:description "Almighty Double-Booking Accounting Program"
:depends-on (#:almighty-money #:almighty-kaikei #:local-time #:mito #:sxql
#:dbi #:dbd-sqlite3)
:components ((:file "reports"))
:build-operation "program-op"
:build-pathname "kaikei"
:entry-point "almighty-kaikei-reports:main")
Add driver
The first change is in the :depends-on list: cl-dbi will compile and load a
driver for your database if you don't already have one loaded. From the cl-dbi
readme:
cl-dbi will load another system on the fly depending on your database's driver:
:dbd-sqlite3 :dbd-mysql :dbd-postgres
You must reference the required one in your system definition if you plan to build an executable (and if you plan to run it on a machine where Quicklisp is not installed).
It will cause a nasty error in your executable if you don't add either the
:dbd-sqlite3, :dbd-mysql, or :dbd-postgres systems to your system
:depends-on. And don't forget to vend get when you add this system!
Add build configuration
Next, we add :build-operation with the argument program-op. ASDF has a
number of predefined operations
(https://asdf.common-lisp.dev/asdf.html#Predefined-operations-of-ASDF-1) for
compiling and loading systems. The default op is load-op. If you don't set it
to program-op, you will simply load the system and get dropped into a REPL.
program-op tells ASDF to make an executable out of the system.
:build-pathname is the name of the executable file we want to create.
:entry-point is the package-qualified name of the function we want to call
when running the executable. More on this in just a second. For now, our system
is all set up with a few changes.
Load & Make System
Now we just need to load and make the system. We're going to do things a little
differently this time, though. What we'll do is make a build.lisp file and
simply load that file. This will give us more flexibility for orchestrating the
next steps. First, the Makefile:
kaikei: build.lisp almighty-kaikei-examples.asd main.lisp reports.lisp
sbcl --load build.lisp
The filenames after kaikei: tell make to run the code that follows if any of
the files listed are updated.
Now for the build.lisp file:
(require :asdf)
;; Force ASDF to only look here for systems.
(asdf:initialize-source-registry `(:source-registry (:tree ,(uiop:getcwd))
:ignore-inherited-configuration))
(progn
(format t "-- LOADING SYSTEM --~%")
(asdf:load-system :almighty-kaikei-examples)
(format t "-- SYSTEM LOADED --~%"))
(progn
(asdf:make "almighty-kaikei-examples")
(format t "-- DONE --~%")
(quit))
We require ASDF in order to use it here. require loads a package if it isn't
already loaded. Usually ASDF is built into modern implementations of Common
Lisp, but this ensures that ASDF is loaded in cases where your implementation
doesn't load it (or have it built in).
The call to asdf:initialize-source-registry is to ensure that when ASDF loads
the system that it resolves dependences using only the systems within our
project directory.
asdf:load-system is the same operation performed with the REPL shortcut ,
load-system.
asdf:make runs the :build-operation we configured earlier.
We run quit because if for some reason you are running make and there are no
changes, then no executable will be made and you'll be thrown into the REPL.
quit is there to conveniently kick us out immediately.
Add almighty-kaikei-examples:main function
The :entry-point in the system is set to a function called main, but we
haven't written that function yet. For now, we're going to make this very
simple:
;; add
(defun main ()
(with-coa (general-ledger "2025-12-1" "2025-12-5")))
Run make
Now, all you need to do is run make to create the binary.
EXECUTABLE WEB APP
We are going to deploy a simple hello world web app to a Ubuntu 24.04 LTS box.
We'll make it using the Fuka Stack: clack, lack, woo, and myway. We'll
use vend for our dependencies. After we setup a small hello world example of
using the Fuka Stack, we'll setup the server, upload the code, build an
executable, and then run that executable as a service via systemd.
If you use Ubuntu 24.04 LTS on an x86-64 processor–the same OS and hardware as the server–you might even be able to skip the step of uploading the code to the server, building the executable on your machine and then uploading the executable to the server.
The purpose is not to get into the weeds of how to make a web framework. When we get to the part about the code, we'll focus on the parts relevant to deploying, not to setting up the framework.
Requirements
- An SSH Key.
- A server with Ubuntu 24.04 LTS.
vendinstalled on your computer.
Server Setup
First, let's start with setting up the server. I used a Linode VPS for this example, but you can use any Ubuntu 24.04 LTS machine.
Add non-root user
First, SSH into your server. Once you're in, run this command, replacing
yourusername with one of your choosing.
adduser yourusername
Then, give that username sudo permissions.
usermod -aG sudo yourusername
Then you need to set up SSH for the new username. Exit the SSH session. On your local machine, run this command in the terminal:
ssh-copy-id yourusername@your-server-ip
Install Roswell dependencies, libev4, Caddy
In the terminal, run this command:
sudo apt install git build-essential automake autoconf libcurl4-openssl-dev zlib1g-dev libev4 -y
Install Roswell
We use Roswell so that we can get the latest version of SBCL or use some other implementation later. While Common Lisp isn't getting updates, implementations update their compilers, adding optimizations, etc.
In the terminal, run these commands:
git clone -b release https://github.com/roswell/roswell.git
cd roswell
sh bootstrap
./configure
make
sudo make install
cd ..
rm -rf roswell
Install Caddy
Caddy is a reverse proxy that will let use run our Woo server on port 5000 but be accessible in the browser from https://the-ip-address-or-domain-name rather than https://the-ip-address-or-domain-name:5000. I like it because it takes care of SSL certificates automatically, so when you use it, https:// just works.
Run the following commands:
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy
Open up /etc/caddy/Caddyfile with your favorite editor (such as nano). There
should already be some default stuff there. Create a block like this:
your-server-ip-address-or-domain-name {
reverse_proxy localhost:5000
}
You need to enable and start the service after your changes.
sudo systemctl enable caddy
sudo systemctl start caddy
(Optional) Change Hostname
The server terminal should show something like yourusername@localhost. You can
change localhost to something of your choosing.
Run this command:
sudo hostnamectl set-hostname almightylisp
Then use our favorite editor to sudo open /etc/hosts. Add 127.0.0.1
yourpreferredhostname, save, and close. Then run exec $SHELL or just restart
the SSH session.
With that, you are ready to move on to the code.
The Code
Now that we have the server setup, we'll get our code written.
First, on your local machine, make a new project root directory called
fuka-stack. Inside it, you can also make another directory called src.
Build Script
The script for building the executable is going to look similar to our previous
build.lisp file. However, this one is a .ros file, used by Roswell to build
the executable.
Open a buffer in fuka-stack/fuka-stack-app.ros, add the following code, and
then save:
#!/bin/sh
#|-*- mode:lisp -*-|#
exec ros -Q -- $0 "$@"
|#
;; Roswell script header (above) makes this executable
(defpackage #:fuka-stack-script
(:use #:cl)
(:export #:main))
(in-package #:fuka-stack-script)
;; Load your system
(asdf:initialize-source-registry
`(:source-registry
(:tree ,(uiop:getcwd))
:ignore-inherited-configuration))
(asdf:load-system "fuka-stack")
;; Environment variable utilities.
(defun get-env-int (env default)
(let ((env (uiop:getenv env)))
(if env
(parse-integer env :junk-allowed t)
default)))
(defun get-env (env default)
(or (uiop:getenv env) default))
(defun envp (env)
(when (string-equal (uiop:getenvp env) "true")
t))
(defun main ()
(let ((host (get-env "HOST" "127.0.0.1"))
(port (get-env-int "PORT" 5000))
(debugp (envp "DEBUGP")))
(handler-case
(progn
(clack:clackup (lack:builder fuka-stack:*component*) :server :woo
:address host
:port port
:debug debugp)
(sleep most-positive-fixnum))
(error (c)
(format *error-output* "Aborting. ~a ~&" c)
(force-output *error-output*)
(uiop:quit 1)))))
The .ros file needs to have its own main function. For the purposes of
deploying a web app, all you need to understand is this main function. That
function uses uiop:getenv via get-env and get-env-int helpers to–you
guessed it–get the environment variables HOST and PORT, defaulting to
127.0.01 and 5000. If the DEBUGP environment variable is set to true,
then the clack:clackup :debug setting will be set to t, otherwise nil.
Later we'll change the variables on the production environment, but this allows
you to try this code out on your computer.
sleep is a function we haven't seen before. Usually, after loading the system
and calling clack:clackup to start the server, it will remain on (usually you
save it to a parameter so you can turn it off with clack:stop).
When you run the executable built with this code, if you don't include the call
to (sleep most-positive-fixnum), systemd will assume your executable is done
working and stop the service, turning off your server.
Service & Env files
We will need to run our executable as a service. If we don't, the server will block our terminal session, and if there is an error, it will simply crash the server. Systemd can be configured to restart any crashed services, and the service won't block our terminal session, either.
Open a buffer in fuka-stack/fuka-stack-app.service, add the following code,
and then save.
[Unit]
Description=Fuka Base App
After=network.target
[Service]
Type=simple
# Replace with the username you create when setting up the server
User=micah
# ...same here
WorkingDirectory=/home/micah/.local/bin/
EnvironmentFile=/etc/systemd/system/fuka-stack-app.env
# ...and here
ExecStart=/home/micah/.local/bin/fuka-stack
TimeoutStartSec=120
TimeoutStopSec=10
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
We also need environment variables set up, which we can do in a separate file
(notice EnvironmentFile above).
Open a buffer in fuka-stack/fuka-stack-app.env, add the following code, and
then save.
HOST=0.0.0.0
PORT=5000
DEBUGP=false
Makefile
The makefile includes both make and make install commands. Open a buffer in
fuka-stack/makefile and save this code inside:
fuka-stack: build.lisp *.asd *.lisp src/*.lisp
ros build fuka-stack-app.ros
install:
mv fuka-stack-app ~/.local/bin/fuka-stack-app
sudo cp fuka-stack-app.service /etc/systemd/system/fuka-stack-app.service
sudo cp fuka-stack-app.env /etc/systemd/system/fuka-stack-app.env
sudo systemctl daemon-reload
sudo systemctl enable fuka-stack-app.service
run:
sudo systemctl start fuka-stack-app.service
make will run Roswell's version of SBCL to build the executable. If you want
to use/test this script locally you need to install Roswell.
make install will move the fuka-stack-app executable, setup the systemd
service and environment variables, and then start the service.
Hello World Code
Now that we have our server set up, and we have all we need to build an executable and start it as a service, we can make our hello-world web app.
In the project root directory, make a file called fuka-stack.asd and place the
below code into it.
(defsystem "fuka-stack"
:author "Micah Killian <[email protected]>"
:maintainer "Micah Killian <[email protected]>"
:description "Fuka Stack hello world deploy example"
:license "MIT"
:version "0.1"
:depends-on (:clack
:woo
:myway
:lack
:lack-component
:lack-request
:lack-response)
:components ((:module "src"
:components
((:file "main")))))
Notice that we don't have any of the build options like we had for the previous executable CLI app. That's because we'll be using Roswell to build the executable.
Save this file and then be sure to run vend get to download the dependencies
into your project.
Finally, we have the main body of the app. I won't be explaining the mechanics of the stack–that is the subject of a future book.
Open a buffer in fuka-stack/src/main.lisp, add this code, and save it.
(defpackage #:fuka-stack
(:use #:cl)
(:nicknames #:fuka-stack/main))
(in-package #:fuka-stack/main)
;; Lack setup
(defclass fuka-component (lack/component::lack-component)
((routes :initarg :routes :accessor component-routes :initform (myway:make-mapper))))
(defparameter *component*
(make-instance 'fuka-component))
(defparameter *request* nil)
(defparameter *response* nil)
(defmethod lack/component:call ((component fuka-component) env)
(let ((*request* (lack/request:make-request env)))
(multiple-value-bind (response foundp)
(myway:dispatch (slot-value component 'routes)
(lack/request:request-path-info *request*)
:method (lack/request:request-method *request*))
(if foundp
(if (functionp response)
response
(destructuring-bind (status headers body) response
(lack/response:finalize-response (lack/response:make-response status headers body))))
(lack/response:finalize-response (lack/response:make-response 404 '(:content-type "text/html") '("Not found")))))))
;; Clack response utility
(defun http-response (body &key (code 200) (headers nil))
(let ((headers (append `(:content-type "text/html; charset=utf-8"
:content-length ,(length body))
headers)))
`(,code ,headers (,body))))
;; Myway routing utilities
(defun make-endpoint (fn)
(lambda (params)
(funcall (symbol-function fn) params)))
(defun route (method routing-rule endpoint &optional (mapper (component-routes *component*)))
(myway:connect mapper
routing-rule
(make-endpoint endpoint)
:method method))
;; Index controller and route
(defun index (params)
(declare (ignore params))
(http-response (format nil
"Hello, world! This is the path: ~a. This is the HTTP request method: ~a."
(lack/request:request-path-info *request*)
(lack/request:request-method *request*))))
(route :GET "/" 'index)
The Deploy
You are on the precipice of a new beginning, my friend. Your Common Lisp web
development life arch begins with the next two steps. It's time to upload your
code to the server. If you are comfortable using git, you could use it now. For
the sake of simplicity, we'll just upload with the rsync command.
From your local machine, run this command:
rsync -avz --progress ~/path/to/your/local/version/of/fuka-stack yourusername@domain-name-or-ip-address-to-your-server:/home/yourusername/fuka-stack
Now all you need to do is ssh back into your server, navigate to the fuka-stack
directory, and run the make, make install, and make run commands in order.
If all goes well (and we know things always go well with deploys), you should
now be able to access your hello-world example in your browser with
https://your-domain-name-or-server-ip-address.
Congratulations, you have successfully deployed a Common Lisp web app!
Summary
There are a number of things we could have done differently. We could have
skipped Roswell and just used whatever version of SBCL was available on the
Ubuntu 24.04 LTS package repository. We could have compiled the latest one from
source. We could have built the executable in about three other ways. We could
have used nginx instead of Caddy. We could have used hunchentoot instead of
clack and friends. We could have used Docker to set up the entire environment
on the server and simply run the code from that environment.
The principles to keep in mind when deploying a Common Lisp web app are these:
- You need to have some entry point–the call to start the server.
- You need to run that call someway as a service to enable automatic restarts.
- You will need to call
sleepin the entry point to keep the server thread open in the background. - You want to turn off the debugger in prod.