DEPLOYING

First, we need to configure the system. We already did that earlier in our almighty-accounting system definition.

(defsystem "almighty-accounting"
  :author "Micah Killian"
  :version "0.0.1"
  :description "A rudimentary accounting system controlled via CLI."
  :depends-on (#:almighty-money
               #:almighty-kaikei
               #:local-time
               #:mito
               #:sxql
               #:dbi
               #:dbd-sqlite3
               #:clingon)
  :serial t
  :components ((:module "src"
                :serial t
                :components
                ((:file "db")
                 (:file "transactions")
                 (:file "views")
                 (:file "commands")
                 (:file "main"))))
  :build-operation "program-op"
  :build-pathname "almighty-accounting"
  :entry-point "almighty-accounting:main")

The last three lines are the parts we need to build the executable.

We pass "program-op" to :build-operation. ASDF has a number of predefined operations (https://asdf.common-lisp.dev/asdf.html#Predefined-operations-of-ASDF-1) for compiling and loading systems. The default op is load-op. If you don't set it to program-op, you will simply load the system and get dropped into a REPL. program-op tells ASDF to make an executable out of the system.

:build-pathname is the name of the executable file we want to create.

:entry-point is the package-qualified name of the function we want to call when running the executable.

In the previous chapter, we didn't define the main.lisp file, so let's do that now.

Open a buffer to almighty-accounting/src/main.lisp and add the following:

(defpackage #:almighty-accounting
  (:use #:cl)
  (:nicknames #:almighty-accounting/main)
  (:local-nicknames (#:db #:almighty-accounting/db)
                    (#:cli #:clingon)
                    (#:commands #:almighty-accounting/commands))
  (:export
   #:main))

(in-package #:almighty-accounting)

(defun main ()
  (let ((app (commands:top-level/command)))
    (db:with-chart-of-accounts #p"data/data.db"
      (cli:run app))))

That's all we'll need.

Now we just need to load and make the system. We're going to do things a little differently this time, though. What we'll do is make a build.lisp file and simply load that file. This will give us more flexibility for orchestrating the next steps. To load build.lisp, we'll use a Makefile:

almighty-accounting: build.lisp almighty-accounting.asd src/*.lisp
        sbcl --load build.lisp 

The filenames after almighty-accounting tell make to run the code that follows if any of the files listed are updated.

Now for the build.lisp file:

(require :asdf)

;;; Force ASDF to only look here for systems.
(asdf:initialize-source-registry `(:source-registry (:tree ,(uiop:getcwd))
 :ignore-inherited-configuration))

(progn
  (format t "-- LOADING SYSTEM --~%")
  (asdf:load-system :almighty-accounting)
  (format t "-- SYSTEM LOADED --~%"))

(progn
  (asdf:make "almighty-accounting")
  (format t "-- DONE --~%")
  (quit))

We require ASDF in order to use it here. require loads a package if it isn't already loaded. Usually ASDF is built into modern implementations of Common Lisp, but this ensures that ASDF is loaded in cases where your implementation doesn't load it (or have it built in).

The call to asdf:initialize-source-registry is to ensure that when ASDF loads the system that it resolves dependences using only the systems within our project directory. This is important because we used vend for our dependencies. We want asdf to use only the versions of those dependencies, not go hunting down dependencies in other directories (which it will).

asdf:load-system is the same operation performed with the REPL shortcut , load-system.

asdf:make runs the :build-operation we configured earlier.

We run quit because if for some reason you are running make and there are no changes, then no executable will be made and you'll be thrown into the REPL. quit is there to conveniently kick us out immediately.

Now, all you need to do is run make in the almighty-accounting project root directory to create the binary.

DEPLOY! (that means run make)

Once the build runs successfully, you should have a new file called almighty-accounting in your almighty-accounting project root directory. You can give it a spin…

But huh…no…no no no no…

We have a bug!

When you tried using the application to make a cash-sale, etc. you probably have something like this happen:

micah@almighty almighty-accounting % ./almighty-accounting cash-sale
huh
Name of good: Unknown good: huh. Must be one of:
beef
chicken
rice
cheese
bread
beef
Name of good:
beef
Quantity: Invalid quantity: beef. Quantity must be a positive integer.
2
More? (y or n) y
Quantity:
rice
Name of good:
n
Quantity: Invalid quantity: n. Quantity must be a positive integer.
2

More? (y or n) n
Quantity: Transaction completed. Total: $140.00

Right from the beginning, you'll notice that my "huh" comes before "Name of good:". That's because when I first started the application, "Name of good:" didn't print to the terminal. The rest of the output is similarly odd.

But it worked fine in the REPL! What's going on?

The problem is read-line: format t writes to *standard-output*, but Lisp streams are "batched" or buffered and nothing forces that buffer out to the terminal before read-line blocks waiting for input. We didn't have any trouble testing in the REPL because it does the flushing of the buffer for us, but when we use the application in the terminal we need to force it to output. Conveniently, there is a function called force-output that does just that.

Under every format call before a read-line, add force-output.

(defun ask-good-quantity ()
  (format t "~&Quantity: ")
  (force-output)                        ; <- Add this
  (let* ((user-input (read-line))
         ...)
    ...))
(defun ask-good-name ()
  (format t "~&Name of good: ")
  (force-output)                        ; <- Add this
  (let* ((user-input (read-line))
         ...)
    ...))
(defun ask-number-of-nights ()
  (format t "~&Number of nights: ")
  (force-output)                        ; <- Add this
  (let* ((user-input (read-line))
         ...)
    ...))

Make sure all files in your project are saved and then rerun make again.

Now that the executable is updated, just run the application. Try this:

./almighty-accounting cash-sale

If it worked, congratulations, you have now deployed your accounting application. You are officially ready to ship real CLI applications!

First, SSH into your server. Once you're in, run this command, replacing yourusername with one of your choosing.

adduser yourusername

Then, give that username sudo permissions.

usermod -aG sudo yourusername

Then you need to set up SSH for the new username. Exit the SSH session. On your local machine, run this command in the terminal:

ssh-copy-id yourusername@your-server-ip

In the terminal, run this command:

sudo apt install git build-essential automake autoconf libcurl4-openssl-dev zlib1g-dev libev4 -y

We use Roswell so that we can get the latest version of SBCL or use some other implementation later. While Common Lisp isn't getting updates, implementations update their compilers, adding optimizations, etc.

In the terminal, run these commands:

git clone -b release https://github.com/roswell/roswell.git
cd roswell
sh bootstrap
./configure
make
sudo make install
cd ..
rm -rf roswell 

Caddy is a reverse proxy that will let us run our Woo server on port 5000 but be accessible in the browser from https://the-ip-address-or-domain-name rather than https://the-ip-address-or-domain-name:5000. I like it because it takes care of SSL certificates automatically, so when you use it, https:// just works.

Run the following commands:

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

Open up /etc/caddy/Caddyfile with your favorite editor (such as nano). There should already be some default stuff there. Create a block like this:

your-server-ip-address-or-domain-name {
    reverse_proxy localhost:5000
}

You need to enable and start the service after your changes.

sudo systemctl enable caddy
sudo systemctl start caddy

The server terminal should show something like yourusername@localhost. You can change localhost to something of your choosing.

Run this command:

sudo hostnamectl set-hostname almightylisp

Then use our favorite editor to sudo open /etc/hosts. Add 127.0.0.1 yourpreferredhostname, save, and close. Then run exec $SHELL or just restart the SSH session.

With that, you are ready to move on to the code.

The script for building the executable is going to look similar to our previous build.lisp file. However, this one is a .ros file, used by Roswell to build the executable.

Open a buffer in fuka-stack/fuka-stack-app.ros, add the following code, and then save:

#!/bin/sh
#|-*- mode:lisp -*-|#
exec ros -Q -- $0 "$@"
|#

;;; Roswell script header (above) makes this executable

(defpackage #:fuka-stack-script
  (:use #:cl)
  (:export #:main))
(in-package #:fuka-stack-script)

;;; Load your system
(asdf:initialize-source-registry
 `(:source-registry
   (:tree ,(uiop:getcwd))
   :ignore-inherited-configuration))

(asdf:load-system "fuka-stack")

;;; Environment variable utilities.
(defun get-env-int (env default)
  (let ((env (uiop:getenv env)))
    (if env
        (parse-integer env :junk-allowed t)
        default)))

(defun get-env (env default)
  (or (uiop:getenv env) default))

(defun envp (env)
  (when (string-equal (uiop:getenvp env) "true")
        t))

(defun main ()
  (let ((host (get-env "HOST" "127.0.0.1"))
        (port (get-env-int "PORT" 5000))
        (debugp (envp "DEBUGP")))
    (handler-case
        (progn
          (clack:clackup (lack:builder fuka-stack:*component*) :server :woo
                                                               :address host
                                                               :port port
                                                               :debug debugp)
          (sleep most-positive-fixnum))
      (error (c)
        (format *error-output* "Aborting. ~a ~&" c)
        (force-output *error-output*)
        (uiop:quit 1)))))

The .ros file needs to have its own main function. For the purposes of deploying a web app, all you need to understand is this main function. That function uses uiop:getenv via get-env and get-env-int helpers to–you guessed it–get the environment variables HOST and PORT, defaulting to 127.0.0.1 and 5000. If the DEBUGP environment variable is set to true, then the clack:clackup :debug setting will be set to t, otherwise nil. Later we'll change the variables on the production environment, but this allows you to try this code out on your computer.

sleep is a function we haven't seen before. Usually, after loading the system and calling clack:clackup to start the server, it will remain on (usually you save it to a parameter so you can turn it off with clack:stop).

When you run the executable built with this code, if you don't include the call to (sleep most-positive-fixnum), systemd will assume your executable is done working and stop the service, turning off your server.

We will need to run our executable as a service. If we don't, the server will block our terminal session, and if there is an error, it will simply crash the server. Systemd can be configured to restart any crashed services, and the service won't block our terminal session, either.

Open a buffer in fuka-stack/fuka-stack-app.service, add the following code, and then save.

[Unit]
Description=Fuka Base App
After=network.target

[Service]
Type=simple
# Replace with the username you create when setting up the server
User=micah
# ...same here
WorkingDirectory=~/.local/bin/
EnvironmentFile=/etc/systemd/system/fuka-stack-app.env

# ...and here
ExecStart=/home/micah/.local/bin/fuka-stack

TimeoutStartSec=120
TimeoutStopSec=10
KillMode=process
KillSignal=SIGTERM

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

We also need environment variables set up, which we can do in a separate file (notice EnvironmentFile above).

Open a buffer in fuka-stack/fuka-stack-app.env, add the following code, and then save.

HOST=0.0.0.0
PORT=5000
DEBUGP=false

The makefile includes both make and make install commands. Open a buffer in fuka-stack/makefile and save this code inside:

fuka-stack: *.ros *.asd *.lisp src/*.lisp
        ros build fuka-stack-app.ros
install:
        mv fuka-stack-app ~/.local/bin/fuka-stack-app
        sudo cp fuka-stack-app.service /etc/systemd/system/fuka-stack-app.service
        sudo cp fuka-stack-app.env /etc/systemd/system/fuka-stack-app.env
        sudo systemctl daemon-reload
        sudo systemctl enable fuka-stack-app.service
run:
        sudo systemctl start fuka-stack-app.service

make will run Roswell's version of SBCL to build the executable. If you want to use/test this script locally you need to install Roswell.

make install will move the fuka-stack-app executable and setup the systemd service and environment variables. make run will turn the service on.

Now that we have our server set up, and we have all we need to build an executable and start it as a service, we can make our hello-world web app.

In the project root directory, make a file called fuka-stack.asd and place the below code into it.

(defsystem "fuka-stack"
  :author "Micah Killian <[email protected]>"
  :maintainer "Micah Killian <[email protected]>"
  :description "Fuka Stack hello world deploy example"
  :license "MIT"
  :version "0.1"
  :depends-on (:clack
               :woo 
               :myway
               :lack
               :lack-component
               :lack-request
               :lack-response)
  :components ((:module "src"
                :components
                ((:file "main")))))

Notice that we don't have any of the build options like we had for the previous executable CLI app. That's because we'll be using Roswell to build the executable.

Save this file and then be sure to run vend get to download the dependencies into your project.

Finally, we have the main body of the app. I won't be explaining the mechanics of the stack–that is the subject of a future book.

Open a buffer in fuka-stack/src/main.lisp, add this code, and save it.

(defpackage #:fuka-stack
  (:use #:cl)
  (:nicknames #:fuka-stack/main))
(in-package #:fuka-stack/main)

;;; Lack setup
(defclass fuka-component (lack/component::lack-component)
  ((routes :initarg :routes :accessor component-routes :initform (myway:make-mapper))))

(defparameter *component*
  (make-instance 'fuka-component))

(defparameter *request* nil)
(defparameter *response* nil)

(defmethod lack/component:call ((component fuka-component) env)
  (let ((*request* (lack/request:make-request env)))
    (multiple-value-bind (response foundp)
        (myway:dispatch (slot-value component 'routes)
                        (lack/request:request-path-info *request*)
                        :method (lack/request:request-method *request*))
      (if foundp
          (if (functionp response)
              response
              (destructuring-bind (status headers body) response
                (lack/response:finalize-response (lack/response:make-response status headers body))))
          (lack/response:finalize-response (lack/response:make-response 404 '(:content-type "text/html") '("Not found")))))))

;;; Clack response utility
(defun http-response (body &key (code 200) (headers nil))
  (let ((headers (append `(:content-type "text/html; charset=utf-8"
                           :content-length ,(length body))
                         headers)))
    `(,code ,headers (,body))))


;;; Myway routing utilities
(defun make-endpoint (fn)
  (lambda (params)
    (funcall (symbol-function fn) params)))

(defun route (method routing-rule endpoint &optional (mapper (component-routes *component*)))
  (myway:connect mapper
                 routing-rule
                 (make-endpoint endpoint)
                 :method method))


;;; Index controller and route
(defun index (params)
  (declare (ignore params))
  (http-response (format nil
                         "Hello, world! This is the path: ~a. This is the HTTP request method: ~a."
                         (lack/request:request-path-info *request*)
                         (lack/request:request-method *request*))))

(route :GET "/" 'index)