Your First API

Welcome Back

This is the fourth part in the tutorial. By now, you’ve built a simple service that vends out friendly greetings by name and can handle a few content types.

We’re going to leave our greeting service alone for the time being and turn our attention to a REST API for some entities. To keep things simple, we’re going to keep the entities in memory. Even so, we will use the same kind of techniques you would use in a real application.

What You Will Learn

After reading this guide, you will be able to:

Use path parameters to identify items
Generate URLs for new items
Encode and decode entity bodies

Guide Assumptions

This guide is for beginners who are new to Pedestal and may be new to Clojure. It doesn’t assume any prior experience with a Clojure-based web framework. You should be familiar with the basics of HTTP: URLs, response codes, and content types.

You do not need experience with any particular database or persistence framework.

This guide also assumes that you are in a Unix-like development environment, with Java installed. We’ve tested it on Mac OS X and Linux (any flavor) with great results. We haven’t yet tried it on the Windows Subsystem for Linux, but would love to hear from you if you’ve succeeded with it there.

Getting Help if You’re Stuck

We’ll take this in small steps. If you get stuck at any point in this guide, please submit an issue about this guide or, hop over to the Pedestal Users mailing list and raise your hand there. You can also get help from the #pedestal channel on the Clojurians Slack.

Where We Are Going

We will start a new project this time. The service will be a simple "to-do" list that persists only in memory. We will use the same techniques that we would use to interact with a real database.

We’re going to make a REST style API. Our API will have the following routes:

Route	Verb	Action	Response
/todo	POST	Create a new list	201 with URL of new list
/todo	GET	Return query form	200 with static page
/todo/:list-id	GET	View a list	200 with all items
/todo/:list-id/:item-id	GET	View an item	200 with item
/todo/:list-id/:item-id	PUT	Update an item	200 with updated item

Route

Verb

Action

Response

/todo

POST

Create a new list

201 with URL of new list

/todo

GET

Return query form

200 with static page

/todo/:list-id

GET

View a list

200 with all items

/todo/:list-id/:item-id

GET

View an item

200 with item

/todo/:list-id/:item-id

PUT

Update an item

200 with updated item

Setting Up

We will start this project with another empty directory. (Still avoiding the template projects for now.)

I’m going to call my project 'todo-api'. Feel free to call yours something different, but it’s up to you to do the mental translation from here on out.

$ mkdir todo-api
$ cd todo-api
$ mkdir src

This time, we’ll set up our deps.edn file first. It will be almost exactly the same as our last one:

deps.edn

{:paths ["src"]
 :deps  {io.pedestal/pedestal.jetty {:mvn/version "0.7.1"}
         org.clojure/data.json      {:mvn/version "2.5.0"}
         org.slf4j/slf4j-simple     {:mvn/version "2.0.10"}}}

Our new source file will start off with the usual namespace declaration:

src/main.clj

(ns main
  (:require [io.pedestal.http :as http]
            [io.pedestal.http.route :as route]
            [io.pedestal.test :as test]))

Partial Functions

We’re going to use some of the same structures from previous tutorials, such as an ok function to build a successful response, but with some differences:

src/main.clj

(defn response [status body & {:as headers}]
  {:status status :body body :headers headers})

(def ok (partial response 200))
(def created (partial response 201))
(def accepted (partial response 202))

partial is a Clojure function that takes a function and returns a new function that needs fewer arguments - you supply the initial arguments in the call to partial. The returned function keeps track of those initial arguments, and when invoked, "glues together" the initial arguments and any additional arguments provided to it, and calls the original function: response in this case.

Although created with def, the symbols ok, created, and accepted, are all functions, as if they were created using defn.

Why do any of this? For all that web developers generally know the status codes in their sleep ^[1], it is more intent of the code that calls these functions is more semantically visible. And you probably did go look up code 201.

Routes Up Front

Second, we’re going to define all our routes right away, but have them all use an echo handler so we can test the routes separately from everything else^[2].

src/main.clj

(def echo
  {:name :echo
   :enter
   (fn [context]
     (let [request (:request context)
           response (ok context)]
       (assoc context :response response)))})

(def routes
  (route/expand-routes
    #{["/todo" :post echo :route-name :list-create]
      ["/todo" :get echo :route-name :list-query-form]
      ["/todo/:list-id" :get echo :route-name :list-view]
      ["/todo/:list-id" :post echo :route-name :list-item-create]
      ["/todo/:list-id/:item-id" :get echo :route-name :list-item-view]
      ["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
      ["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))

Pedestal makes it possible to invoke these routes directly from the REPL, without involving HTTP at all. This is a nice way to break your work up into incremental steps. It goes together nicely with the interactive development support we introduced in the Hello World, With Parameters tutorial.

Get ready for a ton of output:

> clj
Clojure 1.11.1
user=> (require 'main)
nil
user=> (main/start-dev)
[main] INFO org.eclipse.jetty.server.Server - jetty-11.0.18; built: 2023-10-27T02:14:36.036Z; git: 5a9a771a9fbcb9d36993630850f612581b78c13f; jvm 11.0.19+7-LTS
[main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@6c8f2e4e{/,null,AVAILABLE}
[main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@1432a683{HTTP/1.1, (http/1.1, h2c)}{localhost:8890}
[main] INFO org.eclipse.jetty.server.Server - Started Server@b025f2f{STARTING}[11.0.18,sto=0] @57816ms
#:io.pedestal.http{:port 8890, :service-fn #object[io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__14892 0x515f104c "io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__14892@515f104c"], :host "localhost", :type :jetty, :start-fn #object[io.pedestal.http.jetty$server$fn__15680 0x1dde2d1 "io.pedestal.http.jetty$server$fn__15680@1dde2d1"], :interceptors [#Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.route/path-params-decoder}], :routes ({:path "/todo/:list-id", :method :get, :path-constraints {:list-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)", :path-parts ["todo" :list-id], :interceptors [#Interceptor{:name :entity-render} #Interceptor{:name :database-interceptor} #Interceptor{:name :list-view}], :route-name :list-view, :path-params [:list-id]} {:path "/todo/:list-id", :method :post, :path-constraints {:list-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)", :path-parts ["todo" :list-id], :interceptors [#Interceptor{:name :entity-render} #Interceptor{:name :list-item-view} #Interceptor{:name :database-interceptor} #Interceptor{:name :list-item-create}], :route-name :list-item-create, :path-params [:list-id]} {:path "/todo", :method :post, :path-re #"/\Qtodo\E", :path-parts ["todo"], :interceptors [#Interceptor{:name :database-interceptor} #Interceptor{:name :list-create}], :route-name :list-create, :path-params []} {:path "/todo", :method :get, :path-re #"/\Qtodo\E", :path-parts ["todo"], :interceptors [#Interceptor{:name :echo}], :route-name :list-query-form, :path-params []} {:path "/todo/:list-id/:item-id", :method :delete, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :echo}], :route-name :list-item-delete, :path-params [:list-id :item-id]} {:path "/todo/:list-id/:item-id", :method :get, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :entity-render} #Interceptor{:name :list-item-view} #Interceptor{:name :database-interceptor}], :route-name :database-interceptor, :path-params [:list-id :item-id]} {:path "/todo/:list-id/:item-id", :method :put, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :echo}], :route-name :list-item-update, :path-params [:list-id :item-id]}), :servlet #object[io.pedestal.http.servlet.FnServlet 0x47caa1ff "io.pedestal.http.servlet.FnServlet@47caa1ff"], :server #object[org.eclipse.jetty.server.Server 0xb025f2f "Server@b025f2f{STARTED}[11.0.18,sto=0]"], :join? false, :stop-fn #object[io.pedestal.http.jetty$server$fn__15682 0x7330574d "io.pedestal.http.jetty$server$fn__15682@7330574d"]}
user=>

Poking the Routes

Let’s try out the routes that we defined in the main namespace. We’ll do that using response-for. This function exercises our server without going through any actual HTTP requests.

In Pedestal, the :io.pedestal.http/service-fn key of the service map contains the service function which is essentially the bridge between Servlet APIs and Pedestal’s interceptor chain.

By extracting the service-fn (which we keep handy when running in development mode), we can pass it to response-for along with the HTTP verb (as a keyword), and the URL:

user=> (require '[io.pedestal.test :as test])
nil
user=> (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo")
[main] INFO io.pedestal.http - {:msg "GET /todo", :line 80}
{:status 200, :body "{:io.pedestal.interceptor.chain/stack (#Interceptor{:name :echo} #Interceptor{:name :io.pedestal.http.route/path-params-decoder} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/ring-response} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol \"HTTP/1.1\", :async-supported? true, :remote-addr \"127.0.0.1\", :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x3e1ff1cf \"io.pedestal.test$test_servlet_response$reify__15083@3e1ff1cf\"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x389588b2 \"io.pedestal.http.servlet.FnServlet@389588b2\"], :headers {\"content-length\" \"0\", \"content-type\" \"\"}, :server-port -1, :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x57e9122f \"io.pedestal.test$test_servlet_request$reify__15071@57e9122f\"], :content-length 0, :content-type \"\", :path-info \"/todo\", :character-encoding \"UTF-8\", :url-for #object[clojure.lang.Delay 0x41c22d59 {:status :pending, :val nil}], :uri \"/todo\", :server-name nil, :query-string nil, :path-params {}, :body #object[io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a 0x5ea52724 \"io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a@5ea52724\"], :scheme nil, :request-method :get, :context-path \"\"}, :bindings {#'io.pedestal.http.route/*url-for* #object[clojure.lang.Delay 0x41c22d59 {:status :pending, :val nil}]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x7dfccbf2 \"io.pedestal.http.impl.servlet_interceptor$start_servlet_async@7dfccbf2\"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810 0x1fe24f79 \"io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810@1fe24f79\"]), :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x3e1ff1cf \"io.pedestal.test$test_servlet_response$reify__15083@3e1ff1cf\"], :route {:path \"/todo\", :method :get, :path-re #\"/\\Qtodo\\E\", :path-parts [\"todo\"], :interceptors [#Interceptor{:name :echo}], :route-name :list-query-form, :path-params {}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[clojure.core$constantly$fn__5740 0x4f9a0139 \"clojure.core$constantly$fn__5740@4f9a0139\"]}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x389588b2 \"io.pedestal.http.servlet.FnServlet@389588b2\"], :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x57e9122f \"io.pedestal.test$test_servlet_request$reify__15071@57e9122f\"], :io.pedestal.interceptor.chain/queue #object[clojure.lang.PersistentQueue 0x38f183e9 \"clojure.lang.PersistentQueue@1\"], :url-for #object[clojure.lang.Delay 0x41c22d59 {:status :pending, :val nil}], :io.pedestal.interceptor.chain/execution-id 1, :servlet-config nil, :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x23497c5e \"io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@23497c5e\"]}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "X-Download-Options" "noopen", "X-Permitted-Cross-Domain-Policies" "none", "Content-Security-Policy" "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", "Content-Type" "application/edn"}}
user=>

Part of this output — [main] INFO … — comes from built-in Pedestal interceptors for logging requests; the logging output gets mixed into the output from the REPL.

The response is somewhat overwhelmingly verbose because we are using the echo interceptor, so the entire incoming request converted to a string and is nested inside the response. If we omit the response body, it’s easier to observe the most useful parts of the response:

user=> (dissoc *1 :body)
{:status 200, :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "X-Download-Options" "noopen", "X-Permitted-Cross-Domain-Policies" "none", "Content-Security-Policy" "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", "Content-Type" "application/edn"}}

Inside the REPL, Clojure keeps track of the most recent result it printed in the symbol *1. That makes it a snap to dig down into that result. There’s also *2 and *3.

Now we can see the status code (200, OK) and the headers. Not too bad. What happens if we ask for a route that doesn’t exist? Or a verb that isn’t supported on a real route?

user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/no-such-route") :body)
[main] INFO io.pedestal.http - {:msg "GET /no-such-route", :line 80}
{:status 404, :headers {"Content-Type" "text/plain"}}
user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :delete "/todo") :body)
[main] INFO io.pedestal.http - {:msg "DELETE /todo", :line 80}
{:status 404, :headers {"Content-Type" "text/plain"}}
user=>

This 404 response comes from Pedestal itself. That is the default response when the router cannot find a route that matches the URL pattern and passes constraints. (For much more about routers, see the Routing Quick Reference).

Path Parameters

Some of our routes have parameters in them. Those look like keywords embedded in the URL patterns. That tells Pedestal to match any value in that position (except a slash!) and bind it to that parameter in the request map. We should be able to see those parameters in the requests that echo will send back to us. This is one of the ways we can make sure that routes do what we think they are going to do, before we write the real logic.

user=> (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo/abcdef/12345")
[main] INFO io.pedestal.http - {:msg "GET /todo/abcdef/12345", :line 80}
{:status 200, :body "{:io.pedestal.interceptor.chain/stack (#Interceptor{:name :echo} #Interceptor{:name :io.pedestal.http.route/path-params-decoder} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/ring-response} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol \"HTTP/1.1\", :async-supported? true, :remote-addr \"127.0.0.1\", :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x29c808c8 \"io.pedestal.test$test_servlet_response$reify__15083@29c808c8\"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x265b5f05 \"io.pedestal.http.servlet.FnServlet@265b5f05\"], :headers {\"content-length\" \"0\", \"content-type\" \"\"}, :server-port -1, :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x6916566d \"io.pedestal.test$test_servlet_request$reify__15071@6916566d\"], :content-length 0, :content-type \"\", :path-info \"/todo/abcdef/12345\", :character-encoding \"UTF-8\", :url-for #object[clojure.lang.Delay 0x614c7175 {:status :pending, :val nil}], :uri \"/todo/abcdef/12345\", :server-name nil, :query-string nil, :path-params {:list-id \"abcdef\", :item-id \"12345\"}, :body #object[io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a 0x82bbbdc \"io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a@82bbbdc\"], :scheme nil, :request-method :get, :context-path \"\"}, :bindings {#'io.pedestal.http.route/*url-for* #object[clojure.lang.Delay 0x614c7175 {:status :pending, :val nil}]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x7dfccbf2 \"io.pedestal.http.impl.servlet_interceptor$start_servlet_async@7dfccbf2\"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810 0x74f47eab \"io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810@74f47eab\"]), :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x29c808c8 \"io.pedestal.test$test_servlet_response$reify__15083@29c808c8\"], :route {:path \"/todo/:list-id/:item-id\", :method :get, :path-constraints {:list-id \"([^/]+)\", :item-id \"([^/]+)\"}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[io.pedestal.http.route.prefix_tree$add_satisfies_constraints_QMARK_$fn__11533 0x2f8afb66 \"io.pedestal.http.route.prefix_tree$add_satisfies_constraints_QMARK_$fn__11533@2f8afb66\"], :path-re #\"/\\Qtodo\\E/([^/]+)/([^/]+)\", :path-parts [\"todo\" :list-id :item-id], :interceptors [#Interceptor{:name :echo}], :route-name :list-item-view, :path-params {:list-id \"abcdef\", :item-id \"12345\"}}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x265b5f05 \"io.pedestal.http.servlet.FnServlet@265b5f05\"], :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x6916566d \"io.pedestal.test$test_servlet_request$reify__15071@6916566d\"], :io.pedestal.interceptor.chain/queue #object[clojure.lang.PersistentQueue 0x2d4fe03b \"clojure.lang.PersistentQueue@1\"], :url-for #object[clojure.lang.Delay 0x614c7175 {:status :pending, :val nil}], :io.pedestal.interceptor.chain/execution-id 9, :servlet-config nil, :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x23497c5e \"io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@23497c5e\"]}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "X-Download-Options" "noopen", "X-Permitted-Cross-Domain-Policies" "none", "Content-Security-Policy" "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", "Content-Type" "application/edn"}}

The interesting part here is kind of buried inside the request map (which is a string inside the response body). We can pull it out with a bit of Clojure code, but I’ll just highlight it here. (If you’re interested in an exercise, try using clojure.edn to extract the request.)

Inside the request, there is a map :path-params which looks like this:

:path-params {:list-id "abcdef", :item-id "12345"}}

Pedestal extracted these parameters from the URL and bound them to the request for us. Note that the values are always strings, because URLs are strings. If we want to treat "12345" as a number, that’s up to us. But our application had better not break if some sly user sends us up a request with something other than numbers there.

Create List

The first request to handle is creating lists. This is a POST to the /todo URL. In the interest of simplicity, the only part of a request body we need is a name field. Also in the interest of simplicity, we are not going to deal with content negotiation yet.

What do we need to do in order to create a list?

Make a new data structure for the list.
Add it to our repository.

Without getting too far ahead of ourselves, we’d like to avoid doing all of the above in a single handler function, and instead, break this up into several interceptors, each of which is (as much as possible) side-effect free and easily testable.

Let’s break the behavior up into multiple steps, and give each step a provisional name:

A attaches a database connection to the request.
B looks up an item of interest and attaches it to the request.
C makes some decisions and attaches the result of those decisions to the request.
D executes a transaction with those decisions.

This way, A and D are quite generic. B is somewhat specific to the particular route — is it a list or a list item that’s being looked up?. That leaves C which is most specific to a single route, and might event be a simple handler function.

For this guide, we are going to cheat and create an in-memory repository:

src/main.clj

(defonce database (atom {}))                                (1)

1	Using `defonce` here ensures that the value, the database atom, will survive when we reload the namespace.

And the interceptor to snapshot the database and handle transactions:

src/main.clj

(def db-interceptor
  {:name :database-interceptor
   :enter
   (fn [context]
     (update context :request assoc :database @database))   (1)
   :leave
   (fn [context]
     (if-let [[op & args] (:tx-data context)]               (2)
       (do
         (apply swap! database op args)                     (3)
         (assoc-in context [:request :database] @database)) (4)
       context))})                                          (5)

1	`@database` gets the current value of the atom. We attach it to the request inside the context. This is step `A` from above.
2	The :tx-data key on the context may be set by other interceptors. This is step `D` from above.
3	The `do` block is a sure sign of a side effect. The `swap!` here is our stand-in for a real database transaction.
4	We attach the updated database value in case any other interceptors need to use it. We use the latest version of the database (via `@database`), not the value in the request, to account for any concurrent threads changing the database simultaneously.
5	If there wasn’t any tx-data, we just return the context without changes. If you forget this and return nil, every request will respond with 404.

Now we need a handler to receive the request, extract a name, and create the actual list.

src/main.clj

(defn make-list [nm]
  {:name nm
   :items {}})

(defn make-list-item [nm]
  {:name nm
   :done? false})

(def list-create
  {:name :list-create
   :enter
   (fn [context]
     (let [nm (get-in context [:request :query-params :name] "Unnamed List") (1)
           new-list (make-list nm)
           db-id (str (gensym "l"))]                        (2)
       (assoc context :tx-data [assoc db-id new-list])))})  (3)

(def routes
  (route/expand-routes
    #{["/todo" :post [db-interceptor list-create]]          (4)
      ["/todo" :get echo :route-name :list-query-form]
      ["/todo/:list-id" :get echo :route-name :list-view]
      ["/todo/:list-id" :post echo :route-name :list-item-create]
      ["/todo/:list-id/:item-id" :get echo :route-name :list-item-view]
      ["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
      ["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))

1	If there was a query parameter, use it. Otherwise use a placeholder.
2	Generate a unique ID. In a real service, the DB itself would do this for us.
3	Don’t modify the DB here. Attach data to the context that the db-interceptor will use.
4	We modify the route definition to use both new interceptors, in order.

There are a couple of things to notice about this updated route definition. First, instead of using echo here, we have a vector with our db-interceptor and list-create interceptors. Recall from the introduction to interceptors that interceptors have two functions. The :enter functions get called from left to right, then the :leave functions get called from right to left. That means calls to the db-interceptor bracket the call to the list-create interceptor.

The second thing about the new route is that we removed the :route-name :list-create clause. Pedestal requires that every route must have a unique name. When we used the echo interceptor everywhere, there was no way for Pedestal to create unique, meaningful names for the routes so we had to help it out. Now, list-create has it’s own name so we can let Pedestal do it’s default behavior: use the name of the last interceptor in the vector.

When you use a function in a route, Pedestal also can’t pick a good name for you. There are quoting forms you can read about in the routing reference that offer some tricks for that.

First Attempt

Let’s give this a try. Here’s a new helper function to make it easier to try requests out interactively:

src/main.clj

(defn test-request [verb url]
  (io.pedestal.test/response-for (::http/service-fn @server) verb url))

With this new helper we can try out a lot of requests from the REPL. Don’t forget to reload the main namespace in the REPL with (require :reload 'main) and restart the server with (main/restart) before you do.

user> (main/test-request :post "/todo?name=A-List")
{:status 404, :body "Not Found", :headers {"Content-Type" "text/plain"}}

Uh oh. What happened here? We just tried this route a minute ago. Was it the query parameter?

user> (main/test-request :post "/todo")
{:status 404, :body "Not Found", :headers {"Content-Type" "text/plain"}}

Nope.

If you check the contents of your in-memory database by running @main/database in the REPL, you’ll see the new item in there:

user=> @main/database
{"l15385" {:name "A-List", :items {}}}

So why did Pedestal return 404 response?

The problem is that neither of the list-create and db-interceptor interceptors attached any kind of response to the context. Pedestal looked at the context after running all the interceptors, saw that there was no response, and sent the 404 as a fallback. We need to attach a response somewhere.

url-for

In REST apis, it is expected that a POST request that creates an entity will then return a response with a 201 Created status code, and include a Location header that points to the new resource.

We have defined a route for that, :get /todo/:list-id, which we’ve named :list-view, and, knowing the list id, it’s quite straight forward to assemble that string.

But should our handler code be responsible for doing so? Anyone with experience in web development knows that things shift around constantly, and should a change to the routing necessitate changes to some scattered collection of handlers? Even if the routing table is stable, we want to avoid duplicated code.

Pedestal provides the url-for function to generate a URL from route name, and will even interpolate parameters into the URL string. So our list-create interceptor can point the client over to a route that will render the new list.

Of course, it would be nice if the response body also included the current state of the just-created list. That saves the client a round trip and saves the service a request. For now, we can do this all in a single interceptor.

src/main.clj

(def list-create
  {:name :list-create
   :enter
   (fn [context]
     (let [nm (get-in context [:request :query-params :name] "Unnamed List")
           new-list (make-list nm)
           db-id (str (gensym "l"))
           url (route/url-for :list-view :params {:list-id db-id})] (1)
       (assoc context
              :response (created new-list "Location" url)
              :tx-data [assoc db-id new-list])))})

1	Generate URL for route :list-view and the new ID we just generated. Pedestal uses some hidden machinery^[3] to get the routing table, which is needed to construct the URL.

And the results (manually formatted for readability):

user> (main/test-request :post "/todo?name=A")
{:status 201,
 :body "{:name \"A\", :items []}", (1)
 :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains",
           "X-Frame-Options" "DENY",
           "X-Content-Type-Options" "nosniff",
           "X-XSS-Protection" "1; mode=block",
           "Location" "/todo/l30488", (2)
           "Content-Type" "application/edn"}} (3)

1	The created list, as map encoded as a string.
2	The location, generated by `url-for`.
3	The Clojure map was converting a string by a default Pedestal interceptor, which also applied the matching content type.

In general, adding something to the database might cause it to change. Sometimes there are triggers or stored procedures that change it. Other times it’s just an ID being assigned. Usually, instead of attaching the new entity here, I would attach its ID to the context and use another interceptor later in the chain to look up the entity after db-interceptor executes the transaction.

This becomes especially important when you are updating existing entities that could have concurrent changes from many sources.

View List

Now that we’ve done one route, this will go a bit faster. The connection from route to interceptor to entity should be a bit more evident at this point.

To view a list, we need to look it up from the "database". If it doesn’t exist, we need to return a 404 (which can be done by omitting a response body). If it does exist, we send it in the response.

src/main.clj

(defn find-list-by-id [dbval db-id]
  (get dbval db-id))                                        (1)

(def list-view
  {:name :list-view
   :enter
   (fn [context]
     (let [db-id (get-in context [:request :path-params :list-id]) (2)
           the-list (when db-id
                      (find-list-by-id                      (3)
                        (get-in context [:request :database])
                        db-id))]
       (cond-> context                                      (4)
         the-list (assoc :result the-list))))})

1	Query the in-memory database.
2	Extract the path parameter, if present.
3	Lookup the matching list if the parameter was provided.
4	If a list was found, add it to the context as :result; otherwise do nothing resulting in a 404 response.

Add Item to List

As we build up a set of interceptors and utility functions, we find that we can go ever faster. Next up, we’ll look at adding a new item to an existing list.

To add an item to a list we will perform the following steps:

Check that the list exists
Create a new item for the list
Add that item to the list
Return a response with the new item

Someone with a seasoned eye may notice that we have multiple steps that could cause problems in heavily used, multi-threaded server:

What if the list is deleted by some other thread between steps 1 and 2?
What if another item is added to the list between steps 1 and 3 (do we lose one of the items?)

This is the kind of thing that is really important to do atomically. We can take advantage of that Clojure atom to execute a function atomically. That’s what the swap! call does for us.

But since that function might not succeed, it means we don’t know enough to create the response inside the list-item-create interceptor. We need to defer response creation until after the db-interceptor runs its :leave function. Basically, we need something to look up an item by its list ID and item ID, then render the result.

That’s exactly what we would do for the :list-item-view route! We can get some reuse here. If list-item-create can add the item to the database, then the item will exist. If the item can’t be added, it will result in a 404 response.

We just need to do something a little funky-looking with the list-item-view interceptor. We need to put its behavior in the :leave function instead of the :enter function.

Here’s the pair of new interceptors, plus the updated route table.

src/main.clj

(defn find-list-item-by-ids [dbval list-id item-id]
  (get-in dbval [list-id :items item-id] nil))

(def entity-render                                          (1)
  {:name :entity-render
   :leave
   (fn [context]
     (if-let [item (:result context)]
       (assoc context :response (ok item))
       context))})

(def list-item-view                                         (2)
  {:name :list-item-view
   :leave
   (fn [context]
     (let [list-id (get-in context [:request :path-params :list-id])
           item-id (and list-id
                        (get-in context [:request :path-params :item-id]))
           item (and item-id
                     (find-list-item-by-ids (get-in context [:request :database]) list-id item-id))]
       (cond-> context
         item (assoc :result item))))})                     (3)

(defn list-item-add
  [dbval list-id item-id new-item]
  (if (contains? dbval list-id)
    (assoc-in dbval [list-id :items item-id] new-item)
    dbval))

(def list-item-create                                       (4)
  {:name :list-item-create
   :enter
   (fn [context]
     (if-let [list-id (get-in context [:request :path-params :list-id])]
       (let [nm (get-in context [:request :query-params :name] "Unnamed Item")
             new-item (make-list-item nm)
             item-id (str (gensym "i"))]
         (-> context
             (assoc :tx-data [list-item-add list-id item-id new-item])
             (assoc-in [:request :path-params :item-id] item-id))) (5)
       context))})

(def routes
  (route/expand-routes
    #{["/todo" :post [db-interceptor list-create]]
      ["/todo" :get echo :route-name :list-query-form]
      ["/todo/:list-id" :get [entity-render db-interceptor list-view]]
      ["/todo/:list-id" :post [entity-render list-item-view db-interceptor list-item-create]]
      ["/todo/:list-id/:item-id" :get [entity-render list-item-view db-interceptor]]
      ["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
      ["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))

1	New generic interceptor that provides a 200 OK response with EDN body out of any Clojure data structure.
2	New interceptor that looks up list items but doesn’t attach a response (since that’s `entity-render’s job).
3	Instead, `list-item-view` attaches the result to the context where `entity-render` expects to find it.
4	This interceptor creates an item and sets up tx-data to be executed.
5	This is where `list-item-create` passes a parameter to `list-item-view`.

To understand the interceptors for the :list-item-create route, we need to think about :enter functions from left to right and :leave functions from right to left.

Phase Interceptor Action

Phase	Interceptor	Action
Enter	`entity-render`	None.
Enter	`list-item-view`	None.
Enter	`db-interceptor`	Attach the database value to the context.
Enter	`list-item-create`	Attach tx-data to the context and a path-param to identify the item.
Leave	`list-item-create`	None.
Leave	`db-interceptor`	Execute the tx-data, attach updated DB value to context.
Leave	`list-item-view`	Find the (new) item by path-params, attach it to the context as :result.
Leave	`entity-render`	Find :result on the context, make an OK response.

Enter

entity-render

None.

Enter

list-item-view

None.

Enter

db-interceptor

Attach the database value to the context.

Enter

list-item-create

Attach tx-data to the context and a path-param to identify the item.

Leave

list-item-create

None.

Leave

db-interceptor

Execute the tx-data, attach updated DB value to context.

Leave

list-item-view

Find the (new) item by path-params, attach it to the context as :result.

Leave

entity-render

Find :result on the context, make an OK response.

A defect here is that when adding a list item, we don’t return a 201 response and a Location header as we do when adding a list. Implementing that should be an additional section in this tutorial.

Splitting the work into these pipelined stages allows us to make very small, very targeted functions that collaborate via the context.

The rest of the routes follow these same basic patterns. Take a few minutes to see if you can create the interceptors for the next few.

The Whole Shebang

Here is the final version of the code. Everything is cleaned, folded, and sorted nicely.

src/main.clj

(ns main
  (:require [io.pedestal.http :as http]
            [io.pedestal.http.route :as route]
            [io.pedestal.test :as test]))

(defn response [status body & {:as headers}]
  {:status status :body body :headers headers})

(def ok (partial response 200))
(def created (partial response 201))
(def accepted (partial response 202))

;;;
;;; "Database" functions
;;;
(defonce database (atom {}))

(defn find-list-by-id [dbval db-id]
  (get dbval db-id))

(defn find-list-item-by-ids [dbval list-id item-id]
  (get-in dbval [list-id :items item-id] nil))

(defn list-item-add
  [dbval list-id item-id new-item]
  (if (contains? dbval list-id)
    (assoc-in dbval [list-id :items item-id] new-item)
    dbval))

(def db-interceptor
  {:name :database-interceptor
   :enter
   (fn [context]
     (update context :request assoc :database @database))
   :leave
   (fn [context]
     (if-let [[op & args] (:tx-data context)]
       (do
         (apply swap! database op args)
         (assoc-in context [:request :database] @database))
       context))})

;;;
;;; Domain functions
;;;
(defn make-list [nm]
  {:name nm
   :items {}})

(defn make-list-item [nm]
  {:name nm
   :done? false})

;;;
;;; API Interceptors
;;;
(def echo
  {:name :echo
   :enter
   (fn [context]
     (let [request (:request context)
           response (ok context)]
       (assoc context :response response)))})

(def entity-render
  {:name :entity-render
   :leave
   (fn [context]
     (if-let [item (:result context)]
       (assoc context :response (ok item))
       context))})

(def list-create
  {:name :list-create
   :enter
   (fn [context]
     (let [nm (get-in context [:request :query-params :name] "Unnamed List")
           new-list (make-list nm)
           db-id (str (gensym "l"))
           url (route/url-for :list-view :params {:list-id db-id})]
       (assoc context
              :response (created new-list "Location" url)
              :tx-data [assoc db-id new-list])))})

(def list-view
  {:name :list-view
   :enter
   (fn [context]
     (let [db-id (get-in context [:request :path-params :list-id])
           the-list (when db-id
                      (find-list-by-id
                        (get-in context [:request :database])
                        db-id))]
       (cond-> context
         the-list (assoc :result the-list))))})

(def list-item-view
  {:name :list-item-view
   :leave
   (fn [context]
     (let [list-id (get-in context [:request :path-params :list-id])
           item-id (and list-id
                        (get-in context [:request :path-params :item-id]))
           item (and item-id
                     (find-list-item-by-ids (get-in context [:request :database]) list-id item-id))]
       (cond-> context
         item (assoc :result item))))})

(def list-item-create
  {:name :list-item-create
   :enter
   (fn [context]
     (if-let [list-id (get-in context [:request :path-params :list-id])]
       (let [nm (get-in context [:request :query-params :name] "Unnamed Item")
             new-item (make-list-item nm)
             item-id (str (gensym "i"))]
         (-> context
             (assoc :tx-data [list-item-add list-id item-id new-item])
             (assoc-in [:request :path-params :item-id] item-id)))
       context))})

(def routes
  (route/expand-routes
    #{["/todo" :post [db-interceptor list-create]]
      ["/todo" :get echo :route-name :list-query-form]
      ["/todo/:list-id" :get [entity-render db-interceptor list-view]]
      ["/todo/:list-id" :post [entity-render list-item-view db-interceptor list-item-create]]
      ["/todo/:list-id/:item-id" :get [entity-render list-item-view db-interceptor]]
      ["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
      ["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))

(def service-map
  {::http/routes routes
   ::http/type :jetty
   ::http/port 8890})

(defn start []
  (http/start (http/create-server service-map)))

;; For interactive development
(defonce server (atom nil))

(defn start-dev []
  (reset! server
          (http/start (http/create-server
                        (assoc service-map
                               ::http/join? false)))))

(defn stop-dev []
  (http/stop @server))

(defn restart []
  (stop-dev)
  (start-dev))

(defn test-request [verb url]
  (io.pedestal.test/response-for (::http/service-fn @server) verb url))

The Path So Far

In this guide, we have built a new API with an in-memory database. You’re ready to go get funding and launch!

We covered some more advanced concepts with interceptors this time:

Passing results from one interceptor to another
Reusing interceptors in different routes
Separating query from transaction with :enter and :leave functions
Testing our routes interactively at a REPL.

Where To Next?

Congratulations! You’ve finished the whole getting stated trail.

To go deep on some of these topics, check out any of the following references:

1. But do you know what status code 418 is for?

2. See the Hello World With Content Types tutorial for an explanation of the echo interceptor.

3. A private, thread-bound var.