Pedestal

Your First API

Welcome Back

This is the fourth part in the Getting Started trail. By now, you’ve built a simple service that vends out friendly greetings by name and can handle a few content types.

We’re going to leave our greeting service alone for the time being and turn our attention to a REST API for some entities. To keep things simple, we’re going to keep the entities in memory. Even so, we will use the same kind of techniques you would use in a real application.

What You Will Learn

After reading this guide, you will be able to:

  • Use path parameters to identify items

  • Generate URLs for new items

  • Encode and decode entity bodies

Guide Assumptions

Like the other guides in the Getting Started, this guide is for beginners who are new to Pedestal and may be new to Clojure. It doesn’t assume any prior experience with a Clojure-based web framework. You should be familiar with the basics of HTTP: URLs, response codes, and content types.

You do not need experience with any particular database or persistence framework.

If you like to jump straight in to the deep end, you might be interested in the Pedestal Crash Course which assumes you know quite a bit about Clojure and web frameworks.

This guide also assumes that you are in a Unix-like development environment, with Java installed. We’ve tested it on Mac OS X and Linux (any flavor) with great results. We haven’t yet tried it on the Windows Subsystem for Linux, but would love to hear from you if you’ve succeeded with it there.

Getting Help if You’re Stuck

We’ll take this in small steps. If you get stuck at any point in this guide, please submit an issue about this guide or hop over to the mailing list and raise your hand there.

Where We Are Going

We will start a new project this time. The service will be a simple "to-do" list that we keep in memory. Even though we are keeping the data in memory, we will use the same techniques that we would use to interact with a real database.

We’re going to make a REST style API. Our API will have the following routes:

Route Verb Action Response

/todo

POST

Create a new list

201 with URL of new list

/todo

GET

Return query form

200 with static page

/todo/:list-id

GET

View an list

200 with all items

/todo/:list-id/:item-id

GET

View an item

200 with item

/todo/:list-id/:item-id

PUT

Update an item

200 with updated item

Setting Up

We will start this project with another empty directory. (Still avoiding the template projects for now.)

I’m going to call my project 'todo-api'. Feel free to call yours something different, but it’s up to you to do the mental translation from here on out.

$ mkdir todo-api
$ cd todo-api
$ mkdir src

This time, we’ll set up our build.boot file first. It will be almost exactly the same as our last one:

build.boot
(set-env!
 :resource-paths #{"src"}
 :dependencies   '[[io.pedestal/pedestal.service "0.5.1"]
                   [io.pedestal/pedestal.route   "0.5.1"]
                   [io.pedestal/pedestal.jetty   "0.5.1"]
                   [org.clojure/data.json        "0.2.6"]
                   [org.slf4j/slf4j-simple       "1.7.21"]])

Our new source file will start off with the usual namespace declaration:

src/main.clj
(ns main
  (:require [io.pedestal.http :as http]
            [io.pedestal.http.route :as route]
            [io.pedestal.test :as test]))

There are a few things we’re going to do differently this time around.

Partial Functions

First, you’ll notice something different with the ok function. partial is a Clojure function that takes a function and returns a new function that needs fewer arguments. (If you know functional programming, it curries a function.) In this case, we’re saying that ok is exactly like response with the first argument pinned to the value 200. Where response wanted two arguments, ok only needs the remaining argument for the body. I like this technique to make a family of functions that vary in one parameter. We’ll use this later to make other functions for different HTTP status codes.

src/main.clj
(defn response [status body & {:as headers}]
  {:status status :body body :headers headers})

(def ok       (partial response 200))
(def created  (partial response 201))
(def accepted (partial response 202))

Obviously we could just pass in the HTTP status codes from wherever we generate the responses. I think the named functions communicate my intentions better though. As we enhance these, they will form a little language for HTTP responses. No framework needed (yet).

Routes Up Front

Second, we’re going to define all our routes right away, but have them all use an echo handler so we can test the routes separately from everything else. (See Hello World, With Content Types for an explanation of the echo interceptor.)

src/main.clj
(def echo
  {:name :echo
   :enter
   (fn [context]
     (let [request (:request context)
           response (ok context)]
       (assoc context :response response)))})

(def routes
  (route/expand-routes
   #{["/todo"                    :post   echo :route-name :list-create]
     ["/todo"                    :get    echo :route-name :list-query-form]
     ["/todo/:list-id"           :get    echo :route-name :list-view]
     ["/todo/:list-id"           :post   echo :route-name :list-item-create]
     ["/todo/:list-id/:item-id"  :get    echo :route-name :list-item-view]
     ["/todo/:list-id/:item-id"  :put    echo :route-name :list-item-update]
     ["/todo/:list-id/:item-id"  :delete echo :route-name :list-item-delete]}))

We will be able to poke each of the routes and check that the parameters all work. This is a nice way to break your work up into incremental steps. It goes together nicely with interactive development

Interactive Development

If you’ve worked through all the guides so far, you might have gotten sick of killing your service and running boot repl over and over again. In real Clojure development, we rarely restart the REPL. Instead, we make our system friendly to interactive development. The main problem so far is that starting the HTTP service doesn’t return. We can fix that by adding a key to the map that says "return instead of waiting for the server to stop."

src/main.clj
(def service-map
  {::http/routes routes
   ::http/type   :jetty
   ::http/port   8890})

(defn start []
  (http/start (http/create-server service-map)))

;; For interactive development
(defonce server (atom nil))                                                             (1)

(defn start-dev []
  (reset! server                                                                        (2)
          (http/start (http/create-server
                       (assoc service-map
                              ::http/join? false)))))                                   (3)

(defn stop-dev []
  (http/stop @server))

(defn restart []                                                                        (4)
  (stop-dev)
  (start-dev))
1 defonce here means we can recompile this file in the same REPL without overwriting the value in this atom.
2 reset! replaces the current value in the atom with the new value.
3 Tell Pedestal not to wait for the server to exit. You set join? to true when running "for real" so that your main function does exit and terminate the process.
4 This is a quick way to bounce the server.

The key ::http/join? does the trick. We can now run start-dev in our REPL sessions. Instead of waiting forever, the REPL thread now returns, prints the value of the server (an ugly mess!) and lets us continue interacting. This is how we are going to start things in dev from now on.

Get ready for a ton of output:

$ boot repl
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: deps.toolsanalyzerjvm.v0v6v6.deps.toolsanalyzer.v0v6v5.clojure.tools.analyzer.utils, being replaced by: #'deps.toolsanalyzerjvm.v0v6v6.deps.toolsanalyzer.v0v6v5.clojure.tools.analyzer.utils/boolean?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: deps.toolsanalyzerjvm.v0v6v6.deps.toolsanalyzer.v0v6v5.clojure.tools.analyzer, being replaced by: #'deps.toolsanalyzerjvm.v0v6v6.deps.toolsanalyzer.v0v6v5.clojure.tools.analyzer.utils/boolean?
nREPL server started on port 37231 on host 127.0.0.1 - nrepl://127.0.0.1:37231
REPL-y 0.3.7, nREPL 0.2.12
Clojure 1.9.0-alpha10
OpenJDK 64-Bit Server VM 1.8.0_91-8u91-b14-3ubuntu1~16.04.1-b14
        Exit: Control+D or (exit) or (quit)
    Commands: (user/help)
        Docs: (doc function-name-here)
              (find-doc "part-of-name-here")
Find by Name: (find-name "part-of-name-here")
      Source: (source function-name-here)
     Javadoc: (javadoc java-object-or-class-here)
    Examples from clojuredocs.org: [clojuredocs or cdoc]
              (user/clojuredocs name-here)
              (user/clojuredocs "ns-here" "name-here")
boot.user=> (require 'main)
WARNING: seqable? already refers to: #'clojure.core/seqable? in namespace: clojure.core.incubator, being replaced by: #'clojure.core.incubator/seqable?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/boolean?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer, being replaced by: #'clojure.tools.analyzer.utils/boolean?
WARNING: bounded-count already refers to: #'clojure.core/bounded-count in namespace: clojure.core.async, being replaced by: #'clojure.core.async/bounded-count
nil
boot.user=> (main/start-dev)
#:io.pedestal.http{:routes ({:path "/todo", :method :post, :path-re #"/\Qtodo\E", :path-parts ["todo"], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-create, :path-params []} {:path "/todo", :method :get, :path-re #"/\Qtodo\E", :path-parts ["todo"], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-query-form, :path-params []} {:path "/todo/:list-id", :method :post, :path-constraints {:list-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)", :path-parts ["todo" :list-id], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-item-create, :path-params [:list-id]} {:path "/todo/:list-id", :method :get, :path-constraints {:list-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)", :path-parts ["todo" :list-id], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-view, :path-params [:list-id]} {:path "/todo/:list-id/:item-id", :method :delete, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-item-delete, :path-params [:list-id :item-id]} {:path "/todo/:list-id/:item-id", :method :get, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-item-view, :path-params [:list-id :item-id]} {:path "/todo/:list-id/:item-id", :method :put, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-item-update, :path-params [:list-id :item-id]}), :server #object[org.eclipse.jetty.server.Server 0x61cb2f0c "org.eclipse.jetty.server.Server@61cb2f0c"], :stop-fn #object[io.pedestal.http.jetty$server$fn__29983 0x234f179c "io.pedestal.http.jetty$server$fn__29983@234f179c"], :type :jetty, :port 8890, :servlet #object[io.pedestal.http.servlet.FnServlet 0x66b335ba "io.pedestal.http.servlet.FnServlet@66b335ba"], :join? false, :service-fn #object[io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__28783 0x356e95fa "io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__28783@356e95fa"], :interceptors [#Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/router}], :start-fn #object[io.pedestal.http.jetty$server$fn__29981 0x11a761bb "io.pedestal.http.jetty$server$fn__29981@11a761bb"]}
boot.user=>

Whatever happened to "no news is good news?" Believe it or not, that is all successful output. The big glob at the end is the value of the server. It’s a big map with all the state in it. The most important thing to do with it is hang on to it so we can stop the server later. We do that by putting it in an atom.

Now we can use start-dev, stop-dev, and restart as we change routes and interceptors.

Poking the Routes

We now have a server instance bound to the atom server in the main namespace and we have some routes. Let’s try them out using io.pedestal.test/response-for. This function exercises our server without going through any actual HTTP requests. We give it a "service function" (which is one of the values in the server map), an HTTP verb, and a URL.

boot.user=> (require '[io.pedestal.test :as test])
nil
boot.user=> (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo")
{:status 200, :body "{:io.pedestal.interceptor.chain/stack (#Interceptor{:name :main/echo} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/ring-response} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol \"HTTP/1.1\", :async-supported? true, :remote-addr \"127.0.0.1\", :servlet-response #object[io.pedestal.test$test_servlet_response$reify__30082 0x57125cf0 \"io.pedestal.test$test_servlet_response$reify__30082@57125cf0\"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x598fc817 \"io.pedestal.http.servlet.FnServlet@598fc817\"], :headers {\"content-length\" \"0\", \"content-type\" \"\"}, :server-port -1, :servlet-request #object[io.pedestal.test$test_servlet_request$reify__30070 0x6ecbde88 \"io.pedestal.test$test_servlet_request$reify__30070@6ecbde88\"], :content-length 0, :content-type \"\", :path-info \"/todo\", :character-encoding \"UTF-8\", :url-for #object[io.pedestal.http.route$url_for_routes$fn__25896 0x2f31b5a3 \"io.pedestal.http.route$url_for_routes$fn__25896@2f31b5a3\"], :uri \"/todo\", :server-name nil, :query-string nil, :path-params {}, :body #object[io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a 0x6a818969 \"io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a@6a818969\"], :scheme nil, :request-method :get}, :bindings {#'io.pedestal.http.route/*url-for* #object[io.pedestal.http.route$url_for_routes$fn__25896 0x2f31b5a3 \"io.pedestal.http.route$url_for_routes$fn__25896@2f31b5a3\"]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x755fc9f2 \"io.pedestal.http.impl.servlet_interceptor$start_servlet_async@755fc9f2\"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__28758 0x4de92b5e \"io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__28758@4de92b5e\"]), :servlet-response #object[io.pedestal.test$test_servlet_response$reify__30082 0x57125cf0 \"io.pedestal.test$test_servlet_response$reify__30082@57125cf0\"], :route {:path \"/todo\", :method :get, :path-re #\"/\\Qtodo\\E\", :path-parts [\"todo\"], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-query-form, :path-params {}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[clojure.core$constantly$fn__6443 0x4afe047b \"clojure.core$constantly$fn__6443@4afe047b\"]}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x598fc817 \"io.pedestal.http.servlet.FnServlet@598fc817\"], :servlet-request #object[io.pedestal.test$test_servlet_request$reify__30070 0x6ecbde88 \"io.pedestal.test$test_servlet_request$reify__30070@6ecbde88\"], :io.pedestal.interceptor.chain/queue #object[clojure.lang.PersistentQueue 0x7de683f5 \"clojure.lang.PersistentQueue@1\"], :url-for #object[io.pedestal.http.route$url_for_routes$fn__25896 0x2f31b5a3 \"io.pedestal.http.route$url_for_routes$fn__25896@2f31b5a3\"], :io.pedestal.interceptor.chain/execution-id 2, :servlet-config nil, :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x66502d81 \"io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@66502d81\"]}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "Content-Type" "application/edn"}}
boot.user=>

The response is pretty large because we are using the echo interceptor, so the whole request is nested inside the response. If we omit the response body, it isn’t quite so hard to read.

boot.user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo") :body)
{:status 200, :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "Content-Type" "application/edn"}}
boot.user=>

Now we can see the status code (200, OK) and the headers. Not too bad. What happens if we ask for a route that doesn’t exist? Or a verb that isn’t supported on a real route?

boot.user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/no-such-route") :body)
{:status 404, :headers {"Content-Type" "text/plain"}}
boot.user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :delete "/todo") :body)
{:status 404, :headers {"Content-Type" "text/plain"}}
boot.user=>

This 404 response comes from Pedestal itself. That is the default response when the router cannot find a route that matches the URL pattern and passes constraints. (For much more about routers, see the Routing and Linking references.)

Path Parameters

Some of our routes have parameters in them. Those look like keywords embedded in the URL patterns. That tells Pedestal to match any value in that position (except a slash!) and bind it to that parameter in the request map. We should be able to see those parameters in the requests that echo will send back to us. This is one of the ways we can make sure that routes do what we think they are going to do, before we write the real logic.

boot.user> (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo/abcdef/12345")
{:status 200, :body "{:io.pedestal.interceptor.chain/stack (#Interceptor{:name :main/echo} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/ring-response} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol \"HTTP/1.1\", :async-supported? true, :remote-addr \"127.0.0.1\", :servlet-response #object[io.pedestal.test$test_servlet_response$reify__30081 0x3a70b3e3 \"io.pedestal.test$test_servlet_response$reify__30081@3a70b3e3\"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x7a707abd \"io.pedestal.http.servlet.FnServlet@7a707abd\"], :headers {\"content-length\" \"0\", \"content-type\" \"\"}, :server-port -1, :servlet-request #object[io.pedestal.test$test_servlet_request$reify__30069 0x55c45773 \"io.pedestal.test$test_servlet_request$reify__30069@55c45773\"], :content-length 0, :content-type \"\", :path-info \"/todo/abcdef/12345\", :character-encoding \"UTF-8\", :url-for #object[io.pedestal.http.route$url_for_routes$fn__25901 0x2e0ca26f \"io.pedestal.http.route$url_for_routes$fn__25901@2e0ca26f\"], :uri \"/todo/abcdef/12345\", :server-name nil, :query-string nil, :path-params {:list-id \"abcdef\", :item-id \"12345\"}, :body #object[io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a 0x115ce3e6 \"io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a@115ce3e6\"], :scheme nil, :request-method :get}, :bindings {#'io.pedestal.http.route/*url-for* #object[io.pedestal.http.route$url_for_routes$fn__25901 0x2e0ca26f \"io.pedestal.http.route$url_for_routes$fn__25901@2e0ca26f\"]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x7a0e6e15 \"io.pedestal.http.impl.servlet_interceptor$start_servlet_async@7a0e6e15\"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__28763 0x6800c6e2 \"io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__28763@6800c6e2\"]), :servlet-response #object[io.pedestal.test$test_servlet_response$reify__30081 0x3a70b3e3 \"io.pedestal.test$test_servlet_response$reify__30081@3a70b3e3\"], :route {:path \"/todo/:list-id/:item-id\", :method :get, :path-constraints {:list-id \"([^/]+)\", :item-id \"([^/]+)\"}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[io.pedestal.http.route.prefix_tree$add_satisfies_constraints_QMARK_$fn__25780 0x2d81021d \"io.pedestal.http.route.prefix_tree$add_satisfies_constraints_QMARK_$fn__25780@2d81021d\"], :path-re #\"/\\Qtodo\\E/([^/]+)/([^/]+)\", :path-parts [\"todo\" :list-id :item-id], :interceptors [#Interceptor{:name :main/echo}], :route-name :list-item-view, :path-params {:list-id \"abcdef\", :item-id \"12345\"}}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x7a707abd \"io.pedestal.http.servlet.FnServlet@7a707abd\"], :servlet-request #object[io.pedestal.test$test_servlet_request$reify__30069 0x55c45773 \"io.pedestal.test$test_servlet_request$reify__30069@55c45773\"], :io.pedestal.interceptor.chain/queue #object[clojure.lang.PersistentQueue 0x1f087b5e \"clojure.lang.PersistentQueue@1\"], :url-for #object[io.pedestal.http.route$url_for_routes$fn__25901 0x2e0ca26f \"io.pedestal.http.route$url_for_routes$fn__25901@2e0ca26f\"], :io.pedestal.interceptor.chain/execution-id 3, :servlet-config nil, :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0xffd0b98 \"io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@ffd0b98\"]}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "Content-Type" "application/edn"}}
boot.user>

The interesting part here is kind of buried inside the request map (which is a string inside the response body). We can pull it out with a bit of Clojure code, but I’ll just highlight it here. (If you’re interested in an exercise, try using clojure.edn to extract the request.)

Inside the request, there is a map :path-params which looks like this:

:path-params {:list-id "abcdef", :item-id "12345"}}

Pedestal extracted these parameters from the URL and bound them to the request for us. Note that the values are always strings, because URLs are strings. If we want to treat "12345" as a number, that’s up to us. But our application had better not break if some sly user sends us up a request with something other than numbers there.

Create List

The first request to handle is creating lists. This is a POST to the "/todo" URL. In the interest of simplicity, the only part of a request body we need is a name field. Also in the interest of simplicity, we are not going to deal with content negotiation yet.

What do we need to do in order to create a list?

  1. Make a new data structure for the list.

  2. Add it to our repository.

We could do this all in one handler function. But we should take advantage of the interceptor construct to separate concerns. A very typical request lifecycle in Pedestal looks like this:

  1. A attaches a database connection to the request.

  2. B looks up an item of interest and attaches it to the request.

  3. C makes decisions and attaches the result of those decisions to the request.

  4. D executes a transaction with those decisions.

This way, A and D are quite generic. They can even be the enter and leave functions of the same interceptor. B is lightly parameterized. Only C is unique to each request type.

(By the way, when using Datomic, A also grabs the current database value and attaches it to the request as well.)

For this guide, we are going to cheat and create an in-memory repository:

src/main.clj
(defonce database (atom {}))

And the interceptor to snapshot the database and handle transactions:

src/main.clj
(def db-interceptor
  {:name :database-interceptor
   :enter
   (fn [context]
     (update context :request assoc :database @database))                               (1)
   :leave
   (fn [context]
     (if-let [[op & args] (:tx-data context)]                                           (2)
       (do
         (apply swap! database op args)                                                 (3)
         (assoc-in context [:request :database] @database))                             (4)
       context))})                                                                      (5)
1 @database gets the current value of the atom. We attach it to the request inside the context.
2 The :tx-data key on the context may be set by other interceptors.
3 The do block is a sure sign of a side effect. The swap! here is our stand-in for a real database transaction.
4 We attach the updated database value in case any other interceptors need to use it.
5 If there wasn’t any tx-data, we just return the context without changes. If you forget this and return nil, every request will respond with 404.

Now we need a handler to receive the request, extract a name, and create the actual list.

src/main.clj
(defn make-list [nm]
  {:name  nm
   :items {}})

(defn make-list-item [nm]
  {:name  nm
   :done? false})

(def list-create
  {:name :list-create
   :enter
   (fn [context]
     (let [nm       (get-in context [:request :query-params :name] "Unnamed List")      (1)
           new-list (make-list nm)
           db-id    (str (gensym "l"))]                                                 (2)
       (assoc context :tx-data [assoc db-id new-list])))})                              (3)

(def routes
  (route/expand-routes
   #{["/todo"                    :post   [db-interceptor list-create]]                  (4)
     ["/todo"                    :get    echo :route-name :list-query-form]
     ["/todo/:list-id"           :get    echo :route-name :list-view]
     ["/todo/:list-id"           :post   echo :route-name :list-item-create]
     ["/todo/:list-id/:item-id"  :get    echo :route-name :list-item-view]
     ["/todo/:list-id/:item-id"  :put    echo :route-name :list-item-update]
     ["/todo/:list-id/:item-id"  :delete echo :route-name :list-item-delete]}))
1 If there was a query parameter, use it. Otherwise use a placeholder.
2 Generate a unique ID. In a real service, the DB itself would do this for us.
3 Don’t modify the DB here. Attach data to the context that the db-interceptor will use.
4 We modify the route definition to use both new interceptors, in order.

There are a couple of things to notice about this updated route definition. First, instead of using echo here, we have a vector with our db-interceptor and list-create interceptors. Recall from the introduction to interceptors that interceptors have two functions. The :enter functions get called from left to right, then the :leave functions get called from right to left. That means calls to the db-interceptor bracket the call to the list-create interceptor.

The second thing about the new route is that we removed the :route-name :list-create clause. Pedestal requires that every route must have a unique name. When we used the echo interceptor everywhere, there was no way for Pedestal to create unique, meaningful names for the routes so we had to help it out. Now, list-create has it’s own name so we can let Pedestal do it’s default behavior: use the name of the last interceptor in the vector.

(Incidentally, when you use a function in a route, Pedestal also can’t pick a good name for you. A function value doesn’t carry any reference back to it’s original definition. There are quoting forms you can read about in the routing and linking reference that offer some tricks for that.)

Let’s give it a try. Here’s a new helper function to make it easier to try requests out interactively:

src/main.clj
(defn test-request [verb url]
  (io.pedestal.test/response-for (::http/service-fn @server) verb url))

With this new helper we can try out a lot of requests from the REPL. Don’t forget to reload the main namespace in the REPL with (require :reload 'main) and restart the server with (main/restart) before you do.

boot.user> (main/test-request :post "/todo?name=A-List")
{:status 404, :body "Not Found", :headers {"Content-Type" "text/plain"}}

Uh oh. What happened here? We just tried this route a minute ago. Was it the query parameter?

boot.user> (main/test-request :post "/todo")
{:status 404, :body "Not Found", :headers {"Content-Type" "text/plain"}}

Nope.

If you check the contents of your in-memory database by running @main/database in the REPL, you’ll see the new item in there. So why did Pedestal complain about a 404?

The problem is that neither the list-create or db-interceptor interceptor attached any kind of response to the context. Pedestal looked at the context after running all the interceptors, saw that there was no response, and sent the 404 as a fallback. We need to attach a response somewhere.

Url-for

The usual thing in REST APIs is to return a 201 status code, with a "Location" header that identifies where the new resource lives. Pedestal provides a function to generate a URL by route name and even interpolate parameters into it. So our list-create interceptor can point the client over to a route that will render the new list.

Of course, it would be nice if the response body also included the current state of the brand new list. That saves the client a round trip and saves the service a request. For now, we can do this all in one interceptor.

src/main.clj
(def list-create
  {:name :list-create
   :enter
   (fn [context]
     (let [nm       (get-in context [:request :query-params :name] "Unnamed List")
           new-list (make-list nm)
           db-id    (str (gensym "l"))
           url      (route/url-for :list-view :params {:list-id db-id})]                (1)
       (assoc context
              :response (created new-list "Location" url)
              :tx-data [assoc db-id new-list])))})
1 Generate URL for route :list-view and the new ID we just generated.

And the results:

boot.user> (main/test-request :post "/todo?name=A")
{:status 201, :body "{:name \"A\", :items []}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "Location" "/todo/l30488", "Content-Type" "application/edn"}}

One quick note about the response body: in general, adding something to the database might cause it to change. Sometimes there are triggers or stored procedures that change it. Other times it’s just an ID being assigned. Usually, instead of attaching the new entity here, I would attach its ID to the context and use another interceptor later in the chain to look up the entity after db-interceptor executes the transaction.

This becomes especially important when you are updating existing entities that could have concurrent changes from many sources.

View List

Now that we’ve done one route, this will go a bit faster. The connection from route to interceptor to entity should be a bit more evident at this point.

To view a list, we need to look it up from the "database". If it doesn’t exist, we need to return a 404 (which can be done by omitting a response body). If it does exist, we send it in the response.

src/main.clj
(defn find-list-by-id [dbval db-id]
  (get dbval db-id))                                                                    (1)

(def list-view
  {:name :list-view
   :enter
   (fn [context]
     (if-let [db-id (get-in context [:request :path-params :list-id])]                  (2)
       (if-let [the-list (find-list-by-id (get-in context [:request :database]) db-id)] (3)
         (assoc context :result the-list)                                               (4)
         context)
       context))})
1 "Query" our fake database.
2 If there was a path parameter, we will use it. Otherwise, 404.
3 If the path parameter names an actual list, we will use it. Otherwise, 404.
4 Everything is OK, so return the list.

Add Item to List

Even faster this time! We need to check that the list exists, modify it, and return the new item.

I hope that some alarm bells are going off for you now. I just described a sequence of actions. That means time could pass while executing those actions. What happens if the list is there when we check it but gets deleted before we try to add the new item?

This is the kind of thing that is really important to do atomically. We can take advantage of that Clojure atom to execute a function atomically. That’s what the swap! call does for us.

But since that function might not succeed, it means we don’t know enough to create the response inside the list-item-create interceptor. We need to defer response creation until after the db-interceptor runs its leave function. Basically, we need something to look up an item by it’s list ID and item ID, then render the result.

That’s exactly what we would do for the :list-item-view route! We can get some reuse here. If list-item-create can add the item to the DB, then the item will exist. Otherwise it will be a 404. We just need to do something a little funky-looking with the list-item-view interceptor. We need to put it’s behavior in the leave function instead of the enter function.

Here’s the pair of new interceptors, plus the updated route table.

src/main.clj
(def entity-render                                                                      (1)
  {:name :entity-render
   :leave
   (fn [context]
     (if-let [item (:result context)]
       (assoc context :response (ok item))
       context))})

(defn find-list-item-by-ids [dbval list-id item-id]
  (get-in dbval [list-id :items item-id] nil))

(def list-item-view                                                                     (2)
  {:name :list-item-view
   :leave
   (fn [context]
     (if-let [list-id (get-in context [:request :path-params :list-id])]
       (if-let [item-id (get-in context [:request :path-params :item-id])]
         (if-let [item (find-list-item-by-ids (get-in context [:request :database]) list-id item-id)]
           (assoc context :result item)                                                 (3)
           context)
         context)
       context))})

(defn list-item-add
  [dbval list-id item-id new-item]
  (if (contains? dbval list-id)
    (assoc-in dbval [list-id :items item-id] new-item)
    dbval))

(def list-item-create                                                                   (4)
  {:name :list-item-create
   :enter
   (fn [context]
     (if-let [list-id (get-in context [:request :path-params :list-id])]
       (let [nm       (get-in context [:request :query-params :name] "Unnamed Item")
             new-item (make-list-item nm)
             item-id  (str (gensym "i"))]
         (-> context
             (assoc :tx-data  [list-item-add list-id item-id new-item])
             (assoc-in [:request :path-params :item-id] item-id)))                      (5)
       context))})

(def routes
  (route/expand-routes
   #{["/todo"                    :post   [db-interceptor list-create]]
     ["/todo"                    :get    echo :route-name :list-query-form]
     ["/todo/:list-id"           :get    [entity-render db-interceptor list-view]]
     ["/todo/:list-id"           :post   [entity-render list-item-view db-interceptor list-item-create]]
     ["/todo/:list-id/:item-id"  :get    [entity-render list-item-view db-interceptor]]
     ["/todo/:list-id/:item-id"  :put    echo :route-name :list-item-update]
     ["/todo/:list-id/:item-id"  :delete echo :route-name :list-item-delete]}))
1 New generic interceptor that makes an 200 OK response with EDN body out of any Clojure data structure.
2 New interceptor that looks up list items but doesn’t attach a response.
3 Instead, list-item-view attaches the result to the context where entity-render expects to find it.
4 This interceptor creates an item and sets up tx-data to be executed.
5 This is where list-item-create passes a parameter to list-item-view.

To understand the interceptors for the :list-item-create route, we need to think about enter functions from left to right and leave functions from right to left.

Phase Interceptor Action

Enter

`entity-render `

None.

Enter

`list-item-view `

None.

Enter

`db-interceptor `

Attach the DB value to the context.

Enter

`list-item-create `

Attach tx-data to the context and a path-param to identify the item.

Leave

`list-item-create `

None.

Leave

`db-interceptor `

Execute the tx-data, attach updated DB value to context.

Leave

`list-item-view `

Find the (new) item by path-params, attach it to the context as :result.

Leave

`entity-render `

Find :result on the context, make an OK response.

Splitting the work into these pipelined stages allows us to make very small, very targeted functions that collaborate via the context.

(By the way, that repetetively nesting structure in list-item-view is a perfect candidate for a Clojure macro. I’ll leave that as an exercise for you, Dear Reader.)

The rest of the routes follow these same basic patterns. Take a few minutes to see if you can create the interceptors for the next few.

The Whole Shebang

Here is the final version of the code. Everything is cleaned, folded, and sorted nicely.

src/main.clj
(ns main
  (:require [io.pedestal.http :as http]
            [io.pedestal.http.route :as route]
            [io.pedestal.test :as test]))

(defn response [status body & {:as headers}]
  {:status status :body body :headers headers})

(def ok       (partial response 200))
(def created  (partial response 201))
(def accepted (partial response 202))

;;;
;;; "Database" functions
;;;
(defonce database (atom {}))

(defn find-list-by-id [dbval db-id]
  (get dbval db-id))

(defn find-list-item-by-ids [dbval list-id item-id]
  (get-in dbval [list-id :items item-id] nil))

(defn list-item-add
  [dbval list-id item-id new-item]
  (if (contains? dbval list-id)
    (assoc-in dbval [list-id :items item-id] new-item)
    dbval))

(def db-interceptor
  {:name :database-interceptor
   :enter
   (fn [context]
     (update context :request assoc :database @database))
   :leave
   (fn [context]
     (if-let [[op & args] (:tx-data context)]
       (do
         (apply swap! database op args)
         (assoc-in context [:request :database] @database))
       context))})

;;;
;;; Domain functions
;;;
(defn make-list [nm]
  {:name  nm
   :items {}})

(defn make-list-item [nm]
  {:name  nm
   :done? false})

;;;
;;; API Interceptors
;;;
(def echo
  {:name :echo
   :enter
   (fn [context]
     (let [request (:request context)
           response (ok context)]
       (assoc context :response response)))})

(def entity-render
  {:name :entity-render
   :leave
   (fn [context]
     (if-let [item (:result context)]
       (assoc context :response (ok item))
       context))})

(def list-create
  {:name :list-create
   :enter
   (fn [context]
     (let [nm       (get-in context [:request :query-params :name] "Unnamed List")
           new-list (make-list nm)
           db-id    (str (gensym "l"))
           url      (route/url-for :list-view :params {:list-id db-id})]
       (assoc context
              :response (created new-list "Location" url)
              :tx-data [assoc db-id new-list])))})

(def list-view
  {:name :list-view
   :enter
   (fn [context]
     (if-let [db-id (get-in context [:request :path-params :list-id])]
       (if-let [the-list (find-list-by-id (get-in context [:request :database]) db-id)]
         (assoc context :result the-list)
         context)
       context))})

(def list-item-view
  {:name :list-item-view
   :leave
   (fn [context]
     (if-let [list-id (get-in context [:request :path-params :list-id])]
       (if-let [item-id (get-in context [:request :path-params :item-id])]
         (if-let [item (find-list-item-by-ids (get-in context [:request :database]) list-id item-id)]
           (assoc context :result item)
           context)
         context)
       context))})

(def list-item-create
  {:name :list-item-create
   :enter
   (fn [context]
     (if-let [list-id (get-in context [:request :path-params :list-id])]
       (let [nm       (get-in context [:request :query-params :name] "Unnamed Item")
             new-item (make-list-item nm)
             item-id  (str (gensym "i"))]
         (-> context
             (assoc :tx-data  [list-item-add list-id item-id new-item])
             (assoc-in [:request :path-params :item-id] item-id)))
       context))})

(def routes
  (route/expand-routes
   #{["/todo"                    :post   [db-interceptor list-create]]
     ["/todo"                    :get    echo :route-name :list-query-form]
     ["/todo/:list-id"           :get    [entity-render db-interceptor list-view]]
     ["/todo/:list-id"           :post   [entity-render list-item-view db-interceptor list-item-create]]
     ["/todo/:list-id/:item-id"  :get    [entity-render list-item-view db-interceptor]]
     ["/todo/:list-id/:item-id"  :put    echo :route-name :list-item-update]
     ["/todo/:list-id/:item-id"  :delete echo :route-name :list-item-delete]}))

(def service-map
  {::http/routes routes
   ::http/type   :jetty
   ::http/port   8890})

(defn start []
  (http/start (http/create-server service-map)))

;; For interactive development
(defonce server (atom nil))

(defn start-dev []
  (reset! server
          (http/start (http/create-server
                       (assoc service-map
                              ::http/join? false)))))

(defn stop-dev []
  (http/stop @server))

(defn restart []
  (stop-dev)
  (start-dev))

(defn test-request [verb url]
  (io.pedestal.test/response-for (::http/service-fn @server) verb url))

The Path So Far

In this guide, we have built a new API with an in-memory database. You’re ready to go get funding and launch!

We covered some more advanced concepts with interceptors this time:

  • Passing results from one interceptor to another

  • Reusing interceptors in different routes

  • Separating query from transaction with enter and leave functions

  • Testing our routes interactively at a REPL.

Where To Next?

Congratulations! You’ve finished the whole Getting Started trail.

For more guided learning, head back up to the Guides page. Or, to go deep on some of these topics, check out any of the following references: