Your First API
Welcome Back
This is the fourth part in the tutorial. By now, you’ve built a simple service that vends out friendly greetings by name and can handle a few content types.
We’re going to leave our greeting service alone for the time being and turn our attention to a REST API for some entities. To keep things simple, we’re going to keep the entities in memory. Even so, we will use the same kind of techniques you would use in a real application.
What You Will Learn
After reading this guide, you will be able to:
-
Use path parameters to identify items
-
Generate URLs for new items
-
Encode and decode entity bodies
Guide Assumptions
This guide is for beginners who are new to Pedestal and may be new to Clojure. It doesn’t assume any prior experience with a Clojure-based web framework. You should be familiar with the basics of HTTP: URLs, response codes, and content types.
You do not need experience with any particular database or persistence framework.
This guide also assumes that you are in a Unix-like development environment, with Java installed. We’ve tested it on Mac OS X and Linux (any flavor) with great results. We haven’t yet tried it on the Windows Subsystem for Linux, but would love to hear from you if you’ve succeeded with it there.
Getting Help if You’re Stuck
We’ll take this in small steps. If you get stuck at any point in this guide, please submit an issue about this guide or, hop over to the Pedestal Users mailing list and raise your hand there. You can also get help from the #pedestal channel on the Clojurians Slack.
Where We Are Going
We will start a new project this time. The service will be a simple "to-do" list that persists only in memory. We will use the same techniques that we would use to interact with a real database.
We’re going to make a REST style API. Our API will have the following routes:
Route | Verb | Action | Response |
---|---|---|---|
/todo |
POST |
Create a new list |
201 with URL of new list |
/todo |
GET |
Return query form |
200 with static page |
/todo/:list-id |
GET |
View a list |
200 with all items |
/todo/:list-id/:item-id |
GET |
View an item |
200 with item |
/todo/:list-id/:item-id |
PUT |
Update an item |
200 with updated item |
Setting Up
We will start this project with another empty directory. (Still avoiding the template projects for now.)
I’m going to call my project 'todo-api'. Feel free to call yours something different, but it’s up to you to do the mental translation from here on out.
$ mkdir todo-api $ cd todo-api $ mkdir src
This time, we’ll set up our deps.edn
file first. It will be almost
exactly the same as our last one:
{:paths ["src"]
:deps {io.pedestal/pedestal.jetty {:mvn/version "0.8.0-SNAPSHOT"}
org.clojure/data.json {:mvn/version "2.5.0"}
org.slf4j/slf4j-simple {:mvn/version "2.0.10"}}}
Our new source file will start off with the usual namespace declaration:
(ns main
(:require [io.pedestal.http :as http]
[io.pedestal.http.route :as route]
[io.pedestal.test :as test]))
Partial Functions
We’re going to use some of the same structures from
previous tutorials, such
as an ok
function to build a successful response, but with some differences:
(defn response [status body & {:as headers}]
{:status status :body body :headers headers})
(def ok (partial response 200))
(def created (partial response 201))
(def accepted (partial response 202))
partial
is a Clojure function that takes a function and
returns a new function that needs fewer arguments - you supply the initial arguments in the call
to partial
. The returned function keeps track of those initial arguments, and when invoked,
"glues together" the initial arguments and any additional arguments provided to it, and calls the
original function: response
in this case.
Although created with def
, the symbols ok
, created
, and accepted
, are all functions, as if
they were created using defn
.
Routes Up Front
Second, we’re going to define all our routes right away, but have them
all use an echo
handler so we can test the routes separately from
everything else[2].
(def echo
{:name :echo
:enter
(fn [context]
(let [request (:request context)
response (ok context)]
(assoc context :response response)))})
(def routes
(route/expand-routes
#{["/todo" :post echo :route-name :list-create]
["/todo" :get echo :route-name :list-query-form]
["/todo/:list-id" :get echo :route-name :list-view]
["/todo/:list-id" :post echo :route-name :list-item-create]
["/todo/:list-id/:item-id" :get echo :route-name :list-item-view]
["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))
Pedestal makes it possible to invoke these routes directly from the REPL, without involving HTTP at all. This is a nice way to break your work up into incremental steps. It goes together nicely with the interactive development support we introduced in the Hello World, With Parameters tutorial.
Get ready for a ton of output:
> clj Clojure 1.11.1 user=> (require 'main) nil user=> (main/start-dev) [main] INFO org.eclipse.jetty.server.Server - jetty-11.0.18; built: 2023-10-27T02:14:36.036Z; git: 5a9a771a9fbcb9d36993630850f612581b78c13f; jvm 11.0.19+7-LTS [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@6c8f2e4e{/,null,AVAILABLE} [main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@1432a683{HTTP/1.1, (http/1.1, h2c)}{localhost:8890} [main] INFO org.eclipse.jetty.server.Server - Started Server@b025f2f{STARTING}[11.0.18,sto=0] @57816ms #:io.pedestal.http{:port 8890, :service-fn #object[io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__14892 0x515f104c "io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__14892@515f104c"], :host "localhost", :type :jetty, :start-fn #object[io.pedestal.http.jetty$server$fn__15680 0x1dde2d1 "io.pedestal.http.jetty$server$fn__15680@1dde2d1"], :interceptors [#Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.route/path-params-decoder}], :routes ({:path "/todo/:list-id", :method :get, :path-constraints {:list-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)", :path-parts ["todo" :list-id], :interceptors [#Interceptor{:name :entity-render} #Interceptor{:name :database-interceptor} #Interceptor{:name :list-view}], :route-name :list-view, :path-params [:list-id]} {:path "/todo/:list-id", :method :post, :path-constraints {:list-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)", :path-parts ["todo" :list-id], :interceptors [#Interceptor{:name :entity-render} #Interceptor{:name :list-item-view} #Interceptor{:name :database-interceptor} #Interceptor{:name :list-item-create}], :route-name :list-item-create, :path-params [:list-id]} {:path "/todo", :method :post, :path-re #"/\Qtodo\E", :path-parts ["todo"], :interceptors [#Interceptor{:name :database-interceptor} #Interceptor{:name :list-create}], :route-name :list-create, :path-params []} {:path "/todo", :method :get, :path-re #"/\Qtodo\E", :path-parts ["todo"], :interceptors [#Interceptor{:name :echo}], :route-name :list-query-form, :path-params []} {:path "/todo/:list-id/:item-id", :method :delete, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :echo}], :route-name :list-item-delete, :path-params [:list-id :item-id]} {:path "/todo/:list-id/:item-id", :method :get, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :entity-render} #Interceptor{:name :list-item-view} #Interceptor{:name :database-interceptor}], :route-name :database-interceptor, :path-params [:list-id :item-id]} {:path "/todo/:list-id/:item-id", :method :put, :path-constraints {:list-id "([^/]+)", :item-id "([^/]+)"}, :path-re #"/\Qtodo\E/([^/]+)/([^/]+)", :path-parts ["todo" :list-id :item-id], :interceptors [#Interceptor{:name :echo}], :route-name :list-item-update, :path-params [:list-id :item-id]}), :servlet #object[io.pedestal.http.servlet.FnServlet 0x47caa1ff "io.pedestal.http.servlet.FnServlet@47caa1ff"], :server #object[org.eclipse.jetty.server.Server 0xb025f2f "Server@b025f2f{STARTED}[11.0.18,sto=0]"], :join? false, :stop-fn #object[io.pedestal.http.jetty$server$fn__15682 0x7330574d "io.pedestal.http.jetty$server$fn__15682@7330574d"]} user=>
Poking the Routes
Let’s try out the routes that we defined in the main
namespace. We’ll do that using
response-for
. This
function exercises our server without going through any actual HTTP
requests.
In Pedestal, the :io.pedestal.http/service-fn key of the service map contains the service function which is essentially the bridge between Servlet APIs and Pedestal’s interceptor chain.
By extracting the service-fn (which we keep handy when
running in development mode), we can pass it to response-for
along with the HTTP verb (as a keyword), and the URL:
user=> (require '[io.pedestal.test :as test]) nil user=> (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo") [main] INFO io.pedestal.http - {:msg "GET /todo", :line 80} {:status 200, :body "{:io.pedestal.interceptor.chain/stack (#Interceptor{:name :echo} #Interceptor{:name :io.pedestal.http.route/path-params-decoder} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/ring-response} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol \"HTTP/1.1\", :async-supported? true, :remote-addr \"127.0.0.1\", :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x3e1ff1cf \"io.pedestal.test$test_servlet_response$reify__15083@3e1ff1cf\"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x389588b2 \"io.pedestal.http.servlet.FnServlet@389588b2\"], :headers {\"content-length\" \"0\", \"content-type\" \"\"}, :server-port -1, :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x57e9122f \"io.pedestal.test$test_servlet_request$reify__15071@57e9122f\"], :content-length 0, :content-type \"\", :path-info \"/todo\", :character-encoding \"UTF-8\", :url-for #object[clojure.lang.Delay 0x41c22d59 {:status :pending, :val nil}], :uri \"/todo\", :server-name nil, :query-string nil, :path-params {}, :body #object[io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a 0x5ea52724 \"io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a@5ea52724\"], :scheme nil, :request-method :get, :context-path \"\"}, :bindings {#'io.pedestal.http.route/*url-for* #object[clojure.lang.Delay 0x41c22d59 {:status :pending, :val nil}]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x7dfccbf2 \"io.pedestal.http.impl.servlet_interceptor$start_servlet_async@7dfccbf2\"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810 0x1fe24f79 \"io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810@1fe24f79\"]), :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x3e1ff1cf \"io.pedestal.test$test_servlet_response$reify__15083@3e1ff1cf\"], :route {:path \"/todo\", :method :get, :path-re #\"/\\Qtodo\\E\", :path-parts [\"todo\"], :interceptors [#Interceptor{:name :echo}], :route-name :list-query-form, :path-params {}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[clojure.core$constantly$fn__5740 0x4f9a0139 \"clojure.core$constantly$fn__5740@4f9a0139\"]}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x389588b2 \"io.pedestal.http.servlet.FnServlet@389588b2\"], :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x57e9122f \"io.pedestal.test$test_servlet_request$reify__15071@57e9122f\"], :io.pedestal.interceptor.chain/queue #object[clojure.lang.PersistentQueue 0x38f183e9 \"clojure.lang.PersistentQueue@1\"], :url-for #object[clojure.lang.Delay 0x41c22d59 {:status :pending, :val nil}], :io.pedestal.interceptor.chain/execution-id 1, :servlet-config nil, :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x23497c5e \"io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@23497c5e\"]}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "X-Download-Options" "noopen", "X-Permitted-Cross-Domain-Policies" "none", "Content-Security-Policy" "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", "Content-Type" "application/edn"}} user=>
Part of this output — [main] INFO … — comes from built-in Pedestal interceptors
for logging requests; the logging output gets mixed into the output from the REPL.
|
The response is somewhat overwhelmingly verbose because we are using the echo
interceptor, so the entire incoming request converted to a string and is nested inside the response. If we
omit the response body, it’s easier to observe the most useful parts of the response:
user=> (dissoc *1 :body) {:status 200, :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "X-Download-Options" "noopen", "X-Permitted-Cross-Domain-Policies" "none", "Content-Security-Policy" "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", "Content-Type" "application/edn"}}
Inside the REPL, Clojure keeps track of the most recent result it printed in the
symbol *1 . That makes it a snap to dig down into that result. There’s also *2 and *3 .
|
Now we can see the status code (200, OK) and the headers. Not too bad. What happens if we ask for a route that doesn’t exist? Or a verb that isn’t supported on a real route?
user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/no-such-route") :body) [main] INFO io.pedestal.http - {:msg "GET /no-such-route", :line 80} {:status 404, :headers {"Content-Type" "text/plain"}} user=> (dissoc (test/response-for (:io.pedestal.http/service-fn @main/server) :delete "/todo") :body) [main] INFO io.pedestal.http - {:msg "DELETE /todo", :line 80} {:status 404, :headers {"Content-Type" "text/plain"}} user=>
This 404 response comes from Pedestal itself. That is the default response when the router cannot find a route that matches the URL pattern and passes constraints. (For much more about routers, see the Routing Quick Reference).
Path Parameters
Some of our routes have parameters in them. Those look like keywords
embedded in the URL patterns. That tells Pedestal to match any value
in that position (except a slash!) and bind it to that parameter in
the request map. We should be able to
see those parameters in the requests that echo
will send back
to us. This is one of the ways we can make sure that routes do what we
think they are going to do, before we write the real logic.
user=> (test/response-for (:io.pedestal.http/service-fn @main/server) :get "/todo/abcdef/12345") [main] INFO io.pedestal.http - {:msg "GET /todo/abcdef/12345", :line 80} {:status 200, :body "{:io.pedestal.interceptor.chain/stack (#Interceptor{:name :echo} #Interceptor{:name :io.pedestal.http.route/path-params-decoder} #Interceptor{:name :io.pedestal.http.route/router} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/ring-response} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol \"HTTP/1.1\", :async-supported? true, :remote-addr \"127.0.0.1\", :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x29c808c8 \"io.pedestal.test$test_servlet_response$reify__15083@29c808c8\"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x265b5f05 \"io.pedestal.http.servlet.FnServlet@265b5f05\"], :headers {\"content-length\" \"0\", \"content-type\" \"\"}, :server-port -1, :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x6916566d \"io.pedestal.test$test_servlet_request$reify__15071@6916566d\"], :content-length 0, :content-type \"\", :path-info \"/todo/abcdef/12345\", :character-encoding \"UTF-8\", :url-for #object[clojure.lang.Delay 0x614c7175 {:status :pending, :val nil}], :uri \"/todo/abcdef/12345\", :server-name nil, :query-string nil, :path-params {:list-id \"abcdef\", :item-id \"12345\"}, :body #object[io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a 0x82bbbdc \"io.pedestal.test.proxy$javax.servlet.ServletInputStream$ff19274a@82bbbdc\"], :scheme nil, :request-method :get, :context-path \"\"}, :bindings {#'io.pedestal.http.route/*url-for* #object[clojure.lang.Delay 0x614c7175 {:status :pending, :val nil}]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x7dfccbf2 \"io.pedestal.http.impl.servlet_interceptor$start_servlet_async@7dfccbf2\"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810 0x74f47eab \"io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__14810@74f47eab\"]), :servlet-response #object[io.pedestal.test$test_servlet_response$reify__15083 0x29c808c8 \"io.pedestal.test$test_servlet_response$reify__15083@29c808c8\"], :route {:path \"/todo/:list-id/:item-id\", :method :get, :path-constraints {:list-id \"([^/]+)\", :item-id \"([^/]+)\"}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[io.pedestal.http.route.prefix_tree$add_satisfies_constraints_QMARK_$fn__11533 0x2f8afb66 \"io.pedestal.http.route.prefix_tree$add_satisfies_constraints_QMARK_$fn__11533@2f8afb66\"], :path-re #\"/\\Qtodo\\E/([^/]+)/([^/]+)\", :path-parts [\"todo\" :list-id :item-id], :interceptors [#Interceptor{:name :echo}], :route-name :list-item-view, :path-params {:list-id \"abcdef\", :item-id \"12345\"}}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x265b5f05 \"io.pedestal.http.servlet.FnServlet@265b5f05\"], :servlet-request #object[io.pedestal.test$test_servlet_request$reify__15071 0x6916566d \"io.pedestal.test$test_servlet_request$reify__15071@6916566d\"], :io.pedestal.interceptor.chain/queue #object[clojure.lang.PersistentQueue 0x2d4fe03b \"clojure.lang.PersistentQueue@1\"], :url-for #object[clojure.lang.Delay 0x614c7175 {:status :pending, :val nil}], :io.pedestal.interceptor.chain/execution-id 9, :servlet-config nil, :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x23497c5e \"io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@23497c5e\"]}", :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "X-Download-Options" "noopen", "X-Permitted-Cross-Domain-Policies" "none", "Content-Security-Policy" "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", "Content-Type" "application/edn"}}
The interesting part here is kind of buried inside the request map
(which is a string inside the response body). We can pull it out with
a bit of Clojure code, but I’ll just highlight it here. (If you’re
interested in an exercise, try using clojure.edn
to extract the
request.)
Inside the request, there is a map :path-params which looks like this:
:path-params {:list-id "abcdef", :item-id "12345"}}
Pedestal extracted these parameters from the URL and bound them to the request for us. Note that the values are always strings, because URLs are strings. If we want to treat "12345" as a number, that’s up to us. But our application had better not break if some sly user sends us up a request with something other than numbers there.
Create List
The first request to handle is creating lists. This is a POST to the
/todo
URL. In the interest of simplicity, the only part of a request
body we need is a name field. Also in the interest of simplicity, we
are not going to deal with content negotiation yet.
What do we need to do in order to create a list?
-
Make a new data structure for the list.
-
Add it to our repository.
Without getting too far ahead of ourselves, we’d like to avoid doing all of the above in a single handler function, and instead, break this up into several interceptors, each of which is (as much as possible) side-effect free and easily testable.
Let’s break the behavior up into multiple steps, and give each step a provisional name:
-
A
attaches a database connection to the request. -
B
looks up an item of interest and attaches it to the request. -
C
makes some decisions and attaches the result of those decisions to the request. -
D
executes a transaction with those decisions.
This way, A
and D
are quite generic.
B
is somewhat specific to the particular route — is it a list or a list item that’s being looked up?.
That leaves C
which is most specific to a single route, and might event be a simple handler function.
For this guide, we are going to cheat and create an in-memory repository:
(defonce database (atom {})) (1)
1 | Using defonce here ensures that the value, the database atom, will survive when we reload
the namespace. |
And the interceptor to snapshot the database and handle transactions:
(def db-interceptor
{:name :database-interceptor
:enter
(fn [context]
(update context :request assoc :database @database)) (1)
:leave
(fn [context]
(if-let [[op & args] (:tx-data context)] (2)
(do
(apply swap! database op args) (3)
(assoc-in context [:request :database] @database)) (4)
context))}) (5)
1 | @database gets the current value of the atom. We attach it to
the request inside the context. This is step A from above. |
2 | The :tx-data key on the context may be set by other
interceptors. This is step D from above. |
3 | The do block is a sure sign of a side effect. The swap! here is our stand-in for a real database transaction. |
4 | We attach the updated database value in case any other
interceptors need to use it. We use the latest version of the database (via @database ), not the
value in the request, to account for any concurrent threads changing the database simultaneously. |
5 | If there wasn’t any tx-data, we just return the context without changes. If you forget this and return nil, every request will respond with 404. |
Now we need a handler to receive the request, extract a name, and create the actual list.
(defn make-list [nm]
{:name nm
:items {}})
(defn make-list-item [nm]
{:name nm
:done? false})
(def list-create
{:name :list-create
:enter
(fn [context]
(let [nm (get-in context [:request :query-params :name] "Unnamed List") (1)
new-list (make-list nm)
db-id (str (gensym "l"))] (2)
(assoc context :tx-data [assoc db-id new-list])))}) (3)
(def routes
(route/expand-routes
#{["/todo" :post [db-interceptor list-create]] (4)
["/todo" :get echo :route-name :list-query-form]
["/todo/:list-id" :get echo :route-name :list-view]
["/todo/:list-id" :post echo :route-name :list-item-create]
["/todo/:list-id/:item-id" :get echo :route-name :list-item-view]
["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))
1 | If there was a query parameter, use it. Otherwise use a placeholder. |
2 | Generate a unique ID. In a real service, the DB itself would do this for us. |
3 | Don’t modify the DB here. Attach data to the context that the db-interceptor will use. |
4 | We modify the route definition to use both new interceptors, in order. |
There are a couple of things to notice about this updated route
definition. First, instead of using echo
here, we have a vector with
our db-interceptor
and list-create
interceptors. Recall from
the introduction to
interceptors that interceptors have two functions. The :enter
functions get called from left to right, then the :leave functions
get called from right to left. That means calls to the
db-interceptor
bracket the call to the list-create
interceptor.
The second thing about the new route is that we removed the
:route-name :list-create
clause. Pedestal requires that every route
must have a unique name. When we used the echo
interceptor
everywhere, there was no way for Pedestal to create unique, meaningful
names for the routes so we had to help it out. Now, list-create
has
it’s own name so we can let Pedestal do it’s default behavior: use the
name of the last interceptor in the vector.
When you use a function in a route, Pedestal also can’t pick a good name for you. There are quoting forms you can read about in the routing reference that offer some tricks for that. |
First Attempt
Let’s give this a try. Here’s a new helper function to make it easier to try requests out interactively:
(defn test-request [verb url]
(io.pedestal.test/response-for (::http/service-fn @server) verb url))
With this new helper we can try out a lot of requests from the REPL.
Don’t forget to reload the main namespace in the REPL with (require
:reload 'main)
and restart the server with (main/restart)
before
you do.
user> (main/test-request :post "/todo?name=A-List") {:status 404, :body "Not Found", :headers {"Content-Type" "text/plain"}}
Uh oh. What happened here? We just tried this route a minute ago. Was it the query parameter?
user> (main/test-request :post "/todo") {:status 404, :body "Not Found", :headers {"Content-Type" "text/plain"}}
Nope.
If you check the contents of your in-memory database by running
@main/database
in the REPL, you’ll see the new item in there:
user=> @main/database {"l15385" {:name "A-List", :items {}}}
So why did Pedestal return 404 response?
The problem is that neither of the list-create
and db-interceptor
interceptors attached any kind of response to the context. Pedestal
looked at the context after running all the interceptors, saw that
there was no response, and sent the 404 as a fallback. We need to
attach a response somewhere.
url-for
In REST apis, it is expected that a POST request that creates an entity will then return a response with a 201 Created status code, and include a Location header that points to the new resource.
We have defined a route for that, :get /todo/:list-id
, which we’ve named
:list-view, and, knowing the list id, it’s quite straight forward to assemble that string.
But should our handler code be responsible for doing so? Anyone with experience in web development knows that things shift around constantly, and should a change to the routing necessitate changes to some scattered collection of handlers? Even if the routing table is stable, we want to avoid duplicated code.
Pedestal provides the url-for
function to generate a URL from route name,
and will even interpolate parameters into the URL string. So our list-create
interceptor can point the client over to a route that will render the
new list.
Of course, it would be nice if the response body also included the current state of the just-created list. That saves the client a round trip and saves the service a request. For now, we can do this all in a single interceptor.
(def list-create
{:name :list-create
:enter
(fn [context]
(let [nm (get-in context [:request :query-params :name] "Unnamed List")
new-list (make-list nm)
db-id (str (gensym "l"))
url (route/url-for :list-view :params {:list-id db-id})] (1)
(assoc context
:response (created new-list "Location" url)
:tx-data [assoc db-id new-list])))})
1 | Generate URL for route :list-view and the new ID we just generated. Pedestal uses some hidden machinery[3] to get the routing table, which is needed to construct the URL. |
And the results (manually formatted for readability):
user> (main/test-request :post "/todo?name=A") {:status 201, :body "{:name \"A\", :items []}", (1) :headers {"Strict-Transport-Security" "max-age=31536000; includeSubdomains", "X-Frame-Options" "DENY", "X-Content-Type-Options" "nosniff", "X-XSS-Protection" "1; mode=block", "Location" "/todo/l30488", (2) "Content-Type" "application/edn"}} (3)
1 | The created list, as map encoded as a string. |
2 | The location, generated by url-for . |
3 | The Clojure map was converting a string by a default Pedestal interceptor, which also applied the matching content type. |
In general, adding something
to the database might cause it to change. Sometimes there are triggers
or stored procedures that change it. Other times it’s just an ID being
assigned. Usually, instead of attaching the new entity here, I would
attach its ID to the context and use another interceptor later in
the chain to look up the entity after db-interceptor
executes the
transaction.
This becomes especially important when you are updating existing entities that could have concurrent changes from many sources.
View List
Now that we’ve done one route, this will go a bit faster. The connection from route to interceptor to entity should be a bit more evident at this point.
To view a list, we need to look it up from the "database". If it doesn’t exist, we need to return a 404 (which can be done by omitting a response body). If it does exist, we send it in the response.
(defn find-list-by-id [dbval db-id]
(get dbval db-id)) (1)
(def list-view
{:name :list-view
:enter
(fn [context]
(let [db-id (get-in context [:request :path-params :list-id]) (2)
the-list (when db-id
(find-list-by-id (3)
(get-in context [:request :database])
db-id))]
(cond-> context (4)
the-list (assoc :result the-list))))})
1 | Query the in-memory database. |
2 | Extract the path parameter, if present. |
3 | Lookup the matching list if the parameter was provided. |
4 | If a list was found, add it to the context as :result; otherwise do nothing resulting in a 404 response. |
Add Item to List
As we build up a set of interceptors and utility functions, we find that we can go ever faster. Next up, we’ll look at adding a new item to an existing list.
To add an item to a list we will perform the following steps:
-
Check that the list exists
-
Create a new item for the list
-
Add that item to the list
-
Return a response with the new item
Someone with a seasoned eye may notice that we have multiple steps that could cause problems in heavily used, multi-threaded server:
-
What if the list is deleted by some other thread between steps 1 and 2?
-
What if another item is added to the list between steps 1 and 3 (do we lose one of the items?)
This is the kind of thing that is really important to do
atomically. We can take advantage of that Clojure atom to execute a
function atomically. That’s what the swap!
call does for us.
But since that function might not succeed, it means we don’t know
enough to create the response inside the list-item-create
interceptor. We need to defer response creation until after the
db-interceptor
runs its :leave function. Basically, we need
something to look up an item by its list ID and item ID, then render
the result.
That’s exactly what we would do for the :list-item-view route! We
can get some reuse here. If list-item-create
can add the item to the
database, then the item will exist. If the item can’t be added, it will result in a 404 response.
We just need to do something a little funky-looking with the list-item-view
interceptor. We need to put its behavior in the :leave function
instead of the :enter function.
Here’s the pair of new interceptors, plus the updated route table.
(defn find-list-item-by-ids [dbval list-id item-id]
(get-in dbval [list-id :items item-id] nil))
(def entity-render (1)
{:name :entity-render
:leave
(fn [context]
(if-let [item (:result context)]
(assoc context :response (ok item))
context))})
(def list-item-view (2)
{:name :list-item-view
:leave
(fn [context]
(let [list-id (get-in context [:request :path-params :list-id])
item-id (and list-id
(get-in context [:request :path-params :item-id]))
item (and item-id
(find-list-item-by-ids (get-in context [:request :database]) list-id item-id))]
(cond-> context
item (assoc :result item))))}) (3)
(defn list-item-add
[dbval list-id item-id new-item]
(if (contains? dbval list-id)
(assoc-in dbval [list-id :items item-id] new-item)
dbval))
(def list-item-create (4)
{:name :list-item-create
:enter
(fn [context]
(if-let [list-id (get-in context [:request :path-params :list-id])]
(let [nm (get-in context [:request :query-params :name] "Unnamed Item")
new-item (make-list-item nm)
item-id (str (gensym "i"))]
(-> context
(assoc :tx-data [list-item-add list-id item-id new-item])
(assoc-in [:request :path-params :item-id] item-id))) (5)
context))})
(def routes
(route/expand-routes
#{["/todo" :post [db-interceptor list-create]]
["/todo" :get echo :route-name :list-query-form]
["/todo/:list-id" :get [entity-render db-interceptor list-view]]
["/todo/:list-id" :post [entity-render list-item-view db-interceptor list-item-create]]
["/todo/:list-id/:item-id" :get [entity-render list-item-view db-interceptor]]
["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))
1 | New generic interceptor that provides a 200 OK response with EDN body out of any Clojure data structure. |
2 | New interceptor that looks up list items but doesn’t attach a response (since that’s `entity-render’s job). |
3 | Instead, list-item-view attaches the result to the context where
entity-render expects to find it. |
4 | This interceptor creates an item and sets up tx-data to be executed. |
5 | This is where list-item-create passes a parameter to
list-item-view . |
To understand the interceptors for the :list-item-create route, we need to think about :enter functions from left to right and :leave functions from right to left.
Phase | Interceptor | Action |
---|---|---|
Enter |
|
None. |
Enter |
|
None. |
Enter |
|
Attach the database value to the context. |
Enter |
|
Attach tx-data to the context and a path-param to identify the item. |
Leave |
|
None. |
Leave |
|
Execute the tx-data, attach updated DB value to context. |
Leave |
|
Find the (new) item by path-params, attach it to the context as :result. |
Leave |
|
Find :result on the context, make an OK response. |
A defect here is that when adding a list item, we don’t return a 201 response and a Location header as we do when adding a list. Implementing that should be an additional section in this tutorial. |
Splitting the work into these pipelined stages allows us to make very small, very targeted functions that collaborate via the context.
The rest of the routes follow these same basic patterns. Take a few minutes to see if you can create the interceptors for the next few.
The Whole Shebang
Here is the final version of the code. Everything is cleaned, folded, and sorted nicely.
(ns main
(:require [io.pedestal.http :as http]
[io.pedestal.http.route :as route]
[io.pedestal.test :as test]))
(defn response [status body & {:as headers}]
{:status status :body body :headers headers})
(def ok (partial response 200))
(def created (partial response 201))
(def accepted (partial response 202))
;;;
;;; "Database" functions
;;;
(defonce database (atom {}))
(defn find-list-by-id [dbval db-id]
(get dbval db-id))
(defn find-list-item-by-ids [dbval list-id item-id]
(get-in dbval [list-id :items item-id] nil))
(defn list-item-add
[dbval list-id item-id new-item]
(if (contains? dbval list-id)
(assoc-in dbval [list-id :items item-id] new-item)
dbval))
(def db-interceptor
{:name :database-interceptor
:enter
(fn [context]
(update context :request assoc :database @database))
:leave
(fn [context]
(if-let [[op & args] (:tx-data context)]
(do
(apply swap! database op args)
(assoc-in context [:request :database] @database))
context))})
;;;
;;; Domain functions
;;;
(defn make-list [nm]
{:name nm
:items {}})
(defn make-list-item [nm]
{:name nm
:done? false})
;;;
;;; API Interceptors
;;;
(def echo
{:name :echo
:enter
(fn [context]
(let [request (:request context)
response (ok context)]
(assoc context :response response)))})
(def entity-render
{:name :entity-render
:leave
(fn [context]
(if-let [item (:result context)]
(assoc context :response (ok item))
context))})
(def list-create
{:name :list-create
:enter
(fn [context]
(let [nm (get-in context [:request :query-params :name] "Unnamed List")
new-list (make-list nm)
db-id (str (gensym "l"))
url (route/url-for :list-view :params {:list-id db-id})]
(assoc context
:response (created new-list "Location" url)
:tx-data [assoc db-id new-list])))})
(def list-view
{:name :list-view
:enter
(fn [context]
(let [db-id (get-in context [:request :path-params :list-id])
the-list (when db-id
(find-list-by-id
(get-in context [:request :database])
db-id))]
(cond-> context
the-list (assoc :result the-list))))})
(def list-item-view
{:name :list-item-view
:leave
(fn [context]
(let [list-id (get-in context [:request :path-params :list-id])
item-id (and list-id
(get-in context [:request :path-params :item-id]))
item (and item-id
(find-list-item-by-ids (get-in context [:request :database]) list-id item-id))]
(cond-> context
item (assoc :result item))))})
(def list-item-create
{:name :list-item-create
:enter
(fn [context]
(if-let [list-id (get-in context [:request :path-params :list-id])]
(let [nm (get-in context [:request :query-params :name] "Unnamed Item")
new-item (make-list-item nm)
item-id (str (gensym "i"))]
(-> context
(assoc :tx-data [list-item-add list-id item-id new-item])
(assoc-in [:request :path-params :item-id] item-id)))
context))})
(def routes
(route/expand-routes
#{["/todo" :post [db-interceptor list-create]]
["/todo" :get echo :route-name :list-query-form]
["/todo/:list-id" :get [entity-render db-interceptor list-view]]
["/todo/:list-id" :post [entity-render list-item-view db-interceptor list-item-create]]
["/todo/:list-id/:item-id" :get [entity-render list-item-view db-interceptor]]
["/todo/:list-id/:item-id" :put echo :route-name :list-item-update]
["/todo/:list-id/:item-id" :delete echo :route-name :list-item-delete]}))
(def service-map
{::http/routes routes
::http/type :jetty
::http/port 8890})
(defn start []
(http/start (http/create-server service-map)))
;; For interactive development
(defonce server (atom nil))
(defn start-dev []
(reset! server
(http/start (http/create-server
(assoc service-map
::http/join? false)))))
(defn stop-dev []
(http/stop @server))
(defn restart []
(stop-dev)
(start-dev))
(defn test-request [verb url]
(io.pedestal.test/response-for (::http/service-fn @server) verb url))
The Path So Far
In this guide, we have built a new API with an in-memory database. You’re ready to go get funding and launch!
We covered some more advanced concepts with interceptors this time:
-
Passing results from one interceptor to another
-
Reusing interceptors in different routes
-
Separating query from transaction with :enter and :leave functions
-
Testing our routes interactively at a REPL.
Where To Next?
Congratulations! You’ve finished the whole getting stated trail.
To go deep on some of these topics, check out any of the following references: