Pedestal

Hello World, With Parameters

Welcome Back

Now that you’ve gone through the basics in the Hello World guide it’s time to go one level deeper. We’re going to add two features. That will be our vehicle to talk about some important concepts in Pedestal.

What You Will Learn

After reading this guide, you will be able to:

  • Accept query parameters.

  • Use the request map.

  • Apply logic in your handler.

  • Conditionally return an error.

  • Return HTML in your response.

Guide Assumptions

Like Hello World, this guide is for beginners who are new to Pedestal and may be new to Clojure. It doesn’t assume any prior experience with a Clojure-based web framework. You should be familiar with the basics of HTTP: URLs, response codes, and content types.

If you’ve already done some of those other things, you might want to skip ahead to Your First API to start building some logic and multiple routes.

If you like to jump straight in to the deep end, you might be interested in the Pedestal Crash Course which assumes you know quite a bit about Clojure and web frameworks.

This guide also assumes that you are in a Unix-like development environment, with Java installed. We’ve tested it on Mac OS X and Linux (any flavor) with great results. We haven’t yet tried it on the Windows Subsystem for Linux, but would love to hear from you if you’ve succeeded with it there.

Getting Help If You’re Stuck

We’ll take this in small steps. If you get stuck at any point in this guide, please submit an issue about this guide or hop over to the mailing list and raise your hand there.

Where We Are Going

In this guide, we will build on the hello.clj example from last time. We will enhance it to take a single parameter in the URL and return a friendly greeting to that person.

There are some people we don’t want to meet, though. So we’ll also include a list of names that should not be said. Then we can make our handler function a bit smarter. If any forbidden name is uttered, our handler will return a 404 response instead of a cordial greeting.

Before We Begin

If you worked through Hello World, then you already have all the files you need. If not, take a moment to grab the sources from that guide. That will be our starting point for enhancements this time.

Accepting Parameters

HTTP offers a few different ways for the client to send data up to our service. The most basic is probably the URL query parameter. We usually see these on GET requests, but nothing stops a client from using them on other request types.

Example 1. URL with Query Parameter

Because query parameters are so common, Pedestal handles them automatically. You can try that URL with your service right now.

As a reminder, we’re running the server by using the clj tool to start an interactive session (called a REPL, rhymes with ripple) then starting the service in that session. Before we do that, though, let’s discuss interactive development.

Interactive Development

It’s really not optimal to be restarting your REPL in order to restart the service every time you need to make a change. In real Clojure development, we rarely restart the REPL. Instead, we make our system friendly to interactive development. The main problem so far is that starting the HTTP service doesn’t return. We can fix that by adding a key to the map that says "return instead of waiting for the server to stop."

src/hello.clj
(def service-map
  {::http/routes routes
   ::http/type   :jetty
   ::http/port   8890})

(defn start []
  (http/start (http/create-server service-map)))

                                                                                        ;; For interactive development
(defonce server (atom nil))                                                             (1)

(defn start-dev []
  (reset! server                                                                        (2)
          (http/start (http/create-server
                       (assoc service-map
                              ::http/join? false)))))                                   (3)

(defn stop-dev []
  (http/stop @server))

(defn restart []                                                                        (4)
  (stop-dev)
  (start-dev))
1 defonce here means we can recompile this file in the same REPL without overwriting the value in this atom.
2 reset! replaces the current value in the atom with the new value.
3 Tell Pedestal not to wait for the server to exit. You set join? to true when running "for real" so that your main function does exit and terminate the process.
4 This is a quick way to bounce the server.

The key ::http/join? does the trick. We can now run start-dev in our REPL sessions. Instead of waiting forever, the REPL thread now returns, prints the value of the server (an ugly mess!) and lets us continue interacting. This is how we are going to start things in dev from now on. Let’s try it out.

First, let’s start the service.

$ clj
Clojure 1.10.1
user=>
user=> (require 'hello)
nil
user=> (hello/start-dev)
[main] INFO org.eclipse.jetty.util.log - Logging initialized @14152ms to org.eclipse.jetty.util.log.Slf4jLog
[main] INFO org.eclipse.jetty.server.Server - jetty-9.4.18.v20190429; built: 2019-04-29T20:42:08.989Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 1.8.0_222-b10
[main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@7dc155b3{/,null,AVAILABLE}
[main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@71b3c2a7{HTTP/1.1,[http/1.1, h2c]}{localhost:8890}
[main] INFO org.eclipse.jetty.server.Server - Started @14337ms
#:io.pedestal.http{:routes ({:path "/greet", :method :get, :path-re #"/\Qgreet\E", :path-parts ["greet"], :interceptors [#Interceptor{:name }], :route-name :greet, :path-params []}), :server #object[org.eclipse.jetty.server.Server 0x5c8d22de "Server@5c8d22de{STARTED}[9.4.18.v20190429]"], :stop-fn #object[io.pedestal.http.jetty$server$fn__16012 0x17bda3 "io.pedestal.http.jetty$server$fn__16012@17bda3"], :type :jetty, :port 8890, :servlet #object[io.pedestal.http.servlet.FnServlet 0x465b16bb "io.pedestal.http.servlet.FnServlet@465b16bb"], :host "localhost", :join? false, :service-fn #object[io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__15761 0x60df0131 "io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__15761@60df0131"], :interceptors [#Interceptor{:name :io.pedestal.http/log-request} #Interceptor{:name :io.pedestal.http/not-found} #Interceptor{:name :io.pedestal.http.ring-middlewares/content-type-interceptor} #Interceptor{:name :io.pedestal.http.route/query-params} #Interceptor{:name :io.pedestal.http.route/path-params-decoder} #Interceptor{:name :io.pedestal.http.route/method-param} #Interceptor{:name :io.pedestal.http.secure-headers/secure-headers} #Interceptor{:name :io.pedestal.http.route/router}], :start-fn #object[io.pedestal.http.jetty$server$fn__16010 0x43e869ea "io.pedestal.http.jetty$server$fn__16010@43e869ea"]}

Whatever happened to "no news is good news?" Believe it or not, that is all successful output. The big glob at the end is the value of the server. It’s a big map with all the state in it. The most important thing to do with it is hang on to it so we can stop the server later. We do that by putting it in an atom.

Now we can use start-dev, stop-dev, and restart as we make changes.

Moving along

From another window, let’s send a request with a query parameter.

$ curl -i http://localhost:8890/greet\?name=Michael
HTTP/1.1 200 OK
Date: Fri, 20 Sep 2019 15:51:27 GMT
Strict-Transport-Security: max-age=31536000; includeSubdomains
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Content-Security-Policy: object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;
Content-Type: text/plain
Transfer-Encoding: chunked

Hello, world!%

Even though we added a query parameter to our request, Pedestal still found the right route and invoked our respond-hello function. So where did the parameter go? To answer that we have to talk about the mysterious request argument to the handler.

The Request Map

Here’s a reminder of what respond-hello looks like:

(defn respond-hello [request]          (1)
  {:status 200 :body "Hello, world!"}) (2)
1 The function takes one argument, which we call request
2 And it returns a map.

We aren’t using the request argument right now, so we don’t really know what it is. (Well, it’s probably not too hard to guess that it has to do with the HTTP request that Pedestal received. From the title of this section, you probably also deduced that it’s a map!)

Let’s take a look at that request by echoing it back to the client. This is a common debugging trick that you can use.

(defn respond-hello [request]
  {:status 200 :body request})                           (1)
1 Instead of returning the string "Hello, world!" we return the entire request as the response body.

Restart your service with that new definition and use curl. You’ll see something like this:

$ curl -i http://localhost:8890/greet\?name=Michael
HTTP/1.1 200 OK
Date: Fri, 20 Sep 2019 16:04:03 GMT
Strict-Transport-Security: max-age=31536000; includeSubdomains
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Content-Security-Policy: object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;
Content-Type: application/edn
Transfer-Encoding: chunked

{:protocol "HTTP/1.1", :async-supported? true, :remote-addr "127.0.0.1", :params {:name "Michael"}, :servlet-response #object[org.eclipse.jetty.server.Response 0x619acc59 "HTTP/1.1 200 \nDate: Fri, 20 Sep 2019 16:04:03 GMT\r\nStrict-Transport-Security: max-age=31536000; includeSubdomains\r\nX-Frame-Options: DENY\r\nX-Content-Type-Options: nosniff\r\nX-XSS-Protection: 1; mode=block\r\nX-Download-Options: noopen\r\nX-Permitted-Cross-Domain-Policies: none\r\nContent-Security-Policy: object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;\r\nContent-Type: application/edn\r\n\r\n"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x6253df71 "io.pedestal.http.servlet.FnServlet@6253df71"], :headers {"user-agent" "curl/7.54.0", "accept" "*/*", "host" "localhost:8890"}, :server-port 8890, :servlet-request #object[org.eclipse.jetty.server.Request 0x70be4d24 "Request(GET //localhost:8890/greet?name=Michael)@70be4d24"], :query-params {:name "Michael"}, :path-info "/greet", :url-for #object[clojure.lang.Delay 0x13e17f7f {:status :pending, :val nil}], :uri "/greet", :server-name "localhost", :query-string "name=Michael", :path-params [], :body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x273006ca "HttpInputOverHTTP@273006ca[c=0,q=0,[0]=null,s=STREAM]"], :scheme :http, :request-method :get, :context-path ""}

Yikes!

We need to unpack that a bit to make sense out of it. It’s a Clojure map, just printed all in one long line. I’ll reformat it with some line breaks. Syntax highlighting will help us read it too. Feel free to paste this map into your favorite text editor with Clojure syntax highlighting to pick it apart yourself.

I’m also going to omit some of the keys. They will be relevant later, but they’re not interesting now. And anyway, they don’t print very readably because they’re objects rather than data.

Here’s what we are left with.

{
 :protocol         "HTTP/1.1",
 :async-supported? true,
 :remote-addr      "127.0.0.1",
 :path-info        "/greet",
 :uri              "/greet",
 :query-string     "name=Michael",
 :query-params     {:name "Michael"},
 :params           {:name "Michael"},
 :headers          {"user-agent" "curl/7.47.0", "accept" "*/*", "host" "localhost:8890"},
 :server-port      8890,
 :server-name      "localhost",
 :path-params      [],
 :scheme           :http,
 :request-method   :get
}

This is a request map. It really is just a Clojure map. You access its values with get and get-in. All the gory details about request maps are in the reference section. For now, we’ll look at three interesting keys.

The whole query string appears under the :query-string key. That’s the portion of the URL after, but not including, the '?' character. The query string can be a pain to deal with. It is URL-encoded and may have multiple parameters with delimiters. Pedestal slices that up into a map of parameters for you, which is attached to the :query-params key. Each parameter name gets turned into a keyword. Parameter values are always strings, though, so you may still have to parse them a bit.

You see that :params duplicates the same map as :query-params. Pedestal merges all the different kinds of parameters into one map for convenience. Most of the time, I know when I want a query parameter so I just use :query-params, though.

Using the Parameter

Inside our handler function, the :name parameter is available inside a nested map. Let’s dig it out and add it to our response.

src/hello.clj
(defn respond-hello [request]
  (let [nm (get-in request [:query-params :name])]       (1)
    {:status 200 :body (str "Hello, " nm "\n")}))        (2)
1 Extract the parameter, bind it to the name nm.
2 For the response body, concatenate "Hello," and the parameter into a string.

Here we can just grab the value right out of the map and use it in the response.

(You might notice that I’m using nm instead of spelling out name. That’s because I’ve been bitten many times in the past by a subtle issue. There is a core function called name. If I ever change my mind about what to call the parameter, I might end up using greeter in one place but forget and leave name in another place. In that case, I’d be returning a stringified respresentation of the clojure.core/name function instead of my parameter value! I’ve done this often enough that I just avoid shadowing core library symbols as a policy.)

Restart your service and use curl to try a few different requests with parameters.

$ curl http://localhost:8890/greet\?name=Michael
Hello, Michael
$ curl http://localhost:8890/greet\?name=Pankaj
Hello, Pankaj
$ curl http://localhost:8890/greet\?name=Geeta
Hello, Geeta
$ curl http://localhost:8890/greet\?name=ไอรีนซีพีชาไอที
Hello, ไอรีนซีพีชาไอที
$ curl http://localhost:8890/greet\?name=No%20One%20To%20Be%20Trifled%20With
Hello, No One To Be Trifled With
$ curl http://localhost:8890/greet\?name=
Hello,
$ curl http://localhost:8890/greet
Hello,

Dang. We were doing really well right up until the end there. Obviously we need to do something smarter when the input is missing. Bad user input is a way of life on the web, so let’s not make an error out of this. Instead, we just won’t personalize the greeting.

src/hello.clj
(defn respond-hello [request]
  (let [nm   (get-in request [:query-params :name])
        resp (if (empty? nm)                             (1)
               "Hello, world!\n"                         (2)
               (str "Hello, " nm "\n"))]                 (3)
    {:status 200 :body resp}))                           (4)
1 Both nil and a zero-length string count as empty.
2 This is our fallback response.
3 And our personalized greeting.
4 Generating the response.

Let’s try that again:

$ curl http://localhost:8890/greet
Hello, world!
$ curl http://localhost:8890/greet\?name=
Hello, world!
$ curl http://localhost:8890/greet\?name=ไอรีนซีพีชาไอที
Hello, ไอรีนซีพีชาไอที

Much better! But our handler function is getting a bit unwieldy. I don’t mean that it is a whole six lines long, but rather that it mixes too many concerns. This function parses inputs, applies "domain" logic, and deals with creating a response body. It would be better to refactor this to separate those concerns.

src/hello.clj
(defn ok [body]                                          (1)
  {:status 200 :body body})

(defn greeting-for [nm]                                  (2)
  (if (empty? nm)
    "Hello, world!\n"
    (str "Hello, " nm "\n")))

(defn respond-hello [request]                            (3)
  (let [nm   (get-in request [:query-params :name])
        resp (greeting-for nm)]
    (ok resp)))
1 This function can generate any "OK" result.
2 Our logic is now separated.
3 The handler now coordinates the rest.

This version separates the concerns nicely. One big benefit is that the greeting-for function is now easier to test. It’s just a pure function that doesn’t require any HTTP muckery to exercise. Likewise, that ok function is quite easy to test. Both functions are likely to get more complex, but each one will only deal with it’s own complexity.

In this example, we’re treating the case of an empty name the same as if the name parameter just isn’t included. See if you can make the handler more strict so it will return a "400 Bad Request" response if the name parameter is present but zero length.

Conditional Responses

There are some names that should not be spoken. Let’s enhance our service to avoid them. This will be a relatively simple change. We can hardcode the names as a set, then use that set as a function. (Clojure sets are functions that test for the presence of their argument in the set itself.)

This part needs to go before greeting-for because we’ll use it to make a decision in that function.

src/hello.clj
(def unmentionables #{"YHWH" "Voldemort" "Mxyzptlk" "Rumplestiltskin" "曹操"})

Now we need to think about our greeting function. Right now it returns a string. We want to make sure the user gets a 404 if the query names one of the forbidden ones. We could have greeting-for directly return the response map, but that starts to entangle the concerns we just factored out. We’ll do the simplest thing that could possibly work: is greeting-for returns nil, that means the request failed.

src/hello.clj
(defn ok [body]
  {:status 200 :body body})

(defn not-found []
  {:status 404 :body "Not found\n"})

(defn greeting-for [nm]
  (cond
    (unmentionables nm) nil
    (empty? nm)         "Hello, world!\n"
    :else               (str "Hello, " nm "\n")))

(defn respond-hello [request]
  (let [nm   (get-in request [:query-params :name])
        resp (greeting-for nm)]
    (if resp
      (ok resp)
      (not-found))))

Because there are now three possible conditions that can happen in respond-hello, I’ve changed from an if to a cond expression. cond uses pairs of "condition / value" clauses. So if (unmentionables nm) returns any truthy value, then cond returns the nil and stops evaluating. If (empty? nm) returns true, then cond returns the literal string "Hello, world!\n" and nothing else. If neither of those happened, then cond sees the keyword :else as a condition. There’s actually nothing special about :else…​ any truthy value will do. :else is just a convention. Then we build the string the same way as before.

A nested if would also have worked here. In fact cond is a macro that expands into a series of nested if expressions. All three of these conditions are kind of parallel in significance, so I like that they appear to be parallel in the structure of the code. The shape of the code mirrors the shape of my thinking about these cases.

The second thing you’ll notice is that the handler now makes a decision whether to call our ok helper or the new function not-found. I do hope you weren’t expecting much in not-found! It’s as simple as it gets.

Restart your service and give it a try:

$ curl http://localhost:8890/greet\?name=Michael
Hello, Michael
$ curl http://localhost:8890/greet\?name=Rumplestiltskin
Not found
$ curl http://localhost:8890/greet\?name=曹操
Not found
$ curl http://localhost:8890/greet\?name=voldemort
Hello, voldemort
$ curl http://localhost:8890/greet\?name=He%20who%20must%20not%20be%20named
Hello, He who must not be named

Looks like it mostly works, though it has some trouble with different capitalization. Take a look at the docs for clojure.string and see if you can figure out how to make the comparison case-insensitive.

The Whole Shebang

Once again, we built this thing in small steps, so it may seem like there was a lot to deal with. The final product is still pretty small though. Here’s all the final code.

src/hello.clj
(ns hello                                                ;; <1>
  (:require [io.pedestal.http :as http]                  ;; <2>
            [io.pedestal.http.route :as route]))         ;; <3>

(def unmentionables #{"YHWH" "Voldemort" "Mxyzptlk" "Rumplestiltskin" "曹操"})

(defn ok [body]
  {:status 200 :body body})

(defn not-found []
  {:status 404 :body "Not found\n"})

(defn greeting-for [nm]
  (cond
    (unmentionables nm) nil
    (empty? nm)         "Hello, world!\n"
    :else               (str "Hello, " nm "\n")))

(defn respond-hello [request]
  (let [nm   (get-in request [:query-params :name])
        resp (greeting-for nm)]
    (if resp
      (ok resp)
      (not-found))))

(def routes
  (route/expand-routes                                   ;; <1>
   #{["/greet" :get respond-hello :route-name :greet]})) ;; <2>

(def service-map
  {::http/routes routes
   ::http/type   :jetty
   ::http/port   8890})

(defn start []
  (http/start (http/create-server service-map)))

                                                                                        ;; For interactive development
(defonce server (atom nil))                                                             ;; <1>

(defn start-dev []
  (reset! server                                                                        ;; <2>
          (http/start (http/create-server
                       (assoc service-map
                              ::http/join? false)))))                                   ;; <3>

(defn stop-dev []
  (http/stop @server))

(defn restart []                                                                        ;; <4>
  (stop-dev)
  (start-dev))
deps.edn
{:deps
 {io.pedestal/pedestal.service {:mvn/version "0.5.7"}
  io.pedestal/pedestal.route   {:mvn/version "0.5.7"}
  io.pedestal/pedestal.jetty   {:mvn/version "0.5.7"}
  org.slf4j/slf4j-simple       {:mvn/version "1.7.28"}}
 :paths ["src"]}

The Path So Far

In this guide, we built upon Hello World to add:

  • Query parameters

  • General response functions

  • "Domain" logic

  • Conditional responses

We also learned about the request map and how to deal with bad user input.

Where to Next?

So far, almost all our responses have been plain text. (When we returned the request map in the body, the content type was actually application/edn but we can ignore that for now.)

Plain text is ugly, so we want to be able to return HTML. Rich clients don’t like HTML so much, so we also want to return JSON sometimes. The next stop on this trail looks at how Pedestal handles content types and response bodies. This will be our first taste of interceptors.