February 11, 2009

Clojure presentation at GatorJUG

This evening, I delivered a Clojure presentation for GatorJUG, a Java user group.

Thanks to Howard M. Lewis Ship for allowing me to learn from his presentation slides as I was preparing.

Edit: I delivered the presentation again, with small changes, a week later at GatorLUG, a Linux user group. The presentation included corrections (especially Java code formatting) and some additional discussion about Lisp. The improved slides are available in PDF and Open Office presentation format.



I donated $100 so that Rich Hickey can continue working full-time to make Clojure even better. You should read more here and click the “donate” button at the bottom of that page.

January 4, 2009

Compojure security: authentication and authorization

I am writing a Compojure application that will host my journal on EricLavigne.net. In a previous article, I showed how to use PostgreSQL with Compojure. Now that journal articles are stored in the database, it should be possible to add or update journal articles through the website, but I want to make sure that only I can do that. The next step, then, is to deal with security issues, such as authentication, authorization, and encryption. While we’re on the topic of security, I’ll also discuss automatic updates.

Automatic updates

Keeping your computer up-to-date is an important part of security, and is easily accomplished with automatic updates. The idea is described more thoroughly in this article.

Cron is a program that allows you to schedule commands (such as a system update) to be run routinely. The first step is to run the following command.

crontab -e

After running that command, you will be in file editing mode. Type the following line as the last line of that file, then save the file.

0 1 * * * (aptitude -y update && aptitude -y safe-upgrade) 2>&1 >> /var/log/auto_update.log

The line that you added meant that at 1:00 AM (minute 0 of hour 1) of every day of the month (*) and every month (*) and every day of the week (*) you want to run an update command (aptitude -y update …) and record the results of that command in /var/log/auto_update.log.

Password authentication

In this section, I create a login form with username and password fields. In a typical web application, the username and password for each visitor would be stored in the database. A web application for hosting my journal really only needs one authenticated user, so one hard-coded password is enough. For the purpose of demonstration I will follow a path between these two extremes, allowing users to choose any username but setting one hard-coded password.

There is already a working application described in a previous article, so I will focus here on those things that I changed to add authentication: a login form, controllers for logging in and logging out, URL routing for these new components, and a standard page layout that provides access to the new components.

The login view is a form with two text fields, one for the username and one for the password, and a “Log in” button which sends this information to the login controller. The password text field needs type “password” rather than “text” so that the password is not displayed when typed.

The login view

(defn login-view []
  (html
    [:form {:method "post"}
      "User name: "
        [:input {:name "name", :type "text"}]
        [:br]
      "Password: "
        [:input {:name "password", :type "password"}]
        [:br]
      [:input {:type "submit" :value "Log in"}]]))

The login controller receives the username and password from the login form, checks whether these are valid credentials and, if they are valid, authenticates the user by adding the username to the session. Data entered into the login form is stored in params. The session is a reference to a map and can be used to store arbitrary information about one visitor to the website.

Typically, validating a user’s credentials means checking whether some user record in the database has that combination of username and password. Instead, this login controller only checks that the username contains typical characters (letters, numbers, spaces, underscores, and hyphens) and that the password is “secret”.

(defn login-controller [session params]
  (dosync
    (if
      (and
        (= "secret" (params :password))
        ; Username can include letters, numbers,
        ; spaces, underscores, and hyphens.
        (.matches (params :name) "[\\w\\s\\-]+"))
      (do
        (alter session assoc :name (params :name))
        (redirect-to "/articles/"))
      (redirect-to "/login/"))))

The logout controller is very simple – just remove the user’s name from the session so that they are no longer logged in.

(defn logout-controller [session]
  (dosync
    (alter session assoc :name nil)
    (redirect-to "/articles/")))

The URL routing needs three new lines for the login view, login controller, and logout controller. Note that the difference between login view and login controller comes down to the method: GET or POST.

(defservlet journal-servlet
  "Eric Lavigne's Journal"
  (ANY "/articles/" (view-article-list session))
  (ANY "/articles/:title"
    (view-article session (route :title)))
  (GET "/login/" (login-view))
  (POST "/login/" (login-controller session params))
  (ANY "/logout/" (logout-controller session))
  (ANY "/*" (redirect-to "/articles/")))

Each page should tell a user whether they are logged in. If they aren’t logged in, they should have a link to the login page. If they are logged in, they should have a link to logout. Now that there are components that belong on every page, it is time to create a standard page layout to avoid repetition.

(defn page [session title body]
  (html
    [:html
      [:head [:title title]]
      [:body
        [:h1 title]
        body
        [:p
          (if (@session :name)
            (link-to "/logout/"
              (str "Log out " (@session :name)))
            (link-to "/login/" "Log in"))]]]))

The same page layout is used for the article list and for individual articles, and will probably be used for most new pages on the site as well. Besides reducing the amount of code required to define a view, this will give the site a more uniform look. As an example, this is how the article list looks after applying the new page layout.

New page layout with login linkDividerNew page layout when logged in

(defn view-article-list [session]
  (page session "Articles"
    [:dl (mapcat
           (fn [article]
             (list
               [:dt (render-article-link article)]
               [:dd (article :description)]))
           (fetch-articles))]))

Now that password authentication is implemented, site.clj includes the following code. The next step will be authorization for the admin page.

(ns ericlavigne
  (:use compojure.http)
  (:use compojure.html)
  (:require [compojure.jetty :as jetty])
  (:require [clojure.contrib.sql :as sql]))

(defn article-title-to-url-name [title]
  (.replaceAll (.toLowerCase title) "[^a-z0-9]+" "-"))

(defn article-url [article]
  (str
    "/articles/"
    (article-title-to-url-name
      (article :title))))

; Database

(def db {:classname    "org.postgresql.Driver"

         :subprotocol  "postgresql"
         :subname      "production"
         :user         "postgres"})

(defn sql-query [query]
  (sql/with-connection db
    (sql/with-results res
      query (into [] res))))

(defn fetch-articles []
  (sql-query "select * from article"))

(defn fetch-article [title]
  (first
    (filter
      (fn [art]
        (=
          (article-title-to-url-name title)
          (article-title-to-url-name (art :title))))
      (fetch-articles))))

; Renderers

(defn render-article [article]
  [:div
    [:p [:em (article :description)]]
    (article :body)])

(defn render-article-link [article]
  (link-to
    (article-url article)
    (article :title)))

; Layout

(defn page [session title body]
  (html
    [:html
      [:head [:title title]]
      [:body
        [:h1 title]
        body
        [:p
          (if (@session :name)
            (link-to "/logout/"
              (str "Log out " (@session :name)))
            (link-to "/login/" "Log in"))]]]))

; Views

(defn view-article [session title]
  (try
    (let [article (fetch-article title)]
      (page session (article :title)
        (render-article article)))
    (catch Exception ex
      (redirect-to "/articles/"))))

(defn view-article-list [session]
  (page session "Articles"
    [:dl (mapcat
           (fn [article]
             (list
               [:dt (render-article-link article)]
               [:dd (article :description)]))
           (fetch-articles))]))

(defn login-view []
  (html
    [:form {:method "post"}
      "User name: "
        [:input {:name "name", :type "text"}]
        [:br]
      "Password: "
        [:input {:name "password", :type "password"}]
        [:br]
      [:input {:type "submit" :value "Log in"}]]))

; Controllers

(defn login-controller [session params]
  (dosync
    (if
      (and
        (= "secret" (params :password))
        ; Username can include letters, numbers,
        ; spaces, underscores, and hyphens.
        (.matches (params :name) "[\\w\\s\\-]+"))
      (do
        (alter session assoc :name (params :name))
        (redirect-to "/articles/"))
      (redirect-to "/login/"))))

(defn logout-controller [session]
  (dosync
    (alter session assoc :name nil)
    (redirect-to "/articles/")))

; Routing

(defservlet journal-servlet
  "Eric Lavigne's Journal"
  (ANY "/articles/" (view-article-list session))
  (ANY "/articles/:title"
    (view-article session (route :title)))
  (GET "/login/" (login-view))
  (POST "/login/" (login-controller session params))
  (ANY "/logout/" (logout-controller session))
  (ANY "/*" (redirect-to "/articles/")))

; Starting the service

(jetty/defserver journal-server
  {:port 80}
  "/*" journal-servlet)

(jetty/start journal-server)

Admin page authorization

Now it’s time to create the admin page and ensure that only an authorized user (admin) can access that page.

The admin user should have a link to get to the admin page. We can put that link in the standard page layout so that it’s always available, but only to the admin user.

The admin link appears if you are logged in as admin.

Someone who is not logged in gets a “Log in” link. Someone who is logged in as “admin” gets links to both “Log out admin” the new admin page. Everyone else just gets a link to “Log out <name>”.

(defn page [session title body]
  (html
    [:head [:title title]]
    [:body
      [:h1 title]
      body
      (dosync
        (cond
          (not (@session :name))
            [:p (link-to "/login/" "Log in")]
          (= (@session :name) "admin")
            [:div
              [:p (link-to "/admin/" "Admin page")]
              [:p (link-to "/logout/" "Log out admin")]]
          :else
            [:p (link-to "/logout/"
                  (str "Log out " (@session :name)))]))]))

At this point, only the admin user has a link to the admin page, but that doesn’t stop other users from typing the URL into their address bars. We need to protect the admin page from non-admin users. This is most easily done by checking, in the admin view, whether the user is logged in as admin. The problem with that is that later there will be more than one page that is restricted to admin, so that code doesn’t belong in any individual view.

Instead, we can use a controller that checks whether the user is logged in as admin. If the user is logged in as admin, the ensure-admin controller returns the :next keyword to let control pass to another controller or view (such as the admin view). Otherwise, the user is redirected back to the main page.

(defn ensure-admin-controller [session]
  (dosync
    (if (and (@session :name) (= (@session :name) "admin"))
      :next
      (redirect-to "/login/"))))

The new URL routing shows how the ensure-admin controller is used. The ensure-admin controller appears near the end of the URL routing and matches any URL. This means that any request that doesn’t match before that point will be sent to the ensure-admin controller. Non-admin users are redirected at that point, so any URL routing that comes after the ensure-admin controller, such as the admin view, will only be accessible to the admin user.

(defservlet journal-servlet
  "URL routing for Eric Lavigne's Journal"
  (ANY "/articles/" (view-article-list session))
  (ANY "/articles/:title"
    (view-article session (route :title)))
  (GET "/login/" (login-view session))
  (POST "/login/" (login-controller session params))
  (ANY "/logout/" (logout-controller session))
  (ANY "/*" (ensure-admin-controller session))
  (ANY "/admin/" (admin-view session))
  (ANY "/*" (go-home)))

Finally, we need to define the admin view. It can be very simple for now, as the focus is on whether we can protect this view from unauthorized users.

Only the admin user can visit the admin page.

(defn admin-view [session]
  (page session "Admin"
    [:p "Only admin can see this page."]))

Now that password authentication is implemented, site.clj includes the following code. [skip past code]

(ns ericlavigne
  (:use compojure.http)
  (:use compojure.html)
  (:require [compojure.jetty :as jetty])
  (:require [clojure.contrib.sql :as sql]))

(defn article-title-to-url-name [title]
  (.replaceAll (.toLowerCase title) "[^a-z0-9]+" "-"))

(defn article-url [article]
  (str
    "/articles/"
    (article-title-to-url-name
      (article :title))))

(defn go-home []
  (redirect-to "/articles/"))

; Database

(def db {:classname    "org.postgresql.Driver"
         :subprotocol  "postgresql"
         :subname      "production"
         :user         "postgres"})

(defn sql-query [query]
  (sql/with-connection db
    (sql/with-results res
      query (into [] res))))

(defn fetch-articles []
  (sql-query "select * from article"))

(defn fetch-article [title]
  (first
    (filter
      (fn [art]
        (=
          (article-title-to-url-name title)
          (article-title-to-url-name (art :title))))
      (fetch-articles))))

; Renderers

(defn render-article [article]
  [:div

    [:p [:em (article :description)]]
    (article :body)])

(defn render-article-link [article]
  (link-to
    (article-url article)
    (article :title)))

; Layout

(defn page [session title body]
  (html
    [:head [:title title]]
    [:body
      [:h1 title]
      body
      (dosync
        (cond
          (not (@session :name))
            [:p (link-to "/login/" "Log in")]
          (= (@session :name) "admin")
            [:div
              [:p (link-to "/admin/" "Admin page")]
              [:p (link-to "/logout/" "Log out admin")]]
          :else
            [:p (link-to "/logout/"
                  (str "Log out " (@session :name)))]))]))

; Views

(defn view-article [session title]
  (try
    (let [article (fetch-article title)]
      (page session (article :title)
        (render-article article)))
    (catch Exception ex
      (go-home))))

(defn view-article-list [session]
  (page session "Articles"
    [:dl (mapcat
           (fn [article]
             (list
               [:dt (render-article-link article)]
               [:dd (article :description)]))
           (fetch-articles))]))

(defn login-view [session]
  (page session "Log in"
    [:form {:method "post"}
      "User name: "
        [:input {:name "name", :type "text"}]
        [:br]
      "Password: "
        [:input {:name "password", :type "password"}]
        [:br]
      [:input {:type "submit" :value "Log in"}]]))

(defn admin-view [session]
  (page session "Admin"
    [:p "Only admin can see this page."]))

; Controllers

(defn login-controller [session params]
  (dosync
    (if
      (and
        (= "secret" (params :password))
        ; Username can include letters, numbers,
        ; spaces, underscores, and hyphens.
        (.matches (params :name) "[\\w\\s\\-]+"))
      (do
        (alter session assoc :name (params :name))
        (go-home))
      (redirect-to "/login/"))))

(defn logout-controller [session]
  (dosync
    (alter session assoc :name nil)
    (go-home)))

(defn ensure-admin-controller [session]
  (dosync
    (if (and (@session :name) (= (@session :name) "admin"))
      :next
      (go-home))))

; Routing

(defservlet journal-servlet
  "URL routing for Eric Lavigne's Journal"
  (ANY "/articles/" (view-article-list session))
  (ANY "/articles/:title"
    (view-article session (route :title)))
  (GET "/login/" (login-view session))
  (POST "/login/" (login-controller session params))
  (ANY "/logout/" (logout-controller session))
  (ANY "/*" (ensure-admin-controller session))
  (ANY "/admin/" (admin-view session))
  (ANY "/*" (go-home)))

; Starting the service

(jetty/defserver journal-server
  {:port 8080}
  "/*" journal-servlet)

(jetty/start journal-server)

Next steps

I had originally intended to cover encryption with HTTPS. However, this turned out to be a rather difficult topic. Here are some of the resources I found on using HTTPS with Jetty – I may return to this issue for a later article.

Now that the admin page is password protected, it’s time to use that page for site administration, such as adding and editing articles. Adding and editing articles will be the topic of another article.



I donated $100 so that Rich Hickey can continue working full-time to make Clojure even better. You should read more here and click the “donate” button at the bottom of that page.

December 28, 2008

Using PostgreSQL with Compojure

I am writing a Compojure application that will host my journal on EricLavigne.net. In a previous article, I showed how to setup Compojure to run on a Slicehost VPS. The next step is to setup a database to store the journal articles. In this article I will show how to setup a PostgreSQL database, store some example articles in that database, and display those articles in a Compojure application.

Setting up PostgreSQL

The first step is to setup PostgreSQL. This includes installing the postgresql package, starting the database server, and creating a database. The database administration commands used below are in /usr/lib/postgresql/8.3/bin.

apt-get install postgresql

sudo -u postgres initdb -D /var/lib/postgresql/data
sudo -u postgres mkdir /var/lib/postgresql/log
sudo -u postgres pg_ctl -D /var/lib/postgresql/data   \
     -l /var/lib/postgresql/log/postgres.log start
sudo -u postgres createdb -O postgres production

Adding articles to the database

We need some example articles for testing whether the application is working correctly.

First we login to PostgreSQL.

psql -U postgres production

Next, from the PostgreSQL prompt, we create a new table for storing articles.

create table article (
     id           serial primary key,
     title        text not null unique check (trim(title)  ''),
     description  text not null default '',
     body         text not null default '',
     created      timestamp not null default now(),
     updated      timestamp not null default now(),
     published    timestamp null default null
);

Next, create two example articles.

insert into article (title, description, body) values
     ('Article 1', 'My first article',
      '<p>Paragraph 1 in article 1</p><p>Another paragraph</p>'),
     ('Article 2', 'My second article',
      '<p>This article also has a paragraph</p><p>And another</p>');

We’re finished with the PostgreSQL prompt, so we quit the prompt.

\q

Accessing PostgreSQL from Clojure

In order to access PostgreSQL from Clojure, we need a new library.

cd /root/install
wget http://jdbc.postgresql.org/download/postgresql-8.3-604.jdbc4.jar

In order for that library to be accessible to our application, it needs to be added to the classpath in /root/site/run. The new library can be added as the second-to-last line as shown below.

java -cp \
/root/install/compojure/compojure.jar:\
/root/install/compojure/deps:\
/root/install/compojure/deps/clojure.jar:\
/root/install/compojure/deps/clojure-contrib.jar:\
/root/install/compojure/deps/jetty-6.1.14.jar:\
/root/install/compojure/deps/jetty-util-6.1.14.jar:\
/root/install/compojure/deps/servlet-api-2.5-6.1.14.jar:\
/root/install/postgresql-8.3-604.jdbc4.jar:\
. clojure.lang.Repl  site.clj

Finally, the interesting part. I’ll show the full contents of the new /root/site/site.clj and explain the new code later.

; Indicates which libraries we need, and the
; shorter names by which they will be called
(ns site
  (:require [compojure.http.servlet :as servlet])
  (:require [compojure.http.routes :as routes])
  (:require [compojure.http.helpers :as http-helpers])
  (:require [compojure.html :as html])
  (:require [compojure.html.page-helpers :as page-helpers])
  (:require [compojure.server.jetty :as jetty])
  (:require [clojure.contrib.sql :as sql]))

; Information about the PostgreSQL database
(def db {:classname    "org.postgresql.Driver"
         :subprotocol  "postgresql"
         :subname      "production"
         :user         "postgres"})

; Performs a database query and returns the results as a list
(defn sql-query [query]
  (sql/with-connection db
    (sql/with-query-results res
      ; query is string that may contain "?"
      ; which are replaced with later elements
      ; in this array
      [query]
      (into [] res))))

; Fetches all articles
(defn articles []
  (sql-query "select * from article"))

; Converts "My Article" to "my-article" for use in URL
(defn article-title-to-url-name [title]
  (.replaceAll
    (.toLowerCase title)
    "[^a-z0-9]+" "-"))

; Converts map of article attributes to a URL
(defn article-url [article]
  (str
    "/articles/"
    (article-title-to-url-name
      (article :title))))

; Fetch an article with the given title
(defn article [title]
  (first
    (filter
      (fn [art]
        (=
          (article-title-to-url-name
            title)
          (article-title-to-url-name
            (art :title))))
      (articles))))

; HTML page for an article
(defn render-article [article]
  (html/html
    [:head [:title (article :title)]]
    [:body
      [:h1 (article :title)]
      [:p [:em (article :description)]]
      (article :body)]))

; Search for article by title, return HTML or redirect
(defn view-article [title]
  (try
    (render-article (article title))
    (catch Exception ex
      (http-helpers/redirect-to "/articles/"))))

; HTML link to an article
(defn render-article-link [article]
  (page-helpers/link-to
    (article-url article)
    (article :title)))

; HTML page that lists all articles
(defn view-article-list []
  (html/html
    [:head [:title "Articles"]]
    [:body
      [:dl (mapcat
             (fn [article]
               (list
                 [:dt (render-article-link
                        article)]
                 [:dd (article
                        :description)]))
             (articles))]]))

; Mapping between URLs and view functions
(servlet/defservlet journal-servlet
  "Eric Lavigne's Journal"
  (routes/ANY "/articles/"
    (view-article-list))
  (routes/ANY "/articles/:title"
    (view-article (route :title)))
  (routes/ANY "/*"
    (http-helpers/redirect-to "/articles/")))

; Server settings
(jetty/defserver journal-server
  {:port 80}
  "/*" journal-servlet)

; Command to start the server
(jetty/start journal-server)

This first section creates a namespace for my code called “site” and indicates which libraries I am using. I indicate that this file requires code from the compojure.http.servlet namespace, which will be referred to as servlet later in the code. The compojure.http.servlet namespace is defined in compojure/http/servlet.clj of the compojure library, just as the site namespace is defined in site.clj of this project. This naming convention comes from Java packages and enhances Java interoperability.

; Indicates which libraries we need, and the
; shorter names by which they will be called
(ns site
  (:require [compojure.http.servlet :as servlet])
  (:require [compojure.http.routes :as routes])
  (:require [compojure.http.helpers :as http-helpers])
  (:require [compojure.html :as html])
  (:require [compojure.html.page-helpers :as page-helpers])
  (:require [compojure.server.jetty :as jetty])
  (:require [clojure.contrib.sql :as sql]))

Next is a description of the PostgreSQL database, in the format expected by the clojure.contrib.sql library.

(def db {:classname    "org.postgresql.Driver"
         :subprotocol  "postgresql"
         :subname      "production"
         :user         "postgres"})

The following code performs a query against the database and returns the results of that query as a list.

; Performs a database query and returns the results as a list
(defn sql-query [query]
  (sql/with-connection db
    (sql/with-query-results res
      ; query is string that may contain "?"
      ; which are replaced with later elements
      ; in this array
      [query]
      (into [] res))))

The articles function returns a list of all articles from the database.

; Fetches all articles
(defn articles []
  (sql-query "select * from article"))

The URL of an article is based on its title, with spaces replaced by dashes and all letters converted to lowercase. For example, “The green snake” would have a URL of “/articles/the-green-snake”. The function article-title-to-url-name would convert “The green snake” to “the-green-snake”, and the function article-url would convert “The green snake” to “/articles/the-green-snake”.

; Converts "My Article" to "my-article" for use in URL
(defn article-title-to-url-name [title]
  (.replaceAll
    (.toLowerCase title)
    "[^a-z0-9]+" "-"))

; Converts map of article attributes to a URL
(defn article-url [article]
  (str
    "/articles/"
    (article-title-to-url-name
      (article :title))))

The article function retrieves an article from the database based on the title of the article. This function is very inefficient because it retrieves all of the articles and checks which of them match the given title. The database is capable of doing this job much more efficiently, so this function will need to be rewritten later.

; Fetch an article with the given title
(defn article [title]
  (first
    (filter
      (fn [art]
        (=
          (article-title-to-url-name
            title)
          (article-title-to-url-name
            (art :title))))
      (articles))))

The render-article function uses the compojure.html library to produce an HTML page for an article. It should be easy to understand if you’ve used HTML before.

An article

; HTML page for an article
(defn render-article [article]
  (html/html
    [:head [:title (article :title)]]
    [:body
      [:h1 (article :title)]
      [:p [:em (article :description)]]
      (article :body)]))

The view-article function tries to produce an HTML page for an article with a certain title, redirecting to the list of articles if an article with that title cannot be found.

; Search for article by title, return HTML or redirect
(defn view-article [title]
  (try
    (render-article (article title))
    (catch Exception ex
      (http-helpers/redirect-to "/articles/"))))

The render-article-link function produces an HTML snippet to represent a link to an article.

; HTML link to an article
(defn render-article-link [article]
  (page-helpers/link-to
    (article-url article)
    (article :title)))

The view-article-list function produces an HTML page that lists all of the articles with link and description for each.

A list of articles

; HTML page that lists all articles
(defn view-article-list []
  (html/html
    [:head [:title "Articles"]]
    [:body
      [:dl (mapcat
             (fn [article]
               (list
                 [:dt (render-article-link
                        article)]
                 [:dd (article
                        :description)]))
             (articles))]]))

The servlet controls how this application responds to different URLs. The application responds to the “/articles/” URL by listing the available articles, responds to the “/articles/:title” URL (where :title is replaced by the title of an article) by showing that article, and redirects all other URLs to “/articles/”.

; Mapping between URLs and view functions
(servlet/defservlet journal-servlet
  "Eric Lavigne's Journal"
  (routes/ANY "/articles/"
    (view-article-list))
  (routes/ANY "/articles/:title"
    (view-article (route :title)))
  (routes/ANY "/*"
    (http-helpers/redirect-to "/articles/")))

; Server settings
(jetty/defserver journal-server
  {:port 80}
  "/*" journal-servlet)

; Command to start the server
(jetty/start journal-server)

Getting a database setup was a big step in the right direction, but we still need a good way to add new articles. This Compojure application will need an admin panel with forms for easily creating or editing articles, as well as password authentication and SSL for security. These will be the topics of future articles as I prepare for transfering my journal from WordPress to EricLavigne.net.



I donated $100 so that Rich Hickey can continue working full-time to make Clojure even better. You should read more here and click the “donate” button at the bottom of that page.

December 18, 2008

Compojure on a Slicehost VPS

I recently decided to transfer my WordPress journal to a new Compojure application on EricLavigne.net. Compojure is a web development library for Clojure, a Lisp that was designed for tight integration with Java. This article describes how I configured a Slicehost virtual private server (VPS) to run a simple Compojure application. This article should be useful for me as a reference, and I hope that it will help others to get started with Compojure as well. The application will be improved in later articles until it is ready to host my journal.

Getting a slice

The first step is creating a slice. I entered my billing information (credit card required) followed by three other pieces of information: slice size, Linux distribution, and slice name. I chose the smallest slice size because it costs only $20 per month – I can increase this later if necessary. I accepted the default Linux distribution, Ubuntu 8.04.1 LTS (hardy), and you will find it easier to follow along if you make the same choice. The slice name is for your use only – it will identify your slice in SliceManager. My slice’s name is ericlavigne.

Connecting to the slice

When you have finished creating a slice, you will be given an IP address and root password for your slice. You will need an SSH connection to configure your slice. On Linux, I was able to create an SSH connection with the following command (173.45.234.188 is my slice’s IP address). If you are using Windows, you can create an SSH connection with Putty. You will be asked for your password – respond with your slice’s root password.

     ssh root@173.45.234.188

Configuring the slice

The next step is to change your root password.

     passwd

I prefer to use Sun’s Java development kit, so I needed to add “multiverse” to the /etc/apt/sources.list configuration file. The list “main restricted universe” appears several times in /etc/apt/sources.list, and we just need to add the word “multiverse” to the end of that list. Sed can perform that replacement for us.

     sed -ibackup 's/universe/universe multiverse /' /etc/apt/sources.list

Then it’s time to install a few tools and configure one of them.

     apt-get install git-core
     apt-get install sun-java6-bin
     apt-get install ant
     apt-get install screen
     echo "startup_message off" >> ~/.screenrc

Installing Compojure

Now for the main tool – we need to install Compojure. You can ignore the warning about tools.jar.

     mkdir install
     cd install
     git clone git://github.com/weavejester/compojure.git
     cd compojure
     ant

Creating a web application

The following commands create a new directory called “site” for the website and create a script called “run” that will run the web application.

     cd ..
     mkdir site
     cd site
     echo -n "java -cp " >> run
     echo -n "/root/install/compojure/compojure.jar:" >> run
     echo -n "/root/install/compojure/deps:" >> run
     echo -n "/root/install/compojure/deps/clojure.jar:" >> run
     echo -n "/root/install/compojure/deps/clojure-contrib.jar:" >> run
     echo -n "/root/install/compojure/deps/jetty-6.1.14.jar:" >> run
     echo -n "/root/install/compojure/deps/jetty-util-6.1.14.jar:" >> run
     echo -n "/root/install/compojure/deps/servlet-api-2.5-6.1.14.jar:" >> run
     echo ". clojure.lang.Repl  site.clj" >> run
     chmod +x run

Finally, it’s time to create and run a simple web application. It doesn’t need to be any more complicated than “hello world” for now – we can improve it later.

The following command creates a new file called “site.clj” for the web application’s source code.

     nano site.clj

After the above command, you should be running a simple text editor called “nano.” Type the following code (or copy and paste). Then hold ctrl while pressing x to exit the editor. Yes, you do want to save, and the default location (/root/site/site.clj) is the right place.

; Indicate which libraries we need, and the
; shorter names by which they will be called.
(ns site
     (:require [compojure.http.servlet :as servlet])
     (:require [compojure.http.routes :as routes])
     (:require [compojure.server.jetty :as jetty]))

; Respond to any request by saying "Hello."
(servlet/defservlet hello-servlet
     "A hello world web application"
     (routes/ANY "/*" "Hello."))

; This server will run on port 80 and use
; the hello-servlet that is defined above.
(jetty/defserver hello-server
     {:port 80}
     "/*" hello-servlet)

; Command to start the server
(jetty/start hello-server)

Now it’s time to test whether everything is working. Type the following command to start the application.

     ./run

You can view your website in a web browser. My slice’s IP address is 173.45.234.188, so I type the following into the address bar of my browser.


http://173.45.234.188

If everything is working, you should now see the word “Hello.” Congratulations!

Keeping the application running

Just one final issue. The way we have it set up so far, the web application will shutdown when you close your terminal. We need this application to run for months at a time so that everyone can read your greeting. The solution is screen. I recommend reading more about screen in this screen introduction, but for now just follow along as I show how screen’s detach feature will allow your application to continue running when you log off, and how its reattach feature will let you continue using your application’s Clojure prompt when you return.

Hold ctrl and press d to shutdown your web application. Then start your screen session with the following command.

     screen

Now start the application again. It’s the same command, but now screen will keep the application running even if you detach.

     ./run

In order to detach from screen, you can hold down ctrl, press a, release ctrl, and press d. The d is short for detach, and ctrl-a is used for almost all screen commands. While you are detached, the application is still running (try reloading your browser to test this). If you close your terminal while you are using screen, it automatically detaches, so your program keeps running.

Try reattaching to screen with the following command.

     screen -r

Your prompt is back, just like you left it. You will need to do this every time you log into your virtual private server.

This simple web application is just the beginning. Expect more articles on Compojure as I build a web application to host my WordPress journal.



I donated $100 so that Rich Hickey can continue working full-time to make Clojure even better. You should read more here and click the “donate” button at the bottom of that page.

December 16, 2008

Winning a helicopter

I won a helicopter at a gift-stealing game at work today.

helicopter on a table

Rules

Everyone brings a wrapped gift for the gift pile. The gift must cost less than $25 – mine was a board game called Cranium. A round begins by randomly selecting a player who has not yet been selected. That player can either choose a wrapped gift, opening it for everyone to see, or steal someone else’s gift. If a player’s gift is stolen, it is that player’s turn to take a gift from the pile or steal another gift. Each gift can be stolen up to three times. A player can not steal a gift that has already been stolen from that player in the same round. A round is over when a player chooses to open a new gift rather than stealing, and the game is over when all gifts have been opened.

Playing

I started the first round, having drawn the number 1 from a hat, and selected a gift from the pile. It was Helicopter Storm, a remote-controlled helicopter. I had never flown one of these before – it looked like a lot of fun.

helicopter in the box

Scott stole my helicopter a few rounds later. He said that he would enjoy that helicopter for a good ten minutes, right up until one of his cats caught up with it. Those, he laughed, would be very good minutes.

There were a lot of nice gifts, but only two fun gifts: Cranium and the helicopter. Walking out with my own gift, Cranium, wouldn’t have felt right. I wanted that helicopter. I needed a gift that would be stolen from me, so that I would have another shot at the helicopter. Wine was a popular gift. I stole a bottle of Amarula Fruit Cream from Pam.

bottle of amarula fruit cream

It was not until the final round that someone stole my wine. I tried to retrieve the helicopter – Scott blocked me. He could only do that once, so I just had to stay in the game a bit longer. All of the wines had been stolen two or three times, so those wouldn’t work. I have little interest in coffee, but that was also a popular gift. I stole some coffee mugs and coffee from Clint, my boss.

My coffee was stolen quickly, and Scott was unable to block me again. The helicopter was mine!

Now I just need to learn how to fly it.

helicopter on a table

February 9, 2007

Pineapple jerk chicken

  • Ingredients:
    • 3 lbs boneless, skinless chicken breast
    • 8 oz KC Masterpiece spiced Caribean jerk marinade
    • 1/4 cup flour
    • 40 oz crushed, unsweetened pineapple
  • Directions:
    1. Cut chicken into half-inch chunks.
    2. Combine the chicken and marinade and let sit for at least thirty minutes, preferably overnight.
    3. Fry the chicken-marinade mixture until the chicken is fully cooked.
    4. Stir in flour, followed by pineapple.
    5. Continue cooking until the mixture is hot again.
    6. Serve hot.

May be used as a side dish or topping. Good with steamed rice.

January 14, 2007

Fortress: a scientific programming language with implicit concurrency

Sun Microsystems released a preliminary version of its new Fortress scientific programming language this Tuesday.

Fortress has a variety of features specifically designed to support scientific programming. Fortress equations are designed to look like mathematical equations (as in Mathematica) so that computer codes will be readable for other scientists. Fortress allows numbers to have physical units (meters, seconds, ergs, keV) which enhances readability and also allows the compiler to check for unit consistency (a common way for scientists and engineers to check their handwritten calculations). Fortress has compile-time safety checks similar to those found in OCaml to make programs less error prone. Fortress comes with a test framework (unit tests and assertions) to facilitate the debugging process.

Fortress’s most interesting feature is implicit concurrency, which allows Fortress programs to easily take advantage of speed boosts from multi-core architectures and computing clusters. A Fortress program can run on a computing cluster without any parallel-processing related instructions because many Fortress commands are parallel by default. For loops, the equivalent of DO loops in Fortran, are automatically split across multiple processors.

Fortress is still at a very early stage of development, but I have a feeling that it will be an excellent tool for speed-critical, scientific programming (such as nuclear radiation simulation) a few years down the road. Of course, I don’t have the patience to wait that long – Fortress is already running on one of my computers, and I was among the first fifty programmers to sign on as “observers” of the project :-)

January 7, 2007

F# is my new favorite programming language

Yesterday I discovered F# (pronounced F Sharp), a new language developed by Don Syme. F# is heavily based on OCaml, which was already my favorite high-performance language, and also provides easy access to Microsoft’s .Net Framework, an incredible set of libraries which make F# useful for a much wider variety of applications than OCaml.

OCaml is a very flexible and concise language. OCaml supports rapid program development with interactive evaluation (Matlab style development), a resistance to crashing, and functional programming constructs, yet it also produces programs that are about 90% as fast as highly optimized C++ code.

Don Syme started with the already amazing OCaml programming language and took it forward by leaps and bounds. He cleaned up the syntax a bit. He added better support for threading, an important enhancement as personal computers start to increase the number of cores rather than the speed per core. More important than these improvements to the core language, from my perspective, is access to Microsoft’s .NET platform. In addition to a wealth of libraries, this means that F# (OCaml + .NET) can interface with C# code, take advantage of the Visual Studio IDE, and run on any .NET supported platform (Windows, Mac, Linux, XBox, and even microcontrollers).

As a nice fringe benefit, I’m noticing that the want ads are full of C# jobs. Now that OCaml has joined the .NET family, an F# job could be just around the corner.

December 20, 2006

Online postage adventure

The US Postal Service had a great idea: selling mailing labels online, postage included, and letting customers print the labels from the comfort of their homes. As much as I love the idea, I ran into a lot of problems when actually using the service.

This Monday was the first time that I created a mailing label online. I entered my shipping information four times before I was finally able to print out a mailing label, at which point I paid more than double the usual price for postage. Happy customer? Not really…

The first time I filled out the form, I ran into a question about weight, with separate fields for pounds and ounces. This format implied that I needed an answer accurate to within about an ounce (no, I didn’t have a postal scale lying around). I determined that my cell phone was 2.7 ounces and, after much fiddling with strings and coins, determined that my package weighed a bit less than the 2.7 ounce cell phone. I rounded up a bit and said that my package was 3 ounces. Wouldn’t it have been nice to know that any package under one pound would be the same price?

After providing the weight and a variety of other information such as sender address, recipient address, and how much insurance I wanted (with no indication of the cost), I moved to the next page. This was my first indication that printing my own label would actually cost me extra money: Media Mail was not on the list of service options. I chose the cheapest option, Priority Mail, agreeing to pay more than double the usual cost of shipping a CD. Perhaps I was lured in by the thought that I was almost done, how far from the truth…

The next page asked for my username and password. This disgusted me even more than being asked to pay double. Of course USPS needs me to have a permanent account before they can sell me some postage. That makes about as much sense as carrying around a Kash n’ Karry savings card just to buy items at the marked price.

I went through the process of creating an account, capitalizing the first letter of my password and changing a punctuation mark into a number so this silly site would consider my password secure… and then it wants me to create security questions. If I forget the bizarre password that they made me create, I can still access my account just by telling them my mother’s maiden name? Don’t they know that such information is publicly available? I type random keys after each security question so no one can use this backdoor into my new and totally unnecessary account, then I move on to the next page, expecting to enter my credit card information… only to find myself back on page 1. USPS forgot everything I had told it about my package!

After the second time entering all that information, I had to stop partway through to take care of something. Apparently leaving the keyboard for a couple minutes is enough for my session to time out – back to the beginning again.

On the third time entering all that information, I actually got to the point of creating a PDF of my mailing label… and Acrobat Reader froze up. After so many months of good performance by Acrobat, and after so many problems with the USPS site, you can bet I won’t blame Acrobat when one more thing goes wrong!

And on the fourth time it just worked – paying double the cost of Media Mail, of course.

Strangely enough, I was not discouraged by this experience. Rather, I was determined to master the art of online postage, saving myself countless trips to the post office. The adventure continued! A couple days later, I needed to mail a PHP/MySQL book that I had sold on Amazon Marketplace. Trying USPS’s online service, Click-N-Ship, wasn’t even a consideration, but I knew that there were several 3rd party vendors for the same service. Surely one of them would offer better service. The four options were eBay, stamps.com, endicia, and Pitney Bowes.

eBay was the only one of the four vendors that did not require monthly fees (I am wary of free trials, especially risk-free trials with lots of incentive gifts). eBay also doesn’t require any software installation (viruses, registry corruption, security vulnerabilities…) Unfortunately, eBay also requires that payments be made via PayPal, which would have taken too long to arrange. None of the other vendors really stood out, so I took the next on the list: stamps.com.

Stamps.com requires a monthly fee for using the service. I signed up for their free, 30-day, no-risk trial, expecting to receive a bunch of free stamps and a postal scale in the mail. My suspicion was aroused when I realized that cancelling my no-risk trial would involve a phone call. I was imagining being put on hold forever, so I tried calling the number before signing up. I reached an automated system which asked the reason for my call (cancelling an account) then asked for my phone number. It told me that this phone number did not correspond to one of their accounts. Could their account cancellation actually be semi-automated? My suspicions were significantly reduced, and I finished the account activation… then they asked me to download their software.

Downloading software from a company that I don’t trust… my suspicion was back and stronger than ever. It could be adware. It could be designed to steal passwords. It could have accidental errors that cause registry corruption or create security vulnerabilities on my computer. I don’t install software lightly! I started searching for reviews of the stamps.com software and found the following gems at download.com:

“You pay for what you get! You Definatly pay!”

“False advertising, promises come slowly”

“Nothing Good about stamps.com unless your a millionaire!”

“After waiting for 45 minutes I finally got a hold of someone. After canceling I asked a bout my $15 refund request, and come to find out I will not be getting my money back! I have to have an exsisting account in order to get a refund. I have just been robbed! $15 isn’t a lot, but my husband and I work hard for our money! Don’t open an account with stamps.com!”

“Can sign up online, but have to call to cancel. First 3 tries I got busy signals. I called again and got through their menu only to be placed on death-hold. Still on hold, it’s been 45 minutes, listening to god-awful muzak all the while. They are trying to get me to hang up, but it’s worth not paying the monthly fee to hang in there and be done with these people. Wish I’d read these reviews before I signed up. :( Don’t sign up, you’ll regret it.”

At this point, my interest in online postage was at an all-time low. I just wanted to drop that account before ending up like all the others. To stamps.com’s credit, I was actually able to cancel in about ten minutes. Maybe they are starting to learn a lesson about how to treat customers, but I was done experimenting. It was time to get this job done the old fashioned way.

The nearest USPS post office was 1.5 miles away. A few minutes in line… another minute for a real person to take my package, charge me regular price ($2.07 for a 1.92 lb package by Media Mail), and take care of everything… A physical post office took care of me in less time than it took to fill out a form online. I know that websites are a better technology, that they could easily beat the convenience of a physical post office, but it’s amazing what so many web designers will do to ruin a customer’s experience online.