Skip to content

Instantly share code, notes, and snippets.

@holyjak
Last active March 19, 2023 04:36
Show Gist options
  • Star 32 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save holyjak/36c6284c047ffb7573e8a34399de27d8 to your computer and use it in GitHub Desktop.
Save holyjak/36c6284c047ffb7573e8a34399de27d8 to your computer and use it in GitHub Desktop.
Babashka HTTP server for serving static files, similar to `python -m http.server` but more flexible :)
#!/usr/bin/env bb
#_" -*- mode: clojure; -*-"
;; Based on https://github.com/babashka/babashka/blob/master/examples/image_viewer.clj
(ns http-server
(:require [babashka.fs :as fs]
[clojure.java.browse :as browse]
[clojure.string :as str]
[clojure.tools.cli :refer [parse-opts]]
[org.httpkit.server :as server]
[hiccup2.core :as html])
(:import [java.net URLDecoder URLEncoder]))
(def cli-options [["-p" "--port PORT" "Port for HTTP server" :default 8090 :parse-fn #(Integer/parseInt %)]
["-d" "--dir DIR" "Directory to serve files from" :default "."]
["-h" "--help" "Print usage info"]])
(def parsed-args (parse-opts *command-line-args* cli-options))
(def opts (:options parsed-args))
(cond
(:help opts)
(do (println "Start a http server for static files in the given dir. Usage:\n" (:summary parsed-args))
(System/exit 0))
(:errors parsed-args)
(do (println "Invalid arguments:\n" (str/join "\n" (:errors parsed-args)))
(System/exit 1))
:else
:continue)
(def port (:port opts))
(def dir (fs/path (:dir opts)))
(assert (fs/directory? dir) (str "The given dir `" dir "` is not a directory."))
(defn index [f]
(let [files (map #(str (.relativize dir %))
(fs/list-dir f))]
{:body (-> [:html
[:head
[:meta {:charset "UTF-8"}]
[:title (str "Index of `" f "`")]]
[:body
[:h1 "Index of " [:code (str f)]]
[:ul
(for [child files]
[:li [:a {:href (URLEncoder/encode (str child))} child (when (fs/directory? (fs/path dir child)) "/")]])]
[:hr]
[:footer {:style {"text-align" "center"}} "Served by http-server.clj"]]]
html/html
str)}))
(defn body [path]
{:body (fs/file path)})
(server/run-server
(fn [{:keys [:uri]}]
(let [f (fs/path dir (str/replace-first (URLDecoder/decode uri) #"^/" ""))
index-file (fs/path f "index.html")]
(cond
(and (fs/directory? f) (fs/readable? index-file))
(body index-file)
(fs/directory? f)
(index f)
(fs/readable? f)
(body f)
:else
{:status 404 :body (str "Not found `" f "` in " dir)})))
{:port port})
(println "Starting http server at " port "for" (str dir))
(browse/browse-url (format "http://localhost:%s/" port))
@(promise)
@holyjak
Copy link
Author

holyjak commented Mar 3, 2021

Tested with babashka v0.2.12

@brdloush
Copy link

brdloush commented Mar 3, 2021

Doesn't seem to work correctly with my home folder.

~ pwd
/home/brdloush
➜  ~ http-server.bb -p 1234                  

Starting http server at  1234 for .

when I perform following cUrl in other window, the request handling just hangs and no response is generated.

curl localhost:1234

It seems to hand in (fs/glob f "*"), not sure why yet.

@brdloush
Copy link

brdloush commented Mar 3, 2021

Minimal error case

(require '[babashka.fs :as fs])
(require '[clojure.string :as str])
(import [java.net URLDecoder])

(let [f (fs/path "." (str/replace-first (URLDecoder/decode "/") #"^/" ""))]
  (fs/glob f "*"))

^^^ if I make a freezer.bb script out of that and run it from various directories, it yields different results:

  • works fine for my /home/brdloush/Download folder
  • freezes under /home/brdloush
  • throws Exception in /tmp

Hopefully there's some alternative and more stable way of listing files in directory 🤞 :)

@brdloush
Copy link

brdloush commented Mar 3, 2021

Other than those few minor issues, it's very nice! 👏

Btw I'd consider adding [:meta {:charset "UTF-8"}] to :head, so that accented characters (etc) work.

@holyjak
Copy link
Author

holyjak commented Mar 3, 2021 via email

@brdloush
Copy link

brdloush commented Mar 3, 2021

What is the exception you get in /tmp?

Sorry, forgot to paste that one. Seems that it's simply caused by my user not having permissions to acces that directory systemd-private-d113bf5b0b0a40888e4e97ed7dd3e012-haveged.service-I3Q8bg. It's a directory owned by root/root and have 40700 permissions (ie. read/write/execute only by owner).

----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  Visiting /tmp/./systemd-private-d113bf5b0b0a40888e4e97ed7dd3e012-haveged.service-I3Q8bg failed
Location: /tmp/a/freezer.bb:6:3

----- Context ------------------------------------------------------------------
2: (require '[clojure.string :as str])
3: (import [java.net URLDecoder])
4: 
5: (let [f (fs/path "." (str/replace-first (URLDecoder/decode "/") #"^/" ""))]
6:   (fs/glob f "*"))
     ^--- Visiting /tmp/./systemd-private-d113bf5b0b0a40888e4e97ed7dd3e012-haveged.service-I3Q8bg failed

----- Locals -------------------------------------------------------------------
f: #object[sun.nio.fs.UnixPath 0x28d737f6 "."]

----- Stack trace --------------------------------------------------------------

I didn't really get the bb - e '(fs/glob f "\*")' also hang? (might need to add a require) question. How is it different from the Minimal error case I posted before? Besides requires/imports, it also needs the actual f, doesn't it?

@holyjak
Copy link
Author

holyjak commented Mar 3, 2021

Sorry, I thought you were quoting from the script, did not get that it was a repro case. Thank you!

@holyjak
Copy link
Author

holyjak commented Mar 3, 2021

@brdloush Are you in Clojurians Slack? Michiel would appreciate help with getting to the bottom of the issue.

Does list-dir work for you? Does running the same snippet from clojure, not bb, give you the same behavior?

@borkdude
Copy link

borkdude commented Mar 3, 2021

You can try bb -e '(babashka.fs/glob "." "*")' vs bb -e '(babashka.fs/list-dir ".")

@brdloush
Copy link

brdloush commented Mar 3, 2021

Hello @borkdude. Thanks for your help.

  1. bb -e '(babashka.fs/glob "." "*")' "freezes" (see bellow).
  2. bb -e '(babashka.fs/list-dir ".") works and is ultra-fast (0,01s user 0,01s system 105% cpu 0,016 total)

I tried using strace for 1) and it seems that glob "." "*" is actually traversing nested directories. So it's not actually frozen, it would just take ages (and a lot of memory) to get the result.

strace bb -e '(babashka.fs/glob "." "*")'  2>&1 | grep "/home/brdloush" | grep -e openat -e lstat     

it very quickly shows output such as this..

openat(AT_FDCWD, "/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect", O_RDONLY) = 24
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit.js", {st_mode=S_IFREG|0664, st_size=26016, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
openat(AT_FDCWD, "/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit", O_RDONLY) = 26
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/caching.js", {st_mode=S_IFREG|0664, st_size=4409, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/handlers.js", {st_mode=S_IFREG|0664, st_size=12612, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/impl", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
openat(AT_FDCWD, "/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/impl", O_RDONLY) = 28
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/impl/decoder.js", {st_mode=S_IFREG|0664, st_size=12914, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/impl/reader.js", {st_mode=S_IFREG|0664, st_size=2131, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/impl/writer.js", {st_mode=S_IFREG|0664, st_size=18656, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/delimiters.js", {st_mode=S_IFREG|0664, st_size=1062, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/types.js", {st_mode=S_IFREG|0664, st_size=37079, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/eq.js", {st_mode=S_IFREG|0664, st_size=5804, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/com/cognitect/transit/util.js", {st_mode=S_IFREG|0664, st_size=4881, ...}) = 0
lstat("/home/brdloush/./projects/reframe-demo/resources/public/js/compiled/test/out/oops", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0

@borkdude
Copy link

borkdude commented Mar 3, 2021

Thanks for checking. Glob should actually not recurse with just * but there might be a bug in the impl:

https://github.com/babashka/fs/blob/5b8cf66c4cc06bfc24615043c8f8c31f14321f2a/src/babashka/fs.cljc#L248

I will check.

@borkdude
Copy link

borkdude commented Mar 3, 2021

Found the issue. Due to an ordering mistake the glob is always recursive. I tracked it here and fixed it. Will be fixed in the next release of babashka. For now you can use glob with :max-depth 1 or fs/list-dir.

@holyjak
Copy link
Author

holyjak commented Mar 3, 2021

Great job troubleshooting this, @brdloush! Thank you for teaching me about strace :)

@brdloush
Copy link

brdloush commented Mar 3, 2021

@holyjak No problem. strace is a handy little beast especially in cases where some application is for example not loading some config file you're trying to feed it. With a help of strace, you often find out you either misplaced your config file, made a typo in its name or path or something similar :) In general, it's nice to see what files the app is trying to access (and whether it succeeds or fails).

@borkdude Thanks a lot for such a quick fix! 👏

@borkdude
Copy link

borkdude commented Mar 9, 2021

The problem with glob scanning all the files in the directory recursively should now be solved in babashka 0.2.13.

@borkdude
Copy link

borkdude commented Mar 9, 2021

I now added the gist to the babashka examples dir:

https://github.com/babashka/babashka/tree/master/examples#file-server

@brdloush
Copy link

FYI: If you want to run this script in headless environment, the (browse/browse-url (format "http://localhost:%s/" port)) might crash. It internally relies on /usr/bin/xdg-open on linux, which might not be available on headless distribution. So perhaps you can wrap the browse-url call into something like

(when-not (str/blank? (:out (sh/sh "which" "xdg-open")))
  (browse/browse-url (format "http://localhost:%s/" port)))

There might be some better/more idiomatic way to check presensce of xdg-open binary. The problematic browse-url function and its dependencies can be seen here https://github.com/clojure/clojure/blob/master/src/clj/clojure/java/browse.clj

@borkdude
Copy link

Babashka itself has a slightly modified version of browse-url which does not depend on java.awt.Desktop. Feel free to PR improvements to that function.

@cassiel
Copy link

cassiel commented Apr 5, 2021

Haven't tried it yet... but maybe you mean text-align on line 51 rather than text-aling?

@holyjak
Copy link
Author

holyjak commented Apr 6, 2021

Thanks, @cassiel, fixed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment