In this post I'll give updates about open source I worked on during May and June 2022.
But first off, I'd like to thank all the sponsors and contributors that make this work possible! Top sponsors: Clojurists Together, Roam Research, Adgoji, Cognitect, Nextjournal.
If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work via the following organizations:
Turn Clojure functions into CLIs!
This is one of my newest projects. It aims to close the gap between good command line UX and calling Clojure functions. It is very much inspired by the clojure CLI, but solves a problem which sometimes causes frustration, especially among Windows users: having to use quotes in a shell. It also offers support for subcommands. One project benefiting from that is neil. I blogged about babashka CLI here.
Serve static assets.
Another new project is http-server, which can be used in Clojure JVM and babashka to serve static assets in an http server.
In June I had the honor and pleasure to give a workshop about clj-kondo at ClojureD. You can work through the material yourself if you'd like here. Feel free to join the clj-kondo channel on Clojurians Slack for questions. Here are some pictures from the event.
CLI to transform between JSON, EDN and Transit, powered with a minimal query language.
The jet binary is now available for Apple Silicon and adds specter as part of the standard library for transforming data. Also, the output is colorized and pretty-printed using puget now.
Configurable EDN/Clojure parser with location metadata. It has been stable for a while and reached version 1.0.0. The API is exposed now in babashka and nbb as well.
Quickdoc is a tool to generate documentation from namespace/var analysis done by clj-kondo. It's fast and spits out an API.md file in the root of your project, so you can immediately view it on Github. It has undergone significant improvements in the last two months. I'm using quickdoc myself in several projects.
Scripting in Clojure on Node.js using SCI.
Added edamame.core, cljs.math, nREPL improvements and now has significant faster startup due to an improvement in SCI.
Clojure/Script Language Server (LSP) implementation.
This project is driven by the static analysis done by clj-kondo and used by many people to get IDE-like features in editors like emacs and VSCode.
I added support for Apple Silicon using Cirrus CI.
Native, fast starting Clojure interpreter for scripting.
Two new version of babashka were released:
0.8.2 and 0.8.156. The last segment of the version number now indicates the release count, so the last release is the 156th release.
Babashka now also has a new Apple Silicon binary built on Cirrus CI. What is very exciting is that babashka can now execute schema from source. Compatibility with malli is underway.
A linter for Clojure code that sparks joy.
New linters:
Clj-kondo now also has a new Apple Silicon binary built on Cirrus CI.
Configurable Clojure interpreter suitable for scripting and Clojure DSLs. Powering babashka, nbb, joyride and many other projects.
New releases: 0.3.5 - 0.3.32
Highlights:
sci.async namespace for asynchronous loading of codedefrecordA collection of ready to be used SCI configurations.
This project contains configurations for reagent, promesa, etc. and are used in nbb, clerk and other projects.
A recent addition was a configuration for cljs.test which is now shared by nbb and joyride.
New releases: 0.1.2 - 0.1.4
Highlights:
Support exec call in GraalVM native-images - this means you can replace the current process with another one.
The Small Clojure Interpreter exposed for usage in browser script tags.
Added support developing CLJS via nREPL. See docs.
Pure Clojure Webdriver protocol implementation.
This project is now compatible with babashka! Most of the work on this project was done by Lee Read. If you appreciate his work on this, or other projects like rewrite-clj, consider sponsoring him.
Brief mentions of miscellaneous other projects I worked on:
Discuss this post here.
Published: 2022-06-30
Babashka CLI is a new library for command line argument parsing. The main ideas:
-X style invocations. For lazy people like me! If you are not familiar with clj -X, read the docs here.:dir foo instead of :dir '"foo"' or who knows how to write the latter in cmd.exe or Powershell.Given the function:
(defn foo [{:keys [force dir] :as m}]
(prn m))
and with a little bit of config in your deps.edn, you can call the function from the command line using:
clj -M:foo --force --dir=src
or:
clj -M:foo --force --dir src
which will then print:
{:force true, :dir "src"}
We did not have to teach babashka CLI anything about the expected arguments.
Another accepted syntax is:
clj -M:foo :force false :dir src
and this is parsed as:
{:force false, :dir "src"}
Booleans, numbers and keywords are auto-coerced, but if you want to make things strict, you can use metadata. E.g. if we want to accept a keyword for the option mode:
clj -M:foo :force false :dir src :mode overwrite
and parse it as:
{:force false, :dir "src" :mode :overwrite}
you can teach babashka CLI using metadata:
(defn foo
{:org.babashka/cli {:coerce {:mode :keyword}}}
[{:keys [force dir mode] :as m}]
(prn m))
A leading colon is also accepted (and auto-coerced as keyword):
clj -M:foo :force false :dir src :mode :overwrite
The metadata format is set up in such a way that libraries need not have a dependency on babashka CLI itself.
Did you notice that the -M invocation now becomes almost identical to -X, but without quotes?
clj -M:foo :force true :dir src :mode :overwrite
clj -X:foo :force true :dir '"src"' :mode :overwrite
Let's look at a recent project, http-server, where I used babashka CLI to serve both -X, and -M needs.
The only argument hints defined there right now are:
(def ^:private cli-opts {:coerce {:port :long}})
although that could have been left out since numbers are auto-coerced.
The -main function simply defers to the clojure exec API function (intended for -X usage) with the parsed arguments:
(defn ^:no-doc -main [& args]
(exec (cli/parse-opts args cli-opts)))
In turn, the exec function adds some light logic making it suitable for command line usage. It prints help when :help is true. Because I'm lazy, I just print the docstring of serve, the function that's going to be called:
(defn exec
"Exec function, intended for command line usage. Same API as serve but
blocks until process receives SIGINT."
{:org.babashka/cli cli-opts}
[opts]
(if (:help opts)
(println (:doc (meta #'serve)))
(do (serve opts)
@(promise))))
Also the exec function blocks, preventing the process from immediately exiting.
Now when I add this function to deps.edn using:
:serve {:deps {org.babashka/http-server {:mvn/version "0.1.3"}}
:main-opts ["-m" "babashka.http-server"]
:exec-fn babashka.http-server/exec}
it can be called both with -M and -X:
$ clj -M:serve --port 1339
or:
$ clj -M:serve :port 1339
or:
$ clj -X:serve :port 1339
And help printing is supported in both styles:
$ clj -M:serve --help
Serves static assets using web server.
Options:
* `:dir` - directory from which to serve assets
* `:port` - port
or:
$ clj -X:serve :help true
The -main function can also be used in babashka scripts:
#!/usr/bin/env bb
(require '[babashka.deps :as deps])
(deps/add-deps
'{:deps {org.babashka/http-server {:mvn/version "0.1.3"}}})
(require '[babashka.http-server :as http-server])
(apply http-server/-main *command-line-args*)
$ http-server --help
$ http-server --port 1339
I hope you're convinced that with very little code, babashka CLI can let you support both -M, -X style invocations and babashka scripts, while improving command line UX!
Discuss this post here.
Published: 2022-06-24
In Q1 of 2022 I ran the babashka survey again. I've done this once before (in November 2020) to see how people are using babashka and what they think could be improved. This year about 200 people responded. That is double the amount since the previous survey!
Here follows every question of the survey with a summary of the reactions.
The first two options were selected in the majority of answers. Unsurprisingly most people are using babashka as a shell-scripting replacement for bash, exactly how it was intended. Babashka tasks is catching on: almost half of babashka users are using it to replace their Makefiles.
About 50% of people used babashka both for personal projects and work. Only personal and only work was about 25% each. Some people use babashka to sneak in some Clojure at their non-Clojure jobs.
This question was free format. Some common themes:
deftypeThis question was also free format. Probably the top 5 most named libraries:
The majority answer here was: none. People seem to be happy with the current selection of libraries.
It's hard to see common patterns here. A random pick:
Note that spec is available in babashka via this
library. Specter also works from source now since :mvn/version "1.1.4". Dealing with time is done via java.time and cljc.java-time is one of the java.time based libraries that work in babashka.
In order of mentions:
It's great that people are using fs and process on the JVM. I didn't expect so many people to be using babashka.curl on the JVM too!
84% of the people didn't care, 12% cared and 4% had a nuanced answer.
Note that bigger binary size in babashka is correlated with a longer compilation time and more strain on CI resources. That is the primary reason I'm trying to keep it down.
The top 7:
Note that for AWS we now also have a source-compatible library: awyeah-api.
A pick of the free-formatted answers:
Lots of thank you's and compliments... *blush*
Favorite pick:
<script> tag.Other answers:
The majority of answers to this survey were not too surprising, yet a good confirmation that babashka is doing what it's supposed to be doing. In individual answers there were a few fun discoveries, like learning which big companies are using babashka.
You can read results from the previous survey here. In that survey I asked users what they thought was currently missing in babashka. You can find those answers here. I think almost all points were addressed in the meanwhile. The improvements requested in this survey feel mostly as finishing touches: documentation, error messages, nREPL improvements, so it's probably safe to say that babashka now covers the most important Clojure scripting use cases.
Hardly anyone mentioned performance as something that should be improved, although that has been an area of focus in the beginning of this year and something I will be looking into in an ongoing basis. Most if not all of that work is happening in SCI, which is also powering nbb and other JS targets.
If you are interested in the raw (anonimized) data for this survey, then send me a message.
Thanks for taking the survey and being part of the babashka community! Also, thanks to everyone making babashka possible. Sponsors, contributors and users.
Discuss this post here.
Published: 2022-05-12