OSS Updates of May - June 2022

In this post I'll give updates about open source I worked on during May and June 2022.

Sponsors

But first off, I'd like to thank all the sponsors and contributors that make this work possible! Top sponsors: Clojurists Together, Roam Research, Adgoji, Cognitect, Nextjournal.

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work via the following organizations:

Projects

Babashka CLI

Turn Clojure functions into CLIs!

This is one of my newest projects. It aims to close the gap between good command line UX and calling Clojure functions. It is very much inspired by the clojure CLI, but solves a problem which sometimes causes frustration, especially among Windows users: having to use quotes in a shell. It also offers support for subcommands. One project benefiting from that is neil. I blogged about babashka CLI here.

Http-server

Serve static assets.

Another new project is http-server, which can be used in Clojure JVM and babashka to serve static assets in an http server.

Clj-kondo workshop

In June I had the honor and pleasure to give a workshop about clj-kondo at ClojureD. You can work through the material yourself if you'd like here. Feel free to join the clj-kondo channel on Clojurians Slack for questions. Here are some pictures from the event.

Jet

CLI to transform between JSON, EDN and Transit, powered with a minimal query language.

Changelog

The jet binary is now available for Apple Silicon and adds specter as part of the standard library for transforming data. Also, the output is colorized and pretty-printed using puget now.

Edamame

Changelog

Configurable EDN/Clojure parser with location metadata. It has been stable for a while and reached version 1.0.0. The API is exposed now in babashka and nbb as well.

Quickdoc

Quickdoc is a tool to generate documentation from namespace/var analysis done by clj-kondo. It's fast and spits out an API.md file in the root of your project, so you can immediately view it on Github. It has undergone significant improvements in the last two months. I'm using quickdoc myself in several projects.

Nbb

Scripting in Clojure on Node.js using SCI.

Changelog

Added edamame.core, cljs.math, nREPL improvements and now has significant faster startup due to an improvement in SCI.

Clojure-lsp

Clojure/Script Language Server (LSP) implementation.

This project is driven by the static analysis done by clj-kondo and used by many people to get IDE-like features in editors like emacs and VSCode.

I added support for Apple Silicon using Cirrus CI.

Babashka

Native, fast starting Clojure interpreter for scripting.

Changelog

Two new version of babashka were released:

0.8.2 and 0.8.156. The last segment of the version number now indicates the release count, so the last release is the 156th release.

Babashka now also has a new Apple Silicon binary built on Cirrus CI. What is very exciting is that babashka can now execute schema from source. Compatibility with malli is underway.

Clj-kondo

A linter for Clojure code that sparks joy.

Changelog

New linters:

Clj-kondo now also has a new Apple Silicon binary built on Cirrus CI.

SCI

Configurable Clojure interpreter suitable for scripting and Clojure DSLs. Powering babashka, nbb, joyride and many other projects.

Changelog

New releases: 0.3.5 - 0.3.32

Highlights:

SCI configs

A collection of ready to be used SCI configurations.

This project contains configurations for reagent, promesa, etc. and are used in nbb, clerk and other projects.

A recent addition was a configuration for cljs.test which is now shared by nbb and joyride.

Process

Changelog

New releases: 0.1.2 - 0.1.4

Highlights:

Support exec call in GraalVM native-images - this means you can replace the current process with another one.

Scittle

The Small Clojure Interpreter exposed for usage in browser script tags.

Added support developing CLJS via nREPL. See docs.

Etaoin

Pure Clojure Webdriver protocol implementation.

This project is now compatible with babashka! Most of the work on this project was done by Lee Read. If you appreciate his work on this, or other projects like rewrite-clj, consider sponsoring him.

Misc

Brief mentions of miscellaneous other projects I worked on:

Discuss this post here.

Published: 2022-06-30

Babashka CLI: turn Clojure functions into CLIs

Babashka CLI is a new library for command line argument parsing. The main ideas:

Given the function:

(defn foo [{:keys [force dir] :as m}]
  (prn m))

and with a little bit of config in your deps.edn, you can call the function from the command line using:

clj -M:foo --force --dir=src

or:

clj -M:foo --force --dir src

which will then print:

{:force true, :dir "src"}

We did not have to teach babashka CLI anything about the expected arguments.

Another accepted syntax is:

clj -M:foo :force false :dir src

and this is parsed as:

{:force false, :dir "src"}

Booleans, numbers and keywords are auto-coerced, but if you want to make things strict, you can use metadata. E.g. if we want to accept a keyword for the option mode:

clj -M:foo :force false :dir src :mode overwrite

and parse it as:

{:force false, :dir "src" :mode :overwrite}

you can teach babashka CLI using metadata:

(defn foo
  {:org.babashka/cli {:coerce {:mode :keyword}}}
  [{:keys [force dir mode] :as m}]
  (prn m))

A leading colon is also accepted (and auto-coerced as keyword):

clj -M:foo :force false :dir src :mode :overwrite

The metadata format is set up in such a way that libraries need not have a dependency on babashka CLI itself.

Did you notice that the -M invocation now becomes almost identical to -X, but without quotes?

clj -M:foo :force true :dir src :mode :overwrite
clj -X:foo :force true :dir '"src"' :mode :overwrite

Let's look at a recent project, http-server, where I used babashka CLI to serve both -X, and -M needs.

The only argument hints defined there right now are:

(def ^:private cli-opts {:coerce {:port :long}})

although that could have been left out since numbers are auto-coerced.

The -main function simply defers to the clojure exec API function (intended for -X usage) with the parsed arguments:

(defn ^:no-doc -main [& args]
  (exec (cli/parse-opts args cli-opts)))

In turn, the exec function adds some light logic making it suitable for command line usage. It prints help when :help is true. Because I'm lazy, I just print the docstring of serve, the function that's going to be called:

(defn exec
  "Exec function, intended for command line usage. Same API as serve but
  blocks until process receives SIGINT."
  {:org.babashka/cli cli-opts}
  [opts]
  (if (:help opts)
    (println (:doc (meta #'serve)))
    (do (serve opts)
        @(promise))))

Also the exec function blocks, preventing the process from immediately exiting.

Now when I add this function to deps.edn using:

:serve {:deps {org.babashka/http-server {:mvn/version "0.1.3"}}
        :main-opts ["-m" "babashka.http-server"]
        :exec-fn babashka.http-server/exec}

it can be called both with -M and -X:

$ clj -M:serve --port 1339

or:

$ clj -M:serve :port 1339

or:

$ clj -X:serve :port 1339

And help printing is supported in both styles:

$ clj -M:serve --help
Serves static assets using web server.
Options:
  * `:dir` - directory from which to serve assets
  * `:port` - port

or:

$ clj -X:serve :help true

The -main function can also be used in babashka scripts:

#!/usr/bin/env bb

(require '[babashka.deps :as deps])
(deps/add-deps
 '{:deps {org.babashka/http-server {:mvn/version "0.1.3"}}})

(require '[babashka.http-server :as http-server])

(apply http-server/-main *command-line-args*)
$ http-server --help
$ http-server --port 1339

I hope you're convinced that with very little code, babashka CLI can let you support both -M, -X style invocations and babashka scripts, while improving command line UX!

Discuss this post here.

Published: 2022-06-24

Babashka survey Q1 2022 results!

In Q1 of 2022 I ran the babashka survey again. I've done this once before (in November 2020) to see how people are using babashka and what they think could be improved. This year about 200 people responded. That is double the amount since the previous survey!

Here follows every question of the survey with a summary of the reactions.

How are you using babashka?

The first two options were selected in the majority of answers. Unsurprisingly most people are using babashka as a shell-scripting replacement for bash, exactly how it was intended. Babashka tasks is catching on: almost half of babashka users are using it to replace their Makefiles.

Are you using babashka for personal projects, at work, or both?

About 50% of people used babashka both for personal projects and work. Only personal and only work was about 25% each. Some people use babashka to sneak in some Clojure at their non-Clojure jobs.

Where do you think babashka should be improved?

This question was free format. Some common themes:

What features / libraries / namespaces in babashka do you use the most?

This question was also free format. Probably the top 5 most named libraries:

What features or namespaces in babashka are redundant and could be left out?

The majority answer here was: none. People seem to be happy with the current selection of libraries.

What features or libraries would you like to see in babashka in the future, if any?

It's hard to see common patterns here. A random pick:

Note that spec is available in babashka via this library. Specter also works from source now since :mvn/version "1.1.4". Dealing with time is done via java.time and cljc.java-time is one of the java.time based libraries that work in babashka.

Are you using babashka.* libraries with Clojure on the JVM?

In order of mentions:

It's great that people are using fs and process on the JVM. I didn't expect so many people to be using babashka.curl on the JVM too!

Is the binary size of babashka important to you?

84% of the people didn't care, 12% cared and 4% had a nuanced answer.

Note that bigger binary size in babashka is correlated with a longer compilation time and more strain on CI resources. That is the primary reason I'm trying to keep it down.

What operating system are you using babashka on?

Which babashka pods are you using, if any?

The top 7:

Note that for AWS we now also have a source-compatible library: awyeah-api.

When would you use babashka instead of JVM Clojure?

A pick of the free-formatted answers:

Any other feedback on babashka you would like to give?

Lots of thank you's and compliments... *blush*

Are you a user of other babashka-related projects? Please share!

Favorite pick:

Other answers:

Conclusion

The majority of answers to this survey were not too surprising, yet a good confirmation that babashka is doing what it's supposed to be doing. In individual answers there were a few fun discoveries, like learning which big companies are using babashka.

You can read results from the previous survey here. In that survey I asked users what they thought was currently missing in babashka. You can find those answers here. I think almost all points were addressed in the meanwhile. The improvements requested in this survey feel mostly as finishing touches: documentation, error messages, nREPL improvements, so it's probably safe to say that babashka now covers the most important Clojure scripting use cases.

Hardly anyone mentioned performance as something that should be improved, although that has been an area of focus in the beginning of this year and something I will be looking into in an ongoing basis. Most if not all of that work is happening in SCI, which is also powering nbb and other JS targets.

If you are interested in the raw (anonimized) data for this survey, then send me a message.

Thanks

Thanks for taking the survey and being part of the babashka community! Also, thanks to everyone making babashka possible. Sponsors, contributors and users.

Discuss this post here.

Published: 2022-05-12

Archive