OSS updates November 2023

In this post I'll give updates about open source I worked on during November 2023.

To see previous OSS updates, go here.

Sponsors

I'd like to thank all the sponsors and contributors that make this work possible! Without you, the below projects would not be as mature or wouldn't exist or be maintained at all.

Open the details section for more info.

Sponsor info Top sponsors:

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!

If you're used to sponsoring through some other means which isn't listed above, please get in touch.

On to the projects that I've been working on!

Advent of Code

It is Advent of Code time of year again. You can solve puzzles in an online squint or cherry playground here.

Change the /squint/ part of the url to /cherry/ to switch ClojureScript dialect versions.

You can read more about the playground here.

Updates

Here are updates about the projects/libraries I've worked on last month.

Other projects

These are (some of the) other projects I'm involved with but little to no activity happened in the past month.

Click for more details

  • grasp: Grep Clojure code using clojure.spec regexes
  • lein-clj-kondo: a leiningen plugin for clj-kondo
  • http-kit: Simple, high-performance event-driven HTTP client+server for Clojure.
  • http-client: babashka's http-client
  • nbb: Scripting in Clojure on Node.js using SCI
  • fs - File system utility library for Clojure
  • deps.clj: A faithful port of the clojure CLI bash script to Clojure
  • babashka.nrepl: The nREPL server from babashka as a library, so it can be used from other SCI-based CLIs
  • rewrite-edn: Utility lib on top of rewrite-clj with common operations to update EDN while preserving whitespace and comments
  • tools-deps-native and tools.bbuild: use tools.deps directly from babashka
  • jet: CLI to transform between JSON, EDN, YAML and Transit using Clojure
  • quickdoc: Quick and minimal API doc generation for Clojure
  • pod-babashka-go-sqlite3: A babashka pod for interacting with sqlite3
  • pod-babashka-fswatcher: babashka filewatcher pod
  • edamame: Configurable EDN/Clojure parser with location metadata
  • lein2deps: leiningen to deps.edn converter
  • scittle: Execute Clojure(Script) directly from browser script tags via SCI
  • sql pods: babashka pods for SQL databases
  • cljs-showcase: Showcase CLJS libs using SCI
  • process: Clojure library for shelling out / spawning sub-processes
  • babashka.book: Babashka manual
  • instaparse-bb
  • rewrite-clj: Rewrite Clojure code and edn
  • pod-babashka-buddy: A pod around buddy core (Cryptographic Api for Clojure).
  • gh-release-artifact: Upload artifacts to Github releases idempotently
  • carve - Remove unused Clojure vars
  • quickblog: Light-weight static blog engine for Clojure and babashka
  • 4ever-clojure - Pure CLJS version of 4clojure, meant to run forever!
  • pod-babashka-lanterna: Interact with clojure-lanterna from babashka
  • joyride: VSCode CLJS scripting and REPL (via SCI)
  • clj2el: transpile Clojure to elisp
  • deflet: make let-expressions REPL-friendly!
  • babashka.json: babashka JSON library/adapter
  • deps.add-lib: Clojure 1.12's add-lib feature for leiningen and/or other environments without a specific version of the clojure CLI

Published: 2023-12-01

Tagged: clojure oss updates

Playing Advent of Code with Squint

In the previous post I described how I built a Cloudflare worker with squint. This worker is part of my attempt to build a playground for squint in which you can play Advent of Code. Advent of Code is a series of programming puzzles published each day through December 1-25. By exercising puzzles solutions on squint, I'm able to detect missing or incompatible features compared to ClojureScript.

Playground

The Advent of Code playground can be loaded here.

This link redirects you to the most recent version and I'll be updating the redirected-to link if necessary.

Puzzle input

To play Advent of Code, you need to download your puzzle input:

(def input (str/trim (js-await (fetch-input 2017 1))))

The fetch-input function is pre-defined in this boilerplate gist and is loaded via the boilerplate query parameter. Advent of Code input is personalized and based on your Advent of Code account. This is why you are asked to fill in your Advent of Code token in the top of the UI. You can obtain this token by visiting Advent of Code, registering + logging in and then find the session cookie in the developer console. Copy paste this value into the UI and hit Save!. After doing so, the session token is saved in local storage for next puzzles you might want to solve. The session token is only saved in your browser. The Cloudflare worker doesn't store anything and only serves as a proxy.

REPL-mode

The playground has two ways of compiling Squint Clojurescript to JS: the normal ES6 mode and REPL-mode. In REPL-mode you're able to incrementally evaluate squint snippets. This works by compiling vars in keys of global objects, much like CLJS does it. E.g.:

(defn foo [])

is compiled to

var squint_core = await import('squint-cljs/core.js');
globalThis.user = globalThis.user || {};
globalThis.user.foo = (function () {
return null;
});
var foo = globalThis.user.foo;

in REPL-mode. This output certainly isn't optimal for JS bundlers like esbuild so it's only intended for development.

By hitting "Compile" the whole editor is compiled + evaluated. The compiled JavaScript is visible in the right output pane. When in REPL-mode, hitting Cmd-Enter (or Windows-Enter) with the cursor after a form will compile only that expression. Use the comment form to evaluate sub-expressions while working towards a complete solution.

While working on your puzzle, the state of the editor is saved to local storage, so if you accidentally close the browser, the input re-appear next time you visit the playground.

Let me know if you're enjoying this and feel free to post issues in the #squint channel on Clojurians Slack!

Published: 2023-11-24

Tagged: clojure squint

Writing a Cloudflare worker with squint and bun

In this post I'll describe how to write a Cloudflare worker with squint and bun.

Introduction

As I tried to build an Advent of Code playground for squint, I wanted to support downloading puzzle input from adventofcode.com. Doing this directly from the playground resulted in CORS issues. Mario Trost in the #adventofcode channel on Clojurians Slack suggested that this could be solved using a Cloudflare worker and indeed it could. Thanks Mario!

A Cloudflare worker is tiny "serverless" application that scales automatically. The first 100k requests a day are free. A worker can be written in JavaScript, among other languages. Workers written in JavaScript can use the Fetch API which comprises of fetch, Request, Response and more standardized stuff, available as global objects in browsers, Node.js, deno, bun and other browser-inspired JavaScript runtimes. These APIs are basically all you need to build a worker.

Squint is a ClojureScript dialect designed for easier JS interop and smaller bundle output. As such, it seems like a good fit for Cloudflare workers. The smaller the code, the better the (cold) startup time.

Bun is a fast all-in-one JavaScript runtime / bundler / toolkit. Bun seems a good tool to use for developing Cloudflare workers since with little boilerplate you can get something running quickly. Bun supports hot reloading of worker code too.

Hello world

To produce a "hello world" worker only the following is necessary:

squint.edn:

{:paths ["src"]
 :extension "js"
 :output-dir "out"}

The Squint CLJS files are going into src and compiled .js files will be written to out.

The only source file:

src/index.cljs:

(ns index)

(defn ^:async handler [{:keys [method] :as req} _env _ctx]
  (js/Response. "hello world"))

(def default
  {:fetch handler})

Beware that your handler isn't going to be called fetch since this will conflict with js/fetch, something that tripped me up. So I just called it handler. Calling a var default in squint will make it the default export, which is something Cloudflare workers use.

Now run bun install squint-cljs and then bun squint watch. In parallel, run bun --hot out/index.js. This will spin up a server. You can make change to the code and new requests will see the changes. This is very convenient for local development and testing.

To bundle everything to a single file, run:

bun build --minify --outdir=dist/out/index.js

Then you can run bun dist/index.js to prove to yourself that the standalone JS works.

The standalone JavaScript file is somewhere around 2 kilobytes. Seriously: 2kb, that's it :)!

To deploy to Cloudflare, I'm using wrangler. It needs a small config file, called wrangler.toml which describes the name of the application and the location of the main JS file:

name = "hello-world"
main = "dist/index.js"
compatibility_date = "2023-09-04"

After writing the config file, you can run bun wrangler deploy. You will be asked to log in to the Cloudflare dashboard, etc. Eventually your application will be running at https://hellow-world.<your-user>.workers.dev.

Proxy

The final worker code can be seen below. The handler looks at an incoming request, and decides whether it's a GET or OPTION request. In the handling of the GET request, the URL params day, year and aoc-token are pulled out. While developing I noticed that the URLSearchParams object implements a Map-like ad-hoc interface, but squint's get function wasn't aware of this, so initially I couldn't use destructuring like this:

(let [{:keys [foo]} (-> (js/URL. "https://foo.com?foo=1") :searchParams)] foo)

to get foo out of this object. Similarly for the Headers object from Response. Both have a get method. I decided to make the squint get function work with any JS datastructure that has a get method. I'm not entirely sure if that is a good idea though since arbitrary methods called get may perform side effects... Let me know in the comments! But after doing so, destructuring worked on the search params.

Then a request is made to https://adventofcode.com to retrieve the input. When doing this from a non-browser, you don't get into any CORS issues. Note that we can nicely use js-await for waiting for promise results. The text that adventofcode.com returned is passed back along with "Access-Control-Allow-Origin" "* headers to satisfy all the CORS ... stuff.

I also noticed the browser playground was doing an OPTION request so I also need to handle those, returning the most permissive headers to satisfy the CORS gods.

src/index.cljs

(ns index)

(defn ^:async handler [{:keys [method] :as req} _env _ctx]
  (if (= :GET method)
    (let [params (-> req :url (js/URL.) :searchParams)
          {:keys [aoc-token day year]} params]
      (if (and aoc-token day year)
        (let [resp (js-await (js/fetch (str "https://adventofcode.com/" year "/day/" day "/input")
                                       {:headers {:cookie (str "session=" aoc-token)}}))
              body (js-await (.text resp))
              resp (js/Response. body {:headers {"Access-Control-Allow-Origin" "*"}})]
          resp)
        (js/Response. "Set 'aoc-token, 'day' and 'year' as a URL query parameter" {:status 400
                                                                                   :headers {"Access-Control-Allow-Origin" "*"}})))
    ;; response for :OPTION
    (js/Response. nil {:status 200
                       :headers {"Access-Control-Allow-Origin" "*"
                                 "Access-Control-Allow-Methods" "GET,HEAD,POST,OPTIONS"
                                 "Access-Control-Allow-Headers" "*"}})))

(def default
  {:fetch handler})

The final production JS is still around 2-3kb. After another `bun wrangler deploy` this browser playground for Advent of Code started working:

Advent of Code playground

You need to insert your Advent of Code session token in the input box, run Compile and after that it's stored in local storage, for other puzzles you might want to solve this year! Give it a spin and let me know what you think.

The source code for the worker is available here.

Published: 2023-11-19

Tagged: clojure squint

Archive