Archive Tags Discuss Feed Twitter About

REPL adventures

A blog mostly about Clojure and ClojureScript.

OSS updates March and April 2025

In this post I'll give updates about open source I worked on during March and April 2025.

To see previous OSS updates, go here.

Sponsors

I'd like to thank all the sponsors and contributors that make this work possible. Without you the below projects would not be as mature or wouldn't exist or be maintained at all! So a sincere thank you to everyone who contributes to the sustainability of these projects.

gratitude

Current top tier sponsors:

Open the details section for more info about sponsoring.

Sponsor info

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!

On to the projects that I've been working on!

Blog posts

I blogged about an important improvement in babashka regarding type hints here.

Interviews

Also I did an interview with Jiri from Clojure Corner by Flexiana, viewable here.

Updates

Here are updates about the projects/libraries I've worked on in the last two months.

Other projects

These are (some of the) other projects I'm involved with but little to no activity happened in the past month.

Click for more details

  • rewrite-edn: Utility lib on top of
  • deps.clj: A faithful port of the clojure CLI bash script to Clojure
  • scittle: Execute Clojure(Script) directly from browser script tags via SCI
  • rewrite-clj: Rewrite Clojure code and edn
  • pod-babashka-go-sqlite3: A babashka pod for interacting with sqlite3
  • tools-deps-native and tools.bbuild: use tools.deps directly from babashka
  • http-client: babashka's http-client
  • http-server: serve static assets
  • bbin: Install any Babashka script or project with one comman
  • sci.configs: A collection of ready to be used SCI configs.
    • Added a configuration for cljs.spec.alpha and related namespaces
  • qualify-methods
    • Initial release of experimental tool to rewrite instance calls to use fully qualified methods (Clojure 1.12 only0
  • neil: A CLI to add common aliases and features to deps.edn-based projects.
  • tools: a set of bbin installable scripts
  • sci.nrepl: nREPL server for SCI projects that run in the browser
  • babashka.json: babashka JSON library/adapter
  • squint-macros: a couple of macros that stand-in for applied-science/js-interop and promesa to make CLJS projects compatible with squint and/or cherry.
  • grasp: Grep Clojure code using clojure.spec regexes
  • lein-clj-kondo: a leiningen plugin for clj-kondo
  • http-kit: Simple, high-performance event-driven HTTP client+server for Clojure.
  • babashka.nrepl: The nREPL server from babashka as a library, so it can be used from other SCI-based CLIs
  • jet: CLI to transform between JSON, EDN, YAML and Transit using Clojure
  • pod-babashka-fswatcher: babashka filewatcher pod
  • lein2deps: leiningen to deps.edn converter
  • cljs-showcase: Showcase CLJS libs using SCI
  • babashka.book: Babashka manual
  • pod-babashka-buddy: A pod around buddy core (Cryptographic Api for Clojure).
  • gh-release-artifact: Upload artifacts to Github releases idempotently
  • carve - Remove unused Clojure vars
  • 4ever-clojure - Pure CLJS version of 4clojure, meant to run forever!
  • pod-babashka-lanterna: Interact with clojure-lanterna from babashka
  • joyride: VSCode CLJS scripting and REPL (via SCI)
  • clj2el: transpile Clojure to elisp
  • deflet: make let-expressions REPL-friendly!
  • deps.add-lib: Clojure 1.12's add-lib feature for leiningen and/or other environments without a specific version of the clojure CLI

Published: 2025-05-02

Tagged: clojure oss updates

Babashka Java interop, reflection and type hints

Consider the following Clojure code:

(def thread-pool (java.util.concurrent.Executors/newCachedThreadPool))
(def fut (.submit thread-pool (fn [] 3)))
@fut ;;=> ?

I didn't use any type hints, but the names of the vars should give you an idea what's happening:

What result would you expect this program to give? My initial guess would be 3.

However, in babashka the result would sometimes be nil (version 1.12.199 on macOS Apple silicon does for example). I've seen this happen in JVM Clojure too in CI (given no type hints, and thus relying on Clojure reflection). I discovered this problem when trying to make bb run the fusebox library and executing its test suite in CI using babashka.

What's the mechanism behind this flakiness? The .submit method in the snippet above is overloaded. When Clojure is doing reflection, it finds the most suitable method for .submit given the instance object and the arguments. In this case the type of the instance object (the thread-pool value) is of type java.util.concurrent.ThreadPoolExecutor, which has three overloads (inherited from java.util.concurrent.AbstractExecutorService):

Only two of those match the number of arguments we used in the snippet, so we are left with:

Clojure's reflector will try to see if the argument type to .submit matches any of these methods. Since we called it with a function, in this case it matches both. If multiple methods match, it will try to pick the most specific method using clojure.lang.Compiler/subsumes. Since Runnable and Callable are two distinct types (one doesn't inherit from the other), Clojure's reflector just returns the first method it deemed most suitable. So it could be either method. The java documentation mentions the submit method with the Runnable argument first, but this isn't necessarily the order in which Java reflection will list those methods. I have found out that the order may even be indeterministic over multiple CI runs and JVM implementations or versions. What the exact cause of this indeterminism is, I don't know, but I found out the hard way that it exists 😅.

So when does the above snippet return nil? When the Runnable overload is chosen, since Runnable is an interface with run method that returns void (which is nil in Clojure). Let me show this using a snippet that uses JVM Clojure type hinting, where we don't rely on reflection:

(set! *warn-on-reflection* true)

(def ^java.util.concurrent.ThreadPoolExecutor thread-pool
  (java.util.concurrent.Executors/newCachedThreadPool))
(def fut (.submit thread-pool ^Runnable (fn [] 3)))
@fut  ;; => nil

Now let's do the same with Callable:

(set! *warn-on-reflection* true)

(def ^java.util.concurrent.ThreadPoolExecutor thread-pool
  (java.util.concurrent.Executors/newCachedThreadPool))
(def fut (.submit thread-pool ^Callable (fn [] 3)))
@fut ;; => 3

So we see the issue: depending on the overload of .submit we get a different value when we dereference the future, either nil or the value returned from the function.

So far, babashka has relied exclusively on runtime reflection to implement Java interop. So we do get into a problem with the above snippet, unless we add type hints. But so far babashka, or more specifically, SCI hasn't made use of type hints to determine the most suitable method. The Clojure compiler does this, but as you know, SCI interprets code and doesn't make use of the clojure Compiler. It does use a forked version of clojure.lang.Reflector, clojure's runtime reflection code, aptly renamed to sci.impl.Reflector. So far, pretty much the only change to that code was making some methods public that were used internally by SCI. To fix the above problem, SCI now actually makes use of type hints and passes these to sci.impl.Reflector. So in the newly published version of babashka, this code:

(def thread-pool (java.util.concurrent.Executors/newCachedThreadPool))
(def fut (.submit thread-pool ^Callable (fn [] 3)))
@fut ;;=> 3

will consistently return 3 and when changing Callable to Runnable, you'll get nil. Note that the above snippet will still have the ambiguous behavior in JVM Clojure, since it doesn't have enough type hints to figure out the right method at compile time and clojure.lang.Reflector does nothing with type hints. Once the Clojure compiler finds out that you're calling .submit on a java.util.concurrent.ExecutorService though, the code will run with the expected method.

Can you think of more methods in the Java standard library which have this ambiguity when relying solely on Clojure reflection? I'd love to hear about those to see if they work reliably in babashka given some extra help through type hints.

Thanks to Tim Pote for thinking along when fixing the issues in babashka and reading a preview of this blog post.

Published: 2025-04-26

Tagged: clojure babashka

OSS updates January and February 2025

In this post I'll give updates about open source I worked on during January and February 2025.

To see previous OSS updates, go here.

Sponsors

I'd like to thank all the sponsors and contributors that make this work possible. Without you, the below projects would not be as mature or wouldn't exist or be maintained at all.

Current top tier sponsors:

Open the details section for more info about sponsoring.

Sponsor info

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!

On to the projects that I've been working on!

Updates

As I'm writing this I'm still recovering from a flu that has kept me bedridden for a good few days, but I'm starting to feel better now.

Here are updates about the projects/libraries I've worked on in the last two months.

Other projects

These are (some of the) other projects I'm involved with but little to no activity happened in the past month.

Click for more details

  • CLI: Turn Clojure functions into CLIs!
  • scittle: Execute Clojure(Script) directly from browser script tags via SCI
  • nbb: Scripting in Clojure on Node.js using SCI
  • rewrite-clj: Rewrite Clojure code and edn
  • pod-babashka-go-sqlite3: A babashka pod for interacting with sqlite3
  • tools-deps-native and tools.bbuild: use tools.deps directly from babashka
  • http-client: babashka's http-client
  • cherry: Experimental ClojureScript to ES6 module compiler
  • http-server: serve static assets
  • bbin: Install any Babashka script or project with one comman
  • sci.configs: A collection of ready to be used SCI configs.
    • Added a configuration for cljs.spec.alpha and related namespaces
  • qualify-methods
    • Initial release of experimental tool to rewrite instance calls to use fully qualified methods (Clojure 1.12 only0
  • clerk: Moldable Live Programming for Clojure
    • Add support for :require-cljs which allows you to use .cljs files for render functions
    • Add support for nREPL for developing render functions
  • process: Clojure library for shelling out / spawning sub-processes
    • Work has started to support prepending output (in support for babashka parallel tasks). Stay tuned.
  • neil: A CLI to add common aliases and features to deps.edn-based projects.
  • quickdoc: Quick and minimal API doc generation for Clojure
  • tools: a set of bbin installable scripts
  • sci.nrepl: nREPL server for SCI projects that run in the browser
  • html: Html generation library inspired by squint's html tag
  • instaparse-bb: Use instaparse from babashka
  • babashka.json: babashka JSON library/adapter
  • squint-macros: a couple of macros that stand-in for applied-science/js-interop and promesa to make CLJS projects compatible with squint and/or cherry.
  • grasp: Grep Clojure code using clojure.spec regexes
  • lein-clj-kondo: a leiningen plugin for clj-kondo
  • http-kit: Simple, high-performance event-driven HTTP client+server for Clojure.
  • babashka.nrepl: The nREPL server from babashka as a library, so it can be used from other SCI-based CLIs
  • jet: CLI to transform between JSON, EDN, YAML and Transit using Clojure
  • pod-babashka-fswatcher: babashka filewatcher pod
  • lein2deps: leiningen to deps.edn converter
  • cljs-showcase: Showcase CLJS libs using SCI
  • babashka.book: Babashka manual
  • pod-babashka-buddy: A pod around buddy core (Cryptographic Api for Clojure).
  • gh-release-artifact: Upload artifacts to Github releases idempotently
  • carve - Remove unused Clojure vars
  • 4ever-clojure - Pure CLJS version of 4clojure, meant to run forever!
  • pod-babashka-lanterna: Interact with clojure-lanterna from babashka
  • joyride: VSCode CLJS scripting and REPL (via SCI)
  • clj2el: transpile Clojure to elisp
  • deflet: make let-expressions REPL-friendly!
  • deps.add-lib: Clojure 1.12's add-lib feature for leiningen and/or other environments without a specific version of the clojure CLI

Published: 2025-02-28

Tagged: clojure oss updates

Archive