People keep asking why Jepsen is written in Clojure, so I figure it’s worth having a referencable answer. I’ve programmed in something like twenty languages. Why choose a Weird Lisp?

Jepsen is built for testing concurrent systems–mostly databases. Because it tests concurrent systems, the language itself needs good support for concurrency. Clojure’s immutable, persistent data structures make it easier to write correct concurrent programs, and the language and runtime have excellent concurrency support: real threads, promises, futures, atoms, locks, queues, cyclic barriers, all of java.util.concurrent, etc. I also considered languages (like Haskell) with more rigorous control over side effects, but decided that Clojure’s less-dogmatic approach was preferable.

Because Jepsen tests databases, it needs broad client support. Almost every database has a JVM client, typically written in Java, and Clojure has decent Java interop.

Because testing is experimental work, I needed a language which was concise, adaptable, and well-suited to prototyping. Clojure is terse, and its syntactic flexibility–in particular, its macro system–work well for that. In particular the threading macros make chained transformations readable, and macros enable re-usable error handling and easy control of resource scopes. The Clojure REPL is really handy for exploring the data a test run produces.

Tests involve representing, transforming, and inspecting complex, nested data structures. Clojure’s data structures and standard library functions are possibly the best I’ve ever seen. I also print a lot of structures to the console and files: Clojure’s data syntax (EDN) is fantastic for this.

Because tests involve manipulating a decent, but not huge, chunk of data, I needed a language with “good enough” performance. Clojure’s certainly not the fastest language out there, but idiomatic Clojure is usually within an order of magnitude or two of Java, and I can shave off the difference where critical. The JVM has excellent profiling tools, and these work well with Clojure.

Jepsen’s (gosh) about a decade old now: I wanted a language with a mature core and emphasis on stability. Clojure is remarkably stable, both in terms of JVM target and the language itself. Libraries don’t “rot” anywhere near as quickly as in Scala or Ruby.

Clojure does have significant drawbacks. It has a small engineering community and no (broadly-accepted, successful) static typing system. Both of these would constrain a large team, but Jepsen’s maintained and used by only 1-3 people at a time. Working with JVM primitives can be frustrating without dropping to Java; I do this on occasion. Some aspects of the polymorphism system are lacking, but these can be worked around with libraries. The error messages are terrible. I have no apologetics for this. ;-)

I prototyped Jepsen in a few different languages before settling on Clojure. A decade in, I think it was a pretty good tradeoff.

Egg Syntax
Egg Syntax on

Hey aphyr! Have missed seeing your posts on twitter, although totally understand the decision to leave.

Some aspects of the polymorphism system are lacking

I’d love to hear a bit more about that – what do you find lacking, and what libs do you use to work around it? I’ve generally been fairly happy with clojure’s polymorphism, so I’m curious.

Thanks! :)

Rich Hickey

Thanks for this! (even if you did call my kid weird)


I have no question about “why something is written in Clojure”. What I’m curious about is why Aphyr now has to explain why anything is written in Clojure? It’s like Michael Schumacher having to explain why he drives manual.

Aphyr on

I’d love to hear a bit more about that – what do you find lacking, and what libs do you use to work around it? I’ve generally been fairly happy with clojure’s polymorphism, so I’m curious.

This comes up fairly infrequently, but I hit it every year or so: there are cases where you really, really want inheritance, and Clojure’s protocol/interface/deftype system are set up to fight you. For instance, take implementing a new variant of Clojure’s core map, vector, or set datatypes. There’s a gazillion functions you have to implement across a dozen interfaces+protocols. Most of them would be sensibly implemented by proxying to another function, or simply add default arguments to a more complete version of the function. In Java you’d inherit from an abstract class, or use default methods in the interfaces. But there’s no real equivalent for this in Clojure.

I wind up using Zach Tellman’s Potemkin a good deal for this–both for defining collections and for its abstract inheritance scheme. Here’s an example from jepsen.history, which implements several flavors of vector with different memory/laziness semantics.

Fave Oled
Fave Oled on

order of magnitude or two of Java 10-100 times? I guess you meant something else

Giovanni Ruggiero
Giovanni Ruggiero on

Thank you for this post (and for Jepsen, of course), but I’m curious: after ten years, would you still use Clojure if you were creating it today?

Aphyr on

10-100 times? I guess you meant something else

Yup! Clojure’s vectors & maps are (as a very loose rule of thumb) about 10x slower than the standard arrays and java.util collections. Primitives are generally boxed, which carries a memory indirection penalty. Clojure’s seq transformations do a lot of intermediate allocation. Reductions always use objects as accumulators, never primitives. That kinda stuff adds up over time. I generally write the simple, slow version first, and then make it faster when the profiler complains.

Aphyr on

… after ten years, would you still use Clojure if you were creating it today?

Heck yeah. Clojure remains my language of choice for most projects. I would like to get into Rust at some point, especially for performance-sensitive code–I imagine some of the checkers in Jepsen could be much faster in Rust. OTOH, I’ve built up some really, really good libraries in Clojure, and it’d be hard to give those up.

Aphyr on

Thanks for this! (even if you did call my kid weird)

Aw, thanks to you Rich. Couldn’t have done any of this work without your good design.

I’m a weird kid too. ;-)


Post a Comment

Comments are moderated. Links have nofollow. Seriously, spammers, give it a rest.

Please avoid writing anything here unless you're a computer. This is also a trap:

Supports Github-flavored Markdown, including [links](, *emphasis*, _underline_, `code`, and > blockquotes. Use ```clj on its own line to start an (e.g.) Clojure code block, and ``` to end the block.